Agent Code Guard

Markdown Structured-Artifact Guard Evidence

1. Executive conclusion

Overall outcome: ACCEPT. Markdown belongs in Agent Code Guard through two separate, conservative inspection anchors, but this research PR activates nothing:

Candidate Outcome Future universal default Future default enabled State
Document physical lines ACCEPT REVIEW when > 800 Yes PASS / REVIEW
Document nonblank lines Reject as the primary variant — — —
Direct-content section physical span ACCEPT REVIEW when > 200 Yes PASS / REVIEW
Heading-subtree span REJECT — — —
Maximum heading depth REJECT / OUT OF SCOPE — — —

REVIEW means inspect structure, not “split automatically.” A review may conclude reviewed; coherent; keep. There is no FAIL threshold. Both admitted defaults should permit a positive project override and explicit disablement. The exact threshold passes; only a greater measurement reviews.

The evidence is 912 Markdown documents from five heterogeneous repositories. At the proposed boundaries, 59 documents (6.47%) review for document size and 29 section findings affect 29 documents (3.18%) for direct section size; those findings are 0.22% of 13,157 measured direct sections. The findings include genuine navigation/cohesion prompts and deliberate coherent keep cases. This is an acceptable REVIEW workload, not a claim that either number is a quality law.

2. Why Markdown was evaluated

Markdown is the first structured candidate because repositories routinely put overview, design, runbook, specification, RFC, and contributor knowledge in it. Agents frequently retrieve or extend only one topic. Ever-growing files and heading-delimited blocks increase navigation and irrelevant context even when the prose is valid Markdown. The concern is maintainability structure, not spelling, grammar, links, accessibility, or formatting style.

3. Candidate metrics

The research tool compared:

All counts use logical physical records: empty file is zero, x and x\n are one, LF and CRLF are equivalent, and a final unterminated line counts once. Whitespace-only lines are blank. Preamble belongs to the document but not to a synthetic section. A headingless document has no sections.

4. Structural parsing semantics

research/markdown_guard_sample.py is a research-only bounded scanner. It is not imported by production and adds no install dependency.

Its explicit top-level outline contract is:

The scanner deliberately measures the document’s top-level Markdown outline. It does not promote headings nested in block quotes or list items into that outline, interpret raw HTML headings, or implement multi-line Setext paragraph titles. Those constructs remain content. The future production contract should retain these bounds unless a fixture demonstrates that broader container semantics materially improve the guard. This is CommonMark-informed, not a claim to be a complete CommonMark parser.

5. Section ownership models

Model Ownership What it measures Evidence result
Heading subtree Heading through the next heading of equal or shallower level An entire structural branch Repeats document size, especially for top-level headings; noisy
Direct content Heading through immediately before the next heading of any level One locally undivided documentation unit Distinct and actionable

Subtree ranges intentionally overlap. Direct ranges do not. Both include the heading syntax; a Setext heading includes its title and underline. Adjacent headings therefore create a one-line ATX or two-line Setext section, not a zero-line finding.

The subtree median was 95 lines, but even >300 reviewed 151 documents (16.56%). Root headings often owned nearly the whole file: OpenTelemetry’s 2,012-line metrics SDK produced a 2,007-line subtree. That says little beyond the already clearer document measurement. Direct content instead isolates a single block such as Wayfarer’s 242-line testing runbook or ripgrep’s 1,063-line FAQ.

6. Corpus and pinned SHAs

All repositories were cloned through gh outside this repository and removed after analysis. Scope was recursive *.md and *.markdown; Git contents were the exact checked-out commits below.

Repository Exact SHA Files Role/category notes and exclusions
stef-k/agent-code-guard e6bfcd5ee18dfa8ce21f153c6da56a8f276845b1 14 README, admission/design/evidence docs, agent skill/policy Markdown; no exclusions
stef-k/Wayfarer 264e74d9e5b9f93c83ce1eff659b399b2eb22cf6 29 README, architecture, runbooks, process docs; excluded wwwroot/lib/** vendored docs (1 file)
BurntSushi/ripgrep 3fce3b5bb0236da2df6d99672afb8a719642eca7 22 README, guide, FAQ, contributor/process docs; excluded CHANGELOG.md
open-telemetry/opentelemetry-specification 1377f53b2bc0683c45169b8f20fd973eb4d59419 193 specification, OTEP/design, process and README documents; excluded CHANGELOG.md
rust-lang/rfcs 354518a8c9025f40be6f730452c1bfe71a12dc22 654 long-form RFC/specification corpus plus repository process docs; no generated or vendored tree identified

The corpus deliberately over-represents specification/RFC documents so that a universal rule must survive coherent long-form material. No sampled file used the .markdown suffix. A first production slice should therefore support .md; .markdown remains a deterministic research input but lacks corpus evidence for default applicability.

The compact reproduction procedure was: clone each repository with gh repo clone OWNER/REPO <outside-repo-path>, check out the recorded SHA, and run the research script from that clone root. The exact measurement forms were:

python <agent-code-guard>/research/markdown_guard_sample.py . --output <scratch.json>
python <agent-code-guard>/research/markdown_guard_sample.py . --exclude 'wwwroot/lib/**' --output <scratch.json>
python <agent-code-guard>/research/markdown_guard_sample.py . --exclude 'CHANGELOG.md' --output <scratch.json>

The first form applied to Agent Code Guard and Rust RFCs, the second to Wayfarer, and the third to ripgrep and OpenTelemetry. Default candidate sets in the JSON were document 300/500/800/1200 and section 100/150/200/240/300; the compact tables additionally calculate the nearby stated boundaries from the emitted per-document rows. Raw JSON was deliberately not committed.

Manual inspection was role-labeled rather than used to create role thresholds:

Role Inspected examples Observed behavior
README / overview Agent Code Guard README (385/48), ripgrep README (541/218), Wayfarer README (233/57) Direct section catches ripgrep’s unusually large local block; totals remain below document default
Guide / tutorial ripgrep GUIDE (1,025/101) Document reviews while disciplined local sections pass
Architecture / design Wayfarer architecture (256/20), OpenTelemetry OTEPs Typically structured; large OTEPs can review for document or code-heavy local units
Specification / RFC OpenTelemetry metrics SDK (2,012/109), Rust RFC boundary/outliers below Document signal remains useful where direct units are disciplined; coherent keep is common
Runbook / process Wayfarer testing (245/242), OpenTelemetry CONTRIBUTING (195/31) Direct span distinguishes the monolithic runbook from the structured process guide
Reference / generated-like ripgrep FAQ (1,063/1,063), OpenTelemetry profile proto (1,676/565) Strongest coherent/no-change pressure; still useful REVIEW, no role exemption

Values in parentheses are document physical lines / maximum direct-section physical span. Changelogs were excluded rather than used as a role threshold.

7. Document-size distributions

Aggregate

Variant n Median P75 P90 P95 P99 Max
All physical lines 912 182 370 657 881 1,722 2,205
Nonblank physical lines 912 136 273 508 723 1,310 1,809

Physical lines by corpus

Corpus n Median P75 P90 P95 P99 Max
Agent Code Guard 14 136 317 385 387 387 387
Wayfarer 29 132 250 443 608 763 763
ripgrep 22 33 57 541 1,025 1,063 1,063
OpenTelemetry 193 136 318 676 1,000 1,676 2,012
Rust RFCs 654 202.5 391 661 916 1,774 2,205

8. Document-size candidate thresholds

Physical boundary Reviews Rate
>400 202 22.15%
>500 147 16.12%
>600 105 11.51%
>800 59 6.47%
>1,000 37 4.06%
>1,200 22 2.41%

The selected >800 boundary is conservative while still catching large ordinary docs and extreme structured specs. Per corpus it reviews 0% Agent Code Guard, 0% Wayfarer, 9.09% ripgrep, 7.25% OpenTelemetry, and 6.57% Rust RFCs. This is broad rather than driven by one repository.

The closest nonblank comparison, >600, reviewed 62 documents (6.80%). The upper-tail membership and rates were similar. Nonblank lines make formatting compression an especially direct gaming path and discount real scrolling and navigation surface. Blank lines are not demonstrated noise, so all physical lines are the simpler and more honest measurement.

9. Document boundary inspections

Position Role Example Measurement Inspection
Below Specification OpenTelemetry trace/tracestate-probability-sampling.md 497 Coherent specification with usable headings; no split needed, but below the conservative default
Around middle candidate RFC Rust RFC 3513-gen-blocks.md 803 Many language-by-language prior-art subsections; navigation is material and REVIEW is useful even if retained
Moderately above RFC Rust RFC 2091-inline-semantic.md 1,199 Long semantic proposal; structured but sufficiently large that extension deserves an outline check
Extreme RFC Rust RFC 1398-kinds-of-allocators.md 2,205 Formal, intentionally comprehensive RFC; coherent keep is plausible, but targeted retrieval cost is real
Extreme Specification OpenTelemetry specification/metrics/sdk.md 2,012 Mature multi-topic specification with a 2,007-line root subtree; document REVIEW is clearer than subtree REVIEW

Total size is coarse, but not redundant with direct section size. A 2,000-line specification can have disciplined 100-line local units while still imposing a large navigation and context surface. The finding must point to the whole file and ask whether its outline, responsibility, and extension location remain clear; it must not demand a split.

10. Section-size distributions

Per document, the metric below is the maximum section span.

Ownership model n Median P75 P90 P95 P99 Max
Direct content 912 46 77 121 158 265 1,063
Heading subtree 912 95 211 395 550 1,258 2,007

11. Section-size candidate thresholds

Affected documents by maximum direct-content span

Boundary Affected documents Document rate
>100 129 14.14%
>120 95 10.42%
>150 55 6.03%
>160 44 4.82%
>200 29 3.18%
>240 17 1.86%
>300 9 0.99%

At >200, per-corpus rates are 0% Agent Code Guard, 3.45% Wayfarer, 9.09% ripgrep, 3.11% OpenTelemetry, and 3.06% Rust RFCs. The higher ripgrep rate is two deliberately large user-facing documents in a small 22-file set, not a large warning count.

Actual direct-section finding volume

The earlier maximum-per-document distribution answers which documents are affected, but a future guard naturally has one possible finding per oversized section. The same pinned rows were therefore counted individually across all 13,157 direct-content sections:

Boundary Oversized sections Section rate Affected documents Document rate Documents with multiple findings
>200 29 0.22% 29 3.18% 0
>240 17 0.13% 17 1.86% 0
>300 9 0.07% 9 0.99% 0

The findings-per-document distributions were 0:883, 1:29 at >200, 0:895, 1:17 at >240, and 0:903, 1:9 at >300. Thus the previously stated 29/3.18% was an affected-document statistic, but it happens to equal the true finding count in this corpus. Per corpus at >200, findings/affected documents were Agent Code Guard 0/0, Wayfarer 1/1, ripgrep 2/2, OpenTelemetry 6/6, and Rust RFCs 20/20.

No sampled document had multiple sections above any of the three serious boundaries, so there was no over-threshold multi-finding set to inspect for redundancy. As a boundary stress check, the only three documents with multiple sections above 150 were inspected:

Document Large direct sections Inspection
Rust RFC 3875, build-std explicit dependencies Cargo subcommands 257; Proposal 195 Distinct responsibilities; if both crossed 200, suppressing either would hide an actionable unit
OpenTelemetry OTEP 0156, columnar encoding ArrowStreamService 239; protocol-buffer appendix 169 Distinct protocol explanation and coherent reference code; second may be a keep, but is not redundant
Rust RFC 3873, build-std context working-group history 242; rustup alternative 162 Separate historical and architectural topics; independent inspection is meaningful

This evidence settles the future output contract: emit every direct-content section whose span is greater than the configured threshold, in deterministic path/start-line order. Do not reduce a document to only its largest offending section. The all-findings contract adds no observed volume at >200, preserves the exact actionable range of each independent local unit, and avoids hiding a second responsibility if future corpora contain two oversized sections. Result aggregation may still present a document-level guard state, but must retain all section findings. No per-document cap is justified.

Heading-subtree span

Boundary Reviews Rate
>200 246 26.97%
>300 151 16.56%
>400 90 9.87%
>500 56 6.14%
>800 25 2.74%

Only a very high subtree boundary produces a conservative rate, at which point it is an indirect, harder-to-explain duplicate of document size. It is rejected rather than shipped alongside direct content.

12. Section boundary inspections

Position Role Example section Span Inspection
Below RFC Rust RFC 2011 Generic assert local block 80 Compact proposal unit; no signal needed
Around 120 RFC Rust RFC 0216 Detailed design 121 Long API/code exposition but one coherent design; useful evidence against a low default
Around 160 RFC Rust RFC 2580 Reference-level explanation 161 Several API definitions and code; inspectable, often coherent; supports a more conservative boundary
Around selected Runbook/process Wayfarer docs/22-Testing.md, Testing 242 One heading owns policies, environment discovery, databases, browser preflight, commands, and CI; meaningful headings would improve targeted retrieval
Above RFC Rust RFC 1479 Detailed design 416 Large code-heavy design; REVIEW useful, but retaining it as one formal unit is defensible
Extreme Reference/FAQ ripgrep FAQ 1,063 Questions use raw HTML headings, outside the bounded outline contract; the huge local unit is real to Markdown-outline tooling and warrants inspection, though conversion may have compatibility costs
Extreme keep Protocol/reference OpenTelemetry profiles Proto Definition 565 432 fenced lines dominate a coherent protocol definition; explicit reviewed; coherent; keep

Direct content best answers “does this local documentation unit deserve inspection?” It finds the mixed-responsibility testing runbook and the unusual single-section FAQ, while policy can retain formal/code-heavy units. The section-finding volume confirms that >200 remains conservative under the all-oversized-sections output contract.

13. Heading-depth evaluation

Maximum depth had median 3, P75/P90 4, P95/P99 5, and max 6. Rates were 31.91% above 3, 7.24% above 4, and 0.88% above 5. Inspection did not show that level 5 itself causes maintainability difficulty. Deep hierarchy was often appropriate in specifications; shallow files could still be monolithic.

This overlaps mature lint/style and accessibility concerns. markdownlint MD001 checks heading increments, while MD025/MD041 cover top-level heading conventions. Google’s heading guidance emphasizes descriptive, logical hierarchy, and GitLab’s depth guidance is partly tied to its own sidebar. Agent Code Guard would add no stable maintainability meaning by warning on a numerical heading level. Heading depth is rejected.

14. Coherent/no-refactor examples

These are useful REVIEWs with a no-change conclusion, not metric failures. Policy must make that outcome first-class.

15. Fenced-code, table, and list noise

Counting fenced code normally produced 29 direct-section reviews at >200. Removing fenced lines reduced that to 11 (1.21%); at >160, it fell from 44 to 18. Examples crossing below included the OpenTelemetry profile proto (565 to 133), Rust Unix socket detailed design (416 to 65), and ripgrep README (218 to 131).

That difference is real, but not a reason to discount code. Large examples still consume navigation/context, code-exclusion would invite hiding content in fences, and the remaining prose-only count would understate a section’s actual surface. The selected metric counts all lines and uses REVIEW policy for coherent examples.

Long tables, bullet lists, checklists, and compatibility/reference mappings also produced legitimate long units. No deterministic table/list discount improved the meaning enough to justify another semantic dimension. No AST complexity or content-type exception is proposed.

16. External-tool precedent

Authoritative references were checked for overlap:

There is little mature threshold precedent to copy. That weakens any claim of mathematical universality, but supports distinct responsibility: the admitted metrics are navigation/reasoning guardrails, not duplicated Markdown lint.

17. Agent usefulness

The findings give agents a deterministic pause before appending to an already large README, design, runbook, or local section. The expected benefit is easier targeted retrieval, clearer topic boundaries, less irrelevant context, and less “append forever” behavior. The corpus demonstrates those situations, but does not measure exact token savings, so none are claimed.

Costs are coherent-spec reviews, pressure to add headings mechanically, doc fragmentation, and warning fatigue. Conservative defaults, REVIEW-only state, exact ranges, and explicit keep language make those costs acceptable.

18. Gaming resistance

Policy for both metrics must prohibit:

Agents should optimize for clearer navigation and responsibility, never the number itself. Overrides and disablement require ordinary user/project authority; agents must not weaken configuration merely to silence a result.

19. Generated and reference documentation

Generated, vendored, changelog-dump, and dependency documentation should normally be excluded through existing scope.exclude, after Git-ignore-aware selection and before guard applicability. Explicitly retained reference/spec documents remain eligible: their size still justifies inspection, and policy allows a coherent keep. Role-specific production thresholds or generated-file heuristics would create ambiguous policy and are rejected.

The corpus excluded one vendored Wayfarer doc and two changelogs. It retained formal RFCs, specifications, protocol definitions, reference tables, and long examples deliberately as noise controls.

20. Admission criteria 1–14

# Criterion Document physical size Direct section size Subtree size Heading depth
1 Deterministic anchor PASS PASS PASS PASS
2 Engineering value PASS: navigation/context surface PASS: local cohesion/navigation QUALIFIED: mostly repeats document FAIL: depth alone weak
3 Broad applicability PASS across sampled roles/repos PASS across sampled roles/repos QUALIFIED FAIL/qualified by style
4 Distinct responsibility PASS: no mature lint equivalent PASS: no mature lint equivalent FAIL: duplicates document metric FAIL: lint overlap
5 Stable semantics PASS: physical records PASS: next heading of any level PASS but coarse PASS measurement, weak meaning
6 Explainable/actionable PASS: exact file/range PASS: heading and exact range QUALIFIED: overlapping/root spans QUALIFIED
7 Useful state model PASS/REVIEW only PASS/REVIEW only Would be REVIEW only No useful Code Guard state
8 Signal-to-noise PASS at >800 PASS at >200 FAIL at useful boundaries FAIL/weak
9 Threshold/config evidence DEFAULT 800 + override/disable DEFAULT 200 + override/disable NOT APPLICABLE NOT APPLICABLE
10 Gaming resistance MITIGATED by policy; count blanks MITIGATED; count all content Similar but no value Meaningless shallowing risk
11 Scope compatibility PASS through ResolvedScope.files PASS through same facts PASS technically PASS technically
12 Architecture fit/cost PASS with bounded separate facts PASS from same scan FAIL necessity/YAGNI FAIL necessity
13 Failure behavior PASS: UTF-8 error; permissive EOF fence PASS: same PASS technically PASS technically
14 Portable/testable PASS; stdlib-only fixtures PASS; stdlib-only fixtures PASS technically PASS technically
  Decision ACCEPT ACCEPT REJECT REJECT / OUT OF SCOPE

The rejected metrics remain explicit; they are not hidden inside the overall Markdown ACCEPT.

21. Threshold and default-enable conclusions

For a future production slice:

Guard Universal default? Default enabled? Override? Disable? FAIL?
Markdown document physical size Yes, 800 Yes Positive integer Yes No
Markdown direct section physical size Yes, 200 Yes Positive integer Yes No

These are conservative inspection guardrails in the D30 philosophy, not mathematical quality laws. Exact thresholds pass. A finding above either value asks the agent to inspect responsibility and navigation, and authorizes a documented coherent keep.

22. Architecture recommendation

The smallest future shape is:

ResolvedScope.files
    -> .md applicability
    -> one lazy bounded Markdown scan per file
    -> immutable document and direct-section facts
    -> document-size and section-size guards
    -> GuardResult / requiredPolicies

Do not add Markdown to DEFAULT_INCLUDE_EXTENSIONS for LOC, reuse executable AnalysisFacts, or create ArtifactFacts. A concrete immutable Markdown document fact should carry path and physical count plus range-qualified heading sections; a section fact needs heading text, level, source range, and physical span. Repeated headings are disambiguated by range. Rejected subtree/depth measurements should not survive into production facts.

The scanner should be lazy and separate because Markdown parse/failure semantics differ from executable syntax. The research implementation indicates that no production dependency is required. If future fixtures expose material container/CommonMark gaps, reconsider an established parser then, with license, maintenance, wheels, dependency tree, and cross-platform cost assessed before changing the contract.

One later issue should deliver the complete vertical slice: facts/scanner, configuration, runner orchestration over final scope, result/text/JSON output, required policies, tests, documentation, packaging, and three-platform CI. No generic registry/plugin groundwork is justified.

23. Final admission outcome

ACCEPT Markdown document physical size and direct-content section physical size for a future production implementation. REJECT document nonblank lines as the primary measurement, heading-subtree section size as redundant/noisy, and heading depth as style-lint territory. Do not ship production behavior in this evidence PR.

24. Remaining risks