Can AI Agents Read Your Repo? A 10-Check Audit With Real Scores From 3 Repositories

I ran the same 10-check audit on three of my own repositories last week, and the result I did not expect was this line:

| 9 | Code readability | recommended | 10/10 | consistent module naming, docstrings 10/10, 6 top-level dirs |

That’s from the repo that scored 36/100 — grade F overall.

The code is clean, the naming is consistent, every module has docstrings. And the repo is functionally invisible to AI agents, because it’s missing every file agents actually read: no llms.txt, no AGENTS.md, no architecture doc, no LICENSE. A developer who asks an agent “what is this project” gets nothing useful back — not because the code is bad, but because the surface layer that agents digest doesn’t exist.

That gap — between code quality and agent readability — is what this post is about.

Your repo has two audiences now

The first audience is human. The second arrived quietly: every developer who asks ChatGPT “what does this repo do”, every Copilot session opened inside the codebase, every Claude question that ends in “recommend a Python tool for X”. The answers come from digested repo content, not from running the code.

The digestion layer is now its own product category. Gitingest — which flattens any GitHub repo into a prompt-friendly digest — passed 15k stars. GitDiagram renders repos as diagrams for exactly this use. MCP servers increasingly act as the interactive version of the same idea, which is why we ship one with the zens-ink package (how it works).

Agents don’t browse your repo like a senior engineer would. They read the README, the manifest, the instruction files, and whatever index you hand them. If that layer is thin, the agent’s answer is thin — or confidently wrong. It’s the same dynamic as llms.txt for websites: the crawler no longer interprets, it ingests, and ingestion rewards explicit structure.

The 10 checks

The audit scores 100 points across ten checks, split into two tiers:

#	Check	Tier	Points
1	README quality	essential	15
2	llms.txt layer	essential	12
3	Agent instructions (AGENTS.md etc.)	essential	8
4	Installable surface (manifest + entry points)	essential	15
5	MCP / agent interface	essential	10
6	Architecture docs	recommended	8
7	License + metadata	recommended	5
8	Repo hygiene (changelog, tests, CI)	recommended	12
9	Code readability	recommended	10
10	Extras (funding, assets)	recommended	5

The essential tier weighs 60 because those checks decide whether an agent can answer anything about the repo. The README check, for example, isn’t “file exists” — it scores length, install command, a fenced usage example, heading structure, and a plain-language one-liner near the top (which becomes the overview sentence agents repeat verbatim).

Three real runs

I audited three of my own repositories — all real projects, none staged for this post.

Run 1: an agent-optimized CLI package (zens-ink)

  Auditing /workspace/zens-ink (10 checks) ...

**Score: 91/100 (91%) — Grade A**

| 1 | README quality      | essential   | 15/15 | README.md: 7,562 chars, 9 H2 sections, 189 lines |
| 2 | llms.txt layer      | essential   | 12/12 | llms.txt (5,934 chars) + llms-full.txt |
| 3 | Agent instructions  | essential   |  8/8  | SKILL.md |
| 4 | Installable surface | essential   | 15/15 | setup.py (entry point OK, lockfile: requirements.txt) |
| 5 | MCP interface       | essential   |  6/10 | 1 MCP server file(s), README documents MCP |
| 7 | License + metadata  | recommended |  5/5  | LICENSE; manifest metadata: OK |
| 8 | Repo hygiene        | recommended | 12/12 | CHANGELOG, 1 test files, CI, .gitignore |
| 9 | Code readability    | recommended | 10/10 | docstrings 7/8, 8 top-level dirs |

This is the repo after a deliberate agent-readiness pass: llms.txt added with a full tool inventory, ARCHITECTURE.md, changelog discipline, tests, CI. The remaining 9-point gap is mostly the MCP check — the server exists but could expose more per-tool documentation in the README.

Run 2: a typical internal dashboard

**Score: 36/100 (36%) — Grade F**

| 1 | README quality      | essential   | 15/15 | README.md: 3,255 chars, 5 H2 sections, 73 lines |
| 2 | llms.txt layer      | essential   |  0/12 | No llms.txt at repo root |
| 3 | Agent instructions  | essential   |  0/8  | No agent instruction file (AGENTS.md/CLAUDE.md/SKILL.md) |
| 4 | Installable surface | essential   |  9/15 | package.json (no entry point, lockfile: package-lock.json) |
| 5 | MCP interface       | essential   |  0/10 | no MCP or structured agent surface found |
| 7 | License + metadata  | recommended |  0/5  | LICENSE: missing |
| 8 | Repo hygiene        | recommended |  2/12 | .gitignore |
| 9 | Code readability    | recommended | 10/10 | consistent naming, docstrings 10/10 |

A perfect README, perfect code — and a grade F. This is the most common shape I’ve seen: the repo was written to pass human review, and it does. Everything an agent needs is absent. package.json with no bin entry also means there’s nothing runnable for an agent to invoke.

Run 3: a content site with an MCP server

**Score: 27/100 (27%) — Grade F**

| 1 | README quality      | essential   |  0/15 | No README found |
| 2 | llms.txt layer      | essential   |  0/12 | No llms.txt at repo root |
| 5 | MCP interface       | essential   |  4/10 | 1 MCP server file(s) |
| 6 | Architecture docs   | recommended |  5/8  | docs/ (4 entries) |
| 8 | Repo hygiene        | recommended |  2/12 | .gitignore |

The inverse failure: this repo has an MCP server — the strongest possible agent surface — but no README, so no agent can discover what the server offers. The site itself serves llms.txt over HTTP and scores 98/100 on site-focused agent checks; the repo behind it scores 27. Two surfaces, one project, both need work.

The pattern: readability ≠ readability

Across the three runs, code readability was the least discriminating check — 10/10, 10/10, 7/10. Modern code is usually fine. The failures cluster in files that cost an afternoon:

• llms.txt (12 points): the two F repos simply don’t have one. The format is the same one we use for website agent optimization: H1 name, blockquote summary, curated link list. For repos, the inventory lists tools and modules with one-line descriptions — that list is what agents cite when recommending your project.
• AGENTS.md or equivalent (8 points): build/test commands and repo conventions. Agents that find them act; agents that don’t guess. Most agents still read CLAUDE.md, SKILL.md, and .cursorrules too.
• Entry points (up to 12 of the installable 15): “runnable” beats “readable” for agent adoption. A CLI entry the agent can execute turns a recommendation into a demonstration.
• Architecture doc (8 points): even a short ARCHITECTURE.md mapping the directory layout prevents the classic agent failure of confidently explaining your code from the wrong mental model.
• LICENSE (5 points): agents skip unlicensed code when the user asks for something they can ship. No LICENSE, no citation.

The fix order (P0/P1/P2)

The audit emits a prioritized plan rather than a raw score. The order that matches the scoring weights:

P0 — ship before anything else. A README with a one-liner, an install command, and one fenced usage example. A package manifest with a real entry point. These are the inputs every digest starts from.

P1 — this week. llms.txt with the full tool/module inventory. AGENTS.md with build and test commands. An architecture doc. These convert “agent can describe the repo” into “agent can use the repo”.

P2 — polish. llms-full.txt for exhaustive detail, tests and CI signals (agents treat green CI as a trust signal the way users do), FUNDING.yml, an assets folder for cards and screenshots that make the repo citable in richer answers.

Run it yourself

The audit ships in the zens-ink Pro package as repo_readiness — pure Python stdlib, no LLM calls, deterministic scoring:

# audit a local checkout
python3 -m zens_ink_pro.repo_readiness /path/to/repo

# audit any public GitHub repo (clones to a temp dir)
python3 -m zens_ink_pro.repo_readiness --github owner/repo

# markdown + JSON reports
python3 -m zens_ink_pro.repo_readiness /path/to/repo --report out.md --json out.json

The checklist itself is free to apply by hand — that’s exactly what I did before automating it. If you maintain an open-source package, the audit pairs naturally with the site-side checks: agent crawler audits for what bots fetch from your site, MCP exposure for what agents can invoke, and this repo audit for what they read when a developer pastes your GitHub link into the chat.

The honest limits

Three caveats worth stating plainly. First, the checks are deterministic heuristics — presence, length thresholds, regex patterns — not semantic understanding; a keyword-stuffed llms.txt scores the same as an honest one. Second, the weights are judgment calls anchored to how digest tools actually consume repos; a team whose agents live in the codebase all day might reweight AGENTS.md higher. Third, a high score guarantees legibility, not accuracy — the agent still needs your files to tell the truth.

What the score does catch, reliably, is the failure mode almost every repo has today: clean code, missing surface. 91 versus 36 versus 27 wasn’t about engineering quality. It was about whether the files agents read exist at all.

FAQ

What does 'agent-readable' mean for a repository?

It means an AI agent asked about your repo — 'what is this', 'which tool should I use for X', 'why is this failing' — can construct an accurate answer from the repo's surface layer alone: README, llms.txt, agent instruction files (AGENTS.md/CLAUDE.md/SKILL.md), the package manifest, and code structure. Tools like Gitingest flatten repos into exactly this digest before an agent ever sees them. A repo can be beautifully engineered and still be agent-unreadable if that layer is missing, because the agent never reaches the elegant parts — it answers from the README, or from nothing.

What is AGENTS.md and how is it different from README?

AGENTS.md is a cross-agent instruction convention adopted across coding agents in 2025-2026: a file at the repo root that tells agents how to work with the repo — build commands, test commands, conventions, areas to avoid. The README describes the project to humans deciding whether to use it; AGENTS.md instructs agents operating inside it. Most agents also still read CLAUDE.md, SKILL.md, .cursorrules, and .github/copilot-instructions.md, which is why the audit accepts any of them.

Does my repo need llms.txt if it already has a good README?

They serve different readers. The README is prose written for humans, optimized for persuasion and skimming. llms.txt is a curated, link-structured inventory — H1 name, blockquote summary, then '- Title: description' lines listing every tool and module — optimized for an agent that needs to cite and navigate. In the audit, llms.txt carries its own 12 points separate from the README's 15, and repos with both consistently produce better agent answers than either alone.

How is repo readiness different from site agent-readiness scores like is-agentic.com?

They audit two different surfaces of the same project. Site readiness scores whether a website is legible to agents — llms.txt served over HTTP, structured data, MCP endpoints. Repo readiness scores whether the code repository is legible — the files an agent reads when a developer points it at your GitHub. A project should pass both: the site gets your product cited, the repo gets your code used. zens.ink scores 98/100 on the site side; this audit covers the repo side.

Can't I just game the score by adding empty files?

Partially, and that's a real limitation. The checks are deterministic and file-oriented — presence, length thresholds, regex patterns — not semantic. An llms.txt stuffed with keywords scores the same as an honest one. The scores are useful the way a sitemap checker is useful: they catch absence and thinness, which are the actual failure modes in practice. All three repos I audited failed on missing and thin files, not on subtle quality. Score the surface honestly, and the agent answers honestly.