The problem
The note usually exists. It is just somewhere nobody looks.
The “Eight Grade” typo is not exotic. Working with government data means a steady supply of them: the suppression rule hiding in a rate column, the category whose meaning narrowed in 2006, the boundary that moved, the two published calculations with one name and only one of them a real trend.
What surprised us is how often the publisher has already written the answer down. We went and checked twenty widely used open datasets. Nineteen ship some kind of guidance. Eighteen ship guidance that changes what you may honestly say about the numbers.
Seven put it anywhere a program following the download link would ever see it.
The rest is in a tab inside the workbook, a footnote hanging off a portal record, a technical guide on a server that blocks automated clients. NJDOE’s pandemic caveat sits on sheet one of sixty-three, titled “Important 2020-2021 Notes”, and no cell in the data mentions it exists. The full survey is here.
What changes
Your agent goes and looks first
Install the plugin and your agent picks up a habit it does not have by default: before it touches a dataset it goes looking for what is already known — the publisher’s notes, the four parsers on GitHub that already hit this, an existing page if one exists. Then it writes down what it worked out, in a format the next person can check.
/plugin marketplace add lyra-forge/marketplace/plugin install ergo@lyra-forgeWhat you get back is a lead, not a verdict. The page tells you where to look and what to look for; you still confirm it. That sounds like a weak promise until you notice the expensive part was never the reading — it was not knowing there was anything to read.
Why it keeps mattering
Models are getting better at this, which is fine
They are getting better fast. One benchmark of realistic analysis work went from 14.55% to 89.95% on identical questions in ten months. We are not betting against that, and a pitch that needed models to stay bad would be a bad pitch.
The part that is not improving is reach. A model cannot know what is in a spreadsheet tab it never opened, on a host that refuses it. When researchers deleted just the documentation-search tool from a system answering questions about US federal surveys, it lost eleven points. Nothing about a bigger model fixes that.
So the useful thing to write down is not what a model could work out on its own. It is the specific fact about this file that nobody outside your project has ever had access to.
What it looks like
One markdown file per dataset
A human reads it top to bottom. A program reads the blocks. Every fragment below is real output from running the validator when this page was built.
Where it goes
Two newsrooms will write different pages, and both be right
A page belongs wherever someone can correct it. If your repository is public and takes pull requests, keep the page there. If it is private — which is usual, since a data team’s repo holds more than its documentation — a public bundle served out of it is readable by anyone and fixable by nobody. Those pages live in a shared directory instead.
Several projects documenting the same census product will disagree about what matters, because their questions differ. The directory clusters pages by subject and refuses to merge or rank them. If you want one authoritative description of a dataset, this is not that, and does not think it exists.
Before you adopt it
Reasons not to
The strongest objection is that a registry rots. Somebody writes twelve issues in a burst of enthusiasm, the dataset changes, nobody updates the page, and two years later a reader trusts a scope that is no longer true — which is worse than no documentation, because it was believed.
That objection lands, and the honest answer is partial. The round trip catches the
half that involves code: delete a workaround, or rename an issue without touching the
anchor, and check fails in CI. Validation records make an untested
caveat visibly untested, and unknowns makes silence say something.
What none of that does is notice that the publisher changed something nobody looked
at. This format does not detect an issue for you. It is a place to put one, and a
mechanism that makes some kinds of rot loud.
Thin adoption evidence
The format is proven against two projects: the NJ DOE datasets behind njschooldata, and a second adopting project's multi-publisher source-contract corpus. That is enough to know it survives contact with real government data. It is not enough to know it works across a hundred datasets, or for a team that did not write it.
No effectiveness measurement
Nothing here shows that a project with data pages makes fewer mistakes than one without. We have not run that test. The benchmarks say handing knowledge over helps a great deal; none of them says this is the right shape to hand it over in.
The spec is draft, and it has moved
Version 0.4, and the changelog records
breaking renames — bite became pitfall one release ago.
Pages declare the format version they conform to and the validator still accepts the
older ones, but the format is not frozen and should not be treated as if it were.
It is documentation, and documentation is work
Nothing here writes itself. The skill lowers the cost of capture a great deal by being present at the moment you learn something, but a page still reflects the attention somebody gave it. A team that will not maintain a README will not maintain this either.
Status