Found reviewing #241.
Each report ships a <report>.md.citations.md sidecar, 1.5 MB across 353 files. It is not a citation list — it is the output of a regex over the report text, and the regex is wrong.
Measured across all 353 sidecars
|
|
| numbered entries |
13,871 |
entries with a broken markdown-link tail (...w](https://doi.org/...) |
194 |
| entries ending in a stray comma |
2,770 |
| sidecars containing at least one malformed entry |
353 / 353 |
| sidecars listing the same reference more than once |
332 / 353 |
In research/traits/ecology/mutualism-deep-research-falcon.md.citations.md the same nine DOIs appear three times over — clean at entries 20-29, again at 30-40 with the markdown link split mid-URL, and again at 41-49 with trailing commas. Entries 1-19 are PaperQA internal keys (cheng2020ecologicalimportanceof pages 13-15) that resolve to nothing outside the provider.
Every sidecar also re-emits the full prompt
All 353 begin by echoing the ~55-line templates/trait_causal_graph_research.md render — objective, required findings, output format — before the list. That is already in the report's own front matter (template_variables), so it is stored three times per trait.
The report's own References section is strictly better
Each report carries a References section mapping PaperQA keys to DOIs, which is what a curator actually needs.
Fix
Either fix the extractor to emit one deduplicated, well-formed reference per line — parsing the report's References section rather than regexing DOIs out of prose — or stop writing the sidecar and let the report's References section be the single source. The second is cheaper and loses nothing identified so far.
Note for whoever picks this up: #244 proposes making --verify assert sidecar existence. If sidecars are dropped instead, that part of #244 goes away — worth deciding this one first.
Found reviewing #241.
Each report ships a
<report>.md.citations.mdsidecar, 1.5 MB across 353 files. It is not a citation list — it is the output of a regex over the report text, and the regex is wrong.Measured across all 353 sidecars
...w](https://doi.org/...)In
research/traits/ecology/mutualism-deep-research-falcon.md.citations.mdthe same nine DOIs appear three times over — clean at entries 20-29, again at 30-40 with the markdown link split mid-URL, and again at 41-49 with trailing commas. Entries 1-19 are PaperQA internal keys (cheng2020ecologicalimportanceof pages 13-15) that resolve to nothing outside the provider.Every sidecar also re-emits the full prompt
All 353 begin by echoing the ~55-line
templates/trait_causal_graph_research.mdrender — objective, required findings, output format — before the list. That is already in the report's own front matter (template_variables), so it is stored three times per trait.The report's own References section is strictly better
Each report carries a References section mapping PaperQA keys to DOIs, which is what a curator actually needs.
Fix
Either fix the extractor to emit one deduplicated, well-formed reference per line — parsing the report's References section rather than regexing DOIs out of prose — or stop writing the sidecar and let the report's References section be the single source. The second is cheaper and loses nothing identified so far.
Note for whoever picks this up: #244 proposes making
--verifyassert sidecar existence. If sidecars are dropped instead, that part of #244 goes away — worth deciding this one first.