The architecture is not what is unclear

Language models and agentic systems are proposed for trading constantly, and the reported performance is almost impossible to compare across papers. A review updated this week sets out why, and the answer is not that the models differ.

The authors conducted a reproducibility audit of execution realism across 30 trade-relevant primary studies, coding each against a common matrix: point-in-time controls, transparency of the temporal split, held-out evaluation, cost and turnover treatment, execution semantics, universe definition, and whether artifacts were released.

The central finding is stated plainly. Across the audited sample, architecture reporting is generally clearer than the evaluation assumptions needed to judge whether a result is economically interpretable or reproducible at all. Papers describe what the agent is, and leave unstated the things that decide whether its returns survive contact with a market.

Those unstated things are familiar to anyone who has run a strategy: when exactly the data was available, whether the split leaks, what the turnover costs, when the order is assumed to fill. A worked example on ten equities is included in the paper as a methodological scaffold rather than a result, to show how explicit friction and timing assumptions compress an active strategy's apparent performance.

The conclusion the authors draw is that the useful next step for the field is not better agent design but clearer reporting standards for execution realism and comparability. That is an unfashionable thing to publish, and it is the same diagnosis this desk has reached from the other direction: the reason two trading papers cannot be compared is rarely the model.