How we plan to benchmark property prediction against DFT

Machine-learned surrogates promise DFT-quality property estimates at a fraction of the cost. Whether that promise holds for a given model is an empirical question, and we don't have an answer yet — Accretion is pre-proof-of-concept, and no prediction engine has been benchmarked.

This post is about the methodology we're building toward, not a result: how we intend to evaluate a property-prediction surrogate once one exists, structured so the numbers can't flatter themselves.

The plan

The design: each surrogate would be a graph neural network over the relaxed crystal structure, with targets computed against standard DFT functionals. The evaluation split matters more than the architecture — composition-disjoint splits (no polymorph of a test composition appears in training) are the honest test; random splits flatter every model and are easy to over-report.

Status

Twelve property classes are scoped for the first evaluation pass, including formation energy, band gap, and ionic conductivity — the last of which we expect to be the hardest, since errors in comparable published work concentrate in structures with correlated ion motion. Building the surrogate and the disjoint-split harness comes before any of this can be measured.

When there are real numbers, they will replace this post — split definitions and evaluation code included, not just a headline error rate.