And adding to previous post. Kevin has just published (in September 2025, does he have time machine?) a paper on problems of testing LLMs. For the first time in my carrier I read it more like a scream from author's soul. Ok, there were articles like this on data reproducibility, but this one I take more personally, probably.
#LLM
https://www.sciencedirect.com/science/article/pii/S0927025625003842
BTW, it worth also reading his post in LinkedIN:
"I think that most ML benchmarks might be measuring the wrong things entirely.
We've been building evaluation tools for ML models in chemistry/materials science for a while now (ChemBench, MaCBench, MatText, etc.), and honestly: I am more and more worried about what we do as a field.
You can take the exact same models, change how you aggregate scores or define your test set, and suddenly the "best" model is completely different. We showed this with ChemBench - depending on which metric you pick, the model rankings can flip around.
In addition, many benchmarks are basically solved at this point: We've hit the noise floor of the underlying DFT calculations. Yet people keep using it and claiming "progress."
A crucial insight is that we are somehow stuck in a datasets-as-benchmarks paradigm: we're using datasets to both define what we want to measure AND to do the measuring. It's like using a ruler to measure itself.
We wrote this paper as therapy - trying to understand why evaluation feels so broken and what we might do about it. Turns out, most crucial evaluation design choices are just... hidden. All these little decisions that completely change your results, but nobody talks about them.
Our small contribution: "evaluation cards" - basically forcing ourselves to document all the weird choices we made and why.
"
(from https://www.linkedin.com/posts/kevin-maik-jablonka_i-think-that-most-ml-benchmarks-might-be-activity-7357688705874075648-jJqA)
