Kaelum Bench

Kaelum Bench v0.2 · Public Calibration

Compare verified benchmark evidence by configuration.

Each row is one authenticated model configuration. Scores remain provisional and link to the exact published run evidence. Missing suites are not treated as zero.

Signed public release

Configuration-specific results, without an overall score

Coverage states which of the five defined suites has published evidence. Results from different provider routes, runtimes, or material parameters remain separate.

v0.2
Updated
16 configurations · 37 runs

16 of 16 configurations

Scroll horizontally to compare suites.

Verified benchmark scores by model configuration and suite
Model configurationProvider routeAccounting KnowledgeHGB ReportingIFRS ReportingFinancial AnalysisAccounting CapstoneCoverage
Kimi K3moonshotai/kimi-k3Max reasoningopenrouter/moonshotai/int4openrouter-tool-free-non-streaming-v691.7444%Model-judgedProvisional88.4333%DeterministicProvisional91.4444%DeterministicProvisional66.5000%DeterministicProvisionalMissing4 / 5
NVIDIA Nemotron 3 Ultranvidia/nemotron-3-ultra-550b-a55bHigh reasoningopenrouter/togetheropenrouter-tool-free-non-streaming-v685.0762%Model-judgedProvisional78.2508%DeterministicProvisional89.6667%DeterministicProvisional61.1701%DeterministicProvisionalMissing4 / 5
Qwen3.6-35B-A3Bqwen/qwen3.6-35b-a3bMax supported reasoningopenrouter/akashml/fp8openrouter-tool-free-non-streaming-v693.2556%Model-judgedProvisional89.3619%DeterministicProvisional90.6667%DeterministicProvisional77.1222%DeterministicProvisionalMissing4 / 5
Claude Sonnet 5 (Max)anthropic/claude-sonnet-5Max reasoningopenrouter/azureopenrouter-tool-free-non-streaming-v6Missing85.6000%DeterministicProvisional90.7778%DeterministicProvisional59.7368%DeterministicProvisionalMissing3 / 5
DeepSeek V4 Flashdeepseek/deepseek-v4-flashXhigh reasoningopenrouter/alibabaopenrouter-tool-free-non-streaming-v6Missing84.5111%DeterministicProvisional90.5556%DeterministicProvisional52.1983%DeterministicProvisionalMissing3 / 5
Gemini 3.5 Flashgoogle/gemini-3.5-flashHigh reasoningopenrouter/google-vertexopenrouter-tool-free-non-streaming-v6Missing92.3778%DeterministicProvisional91.8889%DeterministicProvisional77.7889%DeterministicProvisionalMissing3 / 5
Grok 4.5x-ai/grok-4.5High reasoningopenrouter/xaiopenrouter-tool-free-non-streaming-v6Missing90.6667%DeterministicProvisional91.4444%DeterministicProvisional56.4111%DeterministicProvisionalMissing3 / 5
Kimi K2.6moonshotai/kimi-k2.6Max supported reasoningopenrouter/modelrun/fp4openrouter-tool-free-non-streaming-v694.6556%Model-judgedProvisional90.1778%DeterministicProvisionalMissing87.5427%DeterministicProvisionalMissing3 / 5
MiniMax M3minimax/minimax-m3Max supported reasoningopenrouter/morphopenrouter-tool-free-non-streaming-v6Missing89.2730%DeterministicProvisional88.5000%DeterministicProvisional91.9444%DeterministicProvisionalMissing3 / 5
Claude Sonnet 5 (Max)anthropic/claude-sonnet-5Max reasoningopenrouter/azureopenrouter-tool-free-non-streaming-v693.1556%Model-judgedProvisionalMissingMissingMissingMissing1 / 5
DeepSeek V4 Flashdeepseek/deepseek-v4-flashXhigh reasoningopenrouter/alibabaopenrouter-tool-free-non-streaming-v694.9889%Model-judgedProvisionalMissingMissingMissingMissing1 / 5
Gemini 3.5 Flashgoogle/gemini-3.5-flashHigh reasoningopenrouter/google-vertexopenrouter-tool-free-non-streaming-v590.2667%Model-judgedProvisionalMissingMissingMissingMissing1 / 5
GLM 5.2z-ai/glm-5.2Xhigh reasoningopenrouter/waferopenrouter-tool-free-non-streaming-v6MissingMissingMissing67.7094%DeterministicProvisionalMissing1 / 5
GLM 5.2z-ai/glm-5.2Xhigh reasoningopenrouter/waferopenrouter-tool-free-non-streaming-v496.0222%Model-judgedProvisionalMissingMissingMissingMissing1 / 5
Grok 4.5x-ai/grok-4.5High reasoningopenrouter/xaiopenrouter-tool-free-non-streaming-v495.9206%Model-judgedProvisionalMissingMissingMissingMissing1 / 5
MiniMax M3minimax/minimax-m3Max supported reasoningopenrouter/morphopenrouter-tool-free-non-streaming-v580.5444%Model-judgedProvisionalMissingMissingMissingMissing1 / 5