Kaelum Bench v0.2 · Public Calibration
Llama 4 Maverick
Accounting Knowledge · Run evidence
Run identity
Model and configuration
- Model display name
- Llama 4 Maverick
- Exact model ID
- meta-llama/llama-4-maverick
- Provider
- DeepInfra
- Exact provider route
- openrouter/deepinfra/base
- Runtime ID
- openrouter-tool-free-non-streaming-v8
- Runtime version
- v8
- Reasoning profile
- native-default
- Reasoning configuration
- native-default
- Tool access
- None
- Suite
- Accounting Knowledge
- Validity
- Provisional
Material configuration parameters
- allow_fallbacks
- false
- data_collection
- deny
- max_attempts_per_task
- 1
- max_concurrency
- 1
- max_generation_metadata_lookups
- 1
- max_output_tokens
- 16384
- max_raw_response_bytes
- 16777216
- max_request_bytes
- 1048576
- metadata_timeout_seconds
- 60
- request_timeout_seconds
- 300
- require_parameters
- true
- temperature
- 0
- zdr
- true
Published score
Scoring evidence
- Score
- 431.6 / 600
- Percentage
- 71.9%
- Scoring method
- Hybrid
- Comparison protocol
- Configuration-specific
- Tested-model cost
- $0.0128
- Judge cost
- $1.51
- Total cost
- $1.53
Usage
Tokens and execution
- Input tokens (tested model)
- 35,510
- Output tokens (tested model)
- 7,061
- Total tokens (tested model)
- 42,571
- Attempted tasks
- 60 / 60
- Valid responses
- 60 / 60
- Invalid JSON
- 0
- Schema failures
- 0
- Provider failures
- 0
- Missing responses
- 0
Judge evidence
Independent assessment
- Judge model
- openai/gpt-5.6-sol
- Judge provider
- OpenRouter (Azure EU)
- Judge route
- Not publicly released
- Judge region
- Not publicly released
- Judge input tokens
- 112,782
- Judge output tokens
- 27,099
- Judge tokens
- 139,881
- Judge assessments
- 60
- Assessment not required
- 0
- Judge retries
- 0
Release provenance
Versioned run evidence
- Benchmark ID
- FMB-KNOW-ACC-01
- Benchmark version
- 0.1.0-dev
- Scorer ID
- fmb-live-scorer-v1
- Configuration key
- config-bb2e0f36f468d536259bf55aac3dbccc90ae8d3bae76330120203d122cebc1f6
- Campaign ID
- 5d89e353-30a2-5dc4-a347-e529eb3890cb
- Comparison group ID
- accounting-knowledge-0.1.0-dev-configuration-specific-v1
- Started at
- Completed at
- Run ID
- a24a4d05-27ef-5083-91d0-96148c3a4cfe
- Signed release ID
- 0bae13a5-a0c9-44fb-b6df-6bc83a6847d5
- Release ID
- 80817257-aa7f-4fb9-867b-66e9ed813de9
- Manifest SHA-256
- d82a32b09acf027f94de5fe162ca71dabb3a389a4e2b12f58fa3d1faec73a3c8
- Source aggregate
- fmb-live-runs.json
- Source aggregate SHA-256
- 0ef185b35b7d9f9b891a40de2bc9a9cee0febe77a41ee700458b6b92b4bb2cd3
- Source aggregate bytes
- 17,057
Authenticated publication
Signed release evidence
The sanitized single-run JSON is a derived projection and is not separately signed. The Ed25519 signature authenticates the exact public manifest bytes, including the release inventory. The manifest in turn binds this run's source aggregate by its SHA-256 digest and byte size.
Download sanitized run JSON →Download signed manifest →Download Ed25519 signature →Download public key · kaelum-bench-public-release-20260718-v1 →Download public run index →Download manifest-bound source aggregate →
Verify this evidence
- Verify the detached Ed25519 signature over the domain-prefixed exact manifest bytes using the published public key.
- Confirm that the sidecar payload SHA-256 equals the manifest SHA-256 shown above.
- Hash the source aggregate and compare its SHA-256 and byte size with the signed manifest entry.
- Resolve this run ID through the public run index and match its campaign ID inside that aggregate.
node -e 'const fs=require("node:fs"),c=require("node:crypto"),m=fs.readFileSync("public-manifest.json"),s=JSON.parse(fs.readFileSync("public-manifest-signature.json","utf8")),k=fs.readFileSync("kaelum-bench-public-release-20260718-v1.pem");console.log(c.verify(null,Buffer.concat([Buffer.from("kaelum-bench-fmb-public-release-v1\0"),m]),k,Buffer.from(s.signature_base64,"base64")))'