Kaelum Bench v0.2 · Public Calibration
nemotron-3-ultra-550b-a55b
Valuation & Financial Modeling · Run evidence
Run identity
Model and configuration
- Model display name
- nemotron-3-ultra-550b-a55b
- Exact model ID
- nvidia/nemotron-3-ultra-550b-a55b
- Provider
- Together AI
- Exact provider route
- openrouter/together
- Runtime ID
- openrouter-tool-free-non-streaming-v7
- Runtime version
- v7
- Reasoning profile
- reasoning-max-supported
- Reasoning configuration
- high
- Tool access
- None
- Suite
- Valuation & Financial Modeling
- Validity
- Provisional
Material configuration parameters
- allow_fallbacks
- false
- data_collection
- deny
- max_attempts_per_task
- 1
- max_concurrency
- 1
- max_generation_metadata_lookups
- 1
- max_output_tokens
- 34000
- max_raw_response_bytes
- 16777216
- max_request_bytes
- 1048576
- metadata_timeout_seconds
- 60
- request_timeout_seconds
- 300
- require_parameters
- true
- temperature
- 0
- zdr
- true
Published score
Scoring evidence
- Score
- 240.0 / 600
- Percentage
- 40.0%
- Scoring method
- Deterministic
- Comparison protocol
- Configuration-specific
- Tested-model cost
- $0.06
- Judge cost
- Not applicable
- Total cost
- $0.06
Usage
Tokens and execution
- Input tokens (tested model)
- 38,366
- Output tokens (tested model)
- 10,286
- Total tokens (tested model)
- 48,652
- Attempted tasks
- 60 / 60
- Valid responses
- 60 / 60
- Invalid JSON
- 0
- Schema failures
- 0
- Provider failures
- 0
- Missing responses
- 0
Judge evidence
Independent assessment
Not applicable. This suite uses deterministic scoring and requires no judge.
Release provenance
Versioned run evidence
- Benchmark ID
- FMB-FIN-VAL-01
- Benchmark version
- 0.2.0-dev
- Scorer ID
- fmb-live-scorer-v1
- Configuration key
- config-ef35aa140459d269d244a80213bd2ef39df837b9119cedcba3ff57f968259dd0
- Campaign ID
- d919271f-f357-45f2-8f2e-542eebb28003
- Comparison group ID
- valuation-financial-modeling-0.2.0-dev-configuration-specific-v1
- Started at
- Completed at
- Run ID
- c74b9990-dc30-5a76-8389-9b8ffb6e0d4a
- Signed release ID
- 1dc37e8d-ffda-48f4-a2ad-6fa9b91eb97a
- Release ID
- 000d4759-6fa4-40f9-8cbf-208192bd88b0
- Manifest SHA-256
- b9041383a995dd88f0f1132b1d76cf8ce74f373fd2bf8eb172f3df034fa4c801
- Source aggregate
- fmb-live-runs-fin-val.json
- Source aggregate SHA-256
- 8ddd334ac2780d4459b90769f6e75a1315240ab9e9a404829817a60fa4448a87
- Source aggregate bytes
- 7,235
Authenticated publication
Signed release evidence
The sanitized single-run JSON is a derived projection and is not separately signed. The Ed25519 signature authenticates the exact public manifest bytes, including the release inventory. The manifest in turn binds this run's source aggregate by its SHA-256 digest and byte size.
Download sanitized run JSON →Download signed manifest →Download Ed25519 signature →Download public key · kaelum-bench-public-release-20260718-v1 →Download public run index →Download manifest-bound source aggregate →
Verify this evidence
- Verify the detached Ed25519 signature over the domain-prefixed exact manifest bytes using the published public key.
- Confirm that the sidecar payload SHA-256 equals the manifest SHA-256 shown above.
- Hash the source aggregate and compare its SHA-256 and byte size with the signed manifest entry.
- Resolve this run ID through the public run index and match its campaign ID inside that aggregate.
node -e 'const fs=require("node:fs"),c=require("node:crypto"),m=fs.readFileSync("public-manifest.json"),s=JSON.parse(fs.readFileSync("public-manifest-signature.json","utf8")),k=fs.readFileSync("kaelum-bench-public-release-20260718-v1.pem");console.log(c.verify(null,Buffer.concat([Buffer.from("kaelum-bench-fmb-public-release-v1\0"),m]),k,Buffer.from(s.signature_base64,"base64")))'