Skip to content

From hard questions to reviewable model evidence

Cases span STEM, finance, legal, medical, agents, machine learning, software engineering, geoscience and education. Explore task goals, working methods, reference artifacts and acceptance evidence across specialist domains.

Workbench visuals are domain-specific UI concepts. Open each case for delivery methods and original workflow films.

Financial reasoning with source evidence · workbench concept
Finance · Risk & researchCASE / 01
DELIVERY MODELFinance specialists × source evidence × independent review

Financial reasoning with source evidence

Research, valuation, risk and compliance judgments delivered with answers, evidence, boundaries and review state.

01Multi-year, multi-source context with citation spans02Answer, evidence, counter-evidence and uncertainty delivered together03High-risk disagreement routed to finance adjudication
≈4.2Kreasoning tasks48–64Kcontext>97%evidence alignment
Legal review with citation-level evaluation · workbench concept
Legal · Contracts & researchCASE / 02
DELIVERY MODELJurisdiction scope × citation evidence × blind adjudication

Legal review with citation-level evaluation

Verifiable tasks across contracts, regulations and case material—not unsupported conclusions.

01Contracts, schedules, regulation and revision history reviewed together02Every material claim tied to a text version and citation span03Jurisdiction, rationale and ruling retained for disputes
≈1.3Klong-doc tasks3review layers>96%citation agreement
Medical evidence abstraction and safety evaluation · workbench concept
Medical · Evidence & safetyCASE / 03
DELIVERY MODELMedical qualification × evidence provenance × safety escalation

Medical evidence abstraction and safety evaluation

Literature, guidance and case-style material converted into structured evidence, references and safety boundaries.

01Population, intervention, outcome and evidence grade structured02Dual medical review for high-risk conclusions03Provenance, confidence, safety boundary and escalation retained
≈3.6Kevidence units2×medical review0PHI
Expert-authored STEM challenge sets · workbench concept
STEM · Original challengesCASE / 04
DELIVERY MODELOriginal authoring × reference derivation × verifier-led calibration

Expert-authored STEM challenge sets

Original tasks that combine advanced knowledge, multi-step derivation and verifiable answers beyond direct retrieval.

01Mathematics, physics, chemistry, biology and computer science02Problem, reference, rubric and verifier delivered together03Independent solve, model pilot and contamination-risk screen
≈1.2Koriginal tasks5STEM domains6–18%model pass rate
Long-horizon agent trajectories and failure attribution · workbench concept
LLM · AgentsCASE / 05
DELIVERY MODELCritical-node annotation × dual review × failure taxonomy

Long-horizon agent trajectories and failure attribution

Node-level review from task intake and planning through tool use and final deliverables.

01Complete action–observation–correction chain02Process quality separated from outcome quality03Trainable corrections tied to root cause
≈2.8Klong traces18avg. steps7failure classes
MLE-Bench / MLS-Bench-style evaluation operations · workbench concept
ML engineering & scienceCASE / 06
DELIVERY MODELEnvironment reproduction × ML expert review × independent rerun

MLE-Bench / MLS-Bench-style evaluation operations

Evaluate task completion, experiment reproducibility, method improvement and evidence of generalization.

01Task, environment, data and grader delivered together02Experiments and artifacts remain traceable03Engineering completion separated from scientific validity
420+experiment runs3isolated envs>95%rerun match
Executable program-reconstruction evaluation · workbench concept
ProgramBench · Behavioral reconstructionCASE / 07
DELIVERY MODELBlack-box specification × hidden tests × grader replication

Executable program-reconstruction evaluation

Models receive behavioral specifications and a black-box reference, then rebuild a complete compatible program against hidden tests.

01Reconstruct complete programs from behavior rather than source02Interface, state, error and boundary tests combined03Author, verifier and grader-replication roles separated
360+program tasks26–140hidden tests>98%grader stability
Long-horizon repository engineering evaluation · workbench concept
DeepSWE · FrontierSWECASE / 08
DELIVERY MODELRepository snapshot × long-horizon trace × CI verification

Long-horizon repository engineering evaluation

Repository exploration, multi-file implementation, regression protection and environment verification organized as adjudicable long-horizon work.

01Unfamiliar repository exploration and multi-file implementation02Patch, tests, logs and recovery trace accepted together03Programmatic verifiers plus independent engineering review
310+repository tasks2–14files changed>96%CI reproducibility
Multi-source geoscience interpretation and synthesis · workbench concept
Vertical work · GeoscienceCASE / 09
DELIVERY MODELMulti-source alignment × geoscience interpretation × expert review

Multi-source geoscience interpretation and synthesis

Well, interval, map, report and historical interpretations connected into traceable specialist reasoning tasks.

01Logs, core, intervals, maps and reports aligned together02Conclusions tied to depth, source version and uncertainty03Independent geoscience review for high-impact interpretations
720+interpretation tasks5source types>96%lineage complete
Curriculum-grounded authoring and learning diagnosis · workbench concept
Vertical work · EducationCASE / 10
DELIVERY MODELCurriculum alignment × original authoring × difficulty calibration

Curriculum-grounded authoring and learning diagnosis

Curriculum standards, concept dependencies, original items and explainable scoring organized as reliable education-model data.

01Curriculum, concept dependencies and capability targets aligned02Reference, step rubric and misconception set delivered together03Teacher solve, model pilot and difficulty-bias calibration
1.6K+assessment tasks6difficulty tiers>93%rubric agreement

Build data around your next capability goal

Start with a task goal and a sample. Define the materials, specialists, verification methods and delivery scope together.