trade-study

Design and evaluation framework for scientific simulation studies. Score competing configurations — model formulations, solver choices, measurement strategies, or any design decision — against known ground truth via protocol-driven simulators, proper scoring rules, hierarchical phases, multi-objective Pareto optimization, and Bayesian model stacking.

trade-study is a design and evaluation framework for scientific simulation studies. Users define simulators — protocol-conformant objects that generate (truth, observations) pairs from a configuration — then score competing configurations (model formulations, solver choices, measurement strategies, or any design decision) against known ground truth, so that decisions validated on synthetic benchmarks transfer reliably to real observational data.

Evaluation proceeds through hierarchical phases — from broad experimental design (full factorial, Latin hypercube, or Bayesian-adaptive search) to focused refinement — scored with proper scoring rules (CRPS, WIS, Brier, coverage).

A four-tier observable hierarchy (embedded constraints, penalized objectives, diagnostic metrics, cost axes) structures multi-objective Pareto optimization with hypervolume and IGD+ front-quality metrics. Calibrated ensemble predictions are produced via Bayesian or score-based model stacking. Global sensitivity analysis (Morris screening) identifies which factors matter most.

Available in Python and Julia. In development; pre-release.