First-hand information for everyone
Large language models are powerful but forgetful across long projects. This paper presents Ensemble QSP, a multi-agent system that limits the amount of text each model must hold at once. By keeping the injected context small and structured, the system aims to run multi-session computational modeling workflows without losing earlier work or degrading performance.
The core idea is a three-layer hierarchical memory. The short-term layer holds the immediate working context: a fixed recent conversation window, a capped scratchpad for exact facts and file pointers, and a tiny buffer of recent task outcomes. The mid-term layer is a compact, structured project-state JSON that contains only task-relevant slices (for example, the five most recent active requests and the 20 most recently modified files). The authors report a median mid-term state size of 301 tokens (interquartile range 215–478; maximum 4,050) across 104 runs. The long-term layer stores domain knowledge such as a ~24,000-character handbook, physics checklists, and retrieved source papers that are injected only when needed. Because completed work is evicted, the injected context stays bounded instead of growing with every session.
Ensemble QSP organizes work with a principal investigator (PI) agent overseeing five specialist sub-agents. The specialists have distinct roles: Optimization (parameter fitting), Modeling (writing and implementing equations), Reporting (analysis and write-up), Infrastructure (literature and compute management), and Review (code quality). The system also enforces physical constraints using domain JSON files, physics checklists, and retrieval-augmented comparison to source material to reduce the chance of physically implausible outputs. The authors say this structure is domain-agnostic: adding a new scientific area only requires a new PI configuration.
The paper reports benchmarking on pharmacokinetic-pharmacodynamic (PKPD) tasks with known ground truth. On a 20-dataset model-selection test spanning six candidate model structures, the multi-agent system chose the correct model on the first attempt for all 20 datasets. A single-agent baseline (GitHub Copilot, GHCP) reached 14/20 correct and required nine human-assisted debugging iterations to reach comparable outcomes. On a separate parameter-recovery test using difficult two-compartment PK profiles, the multi-agent system recovered parameters within 10% of ground truth for 14 of 20 profiles (70%), while GHCP did so for 4 of 20 (20%). The authors also note that the multi-agent runs took longer wall-clock time (about 105 minutes versus 18–25 minutes for the baseline) because of coordination overhead.
Important caveats are reported in the paper. The benchmarks used synthetic datasets with known answers, so real-world performance may differ. In simpler model types, the single-agent baseline sometimes reported lower median error after many debugging iterations, which the authors attribute to tighter solver settings adopted during that manual debugging—an uneven comparison. The multi-agent approach trades faster autonomous convergence and robustness on stiff ordinary differential equation problems for greater compute and coordination time. Finally, the system’s claims about generality are limited to the architecture level: the paper shows examples and says adding a new domain needs only a PI configuration, but that still requires domain files and checklists to be prepared.