Learning from deployment experience

Frozen Models, Evolving Expertise Model-Agnostic Learning from Deployment Experience for Multimodal Medical AI

Yexiao He1, Yucheng Tang2, Pengfei Guo2, Yufan He2, Andriy Myronenko2, Can Zhao2, Ang Li1, Daguang Xu2, Dong Yang2
1University of Maryland, College Park 2NVIDIA
Paper Code (coming soon)

Abstract

Large language models (LLMs) and vision-language models (VLMs) are usually frozen after deployment, so they do not learn from the cases they solve. This is especially concerning in medicine, where new clinical evidence, updated guidelines, and new therapies can change established practice. Fine-tuning can update the model, but it requires access to model weights and additional training. Parameter-free methods avoid training, but they may overfit a fixed validation set, lack reliable domain knowledge, or lose visual details by saving experience only as text.

To address these limitations, we present a model-agnostic framework that allows frozen LLMs and VLMs to learn from deployment experience through three forms of external expertise: a Skill that guides reasoning and tool use, a Knowledge Memory that stores reliable facts supported by earlier cases or trusted external evidence, and a Multimodal Knowledge Base that keeps visual examples and guides the model to relate each retrieved case to the current image. Instead of relying on a fixed validation set, a validation strategy keeps an update only if it helps on new cases without degrading performance on earlier ones. Across six benchmarks covering clinical diagnosis, clinical workflows, medical reasoning, and medical and non-medical visual reasoning, and with four open-weight and closed-source base models, our framework improves performance during online deployment by up to 34.2% over the base model on medical tasks, generalizes to unseen cases, transfers to other models without further optimization, and works in non-medical domains.

Method

The base model stays frozen. It answers a stream of cases in batches, and feedback from one batch can affect only later batches. What changes is the external expertise the model reads when it answers a case.

Framework overview: a frozen model uses a Skill, a Knowledge Memory and an MMKB to answer cases; after each batch an optimizer proposes an update that is validated before deployment
Framework overview. (a) A frozen model uses Skill, Knowledge Memory, and MMKB to answer new cases. MMKB supports comparison with visual references and their source answers. (b) The optimizer proposes new Skills and supported knowledge additions from scored trajectories. The candidate update is tested on new and past cases before deployment, and validation feedback guides the next update.

Skill

A natural-language procedure for reasoning and tool use, included in every case. After each batch, an LLM optimizer rewrites the whole document instead of appending to it, which keeps it short and free of conflicting rules.

Knowledge Memory

Reusable facts with supporting evidence and conditions of use. The model queries it through a tool call. A new fact needs several supporting cases, or one case with direct support from a trusted source such as PubMed.

MMKB

A multimodal knowledge base of reference cases with original images and source answers. For each retrieved reference, the model examines how its images support the answer and compares it with the current image.

Validating updates during deployment

A candidate update is compared with the current expertise on the next batch of new cases and on a representative buffer of past cases, with the same model, MMKB, and random seeds. It is accepted only if it helps on the new cases without degrading performance on the past ones.

Results

We evaluate on six benchmarks: AgentClinic, MedChain, MedThink-Bench, MedThinkVQA, VisuLogic, and QCalEval, with four frozen base models: Nemotron-3-Nano-Omni-30B, GPT-4o-mini, Qwen3.8-27B, and GPT-5.6-Sol.

Online improvement during deployment

  • Ours scores above Base in all 24 benchmark–model pairs and achieves the best score in 22 of them.
  • The average gain over Base is 17.6%, compared with 5.6% for ACE, which falls below Base in 7 pairs.
  • Gains reach 34.2% on AgentClinic and 81.9% on VisuLogic, and exceed 16% on QCalEval for every base model.
Table of online performance for Base, ACE and Ours on six benchmarks and four base models
Online performance on the fixed stream. Higher is better for all metrics. The number below each score gives its relative change from Base (green: gain; red: decline). The best score for each base model is shaded.
Online score curves of Base and Ours with GPT-5.6-Sol on five benchmarks and the five MedChain tasks
Online scores of Base and Ours with GPT-5.6-Sol along the fixed stream. The x-axis counts processed cases, and each point is the score over all of them. The shaded area marks where Ours is above Base.

Ablation study

  • Each component accounts for a different part of the gain: removing the Skill hurts multi-step tasks most, and removing MMKB brings the visual tasks back to their Base scores.
  • Validation is necessary: accepting every update lowers the score on all six benchmarks by 21.8% to 37.8% and leaves every score below Base.
  • MMKB gains come from how references are used. With identical references, standard retrieval changes the score by −3.5% to +5.0%, whereas MMKB improves all four image-based tasks by up to 23.9%.
Ablation table removing Skill, Memory, MMKB or validation with GPT-5.6-Sol
Online ablation with GPT-5.6-Sol. The number below each ablated score gives its relative change from Full. −Validation accepts every update without validation.
Bar chart comparing standard retrieval, Initial MMKB and MMKB on four image-based tasks
Standard multimodal retrieval versus MMKB with GPT-5.6-Sol. Initial MMKB does not add deployment cases.

Frozen generalization to unseen cases

  • After learning on the first ~70% of each stream, the frozen expertise scores above Base on the remaining cases in 21 of 24 pairs.
  • The average gain over Base is 20.7%, compared with 8.3% for SkillOpt and 1.9% for ACE.
Table of frozen generalization results for Base, SkillOpt, ACE and Ours
Frozen generalization. Expertise is learned on the first approximately 70% of the fixed stream and evaluated on the remaining cases. The number below each score gives its relative change from Base.

Zero-step cross-model transfer

  • Expertise learned with one model is given unchanged to the other three. It improves the target model in 33 of the 36 transfer results, with an average gain of 20.0%.
  • Expertise learned with the open-weight Qwen3.8-27B also improves the closed-source GPT-5.6-Sol, by 11.8% on AgentClinic and 23.2% on QCalEval.
Bar charts of zero-step transfer with GPT-5.6-Sol and Qwen3.8-27B as the source model
Zero-step transfer with (a) GPT-5.6-Sol and (b) Qwen3.8-27B as the source. Bars show the relative change from each target's Base (the zero line).

Citation

@article{he2026frozen,
  title   = {Frozen Models, Evolving Expertise: Model-Agnostic Learning from
             Deployment Experience for Multimodal Medical AI},
  author  = {He, Yexiao and Tang, Yucheng and Guo, Pengfei and He, Yufan and
             Myronenko, Andriy and Zhao, Can and Li, Ang and Xu, Daguang and
             Yang, Dong},
  journal = {arXiv preprint arXiv:2610.09146},
  year    = {2026}
}