Clinical reasoning · Reinforcement learning
Latent Decision Flow Rewards for Clinical Guideline Reasoning
1Harvard University
Clinical guidelines encode evidence-based recommendations as conditional decision flows. GUIDER uses these flows to supervise accurate clinical reasoning that follows guidelines. It does not require guidelines during inference.
on MedGUIDE
on MedGUIDE
benchmarks
at inference
Abstract
Learning to follow clinical guidelines from latent decision flows.
Clinical practice guidelines encode evidence-based recommendations as conditional decision flows. These flows provide a natural source of process supervision for language models, but two challenges complicate their use. First, a flow that follows a guideline does not necessarily determine the correct answer for a specific patient. We call this problem the path–answer gap. Second, several valid flows can support the same clinical decision. We call this property path multiplicity. Answer rewards therefore provide no information about the decision process. Supervised training on annotated flows restricts the model to one trajectory and reduces answer accuracy, while process rewards based on one reference flow penalize other valid reasoning paths.
We introduce GUIDER, which jointly optimizes a controller and a policy to address these challenges. The controller learns from supervision derived from guidelines, generates plausible decision flows, and uses them to supervise the policy. The policy learns to generate clinical reasoning and final decisions. GUIDER accounts for path multiplicity by aggregating process rewards across several flows from the controller. It links adherence to correct outcomes by awarding process rewards only for correct answers. Joint optimization allows the supervision to evolve with the policy instead of relying on a fixed reference trajectory. Because the policy never receives an oracle decision path, GUIDER does not need access to guidelines during inference.
On MedGUIDE, GUIDER with Qwen3-14B achieves 82.5 weighted accuracy and 73.5 guideline adherence. These results exceed those of the strongest clinical reinforcement learning baseline by 3.3 and 3.1 points, respectively. With Qwen3-14B and Gemma4-12B backbones, GUIDER also outperforms CRPO, MRPO, and FaithMed on three benchmarks from outside the training domain: AMEGA-LLM, RealMedQA, and CancerGUIDE.
The challenge
A correct answer does not reveal the path taken to reach it.
Path–answer gap
A decision flow that follows a guideline does not necessarily determine the correct answer for a specific patient. Process validity complements outcome correctness, but cannot replace it.
Path multiplicity
Several clinically valid decision flows can support the same decision. A score based on one fixed reference path can penalize equally sound alternatives.
Distinct clinical decision flows can support the same answer, while no single flow necessarily specifies the complete answer.
The approach
Supervise the process while preserving an independent policy.
GUIDER jointly optimizes a decision flow controller and a clinical reasoning policy. The controller proposes several plausible flows. The policy independently generates its own flow and final answer.
Generate
A controller learns from supervision derived from guidelines and samples several plausible latent decision flows.
Aggregate
Monte Carlo aggregation weights flows by their scores and rewards agreement across valid alternatives rather than agreement with one oracle trajectory.
Gate
GUIDER grants process credit only when the policy produces the correct final answer, thereby linking process validity to the clinical outcome.
Optimize jointly
The supervision evolves with the policy, but the policy never observes controller flows or guidelines during inference.
Clinical illustration
From patient presentation to a guideline-consistent decision.
GUIDER identifies the clinically relevant decision states and reaches the correct recommendation without reproducing the reference flow word for word.
01 Patient case
A 72-year-old woman with extensive-stage small-cell lung cancer initially responded to cisplatin/etoposide but relapsed after six months. Second-line topotecan also failed after four months. She now has progressive fatigue, weight loss, dyspnea, new liver and bone metastases, worsening liver function, and ECOG performance status 3.
What treatment option should be considered?
02 Gold guideline recommendation
Poor performance status. Palliative and best supportive care ± single-agent chemotherapy/palliative radiotherapy.
03 Generated decision flow
1Relapse or primary progressive disease after systemic therapy for ES-SCLC
2Poor performance status ECOG PS 3
Final answerA
Palliative and best supportive care, with consideration of single-agent chemotherapy or palliative radiotherapy.
Example clinical question and generated decision flow from MedGUIDE. The generated flow identifies the relevant guideline conditions and arrives at the correct treatment recommendation as a concise sequence of decision states.
Open-ended illustration
Reasoning without predefined answer choices.
AMEGA-LLM evaluates responses to open clinical questions. The model must identify factors specific to the patient and compose the final response directly.
01 Patient case
A 65-year-old African American man presents with six months of worsening urinary symptoms, including weak stream, frequency, nocturia, and difficulty initiating urination. He recently developed mild pelvic discomfort and occasional hematuria. He has a history of vasectomy and benign prostatic hyperplasia. His father was diagnosed with prostate cancer at age 74.
AMEGA question
What are significant risk factors for the primary working diagnosis obtainable from the case?
02 Model response
Significant risk factors for the primary working diagnosis of prostate cancer include:
- Age over 65 years
- African American ethnicity
- Family history of prostate cancer father diagnosed at age 74
- Presence of hematuria
- Presence of pelvic pain
- History of vasectomy
- History of benign prostatic hyperplasia
The patient's age, ethnicity, and family history place him in a high-risk group for prostate cancer. The presence of hematuria and pelvic pain raises concern for a more aggressive disease course. The history of vasectomy is a known risk factor for prostate cancer. His BPH history may complicate the interpretation of prostate-related symptoms.
Proposed next step: PSA testing, digital rectal examination, and consideration of prostate MRI if PSA or DRE findings are abnormal.
Example clinical reasoning question and model response from AMEGA-LLM. Unlike MedGUIDE, AMEGA-LLM provides no predefined answer choices. The model therefore presents its clinical reasoning as the final response rather than as a separate decision flow and multiple-choice answer.
Evaluation
Stronger reasoning within and beyond the training domain.
Across two model families, GUIDER improves both answer accuracy and guideline adherence over CRPO, FaithMed, and MRPO.
BenchmarkGUIDERBest baselineGain
MedGUIDEWeighted accuracy82.579.2+3.3
MedGUIDEGuideline adherence73.570.4+3.1
AMEGA-LLMAggregate score22.321.9+0.4
RealMedQAJudge score1.91.8+0.1
CancerGUIDEJudge score2.42.3+0.1
Qwen3-14B results. We report the mean of three random seeds for each fine-tuned method. Higher values indicate better performance.
What we learn
Clinical guidelines can provide rich process supervision without creating a dependency during inference.
- Multiple sampled flows capture useful path diversity; gains largely saturate at four flows.
- Moderately sized decision flows offer enough structure without unnecessary complexity.
- Improvements transfer across MedGUIDE, AMEGA-LLM, RealMedQA, and CancerGUIDE.
Citation
Build on GUIDER.
Code, training scripts, dependencies, and evaluation templates are available in the public repository.
@misc{bhatt2026guider,
title = {Latent Decision Flow Rewards for Clinical Guideline Reasoning},
author = {Bhatt, Gaurav and Zitnik, Marinka},
year = {2026},
note = {Preprint}
}