InfiMed: Low-Resource Medical MLLMs with Advancing Understanding and Reasoning

In recent years, multimodal large language models (MLLMs) have achieved remarkable progress in areas such as visual understanding, mathematical reasoning, and general-purpose dialogue, becoming a major direction in the development of artificial intelligence. By processing multiple modalities such as images and text simultaneously, they have opened up vast possibilities for advancing toward general artificial intelligence.

However, this situation in the medical domain is quite different. Compared with abundant general-domain internet data, medical data is scarce, often lacks complete reasoning chains, and involves high-stakes scenarios demanding explainable reasoning.

Left: comparison between sparse outputs and reflective, information-dense outputs. Right: enhanced exploration (bottom) enables broader and more effective search spaces, while limited exploration (top) leads to narrower, less efficient search spaces.

Furthermore, while general reinforcement learning methods (e.g., RLHF, RLVR) excel in broad tasks, their exploration within medical scenarios has been limited and often fails to consistently enhance model generalization.

To address these issues, we propose the InfiMed training paradigm, which achieves a breakthrough in understanding and reasoning for medical MLLMs under low-resource conditions. Our InfiMed-Series 3B models surpass models with similar-scale like Google’s MedGemma-4B-IT and even surpassing the larger InternVL3-8B, establishes a new state-of-the-art.

Training pipeline of the InfiMed-Series models.

Core Advantages

1. Mixed Data & Multi-Dimensional Rewards: Integrating medical textual data, medical multimodal data, and general multimodal data in the SFT phase, and combining them with multi-dimensional rewards during the RLVR phase, enhances accuracy and generalization despite the limited availability of high-quality medical data.
2. Early Infusion of Reflective CoT: Introducing Reflective Chain-of-Thought during fine-tuning equips the model with self-reflection and generates richer candidate answers for subsequent RLVR, while data quality is safeguarded through rejection sampling and cross-model verification to ensure accuracy and reliability.
3. Sample-Efficient Training: InfiMed-Series models, trained with only 188K SFT and 36K RLVR samples, demonstrate strong understanding and reasoning capabilities on seven medical benchmarks. Notably, InfiMed-RL-3B achieves an average accuracy of 59.2% across seven major medical benchmarks, significantly outperforming the comparable-sized Med-Gemma-4B-IT (54.8%) and even surpassing the larger InternVL3-8B (57.3%), demonstrating its exceptional advantage.

Data composition of the InfiMed-Series models: the SFT phase utilizes 188K samples, followed by the RLVR phase with 36K samples.

Quantitative Results

InfiMed-RL-3B, with an average score of 59.2, demonstrates comprehensive leadership across 7 benchmarks, where MMMU-H&M, OMVQA, and MedXQA refer to MMMU-Health & Medicine, OmniMedVQA, and MedXpertQA-Multimodal, respectively.

It not only outperforms comparable-scale models like Google’s Med-Gemma-4B-IT (54.8) but also surpasses larger models such as HuatuoGPT-V-7B (54.2) and InternVL3-8B (57.3), highlighting its exceptional resource efficiency and reasoning capability.

Qualitative Results

Comparative analysis reveals that InfiMed-RL-3B delivers the most direct and accurate answers. In contrast, InfiMed-SFT-3B sometimes requires lengthy analysis or “reflection” steps to reach the correct conclusion. This validates that the RLVR stage effectively enhances the model’s decision-making efficiency and final accuracy, making it more suitable for high-stakes scenarios such as clinical or diagnostic assistance.

A case study comparing Qwen2.5-VL-3B and the InfiMed model series on Medical Visual Question Answering (VQA). Content highlighted in red indicates errors or irrelevant information, while dark green signifies correct or critical content. Qwen2.5-VL-3B misunderstood the examination required by the patient, leading to an incorrect answer. Both InfiMed-SFT-3B and InfiMed-RL-3B provided the correct response, but InfiMed-RL-3B delivered a more direct, clear, and efficient answer.

Citation Information

If you find this work useful, citations to the following papers are welcome:

@misc{liu2025infimedlowresourcemedicalmllms,
      title={InfiMed: Low-Resource Medical MLLMs with Advancing Understanding and Reasoning}, 
      author={Zeyu Liu and Zhitian Hou and Guanghao Zhu and Zhijie Sang and Congkai Xie and Hongxia Yang},
      year={2025},
      eprint={2505.23867},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2505.23867}, 
}

Acknowledgements

We would like to express our gratitude for the following open-source projects: EasyR1, VERL, LLaMA-Factory, and Qwen2.5-VL.