README.md

metadata

license: apache-2.0
datasets:
  - leduckhai/S-Chain
language:
  - en

S-Chain: Structured Visual Chain-of-Thought for Medicine

---

⭐ If you find this project helpful, please consider giving it a star on GitHub!

---

Khai Le-Duc^{* 1,2✉}, Duy M. H. Nguyen^{* 3,4,24✉}, Phuong T. H. Trinh^{* 5}, Tien-Phat Nguyen^{* 6}, Nghiem T. Diep^{** 3}, An Ngo^{** 7}, Tung Vu^{** 8}, Trinh Vuong⁹, Anh-Tien Nguyen^10,11, Mau Nguyen¹², Van Trung Hoang¹³, Khai-Nguyen Nguyen¹⁴, Hy Nguyen¹⁵, Chris Ngo², Anji Liu¹⁶, Nhat Ho¹⁷, Anne-Christin Hauschild¹¹, Khanh Xuan Nguyen¹⁸, Thanh Nguyen-Tang¹⁹, Pengtao Xie^20,21, Daniel Sonntag^3,22, James Zou²³, Mathias Niepert^4,24, Anh Totti Nguyen^25✉

*Co-first authors; order randomized   |   **Co-second authors
✉ Corresponding Authors

🎓 Affiliations (click to expand)

1. University of Toronto, Canada 2. Knovel Engineering Lab, Singapore 3. German Research Centre for Artificial Intelligence 4. University of Stuttgart, Germany 5. Chonnam National University, South Korea 6. Singapore University of Technology and Design 7. Bucknell University, USA 8. Concordia University, Canada 9. Korea University 10. Justus Liebig University Giessen, Germany 11. University Medical Center Göttingen, Germany 12. Japan Advanced Institute of Science and Technology 13. Hue University, Vietnam 14. College of William & Mary, USA 15. Deakin University, Australia 16. National University of Singapore 17. University of Texas at Austin, USA 18. University of California, Berkeley, USA 19. New Jersey Institute of Technology, USA 20. University of California San Diego, USA 21. MBZUAI, UAE 22. Oldenburg University, Germany 23. Stanford University, USA 24. Max Planck Research School for Intelligent Systems (IMPRS-IS), Germany 25. Auburn University, USA

---

✨ In honor of Hải Thượng Lãn Ông (海上懶翁) – Lê Hữu Trác (黎友晫), the father of Vietnamese traditional medicine ✨

## 🔍 What is S-Chain? S-Chain is the first large-scale dataset of **Structured Visual Chain-of-Thought (SV-CoT)**: each reasoning step is explicitly linked to visual evidence via bounding boxes. This enables training and evaluating *grounded* medical VLM reasoning instead of hallucinated justifications. - **12,000 medical images** with expert bounding boxes. - **700k+ VQA / rationale pairs** across **16 languages**. - Each sample: image, question, answer, stepwise SV-CoT, and per-step visual regions.

We show that supervising VLMs with SV-CoT:

• Improves interpretability
• Improves grounding fidelity (reasoning actually points to the right region)
• Improves robustness across models and languages

📣 News

• [Oct 2025] Updated experiment scripts and checkpoints for ExGra-Med and LLaVA-Med. See the readme for detailed instructions.
• [Oct 2025] Dataset and project site released.

Citation

If you find this work useful, please cite our paper: https://arxiv.org/abs/2510.22728

@article{leduc2025schain,
  title={S-Chain: Structured Visual Chain-of-Thought For Medicine},
  author={Le-Duc, Khai and Trinh, Phuong T. H. and Nguyen, Duy M. H. and Nguyen, Tien-Phat and Diep, Nghiem T. and Ngo, An and Vu, Tung and Vuong, Trinh and Nguyen, Anh-Tien and Nguyen, Mau and Hoang, Van Trung and Nguyen, Khai-Nguyen and Nguyen, Hy and Ngo, Chris and Liu, Anji and Ho, Nhat and Hauschild, Anne-Christin and Nguyen, Khanh Xuan and Nguyen-Tang, Thanh and Xie, Pengtao and Sonntag, Daniel and Zou, James and Niepert, Mathias and Nguyen, Anh Totti},
  journal={arXiv preprint},
  eprint={2510.22728},
  url={https://arxiv.org/abs/2510.22728},
  year={2025}
}

⚖️ Important Notice on Dataset Usage

The S-Chain dataset is provided solely for research and educational purposes. It may contain human or machine annotation errors, as well as potential biases or inconsistencies inherent to medical data. Users are expected to exercise appropriate caution in interpretation and ensure ethical and non-commercial use.