Research Capstone Project: MediNotes

MediNotes: SOAP Note Generation through Ambient Listening, Large Language Model Fine-Tuning, and RAG

• MediNotes is a first-gen GenAI framework that enhances clinical consultations by automating documentation and providing a healthcare-domain–fine-tuned copilot with retrieval-augmented generation (RAG) and ambient listening.
• MediNotes was awarded “Best in Show” as one of the Top Capstone Projects at the University of Chicago showcase.
• This project was a collaboration with UChicago Medicine to advance healthcare AI.
• Building on groundbreaking research from the Microsoft AI team published in Nature, we developed an innovative framework designed to streamline medical documentation and the consultation process, with the goal of alleviating physician burnout.
• By combining cutting-edge technologies like ambient listening, large language model fine-tuning, and retrieval-augmented generation (RAG), MediNotes represents a significant step forward in optimizing healthcare workflows and improving physician efficiency.
• More detail..

Demo Preview:

Live Demo

This may contain error for preview for raw demo because you may need GPU more to 4gb to run the model:
https://medinotes-llm.streamlit.app/search

Code Setup

1. Download the requirements.txt and app.py file.
2. Install the necessary libraries.

!pip install -r requirements.txt  

3. Run the app.

python -m streamlit run app.py

Model:

You can try the fine-tuned model yourself, which converts medical dialogues into SOAP notes:

Model	Hugging Face Link
Full 4-bit quantized model	https://huggingface.co/Yuki20/llama3_8b_aci_3e_full4b
Full model	https://huggingface.co/Yuki20/llama3_8b_aci_3e_full
Adapter only with Unsloth (Mistral 7b)	https://huggingface.co/Yuki20/mistral_7b_aci_3e
Adapter only with Unsloth (Llama3 8b, 5e)	https://huggingface.co/Yuki20/llama3_8b_aci_5e
Adapter only with Unsloth (Llama3 8b)	https://huggingface.co/Yuki20/llama3_8b_aci

Citation

If you find Medinotes useful in your research or applications, please kindly cite:

@inproceedings{leong2024efficient,
  title={Efficient Fine-Tuning of Large Language Models for Automated Medical Documentation},
  author={Leong, HY and Gao, YF and Ji, S and Zhang, Yang and Pamuksuz, Uktu},
  booktitle={2024 IEEE International Conference on Data Science in Intelligent Systems (DSInS)},
  year={2024}, 
  doi={10.1109/DSInS64146.2024.10992195},
  url={https://doi.org/10.1109/DSInS64146.2024.10992195},
  publisher={IEEE}
}

@inproceedings{leong2024medinotes,
  title={A Gen AI Framework for Medical Note Generation},
  author={Leong, HY and Gao, YF and Ji, S},
  booktitle={2024 IEEE International Conference on Artificial Intelligence and Advanced Control (ICAICA)},
  year={2024},
  doi={10.1109/ICAICA63239.2024.10823004},
  url={https://doi.org/10.1109/ICAICA63239.2024.10823004},
  publisher={IEEE}
}

Acknowledgements

You may refer to related work that serves as foundations for our framework and code repository, Aci-bench. Thanks for their wonderful works.