An efficient medical Vision Language Models (VLM) for medical applications [Artificial Intelligence, Computer Vision, Medical Imaging, Digital Healthcare] – PhD in Computer Science (PhD Funded, student worldwide)

Project Description:

Large Language Models (LLMs) like GPT-4 have demonstrated remarkable abilities in a variety of healthcare tasks, including diagnosis, patient management, and treatment planning. Specialized models like Med-PaLM 2 also provides advanced capabilities in processing and understanding medical language. However, despite these advancements, these models still face considerable limitations in the healthcare domain, particularly in tasks such as medical visual question answering (VQA) for disease diagnosis and understanding. This challenge is largely due to the diversity of medical data modalities, the intricacy of medical reports. While vision language models VLMs (LLaVA-med) have made progress in addressing these challenges, the high resource requirements highlight a pressing need to develop techniques that optimize their efficiency, making the model viable for real-time clinical applications with limited resources.

This project aims to develop an efficient Vision-Language Model (VLM) specifically designed for medical applications that not only enhances diagnostic accuracy but also addresses the substantial resource demands currently hindering LM implementation in healthcare settings. The model will focus on integrating domain-specific medical knowledge to reduce the computational cost while maintaining high performance in tasks such as medical image interpretation and clinical decision-making. By addressing both the computational efficiency challenges and the need for accurate multimodal processing, this project will provide a significant step forward in enabling the use of advanced AI in clinical practice, improving patient outcomes while conserving valuable medical resources.

Objectives:

1.    Efficiency Optimization: Design techniques to reduce the resource consumption of the VLM, focusing on model compression, pruning, and efficient training strategies, making the model viable for real-time clinical applications with limited resources.

2.    Multimodal Integration: Develop a model architecture that can seamlessly process and integrate medical images and corresponding clinical text.

3.    Validation and Benchmarking: Test and validate the model on large-scale medical datasets (PMC-15M) as well private datasets, comparing its efficiency and diagnostic accuracy with state-of-the-art models.

If you have any specific questions regarding this studentship, please contact Prof Xujiong Ye at [email protected] 

The studentship will be awarded on the basis of merit for 3.5 years of full-time study to commence on 1 March 2025. 

International applicants need to be aware that you will have to cover the cost of your student visa, healthcare surcharge and other costs of moving to the UK to do a PhD.

·      Applicants for this studentship must have obtained, or be about to obtain, a First or Upper Second Class UK Honours degree, or the equivalent qualifications gained outside the UK, in an appropriate area of science or technology. 

·      Possess a strong academic background, ideally with a bachelor’s degree in computer science, statistics, physics, engineering, or a related field.

·      Have practical experience working on AI for healthcare projects, utilizing PyTorch and/or TensorFlow libraries.

·      Exhibit proficiency in programming languages such as Python, C++, C, or Java.

·      Preference will be given to candidates with prior experience in presenting or preparing scientific manuscripts for publication in journals or conferences.

·      Evidence of ability to engage in scientific research and to work collaboratively as part of a team, including excellent communication skills in both written and spoken English, is required.

·      If English is not your first language you will need to meet the required level as per our guidance at https://www.exeter.ac.uk/pg-research/apply/english/

In the application process you will be asked to upload several documents.

•  CV

•  Letter of application (outlining your academic interests, prior research experience and reasons for wishing to undertake the project).

•  Transcript(s) giving full details of subjects studied and grades/marks obtained (this should be an interim transcript if you are still studying)

•  Names of two referees familiar with your academic work. You are not required to obtain references yourself. We will request references directly from your referees if you are shortlisted.

•  If you are not a national of a majority English-speaking country you will need to submit evidence of your proficiency in English.

The closing date for applications is midnight on 24th November 2024. Interviews will be held virtually on the MS Teams/Zoom in the week commencing 9th December 2024.

If you have any general enquiries about the application process please email [email protected] or phone 0300 555 60 60 (UK callers) / +44 (0) 1392 723044 (EU/International callers). Project-specific queries should be directed to the main supervisor.

For further information regarding this studentship, please visit the following website: https://www.exeter.ac.uk/study/funding/award/?id=5292