Developing a Comprehensive Framework for Clinical Validation of Generative AI in Primary Care: Beyond Performance Metrics

Commercial partner: EMIS

This project aims to develop a robust framework for the clinical evaluation of generative AI large language models (LLMs) in primary care settings, with a specific focus on the critical phase of clinical validation. While existing literature primarily concentrates on algorithmic performance, this research seeks to bridge the gap between technical capabilities and real-world clinical applicability.

The rapid development of generative AI models, particularly large language models (LLMs), has generated significant excitement and expectations within the healthcare sector. These powerful AI systems have demonstrated remarkable abilities to assist clinicians in various tasks, such as summarizing patient records, generating treatment plans, and answering medical queries. However, the clinical translation of these technologies remains a significant challenge, as healthcare professionals and regulators require rigorous evidence of safety, efficacy, and alignment with clinical workflows before widespread adoption.

This project aims to address this gap by developing a comprehensive framework for the clinical evaluation of generative AI LLMs in primary care settings.

Systematic Literature Review:

The first stage of the project will involve a comprehensive review of current evaluation methods for AI in healthcare. This review will identify the strengths and limitations of existing approaches, with a particular focus on gaps in clinical validation methodologies. By synthesizing the current state of the field, the research team will gain a deeper understanding of the challenges and opportunities in assessing the real-world performance of generative AI LLMs in primary care.

Qualitative Research:

In the second stage, the project will leverage qualitative research methods to engage with key stakeholders, including general practitioners (GPs), policymakers, industry representatives, patient representatives, clinical safety specialists, information governance specialists and regulators. Through in-depth interviews, the research team will seek to understand the evidence requirements and concerns of these stakeholders regarding the safe and effective implementation of LLMs in primary care settings. This qualitative data will be instrumental in shaping the design of the subsequent quantitative phase of the research.

Discrete Choice Experiment (DCE):

Building on the insights gained from the literature review and qualitative research, the project will then employ a Discrete Choice Experiment (DCE) to quantify the preferences of healthcare professionals for different aspects of LLM implementation in primary care. We will sample multiple types of healthcare professionals to understand differing preferences. The DCE will present healthcare providers with a series of hypothetical scenarios, with LLMs’ attributes describing: performance, integration, and clinical workflow impact. By analyzing the choices made by participants, we will determine the relative importance of LLMs’ attributes.

The DCE will provide valuable data to inform the development of the clinical evaluation framework, as it will reveal the priorities and trade-offs that healthcare professionals are willing to make when considering the adoption of generative AI LLMs in their clinical practice. This information will be crucial in ensuring that the framework aligns with the needs and preferences of end-users, thereby increasing the likelihood of successful implementation and uptake.

The overarching goal of this project is to develop a comprehensive, evidence-based framework that can guide the clinical evaluation of generative AI LLMs in primary care settings. By addressing the gap between technical capabilities and real-world clinical applicability, this research aims to facilitate the responsible and effective integration of these transformative technologies into the healthcare system.

Apply using course: DPhil Primary Health Care 

MRC INDUSTRIAL CASE STUDENTSHIPS 2025

Designed to nurture the academic entrepreneurs of the future, the Enterprise studentship programme offers a stimulating educational experience as part of the Oxford-MRC DTP cohort, with the additional benefit of working closely with an industrial partner. This will provide entrepreneurial training opportunities and an insight into how commercial science is conducted alongside a superb academic base within the University. Students will work for at least 3 months in the associated company.

ELIGIBILITY

They are open to both UK and non-UK nationals and will follow the UKRI student eligibility requirements. UKRI will normally limit the proportion of international students appointed each year through individual training grants to 30% of the total intake each year.

FUNDING PACKAGE

Each iCASE studentship is fully-funded – it includes four years of stipend at the UKRI stipend level + £2,500 p.a., course fees, and a generous research training support grant.

APPLICATIONS DEADLINE

Applications must be received by 12 noon (UK time) Tuesday 3 December 2024. Details on entry requirements and how to apply can be found below.

For details of entry requirements please go to the Oxford-MRC DTP iCASE 2025 Projects page.

HOW TO APPLY

Before applying for this project we recommend you contact the lead supervisors for informal discussion.

To make a formal application, please complete the University’s online application form for the DPhil course specified under the project description above. Please indicate the iCASE project clearly by inserting ‘iCASE’ before the project title and by using the reference code iCASE. You will need to provide a personal statement (500 words max if applying for a project hosted by one of Medical Sciences departments – please note that this limit might be different if a project is hosted by one of MPLS departments in which case follow their requirement) detailing your interest and fit for the studentship. Note that no project proposal is required for the iCASE studentship applications.

If you wish to apply for a combination of iCASE and other projects within the hosting department, this can be done on the same application form (max number of projects you can apply for on one application depends on the department you wish to apply to). If you wish to apply for iCASE projects within different departments, you will have to make separate applications directly through those departments.

If you have any queries about the iCASE application process (questions about the project should be directed to the lead supervisor), please email [email protected].