Doctor of Engineering (EngD) – Image capture and multimodal AI for interactive radiology assistance (Canon Medical and University of Edinburgh)

The EngD is an alternative to a traditional PhD aimed at students wanting a career in industry. Students spend about 75% of their time working directly with a company in addition to receiving advanced-level training from a broad portfolio of technical and business courses. On completion students are awarded the PhD-equivalent EngD.

This project aims to investigate the development of advanced “smart” software systems capable of processing and reacting to clinician inputs in near real-time, in order to interact with radiological images such as CT and X-Ray scans. This will involve technologies such as voice- and gesture-activated assistants, interactive image analysis tools, and multimodal AI models. Such technology will facilitate expert medical tasks such as image navigation, anatomy/pathology detection, visual question answering, and medical reporting. This has the potential to revolutionise radiology workflows, particularly in the interventional radiology domain. The project will require research into integration of diverse multimodal data sources, likely including the application of large vision and language models (e.g. [1,2]) to medical data. Responsive adaptation to a variety of multi-step clinician inputs may require on-the-fly AI model training and fine-tuning.

[1] Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A.C., Lo, W.Y. and Dollár, P., 2023. Segment anything. arXiv preprint arXiv:2304.02643.

[2] Singhal, K., Azizi, S., Tu, T., Mahdavi, S.S., Wei, J., Chung, H.W., Scales, N., Tanwani, A., Cole-Lewis, H., Pfohl, S. and Payne, P., 2023. Large language models encode clinical knowledge. Nature, 620(7972), pp.172-180.

The student will be fully integrated within the AI Research team at Canon in Edinburgh. The project will be largely desk-based (i.e. computational) although there will be opportunity to experiment with hardware e.g. camera & microphone inputs, medical imaging hardware such as ultrasound scanner. There are options for onsite, remote and hybrid working (the team is currently hybrid). In the case of onsite working the student would be allocated a desk in the team’s area within our open-plan office.

CDT Essential Criteria

A Masters level degree (MEng, MPhys, MSc) at 2.1 or equivalent.

Desire to work collegiately, be involved in outreach, undertake taught and professional skills study.

Project Essential Criteria

Analytical.

Strong numerical skills.

Enthusiastic.

Passionate about AI for healthcare.

Project Desirable Criteria

Python programming experience.

Education (e.g. undergraduate degree or masters) in statistics and/or machine learning techniques.

The CDT

The CDT in Applied Photonics provides a supportive, collaborative environment which values inclusivity and is committed to creating and sustaining a positive and supportive environment for all our applicants, students, and staff. For further information, please see our ED&I statement: https://bit.ly/3gXrcwg.

Forming a supportive cohort is an important part of the programme and our students take part in various professional skills workshops, including Responsible Research and Innovation, and attend outreach training.