An intelligent training system for healthcare practitioners

2018 Research Internship Project


Faculty Name

Ling Guan

Project Title

An intelligent training system for healthcare practitioners

Project Description

We are developing an intelligent training system that combine state-of-the-art machine learning technologies and Augmented Reality (AR) / Virtual Reality (VR) to provide a realistic training environment with objective assessment of the trainee’s performance. As opposed to traditional setting with multitudes of monitors projecting different types of information, the use of AR/VR with machine intelligence as the core analytical tools can help reduce the cognitive burden of the trainee by seamlessly projecting and blending virtual information with the real-world model (e.g. a manikin). Conceptually, the system consists of four integrated core components: Reconstructor, Visualizer, Interactor and Analyzer. Under the supervision of the professor and senior researchers, the student RA will work on intelligent visualizer which is responsible for presenting the output from the Reconstructor to the trainee in a context-aware manner. In AR/VR, information can be represented in multiple ways, such as augmenting information directly on a mannequin alongside making use of a Heads-Up Display (HUD) like gaming. However, with multiple sources of information, it can be easily overwhelming. The Contextual Visualizer will intelligently decide what information should be conveyed to the trainee based on his/her current phase of training, which can be obtained through the Interactor’s current location and additional information obtained from the pre-planned training manual, present the multiple information sources in an organized fashion so that the trainee can study the correlations among these information sources.

Student Responsibility

The RA will join the project and contribute to the development of the intelligent visualizer. Various algorithms have been researched and developed for visualization. However the situation in AR/VA environment with application to training healthcare practitioners is quite unique and different. The student will test and modify existing algorithms in such environment under senior researchers’ instructions. Specifically, the RA will • Read research papers and review HCI methods. • Implement visualizatoin algorithms. • Implement user interface under instructions. • Configure and compile third party libraries. • Convert algorithms from Matlab codes to visual studio projects.

Specific Requirements

Experienced in C/C++/C# programming language in visual studio Experienced in Matlab Basic knowledge of image processing and machine learning Ability to quickly learn about new IT packages and techniques Knowledge of configuration and compilation of third party libraries Exceptional interpersonal and communication skills Skilled at finding innovative solutions

Reseach Internship Application

Ling Guan : An intelligent training system for healthcare practitioners | Tuesday March 27th 2018 06:14 PM