WO2025042304A1 - Xr-clinic simulator system for practicing professional medical skills - Google Patents

Abstract

The invention relates to the field of medicine and computing. A method for giving users safe practical training in professional medical skills includes steps in which: an animated interactive patient is displayed on a display device in real-time to a trainee, said animated interactive patient being generated by a computing system on the basis of a virtual model of a patient, and the trainee interacts with the animated interactive patient by means of at least one input device while practicing professional medical skills, wherein all of the trainee's actions are registered and recorded in a local database, and the computing system analyzes the trainee's actions while the trainee is interacting with the animated interactive patient and provides audio and video feedback in the form of motor responses, vocal reactions and facial expressions of the animated interactive patient in response to all of the trainee's actions such that pathological changes in the patient's condition can be seen, wherein at the end of a training session, the computing system provides an assessment of the trainee's actions on the basis of saved data.

Description

SIMULATION COMPLEX FOR TRAINING PROFESSIONAL MEDICAL SKILLS XR-CLINIC

AREA OF TECHNOLOGY

This technical solution relates to the field of medicine and computer technology, in particular, to methods and systems for safe practical training of the user in professional medical skills.

LEVEL OF TECHNOLOGY

A solution selected as the closest analogue is known from the prior art, RU 2715148 C1, 02/25/2020. This solution relates to the field of computing, namely to simulators using virtual reality. The simulator includes a PC with a machine-readable medium. It consists of a logical part of the simulator and a graphical three-dimensional shell, which are connected to the computer. It also includes peripheral devices for navigation in the virtual environment. The logical part in the form of a software package includes a switching module and an evaluation module connected to each other via a local software interface. The switching VR simulator is designed in such a way that the operator is connected by two-way communication with the instructor and virtual reality glasses. The virtual reality glasses include: a helmet, headphones, a microphone, a manipulator-controller, position trackers and a platform for movement. The virtual reality glasses have a two-way connection to the software system. It consists of a scenario set module, a scenario editor module and a basic software core module. The engine module consists of a simulator module, a switching module, an evaluation module and a voice module.

The proposed solution is aimed at eliminating the shortcomings of the current level of technology and differs from known solutions in that the proposed solution provides high-quality and safe practical training of the user in professional medical skills.

ESSENCE OF THE INVENTION

The technical problem that the claimed solution is aimed at solving is the creation of a method for safe practical training of the user in professional medical skills. Additional embodiments of the present invention are presented in the dependent claims.

The technical result consists in ensuring safe training of medical personnel.

The claimed technical result is achieved by implementing a method for safe practical training of a user in professional medical skills, which includes the following stages: an animated interactive patient is broadcast to the learning user in real time on a display device, wherein the animated interactive patient is generated by a computing system based on a virtual model of the patient, and the learning user, in the course of practicing professional medical skills, interacts with the animated interactive patient by means of at least one input device, wherein all actions of the learning user are recorded and written to a local database, wherein during the interaction of the learning user with the animated interactive patient, the computing system analyzes the actions of the learning user and, in response to all of his actions, reproduces an audio and video stream in the form of motor, speech and facial reactions of the animated interactive patient with the ability to visualize pathological changes in his condition, wherein at the end of the training session, based on previously saved data, the computing system evaluates the correctness of the actions performed by the learning user.

In a particular embodiment of the described method, the display device is virtual reality glasses.

In another particular embodiment of the described method, the computing system is a virtual reality system.

In another particular embodiment of the described method, the actions of the training user are the supply of oxygen.

Another particular embodiment of the described method is the use of a defibrillator.

In another particular embodiment of the described method, the computing system evaluates the correctness of the user's actions based on an algorithm, where each user action for Each training scenario has a predetermined weight, which is characterized by a number.

DETAILED DESCRIPTION OF THE INVENTION

In the following detailed description of the embodiment of the invention, numerous implementation details are set forth in order to provide a clear understanding of the present invention. However, it will be apparent to one skilled in the art how the present invention may be used with or without these implementation details. In other instances, well-known methods, procedures, and components have not been described in detail in order not to unnecessarily obscure the features of the present invention.

In addition, it will be clear from the above description that the invention is not limited to the embodiment shown. Numerous possible modifications, changes, variations and substitutions, preserving the essence and form of the present invention, will be obvious to those skilled in the art.

Computer technologies are actively used in the educational process almost all over the world. The creation of educational computer simulators is one of the key areas in the computerization of education. Full immersion in virtual reality and interaction with its objects is achieved only with the use of special devices. Such devices that provide full immersion in virtual reality and simulate the user's interaction with it using the senses are called virtual reality (VR) systems.

An important factor in increasing the level of immersion in a virtual environment is the ability of the user to interact with elements of the environment and characters in virtual reality.

The use of simulation techniques is driven by the need to ensure patient safety, provide planned practice, and provide sufficient repetitions to develop practical skills.

Demonstrating symptoms and practicing physical examination skills on real patients is difficult because it is difficult to schedule patients with all the symptoms needed to be mastered in time. clinical rotation of students is impossible, most patients refuse to interact with students, as they experience discomfort and pain when examined by students. In patients with acute pathology, in addition to pain, repeated examinations by students can lead to complications, as well as a delay in providing the necessary emergency care. At the same time, acquiring stable skills requires multiple repetitions and the ability to observe symptoms in different patients.

Thus, the acquisition of skills for examination and recognition of pathologies by students of medical universities is significantly hampered, which subsequently leads to incorrect interpretation of information obtained during examination of patients.

Thanks to the proposed technical solution, the learning user sees the environment, interactive objects, an animated interactive patient and can interact with them. This approach increases the depth of immersion, expands the possibilities of interaction, and also allows for the implementation of automated complex objective assessment systems for user actions.

This technical solution is designed for training and assessment of skills of medical personnel and students (users), without risk to patients, in a realistic simulation environment, with a comprehensive automated assessment system based on the principles of national guidelines.

To ensure ease of use of the equipment and high throughput, technologies for maximizing the ergonomics of the system are used. The solution can be implemented using an autonomous virtual reality system consisting of virtual reality glasses and two ergonomic controllers (input devices). It should be noted that this technical solution is multi-platform, since the method can be implemented not only using virtual reality (VR) systems (Oculus), but also by means of computing equipment (on desktop and web versions). All possible implementations of the method are equal and compatible with the session management and training quality tracking system. The proposed technical solution realistically recreates a dynamic virtual environment, as well as an interactive environment, including medical instruments, virtual patients, and equipment.

The solution contains an automated system for objectively assessing the actions of students, in accordance with current recommendations from ministries of health, and also allows for automated calculation of the patient's medical indicators in real time by modeling the patient's reactions to external influences.

Unlike the analog approach (limited discrete change in the patient's condition), this approach allows the user to have complete freedom of action, regardless of the scenario and pathology of the virtual patient.

The solution implements the ability to conduct multi-user simulations with real-time voice transmission and the functionality of demonstrating the simulation process on external monitors for mass training.

The automated user assessment system at the end of each session provides results in the form of points with a detailed breakdown, based on which the level of preparation of the student can be assessed. The automated assessment system also automatically creates a graph of the progress of the entire group and highlights typical errors, which allows you to highlight weak points in knowledge in large groups of students.

The proposed technical solution contains the following training modules:

"Outpatient appointment with a patient"

International name “Outpatient Appointment”;

"Emergency medical care in a hospital setting."

International name: “Emergency Care Unit Hospital emergency and staff response”.

Emotions, reflexes and other virtual visual manifestations of diseases (facial reactions of an animated interactive patient) are created using Face Capture technology. For this, actors are involved, from whose faces emotions are recorded using special cameras. Since, for example, a stroke or myocardial infarction - an experienced doctor, just by looking at the patient's face, can recognize the diagnosis.

The proposed technical solution also contains a library of sounds characteristic of normal and pathological conditions in accordance with clinical cases and a virtual patient model (audio stream, speech reactions).

The use of a specially developed animated interactive patient (virtual patient model) allows for the continuous calculation of all patient indicators in real time, depending on the user's actions, in accordance with the condition and characteristics of the selected virtual patient.

The Emergency Care Unit - hospital emergency and staff response module uses a universal patient model that allows synthesizing all parameters of a virtual patient (e.g. heart rate, respiratory rate, oxygenation, test results, level of consciousness, pupil response, etc.) depending on the actions performed by the user (e.g. administration of drugs, use of a defibrillator, oxygen supply, etc.).

The patient model contains a large number of parameters that determine the patient’s response to certain influences:

Infusomat;

Oxygen;

Defibrillator;

IV drip;

The drug, time, etc. in real time, as well as the dynamics of its indicators:

heart rate;

respiratory rate;

Saturation;

Analysis;

Appearance;

Behavior;

Glucose, etc.

Virtual patients differ from each other in the numerical values of the above parameters and starting indicators.

The indicators change over time and, if the doctor (training user) does not act (or acts incorrectly), they can lead to the death of the patient. The patient model is implemented in such a way that it is possible to add a large number of effects, as well as visual and numerical indicators. Such a model is universal - the creation of a new patient consists of forming numerical values of a set of parameters.

The patient's indicators change continuously, rather than in discrete states that follow a plot, as a result of which the user has complete freedom of action and is not limited by a prescribed plot.

For user interaction with the interactive virtual patient, the following virtual tools can be used in this technical solution:

Glucometer - Measures glucose levels when placed near the patient's finger;

Pulse oximeter - Measures oxygenation and displays measurements on the monitor;

Tonometer - Measures blood pressure;

Patient Monitor - Reads virtual sensor readings and displays data;

ECG electrodes - Measure the patient's pulse and display the measurements on the monitor;

Stethoscope - Allows auscultation of various points on the patient;

Central and peripheral catheters - Allow the administration of drugs using an infusion pump;

Infusomat - Administers drugs at a certain speed and concentration;

Defibrillator - Performs defibrillation using hand-held electrodes;

Urinary catheter - Fills the urine bag;

Oxygen Mask - Delivers oxygen at the specified flow and humidity;

Cannulas - Deliver oxygen at the specified flow and humidity;

Flashlight - Allows you to determine the reaction of the pupils to light.

Medical examination.

It should be noted that the patient model, virtual patient parameters and evaluation system have been developed by international medical experts, and the permissible impacts on the virtual patient in the solution and their impact on the indicators are based on medical articles and textbooks, since only existing data that did not go beyond the educational programs were used.

The permissible impacts on the virtual patient in the simulator and their influence on the indicators are based on medical articles and textbooks, as well as on the basis of standards of medical care and taking into account clinical recommendations (treatment protocols).

Integrated automated assessment system. All actions of the learning user are recorded and entered into a local database. Based on the guidelines, the assessment system includes an algorithm for standard task completion. The student's actions and inactions are ranked according to their importance and the degree of correctness of the clinical case solution.

Verbal interaction with the patient. It is possible to conduct a dialogue using a dialogue interface. The patient's responses are not only presented in writing, but also realistically voiced for maximum immersion in the virtual environment.

The proposed technical solution also contains:

Interface for adjusting data to comply with current recommendations of the local Ministry of Health;

Interface for operational adjustment of assessment parameters depending on educational goals and clinical recommendations;

A cloud database containing a list of drugs, interpretations of examination results, and the correct sequence of actions for the student;

Tracking every action and inaction of the learner to determine the final result.

Additional positive technical effects from the implementation of the proposed technical solution.

The use of a virtual environment does not require a large physical area. Different locations can be visualized in one room. An automated assessment system and a system of intelligent prompts in the training mode allow to reduce the load on the teacher during the training process.

An example of the implementation of this technical solution. An animated interactive patient is broadcast to the learning user in real time via virtual reality glasses. The animated interactive patient is generated by the computing system based on a virtual model of the patient using pre-set scenarios corresponding to a patient with a specified disease. The scenarios are a set of data corresponding to a patient with a specified disease. The data set includes:

- Patient's appearance;

- The patient’s answers to questions within the framework of the dialogue;

- Results of visual examination of the patient;

- Results of manual examination (palpation, percussion, auscultation);

- Results of instrumental examination (body temperature, heart rate, blood pressure, oxygenation, glucose level);

- Results of additional examinations (ECG, ultrasound, X-rays, MRI, blood tests, colonoscopy, etc.);

- Correct diagnosis;

- Correct treatment;

- Assessments in the form of points for each possible action with the patient.

The learning user, using the virtual reality system (input devices), interacts with the animated interactive patient, namely, performs actions aimed at treating the patient and/or identifying pathologies/diseases. In accordance with pre-established algorithms, for all actions/inactions of the learning user, during his interaction with the animated interactive patient, the computing system reproduces an audio and/or video stream in the form of motor, speech and facial reactions of the animated interactive patient, with the ability to visualize (broadcast to virtual reality glasses) pathological changes in his condition.

All actions of the learning user are recorded and written into a local database, and at the end of the learning session, based on previously saved data, the computing system evaluates the correctness of the actions performed by the learning user.

In each scenario, the user is shown the results of any measurements, in accordance with the specified data, and at the end of the session the user receives a score in accordance with the points scored. All scenario data is divided into categories:

- Passport;

- Dialogues;

- Inspection;

- Additional examination;

- Diagnosis;

- Treatment.

Fig. 1 shows an example of a dialogue for the scenario "Acute appendicitis, uncomplicated" (White male, 30 years old. Teacher).

Fig. 2 shows a general diagram of a computing device (200) capable of providing the data processing necessary for implementing the claimed solution.

In general, the device (200) comprises components such as: one or more processors (201), at least one RAM module (202), data storage means (203), input/output interfaces (204), input/output means (205), network interaction means (206).

The processor (201) of the device performs the basic computing operations necessary for the functioning of the device (200) or the functionality of one or more of its components. The processor (201) executes the necessary machine-readable commands contained in the RAM (202).

The RAM module (202) is usually implemented as RAM and contains the necessary software logic to provide the required functionality.

The data storage means (203) can be implemented in the form of HDD, SSD disks, RAID array, network storage, flash memory, optical storage devices (CD, DVD, MD, Blue-Ray disks), etc. The means (203) allows for long-term storage of various types of information, for example, the above-mentioned files with user data sets, databases containing records of time intervals measured for each user, user identifiers, etc.

Interfaces (204) are standard means for connecting and working with the server part, for example, USB, RS232, RJ45, LPT, COM, HDMI, PS/2, Lightning, FireWire, etc. The choice of interfaces (204) depends on the specific design of the device (200), which can be a personal computer, mainframe, server cluster, thin client, smartphone, laptop, etc.

A keyboard must be used as a means of input/output of data (205) in any embodiment of the system implementing the described method. The hardware implementation of the keyboard can be any known: it can be either a built-in keyboard used on a laptop or netbook, or a separate device connected to a desktop computer, server or other computer device. The connection in this case can be either wired, in which the connecting cable of the keyboard is connected to the PS/2 or USB port located on the system unit of the desktop computer, or wireless, in which the keyboard exchanges data via a wireless communication channel, for example, a radio channel, with a base station, which, in turn, is directly connected to the system unit, for example, to one of the USB ports. In addition to the keyboard, the following can also be used as part of the means of input/output of data: a joystick, a display (touch display), a projector, a touchpad, a mouse manipulator, a trackball, a light pen, speakers, a microphone, etc.

Network interaction means (206) are selected from a device that provides network reception and transmission of data, for example, an Ethernet card, a WLAN/Wi-Fi module, a Bluetooth module, a BLE module, an NFC module, an IrDa, an RFID module, a GSM modem, etc. With the help of means (205), data exchange is organized via a wired or wireless data transmission channel, for example, WAN, PAN, LAN, Intranet, Internet, WLAN, WMAN or GSM.

The components of the device (200) are connected via a common data transmission bus (210).

In these application materials, a preferred disclosure of the implementation of the claimed technical solution was presented, which should not be used as limiting other, particular embodiments of its implementation that do not go beyond the scope of the requested scope of legal protection and are obvious to specialists in the relevant field of technology.

AND

Claims

Formula 1. A method for safe practical training of a user in professional medical skills, comprising the stages of: an animated interactive patient is broadcast to the training user in real time on a display device, wherein the animated interactive patient is generated by a computing system based on a virtual model of the patient, and the training user, in the course of practicing professional medical skills, interacts with the animated interactive patient by means of at least one input device, wherein all actions of the training user are recorded and written to a local database, wherein during the interaction of the training user with the animated interactive patient, the computing system analyzes the actions of the training user and, in response to all of his actions, reproduces an audio and video stream in the form of motor, speech and facial reactions of the animated interactive patient with the ability to visualize pathological changes in his condition, wherein at the end of the training session, based on previously saved data, the computing system evaluates the correctness of the actions performed by the training user. 2. The method according to claim 1, wherein the display device is virtual reality glasses. 3. The method according to claim 1, wherein the computing system is a virtual reality system. 4. The method according to claim 1, wherein the actions of the training user are the supply of oxygen. 5. The method according to claim 1, wherein the actions of the learning user are using a defibrillator. 6. The method according to claim 1, in which the computing system evaluates the correctness of the user's actions performed based on an algorithm, where each user action for each training scenario has a predetermined weight, which is characterized by a number.