RU2799123C1 - Method of learning using interaction with physical objects in virtual reality - Google Patents

Abstract

FIELD: computer technology.

SUBSTANCE: present technical solution relates to methods for teaching users the skills of a physical examination to recognize pathologies using a virtual reality simulation system. A method is proposed, which includes the following steps: to a trainee, in real time, in virtual reality glasses, an animated interactive patient is broadcast, which is synchronized along the outlines of the body with a physical mannequin located in close proximity to the trained user, allowing the user in real time to tactilely feel the anatomically reliable body relief and carry out palpation, percussion and auscultation of organs, and the efforts exerted by the user when interacting with the mannequin during palpation, percussion or auscultation are determined using weight sensors and / or magnetic sensors and converted into deformation of the virtual patient's body surface, due to which audio and / or video stream is reproduced in virtual reality glasses in the form of motor, speech and facial reactions of virtual patients to the influence of the trainee, with the possibility of visualizing pathological changes, and the fixed coordinates and pressure forces applied by the user to the mannequin are used by the computer system for automated evaluation of user actions, according to predetermined criteria.

EFFECT: improved learning efficiency.

2 cl

Description

FIELD OF TECHNOLOGY

The present technical solution relates to the field of computer technology, in particular, to methods for teaching users the skills of a physical examination to recognize pathologies using a virtual reality simulation complex.

BACKGROUND OF THE INVENTION

From the prior art, the solution chosen as the closest analogue is known, RU 2715148 C1, 25.02.2020. This solution relates to the field of computer technology, namely to simulators using virtual reality. The simulator includes a PC with a machine-readable media. It consists of a logical part of the simulator and a graphical three-dimensional shell that are connected to a computer. It also includes peripherals for navigating the virtual environment. The logical part in the form of a software package includes a switching module and an evaluation module interconnected by means of a local program interface. The switching VR simulator is designed in such a way that the student is connected by two-way communication with the instructor and virtual reality glasses. The composition of virtual reality glasses includes: a helmet, headphones, a microphone, a manipulator-controller, position trackers, a platform for movement. Virtual reality glasses have a two-way connection to the software system. It consists of a script set module, a script editor module, and a core program core module. The engine module consists of a simulator module, a switching module, an evaluation module, and a voice module.

The proposed solution is aimed at eliminating the shortcomings of the state of the art and differs from the known solutions in that the proposed method combines the advantages of virtual reality and highly realistic tactile simulators.

SUMMARY OF THE INVENTION

The technical problem to be solved by the claimed solution is to create a method for teaching users the skills of a physical examination for recognizing pathologies using a virtual reality simulation complex.

Additional embodiments of the present invention are presented in dependent claims.

The technical result is to ensure safe user training. An additional technical result is to improve the quality of user training.

The claimed technical results are achieved by implementing a method for teaching users the skills of a physical examination to recognize pathologies using a virtual reality simulation complex, including the steps at which:

An animated interactive patient, synchronized along the contours of the body with a physical mannequin, located in close proximity to the trained user, is transmitted to the learning user in real time in virtual reality glasses, allowing the user to tactilely feel the anatomically reliable body relief and perform palpation, percussion in real time and auscultation of organs in accordance with predetermined examination scenarios, and the efforts exerted by the user when interacting with the dummy during palpation, percussion or auscultation are determined using weight sensors and converted into deformation of the virtual patient's body surface, due to which

in virtual reality glasses, an audio and video stream is reproduced in the form of motor, speech and facial reactions of virtual patients to the effects of a trainee, with the ability to visualize pathological changes, and the fixed coordinates and pressure forces applied by the user to the mannequin are used by the computer system for automated evaluation of user actions, according to predetermined criteria.

In a particular embodiment of the described method, the evaluated actions may include the sequence of passing palpation points, the efforts made, the accuracy of the positioning of the user's hands relative to the patient's body, diagnosis, elements of examination and interaction with the patient and the environment.

DETAILED DESCRIPTION OF THE INVENTION

In the following detailed description of the implementation of the invention, numerous implementation details are provided to provide a clear understanding of the present invention. However, one skilled in the art will appreciate how the present invention can be used, both with and without these implementation details. In other instances, well-known methods, procedures, and components have not been described in detail so as not to unnecessarily obscure the features of the present invention.

Furthermore, it will be clear from the foregoing that the invention is not limited to the present implementation. Numerous possible modifications, changes, variations and substitutions that retain the spirit and form of the present invention will be apparent to those skilled in the subject area.

Computer technologies are actively used in the educational process almost all over the world. Full immersion in virtual reality and interaction with its objects is achieved only when using special devices. Such devices that provide complete immersion in virtual reality and simulate human interaction with it using the senses are called virtual reality (VR) systems.

An important factor for increasing the level of immersion in the virtual environment is the possibility of interactive interaction with environmental elements and characters in virtual reality.

The use of simulation systems is due to the need to ensure patient safety by providing planned practice and a sufficient number of repetitions to develop practical skills for students.

Each of the existing modalities of simulation has its limitations, so the technology using simulated patients allows you to practice communication skills, imitate some diseases externally, convey emotions, but does not allow you to simulate acute conditions, practice invasive procedures, and is also limited by the impossibility of simulating a wide range of clinical symptoms.

Simulators for practicing physical examination skills allow you to simulate various clinical findings - palpation, for example, volumetric formations or enlarged internal organs, changes in pulsation; auscultatory signs of diseases - heart murmurs, pulmonary rales, but at the same time do not allow simulating the patient's reaction and, as a result, a number of symptoms associated with pain or reflex movements when pressed or tapped in certain anatomical areas.

Demonstration of symptoms and the possibility of practicing physical examination skills on real patients is difficult, since it is impossible to plan the presence of patients with all the symptoms necessary for mastering during the clinical rotation of students, most patients refuse to interact with students, as they experience discomfort and pain during examination by students. In patients with acute pathology, in addition to pain, repeated examinations by students can lead to complications, as well as to a delay in the provision of necessary emergency care. At the same time, the acquisition of stable skills requires repeated repetition and the ability to observe symptoms in different patients.

Thus, the acquisition of the skills of physical examination and recognition of pathologies by students of medical universities is significantly difficult, which subsequently leads to an incorrect interpretation of the information obtained during the examination of patients.

The objective of the present invention is to develop the skills of physical examination of patients and recognition of pathologies of various organs and systems.

The teaching method is based on the visualization of various symptoms in a virtual environment during the interaction of students with simulators and phantoms that imitate various parts of the body. Thanks to the technologies used, the student has the opportunity to observe the symptoms of various diseases, including pain on palpation, percussion and auscultatory changes. In addition, the student is not limited in time when performing a physical examination and can repeat the examination many times. The student is offered the opportunity to visualize the pathology of organs when the skin transparency mode is turned on, as well as visual cues that guide the course of the examination. At the same time, the possibility of simultaneous perception of information through different channels - sensory and visual, improves the memorization of information (Baddeley, 1974, 2007). The multiplier effect with the simultaneous use of different modalities underlies the Cognitive Theory of Multimedia Learning (Mayer, 1998), students better transfer the knowledge they received in the process of blended (multimodal) learning.

This technical solution makes it possible to achieve sustainable mastery of the skills of physical examination and recognition of diseases.

The proposed method of training includes training in the following methods:

1. Physical examination of patients with acute abdominal pathology;

2. Physical examination of patients with masses in the abdominal cavity;

3. Physical examination of patients with diseases of the respiratory system;

4. Physical examination of patients with diseases of the cardiovascular system.

Due to the proposed technical solution, the student sees the environment, interactive objects, an animated interactive patient and physically feels an anatomically reliable body relief synchronized with the visual image, which improves the memorization of information and its reproduction. This approach increases the depth of immersion, expands the possibilities of interaction, and also allows you to implement systems for automated complex objective assessment of user actions.

To implement this approach, technologies for tracking the user's head and hands, tracking mannequins and sensors for pressing force on the surface of the mannequin are used, as well as specially created algorithms for applying data from all sensors and a content component, including a rendered location, models of the environment, the patient, animation of his movements and reactions. , models of pathological changes, etc.

During the learning process, all user actions are automatically evaluated and at the end of the session, the results are determined in the form of points with a detailed transcript. Evaluated actions may include the sequence of palpation points, percussion, applied efforts, the accuracy of positioning of the user's hands and instruments relative to the patient's body, diagnosis, elements of examination and interaction with the patient and the environment. Results tables are saved and linked to user accounts.

1. Physical examination of patients with acute abdominal pathology

The method allows to simulate various acute pathologies of the abdominal organs with a high degree of reliability. For each pathology, clinical cases have been developed containing complete and reliable information about the patient, the results of his research and possible treatment trajectories. A phantom is used - an anatomically correct mannequin of the torso of an adult or a child, depending on the clinical situation. A library of sounds characteristic of normal and pathological conditions is used in accordance with clinical cases and a physical model. Each sound corresponds to a specific point on the virtual patient's body and is transmitted directly to the virtual reality helmet. The choice of auscultation point is carried out by applying a stethoscope simulator to the corresponding points on the phantom.

Determination of points and force of pressure is carried out using a system for tracking the position of hands in space and a system for tracking the position of a dummy, as well as pressure sensors located on the platform, and is transmitted to an automated evaluation system for interpreting the effort during palpation and scoring for the skill. The location of the applied force is determined using the user's hand tracking system.

To demonstrate the patient's response in conditions where pain of varying severity is provided in the region of the anterior abdominal wall, the virtual patient demonstrates pain on palpation in a certain area. Pain is presented in the form of various variants of the patient's reaction with corresponding sounds, facial expressions and demonstration of contraction of the anterior abdominal wall with lifting of the legs and upper body in response to pressure of various depths.

It is possible to conduct a dialogue using the dialogue interface. The patient's answers are presented not only in writing, but also realistically voiced for maximum immersion in the virtual environment.

The method includes two modes used sequentially - a learning mode and an evaluation mode. In the training mode, the user has access to visual cues: marked areas on the body of a virtual patient available for performing actions (palpation, percussion, auscultation), the direction of the position of the hands. There are no hints in evaluation mode. An automated evaluation of the case solution is performed in accordance with the prescribed protocols. Time limit of sessions depending on the duration and objectives of the exam. An automatic assessment is carried out according to 5 mandatory blocks: Questioning, General examination and objective examination, Additional examination methods, Diagnosis and Treatment.

2. Physical examination of patients with masses in the abdominal cavity

The method allows to simulate with a high degree of certainty various pathologies of the abdominal organs, accompanied by a change in the volume of individual organs of the abdominal cavity. For each pathology, clinical cases have been developed containing complete and reliable information about the patient, the results of his research and possible treatment trajectories. A phantom is used - an anatomically correct imitator of the torso of an adult or a child, with the possibility of installing various modules that imitate volumetric formations, depending on the clinical situation. A library of sounds characteristic of normal and pathological conditions is used in accordance with clinical cases and a physical model. Each sound corresponds to a specific point on the virtual patient's body and is transmitted directly to the virtual reality helmet. The choice of auscultation point is carried out by applying a stethoscope simulator to the corresponding points on the phantom.

Determination of pressure points and force is carried out using a system for tracking the position of hands in space and a system for tracking the position of a dummy, as well as pressure sensors located on the platform, and is transmitted to an automated assessment system for interpreting the effort and identifying pathological formations during palpation and scoring for the practiced skill.

To demonstrate the patient's response in conditions where pain of varying severity is provided in the region of the anterior abdominal wall, the virtual patient demonstrates pain on palpation in a certain area. The pain is presented in the form of various variants of the patient's reaction, accompanied by sounds and demonstration of contraction of the anterior abdominal wall with lifting of the legs and upper body in response to pressure of various depths.

It is possible to conduct a dialogue using the dialogue interface. The patient's answers are presented not only in writing, but also realistically voiced for maximum immersion in the virtual environment.

The method includes two modes used sequentially - a learning mode and an evaluation mode. In the training mode, the user can visualize the internal organs with the corresponding pathological changes. When performing palpation, the organs can change their shape, which is visualized in virtual reality glasses. Visual cues are available: marked areas on the body of a virtual patient available for performing actions (palpation, percussion, auscultation), the direction of the position of the hands. There are no hints in evaluation mode. An automated evaluation of the case solution is performed in accordance with the prescribed protocols. Time limit of sessions depending on the duration and objectives of the exam. An automatic assessment is carried out according to 5 mandatory blocks: Questioning, General examination and objective examination, Additional examination methods, Diagnosis and Treatment;

3. Physical examination of patients with diseases of the respiratory system

The method allows to simulate various pathologies of the organs of the respiratory system with a high degree of reliability. For each pathology, clinical cases have been developed containing complete and reliable information about the patient, the results of his research and possible treatment trajectories. A phantom is used - an anatomically correct imitation of the upper body of an adult or a child, fixed in an upright position. A library of sounds characteristic of normal and pathological conditions is used in accordance with clinical cases and a physical model. Each sound corresponds to a specific point on the virtual patient's body and is transmitted directly to the virtual reality helmet. The choice of auscultation point is carried out by applying a stethoscope simulator to the corresponding points on the phantom.

The determination of the selected auscultation points is carried out using the tracking system for the position of the hands in space and the tracking system for the position of the dummy, and is transmitted to the automated assessment system for interpreting the completeness of the examination and identifying pathological formations during auscultation and scoring for the skill.

It is possible to conduct a dialogue using the dialogue interface. The patient's responses are presented not only in text form, but also realistically voiced for maximum immersion in the virtual environment.

The method includes two modes used sequentially - a learning mode and an evaluation mode. In the training mode, the user can visualize the organs of the respiratory system with the corresponding pathological changes. Visual cues are available: areas on the body of a virtual patient available for performing actions (palpation, percussion, auscultation), the direction of the position of the hands, the position of the stethoscope. There are no hints in evaluation mode. An automated evaluation of the case solution is performed in accordance with the prescribed protocols. Time limit of sessions depending on the duration and objectives of the exam. An automatic assessment is carried out according to 5 mandatory blocks: Questioning, General examination and objective examination, Additional examination methods, Diagnosis and Treatment.

4. Physical examination of patients with diseases of the cardiovascular system

The method allows to simulate various pathologies of the organs of the cardiovascular system with a high degree of reliability. For each pathology, clinical cases have been developed containing complete and reliable information about the patient, the results of his research and possible treatment trajectories. A phantom is used - an anatomically correct imitation of the body of an adult lying on his back - a widespread model of a cardiac patient simulator that most fully and reliably simulates the symptoms of various cardiovascular pathologies - Harvey. A library of symptoms characteristic of normal and pathological conditions is used in accordance with clinical cases and a physical model, including pathological changes in sounds, pulsations at various points, displacement of the heart impulse. Each sound corresponds to a specific point on the virtual patient's body and is transmitted directly to the stethoscope simulator. The choice of auscultation point is carried out by applying a stethoscope simulator to the corresponding points on the phantom.

The determination of the selected areas of chest palpation, pulsation and auscultation points is carried out using a tracking system for the position of the hands in space and a magnetic tracking system built into the dummy, and is transmitted to an automated assessment system for interpreting the completeness of the examination and identifying pathological formations during auscultation and scoring for practiced skill.

It is possible to conduct a dialogue using the dialogue interface. The patient's responses are presented not only in text form, but also realistically voiced for maximum immersion in the virtual environment.

For all these methods, a unique software and hardware part has been developed that combines the physical model of the patient and the virtual one synchronized with it. The developed content is clinically reliable and allows for both training and automatic assessment of manual and clinical skills, which increases the objectivity of the assessment and reduces the burden on the teacher.

The simulator for physical examination of patients with acute abdominal pathology and patients with volumetric formations in the abdominal cavity with a high degree of reliability allows you to simulate more than 60 clinical situations, including the norm and various pathologies of the abdominal organs. The scenarios depend on the physical model that is currently installed in the manikin to increase the realism of the virtual model. The physical configuration of the mannequin before starting the session is set through a special user interface.

To control the physical diaphragm simulator, a stepper motor is used, mounted on the diaphragm drive handle on the dummy, and an electronic control circuit connected to the computer center. The simulation program has the ability at certain moments of the simulation to give control commands to the drive to set the diaphragm simulator in motion. At the same time, the visual content of the simulator is synchronized with the state of the drive.

The base platform (lodge) must include at least 4 pressure sensors (forces) to determine the degree of pressure and an electronic controller for reading, processing and transmitting data from the sensors.

The dummy cradle contains integrated strain-resistive sensors and a control board.

Control unit structure:

1. Microcontroller;

2. Stepper motor driver;

3. Power system;

4. Bridge-type analog-to-digital converters for strain-resistive sensors.

Functionality of the control unit:

1. Amplification, reading and digitization of analog signals of four strain-resistive sensors.

2. Voltage conversion and control of the diaphragm drive stepper motor mounted on the dummy.

3. Data exchange with a computer via the USB interface (transmission of forces on sensors, receiving commands to control the drive).

The hardware part, through which the proposed technical solution is implemented, can be implemented using the following specific technical elements:

• Manikin for abdominal examination of a patient with an integrated breath drive, without changes in the abdominal cavity (to maintain tactile authenticity);

• Points of virtual reality Oculus Quest;

• Computing center (a computing device capable of performing the necessary data processing, for example, a gaming computer with a wireless video transmitter and monitor);

• Stepper motor, electronic control unit, and belt drive to the diaphragm shaft.

One of the examples of the work of the proposed technical solution.

• The actuator mounted on the dummy rotates the standard shaft, simulating the patient's breathing.

• The dummy is mounted on a platform-base (lodgement) equipped with highly sensitive force sensors and an emphasis that repeats the shape of the dummy from the side to relieve the load from the sensors during two-handed palpations.

• The base platform is connected to a computing center (for example, to a gaming computer) via a USB interface and transmits real-time data on the position of the user's hands and the readings of the force sensors.

• A student wearing VR glasses sees the patient lying down, sees his hands in the area around the patient and can perform palpation according to the scenario. The pressing forces during palpation, determined using sensors, are converted into deformation of the surface of the virtual patient's body and used by the automated evaluation system.

• Inside the virtual location there are interfaces for interacting with a virtual patient.

• Inside the virtual location it is possible to enable the mode of visualization of the internal organs of the patient.

The simulator, through which the proposed technical solution is implemented, contains a library of sounds characteristic of normal and pathological conditions in accordance with clinical cases and a physical model. Each sound corresponds to a specific point on the virtual patient's body.

The detection of the degree of pressure is carried out using pressure sensors located on the platform and transmitted to an automated evaluation system for interpreting the effort during palpation and scoring the skill. The location of the applied force is determined using the user's hand tracking system.

Detection of the place of pressure is carried out using a system for tracking the position of hands in space and a system for tracking the position of the dummy. The data is fed into an automated scoring system to interpret the location, area, and pressure vector for palpation and skill scoring.

Detection of the position of the hands is carried out using an optical tracking system of the position of the hands in space based on stereoscopic cameras mounted on a virtual reality helmet. The data is transferred to an automated assessment system for the position of the hands in space during palpation and scoring for the skill.

Simulation of the physics of the reaction of internal organs to pressure. Data on the location and force of pressure on the surface of the abdomen is used to deform and displace the models of the internal organs of a virtual patient in the imaging mode of the internal organs (transparent abdomen).

In addition to physical simulation in a virtual environment, clinical cases have been developed for each case (scenario) containing complete and reliable information about the patient, the results of his examination, examination options and diagnosis.

Simulation of pain in the patient. To demonstrate patient feedback and empathy during examination, in conditions where pain of varying degrees is provided in the anterior abdominal wall, the virtual patient demonstrates pain on palpation in a certain area. The pain is presented in the form of various variants of the patient's sounds, changes in facial expressions and demonstration of contraction of the anterior abdominal wall in response to pressure of varying degrees of depth.

Integrated automated evaluation system. The system is a collection of information from sensors of physical interaction, spatial orientation and actions of the student in a virtual environment. Based on national guidelines, the assessment system contains an algorithm for the reference performance of the task. The ranking of actions and inactions of the student according to the degree of importance and the degree of correctness of the decision of the clinical case is provided.

Palpation anatomical landmarks. The physical part used to model the pathologies of the abdominal organs is widely represented in the USA, Europe and Asia. As a simulation of the physical part, the Limbs & Things abdominal pathology simulator as the most realistic, widely used and validated in leading medical schools in the USA, Europe and Asia, the Kyoto Kagaku mannequin or any other product that meets the specified requirements can be used as a simulation of the physical part.

Verbal interaction with the patient. It is possible to conduct a dialogue using the dialogue interface. The patient's responses are not only presented in writing, but also realistically voiced for maximum immersion in the virtual environment.

Availability of study/exam modes.

In the training mode, the user has access to visual cues: marked areas on the body of a virtual patient for performing actions (palpation, percussion, auscultation), an indicator of the direction of the position of the hands. There are no hints in exam mode. An automated evaluation of the case solution is performed in accordance with the prescribed protocols. Time limit of sessions depending on the duration and objectives of the exam.

Automated complex objective assessment system during the simulation session (according to the OSKE principles):

Automatic assessment is carried out for 5 mandatory blocks:

Survey;

General examination and objective examination;

Additional examination methods;

Diagnosis;

Treatment.

The proposed technical solution is also implemented through:

Data correction interface to comply with current recommendations of the local Ministry of Health;

Interface for prompt adjustment of assessment parameters depending on educational goals and clinical recommendations;

Cloud database containing a list of drugs, interpretation of examination results, the correct sequence of actions of the student;

Tracking each action and inaction of the student to determine the final result.

Visualization of internal organs during palpation. In training mode, visualization of internal organs is available. When performing palpation, the organs can change their shape, which is visualized in virtual reality glasses.

Visualization of the situation of the examination room. The examination and the solution of the clinical case are carried out in a virtual environment (without the use of physical objects of the entourage), which imitates a real examination room.

The ability to perform other manipulations (except for examining the abdomen). When solving a clinical case, the following are also available: visual examination, palpation and auscultation of other organs and systems: cardiovascular, respiratory, lymphatic, digestive, endocrine.

Imitation of breathing in automatic mode. Simulation of breathing automatically in accordance with a pre-set scenario is carried out using an electric drive installed on the mannequin and the corresponding movements of the patient's body in a virtual environment.

Functions and positive effects provided by the proposed technical solution.

Displaying the location of the doctor's office, patient and interfaces in virtual reality.

Tracking the movements of the head and hands of the user and the mannequin, synchronization of positions with the corresponding objects in virtual reality.

Providing the most realistic tactile feedback when interacting with the body of a virtual patient using a highly realistic manikin and integrated electronics, sensors and an electric drive to simulate breathing.

Measuring the coordinates and pressing force on the mannequin and palpation.

Performing training and examination sessions, issuing the result of a comprehensive assessment of user actions.

Space saving. The use of a virtual environment does not require a large physical area. In one office, different locations can be visualized.

Staff savings. An automated assessment system and a system of intelligent prompts in the training mode can reduce the burden on the teacher in the learning process. A patient abdominal simulator is used to perform patient admission and palpation, eliminating the need for real or standardized patients and actors.

High realism. Virtual reality is complemented by the tactile sensations of a realistic simulator with interchangeable pathologies. Anatomical landmarks and visualization of the environment allow you to immerse yourself in the solution of a clinical case.

Ergonomics. All parts of the simulator occupy no more than 1 sq.m. area of the room, sensors are located on the platform and virtual reality glasses, and the mobile stand can move freely if necessary.

Convenience of the interface. The simulator interface is represented by two parts - an instructor (teacher) and a student. The launch is carried out from the instructor's computer, and the equipment is calibrated using an application on a mobile phone. The learner interface is located entirely in a virtual environment and interactively adapted.

medical credibility. The solution uses the physical part of the simulator for examining the abdominal cavity, which has a high evidence base and fully corresponds to the tactile sensations during palpation of a living person. Case filling: dialogues, examination results and behavior of the virtual patient correspond to clinical situations, which are compiled in accordance with clinical recommendations and examination procedures according to the disease profile.

The simulator for the physical examination of patients with diseases of the respiratory system consists of the following components:

• Torso (mannequin) of a life-size human on a stand for placement on a horizontal surface;

• Glasses of virtual reality;

• Simulator of a stethophonendoscope.

The simulator reproduces the sounds characteristic of a given pathology (state) into the headphones of the virtual glasses. The reproduced sound is the sound recorded at a specific auscultatory point of a real patient.

The choice of the reproduced sound occurs automatically when the stethoscope touches the corresponding zone on the mannequin.

The user in virtual glasses sees the simulated patient and elements of his respiratory and cardiovascular systems.

The ability to adjust the degree of transparency of the chest wall to visualize the underlying structures. Reproduction of animation of the studied structures (movement of valves, alveoli, bronchial wall, etc. in a separate window) to form an understanding of the mechanisms of occurrence of sound phenomena.

• Trial mode.

It involves the choice of any state from the library with free listening to any zones in any order, the choice of the degree of transparency of the chest, the display of vital indicators, familiarization with didactic materials. Auscultation points are highlighted.

• Scenario learning mode.

It involves the selection of any state from the library, the examination follows a pre-created algorithm with text and voice descriptions, tips, animation of certain structures in a separate window for a visual understanding of the mechanisms of the appearance of sound phenomena. The transparency of the chest is determined by the scenario (assigned by the Teacher from the web interface).

• Practice mode.

Assumes examination of a virtual patient according to the state selected from the library (assigned by the Teacher from the web interface). Animation for a visual understanding of the mechanisms of the appearance of sound phenomena. The transparency of the chest is determined by the scenario. During the study, the user fills in the protocol of auscultation. At the end of the lesson, an examination protocol is issued indicating correct and incorrect actions, interpretations.

• Exam mode.

It involves examination of a virtual patient specified by the Administrator with filling in the auscultation protocol. Hints off, chest opaque. (The examination case is assigned by the Teacher from the web interface, the result is displayed in the Teacher's web interface)

In Practice mode, automatic evaluation is available, tracking the correct sequence and time of auscultation at each point.

The simulator for physical examination of patients with diseases of the cardiovascular system is a mannequin of an adult Caucasian male with a normosthenic build, mounted on a movable trolley-stand in a horizontal position with an elevated head end.

The computing device (computer) monitors the position of the stethoscope in real time at each point of auscultation with a display in the form of a three-dimensional model of the chest. Tracking is done using a network of reed switches located under the dummy's skin.

The physical location of the stethoscope in one area of auscultation is displayed as an animation of the corresponding area on the screen and on a three-dimensional model in virtual glasses.

The training mode should be accompanied by the display on the monitor of the detailed anatomy, physiology and pathophysiology of the selected program.

The 3D model is displayed from different angles and is accompanied by text comments.

In the learning mode, the following data is displayed on the monitor:

- heart rate;

- NPV;

- BP;

- patient data;

- dynamic ECG (12 leads);

- dynamic PCG;

- pulse wave with the ability to zoom and select the intensity of the image.

Exam mode.

In the exam mode, the monitor screen displays:

- student data;

- the time of the exam;

- the time of fixation of the stethoscope at the point of auscultation;

- selection of additional studies.

When you select a clinical case in the exam mode, the patient data is displayed on the monitor:

- complaints;

- anamnesis of the disease;

- family history;

- past illnesses.

When passing the exam for the selected clinical case, the monitor displays active fields for filling in the examination results by selecting a value from the list:

- NPV;

- BP;

- heart rate;

- neck veins;

- pulse deficit;

- Appearance of the heart area.

- Arterial pulse:

• symmetry;

• rhythm;

• frequency;

• voltage;

• value;

• form.

- Precordial pulsation:

Left ventricle:

• palpated;

• localization;

• square;

• height;

• force;

• resistance;

• form.

Right ventricle.

• palpable.

Pulmonary artery.

• palpated.

- Venous pulse on the jugular vein.

• form.

- Preliminary diagnostic hypothesis.

After passing the clinical case, the computer system generates a final report with a total score and correct answers.

Quantitative parameters necessary for the implementation of the proposed technical solution:

the number of tracked degrees of freedom of the user's head movements: 6 (3 movements, 3 rotations);

number of tracked degrees of freedom of mannequin movements: 6 (3 movements, 3 rotations);

tracking data update rate: from 60 Hz;

range of measured pressing forces on the mannequin: 0-10 kgf;

sensitivity of determining the coordinates of pressing on the surface of the mannequin: 5 mm;

frequency of updating data from force sensors: from 60 Hz;

Input actions required to implement the proposed technical solution:

virtual reality helmet movements;

user's hand movements;

the position and orientation of the manikin;

data from the pressure sensors on the mannequin.

Output reactions required to implement the proposed technical solution:

image in virtual reality glasses;

sound in headphones;

tables of results of simulation sessions.

In these application materials, a preferred disclosure of the implementation of the claimed technical solution was presented, which should not be used as limiting other, private embodiments of its implementation, which do not go beyond the scope of the requested legal protection and are obvious to specialists in the relevant field of technology.

Claims (4)

1. A method for teaching users the skills of a physical examination for recognizing pathologies using a virtual reality simulation complex, including the steps at which: An animated interactive patient, synchronized along the contours of the body with a physical mannequin, located in close proximity to the trained user, is transmitted to the learning user in real time in virtual reality glasses, allowing the user to tactilely feel the anatomically reliable body relief and perform palpation, percussion in real time and auscultation of organs in accordance with predetermined examination scenarios, and the efforts exerted by the user when interacting with the dummy during palpation, percussion or auscultation are determined using weight sensors and converted into deformation of the virtual patient's body surface, due to which in virtual reality glasses, an audio and video stream is reproduced in the form of motor, speech and facial reactions of virtual patients to the effects of a trainee, with the ability to visualize pathological changes, and the fixed coordinates and pressure forces applied by the user to the mannequin are used by the computer system for automated evaluation of user actions, according to predetermined criteria. 2. The method according to claim 1, in which the evaluated actions include the sequence of passing the palpation points, the applied efforts, the accuracy of the positioning of the user's hands relative to the patient's body, the diagnosis, the elements of examination and interaction with the patient and the environment.