CN117976129A - A depth perception training method based on multi-depth cue scenes - Google Patents
A depth perception training method based on multi-depth cue scenes Download PDFInfo
- Publication number
- CN117976129A CN117976129A CN202410384501.8A CN202410384501A CN117976129A CN 117976129 A CN117976129 A CN 117976129A CN 202410384501 A CN202410384501 A CN 202410384501A CN 117976129 A CN117976129 A CN 117976129A
- Authority
- CN
- China
- Prior art keywords
- training
- user
- scene
- depth
- trained
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
- G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
- G16H15/00—ICT specially adapted for medical reports, e.g. generation or transmission thereof
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
- G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
- G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
- G06F3/013—Eye tracking input arrangements
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Health & Medical Sciences (AREA)
- General Physics & Mathematics (AREA)
- Physics & Mathematics (AREA)
- Human Computer Interaction (AREA)
- Epidemiology (AREA)
- General Health & Medical Sciences (AREA)
- Medical Informatics (AREA)
- Primary Health Care (AREA)
- Public Health (AREA)
- Rehabilitation Tools (AREA)
- Eye Examination Apparatus (AREA)
Abstract
本发明公开了一种基于多深度线索场景的深度知觉训练方法，包括：设置眼球追踪器，用于跟踪眼球运动，实现用户与训练场景交互，并采集眼动数据；在虚拟环境中构建显示深度知觉的三个训练场景；对待训练用户进行训练并记录训练反馈数据和眼动数据；得到分析报告；据其调整视觉训练场景，继续训练，直至完成预设的训练任务；评估待训练用户的深度知觉能力，生成训练结果报告。本发明基于多深度线索场景，在虚拟现实环境中针对不同用户的个体差异，使用不同的深度线索和视差进行深度知觉训练，用户能够更直观地感受到深度知觉能力的提升。同时，引入眼球跟踪技术，进一步提升用户的训练效果。
The present invention discloses a depth perception training method based on a multi-depth clue scene, comprising: setting an eye tracker for tracking eye movements, enabling user interaction with the training scene, and collecting eye movement data; constructing three training scenes showing depth perception in a virtual environment; training the user to be trained and recording training feedback data and eye movement data; obtaining an analysis report; adjusting the visual training scene according to the report, and continuing the training until the preset training task is completed; evaluating the depth perception ability of the user to be trained, and generating a training result report. Based on a multi-depth clue scene, the present invention uses different depth cues and parallaxes to perform depth perception training in a virtual reality environment according to the individual differences of different users, so that users can more intuitively feel the improvement of depth perception ability. At the same time, eye tracking technology is introduced to further improve the training effect of users.
Description
Technical Field
The invention relates to a depth perception training method, in particular to a depth perception training method based on a multi-depth cue scene.
Background
Virtual Reality (VR) is an emerging technology that can create immersive and motivational environments, creating scenes of different depth cues according to user demand, enabling users to perform visual training in real-world-like environments.
Depth perception refers to the ability of the human visual system to play an important role in daily life by sensing and understanding the distance and relative positional relationship between objects. The real world is a three-dimensional environment, so it is crucial to accurately discern the relative position and depth between objects. In recent years, more and more research has focused on how to help improve the visual appearance of depth perception defects, especially for stereoscopic defects.
However, stereoscopic vision is only one aspect of depth perception, and users with poor stereoscopic vision do not necessarily have poor depth perception capability, because depth perception is a comprehensive process involving the comprehensive operation of multiple depth cues and cognitive mechanisms, wherein the depth cues contain multiple perception information, which can enhance the perception capability of the user.
There is still a lack of suitable training and assessment methods for depth perception today and no problem of effective feedback.
Disclosure of Invention
The invention aims to: the invention aims to solve the technical problem of providing a depth perception training method based on a multi-depth cue scene aiming at the defects of the prior art.
In order to solve the technical problems, the invention discloses a depth perception training method based on a multi-depth cue scene, which comprises the following steps:
Step 1, an eyeball tracker is arranged and used for tracking eyeball movement, realizing interaction between a user and a training scene according to the eyeball movement and collecting eye movement data;
step 2, constructing three different training scenes for displaying depth perception in a virtual environment by combining binocular depth cues and monocular depth cues;
Step 3, training the user to be trained by using the training scene, and respectively recording training feedback data and eye movement data of the user to be trained in the training scene;
Step 4, respectively obtaining analysis reports by combining training feedback data recorded in the three training scenes with corresponding eye movement data;
Step 5, adjusting a visual training scene according to the analysis report, and continuing training until a preset training task is completed;
And 6, evaluating depth perception capability of the user to be trained according to the analysis report, and generating a training result report.
Further, the step1 of implementing user interaction with a scene specifically includes:
the eyeball motion position and the gazing focus of the user to be trained are tracked and collected in real time through an eyeball tracker;
When the user to be trained gazes at the training scene, namely the menu in the virtual reality environment, the menu function is triggered when the gazing duration meets the preset condition.
Further, the three different training scenes described in step 2, namely the first training scene, the second training scene and the third training scene of depth perception, specifically include:
a first training scene having both binocular and monocular depth cues; wherein, monocular cue, at least include: perspective effect, distribution of highlights and shadows, superposition of relative sizes and contours of objects, binocular cues, comprising at least: motion parallax and binocular parallax of an object;
The second training scene is used for removing all monocular depth cues, namely removing perspective, highlight and shadow distribution, motion parallax, object relative size and contour superposition, and only retaining binocular depth cues, namely motion parallax and binocular parallax of the object;
And the third training scene is used for removing all monocular depth cues, namely removing perspective, highlight and shadow distribution, motion parallax, object relative size and contour superposition, and simultaneously limiting the motion parallax of objects in the binocular depth cues, and only retaining binocular parallax.
Further, the retaining binocular parallax in step 2 specifically includes: modifying object sizes in the scene and modifying linear perspectives.
Further, the modifying the object size in the scene in step 2 is specifically as follows:
In the training scene, when one object leads another object, the proportion of the previous object is adjusted according to the distance from the next object to human eyes along with the change of the distance from the user to be trained, and the method is concretely as follows:
；
Wherein, For the distance between the latter object and the eyes of the user to be trained,Is the distance between two objects,Is the scaling of the previous object.
Further, the modified linear perspective described in step 2 is specifically as follows:
In the training scene, a reference object is introduced as a rear object, the other object is placed at a position closer to a user along an x-axis, the position of the reference object is offset by a preset length on a y-axis in the center of an inclined plane, the two objects are prevented from overlapping on an x-y plane and an x-z plane, the human eyes and the two objects are kept to have consistent visual angle difference, and the position of the former object is adjusted according to the position relation between the human eyes of the user to be trained and the latter object.
Further, the training of the user to be trained in step 3 is specifically as follows:
in a first training scene, a user to be trained judges the position and depth of a target according to the shielding relation between objects in the scene; judging the concave-convex shape and the position of the target by a user to be trained according to the illumination change and the shadow on the object;
In a second training scene, a user to be trained deduces the distance of the user to be trained by observing the relative speed of the object in motion; the user to be trained judges the relative position of the object by observing the track and direction change of the object in motion; the user to be trained judges the distance and depth of the known object by comparing the visual movement difference of the left eye and the right eye;
In a third training scene, the user to be trained judges the distance of the object by comparing the parallax of the left eye and the right eye; the user to be trained judges the depth and the space position of the object by integrating and comparing the visual information of the left eye and the right eye; the user to be trained judges the distance relation and depth difference of the object by observing the parallax gradient;
And (3) during training, setting different grades according to the difficulty by using different training scenes in the step (2), displaying the grades to the user to be trained for judgment, and scoring according to the judgment of the user to be trained.
Further, the analysis report in step 4 at least includes: user training accuracy, user completion time, entropy and stereoscopic sharpness, wherein:
User training accuracy The calculation method is as follows:
；
Wherein, Representing the score of the user to be trained in the training process,Representing the total score in the training process;
The user completion time T is the time for completing one time of training in one level of a scene of the user to be trained;
Entropy, namely, calculating an uncertainty value of a user to be trained through gaze data in eye movement data, wherein the specific calculation method is as follows:
；
Wherein, Shannon entropy representing gaze point,Represents gaze point range probability,Represents the gaze point,Represents Net,Representing the value quantity of the gaze point;
The method for calculating the stereo vision sharpness comprises the following steps:
；
Wherein, Representing stereo acuity,AndRepresenting two object positionsAndAn angle formed with the eye.
Further, in the step 5, according to the analysis report, the visual training scene is adjusted, and training is continued, which specifically includes:
If the analysis report satisfies the following conditions: the training times of the user to be trained in the same scene meet the preset training times, the accuracy of each training, the user completion time and the entropy value all meet the preset requirements, and different parallaxes in the scene can be distinguished as ；
The scene automatically enters the next level after the last test is completed. After all the level tests in one scene are completed, the scene automatically jumps to the next depth scene for training.
Further, the parallax described in step 5The calculation method is as follows:
；
Wherein, Representing the target position,AndRepresenting the position of the left and right eye, respectively,Representing the angle between the left eye and the target,Representing the angle between the right eye and the target,Representing the distance between the left eye and the target,Representing the distance between the right eye and the target.
Advantageous effects
1. According to the invention, through constructing training scenes with different depth cues, immersive experience and visual parallax are provided for a user in a virtual reality environment, so that depth perception is improved. Conventional depth perception training techniques typically provide only a single depth cue, limiting the user's understanding and adaptability to real-world depth perception.
2. According to the invention, the eyeball tracker is introduced to track the eyeball movement of the user, so that the real-time capturing and man-machine interaction of the user information are realized. Compared with the prior art, the invention improves the eye movement data acquisition and man-machine interaction, and provides more accurate and efficient interaction experience for users.
3. According to the invention, the training result report is generated by analyzing the user feedback information and the eye movement data, so that the defects of the prior art are further improved, more comprehensive and personalized training evaluation and feedback are provided for the user, and the depth perception capability of the user is improved.
4. By introducing three levels of difficulty designs, the invention provides a more interesting, diversified and challenging training experience for users. This improvement can better motivate users' enthusiasm and sustained engagement, further enhancing their depth perception capabilities, than the single tediousness of the prior art.
Drawings
The foregoing and/or other advantages of the invention will become more apparent from the following detailed description of the invention when taken in conjunction with the accompanying drawings and detailed description.
Fig. 1 is a general flow chart of the present invention.
Fig. 2 is a schematic diagram of the invention in a linear perspective.
FIG. 3 is a schematic view of the object size according to the present invention.
Fig. 4 is a schematic diagram of a training scenario in the present invention.
Fig. 5 is a schematic diagram of the relative parallax measurement according to the present invention.
FIG. 6 is a flow chart of the adaptive game level adjustment in the present invention.
FIG. 7 is a flow chart of the adaptive adjustment of visual scenes in the present invention.
Detailed Description
The invention provides a depth perception training method based on a multi-depth cue scene, which solves the problems that proper training and evaluation methods are still lacking in depth perception and effective feedback is not available. The technical scheme is as follows:
eye movement module: and adopting an eye movement tracking technology to interact with the scene according to the eye movement information of the user.
Eyeball information acquisition unit: the eyeball tracking technology is used for collecting eyeball information of a tested person in the virtual game environment and recording information such as tested person eyeball fixation information, glance range, fixation times, blink times and the like.
A data storage unit: for storing training data and eye movement data of the user.
A data analysis unit: the method is used for integrating and analyzing training data of the user in the virtual environment with eye movement data.
Report generation unit: a depth perception training result report about the user is generated according to the data analysis result.
In a first aspect, the present invention provides a depth perception training method based on a multi-depth cue scene, where the method is shown in fig. 1, and specifically includes:
the eyeball tracker is arranged to track the eyeball movement of the user, so that the interaction between the user and the scene and the acquisition of the eye movement data are realized;
Displaying a first training scene of depth perception in a virtual environment by combining binocular depth cues and monocular depth cues;
a second training scene that displays depth perception in the virtual environment by removing monocular depth cues while preserving binocular depth cues;
displaying a third training scene of depth perception in the virtual environment by including binocular disparity cues and restricting motion disparity cues;
Three analysis reports are respectively obtained by combining training feedback data of the three scenes with eye movement information of a training process;
Self-adaptively adjusting the visual scene according to the user analysis report;
the depth perception capability of the user is evaluated according to three analysis reports obtained in three scenes, and a training result report is generated.
Further, the position and the fixation focus of the eyeball movement of the user are tracked and acquired in real time through an eyeball tracker in the eye movement module. By analyzing eye tracking data when a user gazes at a start menu in a virtual reality environment, the system is able to recognize the user's intent and trigger the start of a game when the gaze duration meets certain criteria.
In a second aspect, a training scene for displaying depth perception in a virtual reality environment by transforming binocular depth cues and monocular depth cues, comprising:
The first training scene, consisting of binocular and monocular depth cues, includes perspective, distribution of highlights and shadows, motion parallax, object relative size, and contour superposition. Objects in the scene are farther from the user and thus appear smaller, while objects closer to the user appear larger; the targets in the scene can be overlapped or partially blocked, and a user can judge the positions and the depths of the targets according to the blocking relation; the scene has fixed illumination, and a user can judge the concave-convex shape and the position of the target according to illumination change and shadow on the target.
And the second training scene is used for removing all monocular depth cues, including perspective, highlight and shadow distribution, motion parallax, object relative size and contour superposition, and finally only binocular depth cues are reserved. By observing the relative speed of the object in motion, the user deduces the distance of the object. The farther target produces slower visual movement at the same speed, while the nearer target produces faster visual movement; the user perceives the relative position of the object by observing the track and direction change of the object in motion; the user more accurately perceives the distance and depth of the object by comparing the visual motion differences of the left and right eyes.
And a third training scene, which is used for removing all monocular depth cues, including perspective, highlight and shadow distribution, motion parallax, object relative size and contour superposition, and limiting the motion parallax in the binocular depth cues, so as to leave binocular parallax cues. By comparing the parallax of the left and right eyes, the user deduces the distance of the object. The user can perceive the depth and the space position of the target by integrating and comparing the visual information of the left eye and the right eye; by observing the parallax gradient, the user can judge the distance relationship and depth difference of the object.
Further, the retaining binocular disparity includes modifying the object size. The distance of the object from the user and its size can affect the angle of view projected on the user's retina. When one object leads another, the viewing angle of the object increases as the distance from the user decreases. Therefore, in order to make the viewing angle identical between objects, it is necessary to adjust the ratio of the latter ball according to the ratio of the latter object to the distance. The distance between the previous object and the eyes of the user is D, the distance between the previous object and the eyes of the user is D, and the scaling ratio of the previous object is。
Further, the preserving binocular disparity includes modifying the linear perspective. One reference object is introduced as a post object and the other ball is placed closer to the user along the x-axis, and the user, the inclined plane and the two objects must be aligned in order to maintain the same viewing angle. However, this would result in two objects overlapping in the gaze direction, which needs to be avoided. Thus, the position of the reference object is offset by 25cm on the y-axis in the center of the inclined plane. If the previous object is closer to the user only in the x-axis, the perspective between the user and each object will be different. This effect applies not only to the x-y plane but also to the x-z plane. Therefore, the previous object must be consistent with the three-dimensional vector between the user and the reference object while satisfying the required parallax difference.
In a third aspect, a user obtains an analysis result report by combining training feedback data of a scene with eye movement information of a training process, where the analysis result report includes user training accuracy, user completion time, entropy, and stereo vision sharpness, and the analysis result report specifically includes:
User training accuracy (a): (training final score/game total score) ；
User completion time (T): refers to the time for the user to complete one training in one level of one scene
Entropy: gaze data is obtained by an eye tracker and applied to gaze behavior, indicating variability and unpredictability of gaze location. If the gaze locations are random and evenly distributed over all possible locations, the entropy will be high, indicating to the user whether the target is correct or not. But if it aims at a particular location in a predictable way, the value of entropy will be low, indicating that the user has a high probability of determining the target is correct.
Wherein,Shannon entropy representing gaze point,Represents gaze point range probability,Represents the gaze point,Represents a Nate, usually taken as 2,Representing the value quantity of the gaze point;
Stereo visual acuity: in depth perception capability, stereo sharpness is an indispensable measure. There are also 3 levels in each depth scene. In the first level, the required stereoscopic sharpness vision is 400 arcseconds, in the second level, the required stereoscopic sharpness vision is 200 arcseconds, and in the fourth level, the required stereoscopic sharpness is 100 arcseconds. This means that as the level increases, the user's stereoscopic sharpness will increase.
In the formula (2), the amino acid sequence of the formula (2),Representing stereo acuity,AndRepresenting the angle formed by the two target positions a and B with the eyes, the parallax is quantified.
According to the fourth aspect, according to the analysis result report, the training scene is adaptively adjusted and the user is trained, specifically including:
the user needs to test more than 2 times at least in one scene, if the accuracy of the two training processes reaches more than 90%, the time for completing one test is not more than 3 minutes, the entropy can be less than 0.5, and different parallaxes in the scene can be distinguished 。
；
In the formula (3), B represents a target position,AndRepresenting the position of the left and right eye, respectively.
The scene automatically enters the next level after the last test is completed. After all the level tests in one scene are completed, the scene automatically jumps to the next depth scene for training.
Further, at the back end of the virtual reality device, C# programming is used, a condition triggering event is adopted, and when the condition is completed, an adaptive mechanism is triggered, and the next level of the scene is automatically jumped. When a user has tested all levels in one scene, the next depth scene will be automatically skipped. If one of the tasks is not satisfied, the scene is circulated until the next condition is triggered;
In a fifth aspect, the user gives a report of the training results after training for one month. And obtaining the depth perception capability level by using the user training accuracy, the user completion time, the entropy value and the stereo vision sharpness in the training result report, and objectively evaluating the user training effect.
Example 1:
According to the depth perception training method based on the multi-depth cue scene, a VR game can be created in the virtual reality device, the game comprises three scenes with different depth cues, and each scene has three different levels. A group of target monsters with different parallaxes is placed in the game just in front of the user's field of view, and the target monsters are moved by a fixed number of steps (4 seconds per step) 2m away from the user, and in order to obtain a high score, the user needs to accurately identify the nearest target monsters to himself and hit them quickly before they reach a distance point of 1.5 m. And if the accurate monster targets are hit each time, the corresponding score and the accurate prompt sound effect feedback are generated, otherwise, the score is not added, and the wrong sound effect feedback is output. The time until the user hits the next hit from the last hit and the total time to complete the game are recorded.
After each scene training, the user can obtain an analysis report, and indexes such as user training accuracy, user completion time, entropy value, stereo vision sharpness and the like are displayed on the analysis report. Training will last one month, the depth perception capability and the stereo vision sharpness of the user will be greatly improved for one month, the depth perception capability and the stereo vision sharpness of the user will be greatly improved, the user can more accurately judge the position and the distance of the object, and the safety during actions is improved. After all training is completed, the user will get a training result report. The training results report may evaluate the user's depth perception capabilities.
As can be seen from the above description, the embodiment of the present invention provides a depth perception training method based on a multi-depth cue scene. In the invention, depth perception scenes are constructed by changing binocular depth cues and monocular depth cues, and a colorful and attractive powerful environment is provided for users through VR equipment, so that different scenes can be changed, and game level and time are set to increase the interestingness of the virtual game.
Example 2:
the invention provides a depth perception training method based on a virtual reality game, which specifically comprises the following steps:
s1, tracking eyeball movement of a user by installing an eyeball tracker to realize interaction between the user and a scene and acquisition of eye movement data;
S2, displaying a first training scene of depth perception in a virtual environment by combining the binocular depth cue and the monocular depth cue;
s3, reserving a second training scene of depth perception in the virtual environment by removing the monocular depth cues and reserving the binocular depth cues;
s4, displaying a third training scene of depth perception in the virtual environment by containing binocular parallax cues and limiting motion parallax cues;
s5, three analysis reports are respectively obtained by combining training feedback data of the three scenes with eye movement information of a training process;
S6, adaptively adjusting the visual scene according to the user analysis report;
And S7, evaluating the depth perception capability of the user according to three analysis reports obtained in three scenes, and generating a training result report.
Further, the eyeball movement position and the gazing focus of the user are tracked and collected in real time through a high-precision eyeball tracker in the eye movement module. By analyzing eye tracking data when a user gazes at a start menu in a virtual reality environment, the system is able to recognize the user's intent and trigger the start of a game when the gaze duration meets certain criteria.
The training scene for displaying depth perception in a virtual reality environment by transforming binocular depth cues and monocular depth cues described in the present embodiment includes:
the first training scene, consisting of binocular and monocular depth cues, includes perspective, distribution of highlights and shadows, motion parallax, relative object size, and occlusion.
Further, the perspective line clues can enable a user to perceive the distance and the distance according to the convergence degree of the clues; object size is also an important monocular depth cue, objects in the scene are far from the user and thus appear smaller, while objects closer to the user appear larger; the targets in the scene can be overlapped or partially blocked, and the user can judge the positions and the depths of the targets according to the blocking relation. Occlusion cues provide important information about the relative position and distance between objects; the scene has fixed illumination, and a user can judge the concave-convex shape and the position of the target according to illumination change and shadow on the target.
And the second training scene is used for removing all monocular depth cues, including perspective, highlight and shadow distribution, motion parallax, object relative size and contour superposition, and finally only binocular depth cues are reserved. There is motion parallax in the preserved binocular depth cues. Motion parallax is the difference in relative motion in the field of view due to objects in different positions as the user moves himself.
Further, by observing the relative speed of the target in motion, the user can infer the distance of the target. The farther objects produce slower visual movements at the same speed, while the closer objects produce faster visual movements.
The user perceives the protrusion, penetration and relative position of the object by observing the track and direction change of the object in motion.
The user can more accurately perceive the distance and depth of the object by comparing the visual movement differences of the left and right eyes.
And the third training scene is used for removing all monocular depth cues, including perspective, highlight and shadow distribution, motion parallax, object relative size and contour superposition, and limiting the motion parallax in the binocular depth cues, so that binocular parallax cues are left, and the depth cue conditions are gradually reduced.
Further, binocular parallax refers to a visual difference generated by two eyes observing an object at different positions. Binocular parallax is one of the key factors of depth perception, including parallax interpretation depth, stereoscopic vision and parallax gradient, and plays an important role in the perception of the far and near and spatial depth of an object by a user.
By comparing the parallax of the left and right eyes, the user can infer the distance of the object. Such parallax differences may help a user perceive the distance and depth of an object.
The retaining binocular disparity includes modifying the object size. The distance of the object from the user and its size can affect the angle of view projected on the user's retina. When one object leads another, the viewing angle of the object increases as the distance from the user decreases. Therefore, in order to make the viewing angle identical between objects, it is necessary to adjust the ratio of the latter ball according to the ratio of the latter object to the distance.
As shown in FIG. 2, the distance between the previous object and the eyes of the user is D, the distance between the previous object and the eyes of the user is D, and the scaling ratio of the previous object is。
The preserving binocular disparity includes modifying the linear perspective. As shown in fig. 3, one reference object a' is introduced as a rear object and the other object B is placed closer to the user along the x-axis, and the user, the inclined plane and the two objects must be aligned in order to maintain the same viewing angle. However, this would result in two objects overlapping in the gaze direction, which needs to be avoided. Thus, the position of the reference object is offset by 25 cm on the y-axis in the center of the inclined plane. If the previous object is closer to the user only in the x-axis, the perspective between the user and each object will be different. This effect applies not only to the x-y plane but also to the x-z plane. Therefore, the previous object must be consistent with the three-dimensional vector between the user and the reference object while satisfying the required parallax difference.
In this embodiment, a user obtains an analysis report by combining training feedback data of a scene with eye movement information in a training process, where the analysis result report may include user training accuracy, user completion time, entropy value, and stereo vision sharpness, and specifically includes:
user training accuracy: (training final score/game total score) ；
User completion time: the time for the user to finish one training in one level of one scene is referred to;
Entropy: gaze data is obtained by an eye tracker and applied to gaze behavior, indicating variability and unpredictability of gaze location. If the gaze locations are random and evenly distributed over all possible locations, the entropy will be high, indicating to the user whether the target is correct or not. But if it aims at a particular location in a predictable way, the value of entropy will be low, indicating that the user has a high probability of determining the target is correct.
Stereo visual acuity: in depth perception capability, stereo sharpness is an indispensable measure. There are also three levels in each depth scene. In the first level, the required stereoscopic sharpness vision is 400 arcseconds, in the second level, the required stereoscopic sharpness vision is 200 arcseconds, and in the third level, the required stereoscopic sharpness is 100 arcseconds. This means that as the level increases, the user's stereoscopic sharpness will increase.
In the formula (2), the amino acid sequence of the formula (2),Representing stereo acuity,AndRepresenting the angle formed by the two target positions a and B with the eyes, the parallax is quantified. As shown in fig. 5, the inter-pupillary distance is denoted as a, the angle formed by the two target locations a and B with the eyeAndParallax is quantized. The smaller the difference between the two angles, the smaller the distance Δd of the targets a and B. The disparity threshold is determined by reducing the distance Δd between objects multiple times until the depth difference between two objects cannot be discerned, and the stereo sharpness is evaluated.
In this embodiment, according to the analysis result report, the training scenario is adaptively adjusted and the user is trained, which specifically includes:
As shown in FIG. 6, the user needs to test more than 2 times at least in one scene, if the accuracy of training is more than 90% in both times, the test is completed for no more than 3 minutes, the entropy can be less than 0.5, and different parallaxes in the scene can be distinguished 。
；
In the formula (3), B represents a target position,AndRepresenting the position of the left and right eye, respectively.
The scene automatically enters the next level after the last test is completed. After all the level tests in one scene are completed, the scene automatically jumps to the next depth scene for training.
As shown in fig. 7, at the back end of the virtual reality device, using c# programming, a conditional triggering event is employed, and when the above conditions are completed, an adaptive mechanism is triggered, automatically jumping to the next level of the scene. When the user has tested all levels in one scene, the user will automatically jump to the next depth scene. If one of the tasks is not satisfied, the scene is circulated until the next condition is triggered;
in this embodiment, the user gives a report of the training result after training for one month. And obtaining the depth perception capability level by using the user training accuracy, the user completion time, the entropy value and the stereo vision sharpness in the training result report, and objectively evaluating the user training effect.
；
Wherein A represents the accuracy of user training, T represents the user completion time,Representing the stereo sharpness.
Example 3:
the module and the unit provided by the invention comprise:
eye movement module: and adopting an eye movement tracking technology to interact with the scene according to the eye movement information of the user.
Eyeball information acquisition unit: the eyeball tracking technology is used for collecting eyeball information of a tested person in the virtual game environment and recording information such as tested person eyeball fixation information, glance range, fixation times, blink times and the like.
A data storage unit: for storing training data and eye movement data of the user.
A data analysis unit: the method is used for integrating and analyzing training data of the user in the virtual environment with eye movement data.
Report generation unit: a depth perception training result report about the user is generated according to the data analysis result.
Further, the eye movement module starts the game when the user gazes at a start menu of the interface. A gaze is considered to be directed at an object when the duration of the gaze is greater than 100ms and the gaze angle is less than 1 °.
Example 4:
According to the depth perception training method based on the multi-depth cue scene, a VR game can be created in the virtual reality device, the game comprises three scenes with different depth cues, and each scene has three different levels. As shown in fig. 4, a group of target monsters having different parallaxes is placed in the game just in front of the user's field of view, and the target monsters are moved by a fixed number of steps (4 seconds per step), so that in order to obtain a high score, the user needs to accurately identify the nearest target monsters to himself and hit them quickly before they reach a distance point of 1.5 meters. And if the accurate monster targets are hit each time, the corresponding score and the accurate prompt sound effect feedback are generated, otherwise, the score is not added, and the wrong sound effect feedback is output. The time until the user hits the next hit from the last hit and the total time to complete the game are recorded.
There are three different levels of difficulty in a scene and there are scores and timings, and object discrimination in each level is required for stereoscopic sharpness to measure the depth difference between every two consecutive objects. In the first level, the required stereoscopic sharpness vision is 400 arcseconds, while in the fourth level, the required stereoscopic sharpness is 100 arcseconds, the higher the level, the greater the difficulty.
The user needs to test more than 2 times at least in one level in one scene, if the accuracy of the two training processes reaches more than 90%, the time for completing one test is not more than 3 minutes, the target parallax measurement is in the parallax range of the specified scene and the entropy value can be below 0.5, and the scene automatically enters the next level after the last test is completed. After all the level tests in one scene are completed, the scene automatically jumps to the next depth scene for training. If the above requirements are not met, the cost level needs to be re-completed until the requirements are met. After each scene training, the user can obtain an analysis report, and indexes such as user training accuracy, user completion time, entropy value, stereo vision sharpness and the like are displayed on the analysis report. Training will last for one month, and the depth perception capability and the stereo vision sharpness of the user will be greatly improved, so that the user can more accurately judge the position and distance of the object, and the safety during the action is improved. After all training is completed, the user will get a training result report. The training results report may evaluate the user's depth perception capabilities.
In a specific implementation, the present application provides a computer storage medium and a corresponding data processing unit, where the computer storage medium is capable of storing a computer program, where the computer program when executed by the data processing unit may perform some or all of the steps of the depth perception training method based on a multi-depth cue scene provided by the present application. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), a random-access memory (random access memory, RAM), or the like.
It will be apparent to those skilled in the art that the technical solutions in the embodiments of the present invention may be implemented by means of a computer program and its corresponding general hardware platform. Based on such understanding, the technical solutions in the embodiments of the present invention may be embodied essentially or in the form of a computer program, i.e. a software product, which may be stored in a storage medium, and include several instructions to cause a device (which may be a personal computer, a server, a single-chip microcomputer, an MCU or a network device, etc.) including a data processing unit to perform the methods described in the embodiments or some parts of the embodiments of the present invention.
The invention provides a thought and a method for depth perception training based on a multi-depth cue scene, and the method and the way for realizing the technical scheme are numerous, the above is only a preferred embodiment of the invention, and it should be noted that, for those skilled in the art, several improvements and modifications can be made without departing from the principle of the invention, and the improvements and modifications should be regarded as the protection scope of the invention. The components not explicitly described in this embodiment can be implemented by using the prior art.
Claims (10)
1. A depth perception training method based on a multi-depth cue scene, the method comprising:
Step 1, an eyeball tracker is arranged and used for tracking eyeball movement, realizing interaction between a user and a training scene according to the eyeball movement and collecting eye movement data;
step 2, constructing three different training scenes for displaying depth perception in a virtual environment by combining binocular depth cues and monocular depth cues;
Step 3, training the user to be trained by using the training scene, and respectively recording training feedback data and eye movement data of the user to be trained in the training scene;
step 4, respectively obtaining analysis reports by combining training feedback data recorded in the training scene with corresponding eye movement data;
Step 5, adjusting a visual training scene according to the analysis report, and continuing training until a preset training task is completed;
And 6, evaluating depth perception capability of the user to be trained according to the analysis report, and generating a training result report.
2. The depth perception training method based on the multi-depth cue scene as claimed in claim 1, wherein the step 1 of implementing the user interaction with the training scene specifically comprises:
the eyeball motion position and the gazing focus of the user to be trained are tracked and collected in real time through an eyeball tracker;
When the user to be trained gazes at the training scene, namely the menu in the virtual reality environment, the menu function is triggered when the gazing duration meets the preset condition.
3. The depth perception training method based on multi-depth cue scenes according to claim 2, wherein the three different training scenes in step 2, namely a first training scene, a second training scene and a third training scene of depth perception, specifically comprise:
a first training scene having both binocular and monocular depth cues; wherein, monocular cue, at least include: perspective effect, distribution of highlights and shadows, superposition of relative sizes and contours of objects, binocular cues, comprising at least: motion parallax and binocular parallax of an object;
The second training scene is used for removing all monocular depth cues, namely removing perspective, highlight and shadow distribution, motion parallax, object relative size and contour superposition, and only retaining binocular depth cues, namely motion parallax and binocular parallax of the object;
And the third training scene is used for removing all monocular depth cues, namely removing perspective, highlight and shadow distribution, motion parallax, object relative size and contour superposition, and simultaneously limiting the motion parallax of objects in the binocular depth cues, and only retaining binocular parallax.
4. A depth perception training method based on a multi-depth cue scene as claimed in claim 3, wherein the retaining binocular disparity in step 2 specifically comprises: modifying object sizes in the scene and modifying linear perspectives.
5. The depth perception training method based on the multi-depth cue scene as claimed in claim 4, wherein the modifying the object size in the scene in step2 is specifically as follows:
In the training scene, when one object leads another object, the proportion of the previous object is adjusted according to the distance from the next object to human eyes along with the change of the distance from the user to be trained, and the method is concretely as follows:
；
Wherein, For the distance between the latter object and the eyes of the user to be trained,Is the distance between two objects,Is the scaling of the previous object.
6. The depth perception training method based on multi-depth cue scene as claimed in claim 5, wherein the modified linear perspective in step 2 is as follows:
In the training scene, a reference object is introduced as a rear object, the other object is placed at a position closer to a user along an x-axis, the position of the reference object is offset by a preset length on a y-axis in the center of an inclined plane, the two objects are prevented from overlapping on an x-y plane and an x-z plane, the human eyes and the two objects are kept to have consistent visual angle difference, and the position of the former object is adjusted according to the position relation between the human eyes of the user to be trained and the latter object.
7. The depth perception training method based on the multi-depth cue scene as claimed in claim 1, wherein the training of the user to be trained in the step 3 is specifically as follows:
in a first training scene, a user to be trained judges the position and depth of a target according to the shielding relation between objects in the scene; judging the concave-convex shape and the position of the target by a user to be trained according to the illumination change and the shadow on the object;
In a second training scene, a user to be trained deduces the distance of the user to be trained by observing the relative speed of the object in motion; the user to be trained judges the relative position of the object by observing the track and direction change of the object in motion; the user to be trained judges the distance and depth of the known object by comparing the visual movement difference of the left eye and the right eye;
In a third training scene, the user to be trained judges the distance of the object by comparing the parallax of the left eye and the right eye; the user to be trained judges the depth and the space position of the object by integrating and comparing the visual information of the left eye and the right eye; the user to be trained judges the distance relation and depth difference of the object by observing the parallax gradient;
And (3) during training, setting different grades according to the difficulty by using different training scenes in the step (2), displaying the grades to the user to be trained for judgment, and scoring according to the judgment of the user to be trained.
8. The depth perception training method based on multi-depth cue scene as claimed in claim 7, wherein the analysis report in step 4 at least comprises: user training accuracy, user completion time, entropy and stereoscopic sharpness, wherein:
User training accuracy The calculation method is as follows:
；
Wherein, Representing the score of the user to be trained in the training process,Representing the total score in the training process;
The user completion time T is the time for completing one time of training in one level of a scene of the user to be trained;
Entropy, namely, calculating an uncertainty value of a user to be trained through gaze data in eye movement data, wherein the specific calculation method is as follows:
；
Wherein, Shannon entropy representing gaze point,Represents gaze point range probability,Represents the gaze point,Represents Net,Representing the value quantity of the gaze point;
The method for calculating the stereo vision sharpness comprises the following steps:
；
Wherein, Representing stereo acuity,AndRepresenting two object positionsAndAn angle formed with the eye.
9. The depth perception training method based on the multi-depth cue scene as claimed in claim 8, wherein the adjusting the visual training scene according to the analysis report in the step 5, the training is continued, specifically comprises:
If the analysis report satisfies the following conditions: the training times of the user to be trained in the same scene meet the preset training times, the accuracy of each training, the user completion time and the entropy value all meet the preset requirements, and different parallaxes in the scene can be distinguished as ；
The scene automatically enters the next level after the last test is finished; after all the level tests in one scene are completed, the scene automatically jumps to the next depth scene for training.
10. The depth perception training method based on multi-depth cue scene as claimed in claim 9, wherein the parallax is as described in step5The calculation method is as follows:
；
Wherein, Representing the target position,AndRepresenting the position of the left and right eye, respectively,Representing the angle between the left eye and the target,Representing the angle between the right eye and the target,Representing the distance between the left eye and the target,Representing the distance between the right eye and the target.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410384501.8A CN117976129A (en) | 2024-04-01 | 2024-04-01 | A depth perception training method based on multi-depth cue scenes |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202410384501.8A CN117976129A (en) | 2024-04-01 | 2024-04-01 | A depth perception training method based on multi-depth cue scenes |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN117976129A true CN117976129A (en) | 2024-05-03 |
Family
ID=90864984
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202410384501.8A Pending CN117976129A (en) | 2024-04-01 | 2024-04-01 | A depth perception training method based on multi-depth cue scenes |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN117976129A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119339876A (en) * | 2024-09-18 | 2025-01-21 | 深圳市火眼睛晶医疗科技有限公司 | Image-based myopia prevention and control and visual perception training method and system |
Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103239347A (en) * | 2013-05-09 | 2013-08-14 | 北京大学 | System for treating visual dysfunction by adopting ocular dominance regulation |
| CN104853185A (en) * | 2015-06-06 | 2015-08-19 | 吉林大学 | Stereo video comfort evaluation method combining multiple parallaxes with motion |
| CN110180067A (en) * | 2019-05-09 | 2019-08-30 | 北京津发科技股份有限公司 | Cognitive ability training and test method, terminal device and storage medium |
| US20190281211A1 (en) * | 2018-03-06 | 2019-09-12 | Eyedaptic, Inc. | Adaptive System for Autonomous Machine Learning and Control in Wearable Augmented Reality and Virtual Reality Visual Aids |
| CN111202663A (en) * | 2019-12-31 | 2020-05-29 | 浙江工业大学 | Vision training learning system based on VR technique |
| CN111613295A (en) * | 2020-04-29 | 2020-09-01 | 杭州心景科技有限公司 | Memory intervention and evaluation method and device based on virtual imaging and terminal equipment |
| CN112102186A (en) * | 2020-09-07 | 2020-12-18 | 河海大学 | Real-time enhancement method for underwater video image |
| KR20210019883A (en) * | 2019-08-13 | 2021-02-23 | 박시현 | Device, method and program for training ocular motor ability using image |
| CN113101159A (en) * | 2021-04-08 | 2021-07-13 | 杭州深睿博联科技有限公司 | A VR-based stereo vision training and evaluation method and device |
| CN113260300A (en) * | 2018-11-07 | 2021-08-13 | 斯达克实验室公司 | Fixed point gaze motion training system employing visual feedback and related methods |
| CN113797070A (en) * | 2021-10-27 | 2021-12-17 | 中国人民解放军陆军军医大学第一附属医院 | A visual training method and system for treating amblyopia |
| CN115531150A (en) * | 2022-10-27 | 2022-12-30 | 河海大学常州校区 | Stereoscopic vision testing and training method based on virtual reality |
| CN116019693A (en) * | 2023-03-30 | 2023-04-28 | 广州视景医疗软件有限公司 | VR-based stereoscopic vision training method and device |
| CN116942074A (en) * | 2023-09-19 | 2023-10-27 | 广东视明科技发展有限公司 | Visual function evaluation training method based on multi-screen depth multi-focus stack mode |
| CN117618723A (en) * | 2023-12-05 | 2024-03-01 | 波克医疗科技(上海)有限公司 | Be used for improving children's attention training system |
-
2024
- 2024-04-01 CN CN202410384501.8A patent/CN117976129A/en active Pending
Patent Citations (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103239347A (en) * | 2013-05-09 | 2013-08-14 | 北京大学 | System for treating visual dysfunction by adopting ocular dominance regulation |
| CN104853185A (en) * | 2015-06-06 | 2015-08-19 | 吉林大学 | Stereo video comfort evaluation method combining multiple parallaxes with motion |
| US20190281211A1 (en) * | 2018-03-06 | 2019-09-12 | Eyedaptic, Inc. | Adaptive System for Autonomous Machine Learning and Control in Wearable Augmented Reality and Virtual Reality Visual Aids |
| CN113260300A (en) * | 2018-11-07 | 2021-08-13 | 斯达克实验室公司 | Fixed point gaze motion training system employing visual feedback and related methods |
| CN110180067A (en) * | 2019-05-09 | 2019-08-30 | 北京津发科技股份有限公司 | Cognitive ability training and test method, terminal device and storage medium |
| KR20210019883A (en) * | 2019-08-13 | 2021-02-23 | 박시현 | Device, method and program for training ocular motor ability using image |
| CN111202663A (en) * | 2019-12-31 | 2020-05-29 | 浙江工业大学 | Vision training learning system based on VR technique |
| CN111613295A (en) * | 2020-04-29 | 2020-09-01 | 杭州心景科技有限公司 | Memory intervention and evaluation method and device based on virtual imaging and terminal equipment |
| CN112102186A (en) * | 2020-09-07 | 2020-12-18 | 河海大学 | Real-time enhancement method for underwater video image |
| CN113101159A (en) * | 2021-04-08 | 2021-07-13 | 杭州深睿博联科技有限公司 | A VR-based stereo vision training and evaluation method and device |
| CN113797070A (en) * | 2021-10-27 | 2021-12-17 | 中国人民解放军陆军军医大学第一附属医院 | A visual training method and system for treating amblyopia |
| CN115531150A (en) * | 2022-10-27 | 2022-12-30 | 河海大学常州校区 | Stereoscopic vision testing and training method based on virtual reality |
| CN116019693A (en) * | 2023-03-30 | 2023-04-28 | 广州视景医疗软件有限公司 | VR-based stereoscopic vision training method and device |
| CN116942074A (en) * | 2023-09-19 | 2023-10-27 | 广东视明科技发展有限公司 | Visual function evaluation training method based on multi-screen depth multi-focus stack mode |
| CN117618723A (en) * | 2023-12-05 | 2024-03-01 | 波克医疗科技(上海)有限公司 | Be used for improving children's attention training system |
Non-Patent Citations (6)
| Title |
|---|
| JIAMIN PING SID STUDENT MEMBER;等: "Depth perception in shuffleboard: Depth cues effect on depth perception in virtual and augmented reality system", REGULAR CONTRIBUTED PAPER, vol. 28, 31 December 2019 (2019-12-31), pages 164 - 176 * |
| TINGTING ZHANG;等: "Eye movements during change detection: the role of depth of field", COGNITIVE COMPUTATION AND SYSTEMS, vol. 01, no. 02, 31 December 2019 (2019-12-31), pages 55 - 59 * |
| 卢炜;: "斜视弱视患者的双眼视觉状况及其矫治", 眼科, vol. 18, no. 05, 25 September 2009 (2009-09-25), pages 289 - 292 * |
| 周玲;等: "虚拟现实中关于视觉深度感知的讨论", 电子器件, vol. 23, no. 02, 30 June 2000 (2000-06-30), pages 34 - 40 * |
| 彭程: "基于信息论的自顶向下因素与眼动模式相关性分析", 中国优秀硕士学位论文全文数据库 信息科技辑, 15 January 2022 (2022-01-15), pages 10 * |
| 彭程: "基于信息论的自顶向下因素与眼动模式相关性分析", 中国优秀硕士学位论文全文数据库 信息科技辑, 15 January 2022 (2022-01-15), pages 138 - 1724 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN119339876A (en) * | 2024-09-18 | 2025-01-21 | 深圳市火眼睛晶医疗科技有限公司 | Image-based myopia prevention and control and visual perception training method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US11954243B2 (en) | Mapping and localization of a passable world | |
| JP5475893B2 (en) | Apparatus and method for measuring visual fatigue level | |
| US20220219076A1 (en) | Game processing program, game processing method, and game processing device | |
| CN109196447B (en) | Interaction with 3D virtual objects using gestures and multi-DOF controllers | |
| KR101433786B1 (en) | Image display apparatus, game program, and method of controlling game | |
| US9380287B2 (en) | Head mounted system and method to compute and render a stream of digital images using a head mounted display | |
| US10846927B2 (en) | Method and apparatus for displaying a bullet-style comment in a virtual reality system | |
| CN102981616B (en) | The recognition methods of object and system and computer in augmented reality | |
| EP4123425A1 (en) | Eye tracking calibration techniques | |
| US20110018867A1 (en) | Video game machine, gaming image display method, gaming image dispaly program and network game system | |
| Looser et al. | An evaluation of virtual lenses for object selection in augmented reality | |
| Sundstedt et al. | A psychophysical study of fixation behavior in a computer game | |
| Periverzov et al. | IDS: The intent driven selection method for natural user interfaces | |
| Binaee et al. | Movements of the eyes and hands are coordinated by a common predictive strategy | |
| CN107422844A (en) | A kind of information processing method and electronic equipment | |
| CN117976129A (en) | A depth perception training method based on multi-depth cue scenes | |
| EP4140555A1 (en) | Aiming display automation for head mounted display applications | |
| US11977679B1 (en) | VR rehabilitation training method and system with quantitative evaluation function | |
| Heydn et al. | The golden bullet: A comparative study for target acquisition, pointing and shooting | |
| CN114967128B (en) | Sight tracking system and method applied to VR glasses | |
| Wang et al. | Evaluating the effects of real world distraction on user performance in virtual environments | |
| CN116807388B (en) | Dominant eye detection method, device, terminal equipment and medium based on eye movement tracking | |
| Alonso et al. | Virtual ball catching performance in different camera views | |
| Diller et al. | Automatic Viewpoint Selection for Interactive Motor Feedback Using Principle Component Analysis. | |
| JP2025539143A (en) | Method for triggering actions in a metaverse or virtual world |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20240503 |
|
| RJ01 | Rejection of invention patent application after publication |