JP7220043B2 - training system and program - Google Patents

Description

The present invention relates to a technique for training in virtual space.

Techniques for training using VR (virtual reality) are known. For example, Patent Literature 1 describes that, in a bicycle driving simulator, the time during which the subject checks the rear from the start of driving the simulated bicycle to the end thereof is calculated as evaluation data.

JP 2017-9717 A

In some cases, VR is used for training to check the state of objects (equipment, suspicious objects, etc.) included in the virtual space. However, in such training, the object whose condition is checked is not necessarily in a particular direction from the trainee's point of view. For example, when the trainee moves in the virtual space, the direction of the target as seen from the trainee changes as the trainee moves. In the technique described in Patent Document 1, the time at which a specific direction is confirmed is calculated as evaluation data. It is not possible to recognize whether or not the status has been confirmed. Therefore, the behavior of such a trainee cannot be evaluated.
SUMMARY OF THE INVENTION An object of the present invention is to evaluate the behavior of a trainee who confirms the condition of an object in training performed in a virtual space.

The present invention is a training system for evaluating a trainee's behavior in a virtual space, comprising a storage unit that stores a visual field image from the position of the trainee in the virtual space, and a visual field direction of the trainee in the real space. a detecting unit that detects; a specifying unit that specifies a field of view of the trainee in the virtual space based on the position of the trainee in the virtual space and the direction of the field of view; an index value for evaluating the behavior of the trainee who confirms the state of the target based on the positional relationship between the display unit that displays the portion and the position of the line of sight included in the visual field and the position of the target included in the virtual space. A training system training system comprising a calculation unit for calculating

The calculation unit may increase the index value when the line-of-sight position and the target position overlap.

The position of the trainee moves in the virtual space, and the calculation unit calculates the position of the target from the position of the trainee in the virtual space at that time when the line-of-sight position and the position of the target overlap. The index value may be increased when the distance to the position is within a predetermined distance.

The predetermined distance may differ depending on the type of the target.

When the line-of-sight position and the position of the target overlap and the time period during which the line-of-sight position and the position of the target continue to overlap is within a predetermined range, the calculation unit calculates the index You can increase the value.

The predetermined range may differ depending on the type of the target.

The virtual space includes a plurality of objects, and the calculation unit calculates the order in which the priority determined for each of the plurality of objects and the order in which the line-of-sight position and the position of each of the plurality of objects overlap. Based on this, the amount of increase in the index value may be weighted.

The position of the trainee moves in the virtual space, and the calculation unit moves around the trainee based on the amount of change in the line-of-sight direction included in the visual field from the movement direction of the position of the trainee. Other index values for evaluating behavior to be confirmed may be calculated.

The training system provides a model position of a line-of-sight position and a position of the object when confirming the condition of the object, which are generated based on an action of confirming the condition of the object performed by an expert in the virtual space. A storage unit that stores data indicating a relationship may be further provided, and the calculation unit may calculate the index value based on a degree of similarity between the positional relationship and the model positional relationship.

Further, the present invention provides a computer included in a training system for evaluating the behavior of a trainee in a virtual space, which includes a step of detecting the line-of-sight direction of the trainee in the real space, the position of the trainee in the virtual space and the specifying the visual field of the trainee in the virtual space based on the direction of the visual field; Evaluating the behavior of the trainee confirming the state of the target based on the step of displaying a portion included in the field of view, and the positional relationship between the line-of-sight position included in the field of view and the position of the target included in the virtual space. and a step of calculating an index value to be used.

Advantageous Effects of Invention According to the present invention, it is possible to evaluate the behavior of a trainee who confirms the condition of a target during training performed in virtual space.

It is a figure showing an example of composition of training system 1 concerning an embodiment. 2 is a diagram showing an example of the hardware configuration of the display device 10; FIG. 3 is a diagram showing an example of a functional configuration of the display device 10; FIG. It is a top view which shows an example of virtual space. 6 is a flowchart showing an example of display processing; 7 is a flowchart showing an example of evaluation processing for confirming the surroundings; 10 is a flowchart illustrating an example of evaluation processing for object confirmation; 5 is a diagram showing an example of a moving image 50 displayed on the display unit 16. FIG. FIG. 10 is a diagram showing an example of a moving image 50 displayed on the display unit 16 according to a modified example; It is a figure which shows an example of the correspondence table 131 which concerns on a modification. It is a figure which shows an example of the correspondence table 132 which concerns on a modification.

1. Configuration FIG. 1 is a diagram showing an example of the configuration of a training system 1 according to this embodiment. The training system 1 uses VR to assume the presence of objects (suspicious objects, injured persons, etc.) to which attention should be directed at work sites such as security, disaster response, rescue, etc., and pay sufficient attention to the conditions of the objects. It is used to conduct various training related to work that requires A training system 1 evaluates a trainee's behavior in a virtual space. The term “training” refers to educational activities aimed at reaching certain goals. Training does not necessarily require actual action. Training also includes training and learning. The training system 1 includes multiple display devices 10 and a management device 20 . The multiple display devices 10 and the management device 20 are connected via a communication line 30 . Communication line 30 carries communications between these devices. A private network such as a LAN (Local Area Network) may be used for the communication line 30, for example.

FIG. 2 is a diagram showing an example of the hardware configuration of the display device 10. As shown in FIG. The display device 10 provides a virtual space by displaying an image simulating the field of view of the trainee, and records and evaluates the trainee's behavior while viewing the content (hereinafter, simply referred to as behavior). . Note that this content includes, for example, images and sounds. The display device 10 is worn on the trainee's head. For example, a head-mounted wearable terminal may be used as the display device 10 . The display device 10 includes a processor 11 , a memory 12 , a storage 13 , a communication section 14 , an operation section 15 , a display section 16 , a sound output section 17 and a sensor section 18 . These configurations are connected via a bus 19 .

The processor 11 executes various processes by reading programs to the memory 12 and executing them. A CPU (Central Processing Unit) may be used for the processor 11, for example. The memory 12 stores programs executed by the processor 11 . For the memory 12, for example, a ROM (Read Only Memory) or a RAM (Random Access Memory) may be used. The storage 13 stores various data and programs. A flash memory, for example, may be used for the storage 13 . The communication unit 14 is connected to the communication line 30 and performs data communication via the communication line 30 . The operation unit 15 is used for inputting various kinds of information. For example, an operation key may be used for the operation unit 15 . The display unit 16 displays various information. A liquid crystal display, for example, may be used for the display unit 16 . The sound output unit 17 converts the audio signal into sound and outputs the sound. For example, a speaker may be used for the sound output unit 17 .

The sensor unit 18 detects movement of the trainee's head in a coordinate system in real space. This movement includes head orientation. In this embodiment, the orientation of the trainee's head in the real space is detected as the trainee's visual field direction in the real space. The sensor unit 18 may include, for example, a gyroscope, an acceleration sensor, and a geomagnetic sensor. The gyroscope measures the angle or angular velocity of the trainee's head. The acceleration sensor measures the acceleration of the trainee's head. A geomagnetic sensor measures the orientation of the trainee's head.

The display unit 16 displays an image of the virtual space corresponding to the trainee's field of view specified based on the orientation of the head detected by the sensor unit 18 . That is, the trainee can visually recognize the entire virtual space by turning the head left and right in the real space.

In addition, the storage 13 stores in advance moving image data indicating moving images captured in advance using an imaging device (not shown) as information associated with the coordinate system in the virtual space. In this embodiment, the position of the trainee in the virtual space is predetermined for each time, and changes with the passage of time. It simulates the view from the trainee at each position, that is, the animation of the range visible from each position of the trainee in the virtual space. This moving image is composed of a plurality of panoramic images called frames. A panoramic image has a wider angle of view than the human field of view. The angle of view of the panoramic image may be 360 degrees, for example. A moving image may be captured using, for example, a single video camera equipped with a wide-angle lens, or may be generated by combining moving images captured using a plurality of video cameras that capture images in different directions. Also, by using an omnidirectional camera or the like to create a moving image from 360-degree panorama images in all directions, the training may be performed in a situation closer to the real space.

In addition, by setting the direction of 0 degrees as the specific direction in the panoramic image forming the virtual space, the range of -180 degrees to 180 degrees in the panoramic image is defined. In the present embodiment, a moving direction D1 at the trainee's position P0, which will be described later, is used as the specific direction.

Note that the display device 10 may be configured by a housing mounted on the trainee's head and a smartphone attached to the housing. In this case, the smartphone includes the above-described processor 11 , memory 12 , storage 13 , communication section 14 , operation section 15 , display section 16 , sound output section 17 , and sensor section 18 . The smartphone is attached to the housing in such a posture that the trainee can see the display unit 16 when the housing is worn on the trainee's head.

Returning to FIG. 1 , the management device 20 collects and outputs the trainee's evaluation results from each display device 10 . A general-purpose computer, for example, may be used as the management device 20 . The management device 20 may have the same configuration as the processor 11, the memory 12, the storage 13, the communication section 14, the operation section 15, and the display section 16 described above.

FIG. 3 is a diagram showing an example of the functional configuration of the display device 10. As shown in FIG. The display device 10 functions as an identification unit 101 , a measurement unit 102 , a calculation unit 103 and an output unit 104 . These functions are realized by cooperation between the program stored in the memory 12 and the processor 11 executing this program, whereby the processor 11 performs calculations or controls communication by the communication unit 14 .

The identifying unit 101 identifies the trainee's visual field in the virtual space based on the position of the trainee in the virtual space and the visual field direction of the trainee in the real space detected by the sensor unit 18 . An example of a method for specifying the field of view will be described below. Assuming that the orientation of the trainee's head in real space when the video starts playing is the specific direction in the panoramic image, the orientation of the trainee's head (coordinate system in real space) and the direction in the panoramic image (in virtual space coordinate system). Then, the range between one end forming a predetermined angle (for example, 80 degrees) clockwise from a specific direction in the virtual space and the other end forming a predetermined angle (for example, 80 degrees) counterclockwise is the field of view of the trainee in the virtual space. Identify as That is, the field of view when the reproduction of the moving image is started is in the range of -80 degrees to 80 degrees, and a part of the panorama image (-80 degrees to 80 degrees) corresponding to this field of view is displayed on the display unit 16 . After that, the current field of view of the trainee in the virtual space is identified according to the change in the orientation of the trainee's head detected by the sensor unit 18 . For example, when the range of -80 degrees to 80 degrees is specified as the field of view, when the sensor unit 18 detects that the orientation of the trainee's head has changed counterclockwise by 20 degrees, the specifying unit Based on this detection result, 101 identifies a range of −60 degrees to 100 degrees obtained by changing the entire field of view by 20 degrees in the counterclockwise direction as the trainee's current field of view. For the size of the field of view, for example, the size of the field of view of a general person may be used.

The measurement unit 102 measures the distance from the current position of the trainee in the virtual space to the position of the object included in the virtual space. A target is a target whose state is to be confirmed. A target may be an object or an event. For example, when security patrol training is conducted in a virtual space, the target may be facility equipment or articles to be monitored in patrol security, equipment used in an emergency, or an abnormal object or event. For distance measurement, for example, coordinates indicating the position of the trainee and coordinates indicating the position of the target in the virtual space may be used. Further, when the position of the trainee and the position of the target in the virtual space are set in advance as in this embodiment, the distance from the current position of the trainee in the virtual space to the position of the target included in the virtual space is stored in advance. You may make it

The calculation unit 103 calculates an attention index value for evaluating the behavior of the trainee who confirms the state of the target based on the positional relationship between the line-of-sight position included in the visual field specified by the specifying unit 101 and the position of the target included in the virtual space. Calculate The line-of-sight position means a place where the line of sight is poured. In this embodiment, the center of the trainee's visual field is used as the line-of-sight position. As will be described later, eye tracking or the like may be used to more accurately identify the line-of-sight position of the trainee. This attention index value is, for example, a value indicating the proficiency level of the behavior of the trainee who confirms the state of the object. For example, a number may be used as the attention index value. In this case, a larger attention index value may indicate a higher evaluation. However, the attention index value is not necessarily limited to numbers. For example, the attention index value may be represented by characters or symbols.

The attention index value may increase, for example, when the line-of-sight position of the trainee and the position of the target overlap. Note that this “overlapping” state may include not only a completely matching state but also a partially overlapping state. Also, the attention index value may be increased when the line-of-sight position of the trainee overlaps with the target position and the distance measured by the measurement unit 102 is within a predetermined distance. That is, even if the line-of-sight position of the trainee and the target position overlap, the attention index value does not need to be increased when the distance measured by the measurement unit 102 is outside the predetermined distance. For this predetermined distance, for example, a distance that allows the trainee to visually recognize the condition of the object may be used. For example, if the state of the object cannot be visually recognized unless the object is approached, the range of distances that can be regarded as approaching the object may be used as the predetermined distance. Furthermore, the attention index value may be increased when the line-of-sight position of the trainee overlaps with the position of the target, and the duration of the overlapping state of these positions is within a predetermined range. . That is, even when the position of the line of sight of the trainee and the position of the object overlap, the attention index value does not increase if the duration of the overlapping state of these positions is outside the predetermined range. good too. The duration of the overlapping state may be measured using a timer, for example. For this predetermined range, for example, a range of time required for the trainee to visually recognize the target state may be used. For example, if the state of the target cannot be visually recognized unless the target is viewed continuously for a certain amount of time, that time may be used as the predetermined range.

In addition to the attention index value, the calculation unit 103 also calculates a vigilance index value for evaluating the behavior of checking the surroundings of the trainee based on the amount of change in the line-of-sight direction from the movement direction of the position of the trainee. good too. This warning index value is, for example, a value indicating the proficiency level of the trainee's action to check the surroundings. For example, a number may be used as the warning index value. In this case, a higher alert index value may indicate a higher evaluation. However, similarly to the attention index value described above, the warning index value is not necessarily limited to numbers. The movement direction of the position of the trainee means the direction in which the position of the trainee moves in the virtual space. That is, the movement direction is the trainee's traveling direction. The movement direction may be a direction from the trainee's current position to the next position. In the following description, "movement of the position of the trainee" may be simply referred to as "movement of the trainee". The line-of-sight direction of the trainee is, for example, the direction from the trainee's position in the virtual space toward the center of the trainee's visual field. For example, the angle formed by the movement direction and the line-of-sight direction may be used as the amount of change in the line-of-sight direction from the movement direction of the trainee. The caution index value may increase, for example, as the amount of change increases.

The output unit 104 outputs the attention index value calculated by the calculation unit 103 . This attention index value may be output by transmitting it to the management device 20, for example. The timing of outputting the attention index value may be, for example, after the moving image ends. Also, a warning index value may be output together with the attention index value.

2. Operation FIG. 4 is a plan view showing an example of the virtual space. In this virtual space, work training for patrolling the patrol area 40 is carried out. The patrol area 40 includes an ashtray 41 , a fire hydrant lamp 42 , an emergency exit 43 , an AED 44 and a suspicious object 45 . All of these are targets whose states should be checked during patrol. For example, the ashtray 41 should be checked for extinguished cigarettes. The hydrant lamp 42 should be checked to see if it is illuminated. The emergency exit 43 should be checked for obstructions in front of it and for guidance lights. The AED 44 should be checked for removal and battery lamp illumination. Suspicious object 45 should be checked for its existence. A patrol route R is set in advance in the patrol area 40 . In the virtual space, the trainee travels along the patrol route R and performs patrol work. The position of the trainee is set in advance for each time, and moves along the patrol route R as time elapses.

2.1. Display Processing FIG. 5 is a flowchart showing an example of display processing executed by the display device 10 . This display processing is started, for example, when a signal instructing the start of display is input to the display device 10 . This signal may be input by an operation using the operation unit 15, or may be transmitted from the management device 20, for example.

In step S<b>11 , reproduction of moving images is started on the display unit 16 based on the moving image data stored in the storage 13 . Specifically, the angle-of-view portions corresponding to the range of the trainee's visual field are extracted from each panorama image included in the moving image indicated by the moving image data, and displayed on the display unit 16 in order. This moving image simulates the field of view of the trainee who moves on the patrol route R. Therefore, the position of the viewpoint of the camera automatically moves along the patrol route R in the moving image.

In the example shown in FIG. 4, in the initial state, for example, a portion included in the field of view V1 when the trainee faces the moving direction D1 at the position P0 is extracted and displayed from the panorama image included in the moving image. Here, 80 degrees to the left and right are used as the angle indicating the size of the field of view. In this case, the visual field V1 has one end forming an angle of 80 degrees clockwise in FIG. 4 and the other end forming an angle of 80 degrees counterclockwise in FIG. range between In FIG. 4, the distance in the depth direction of the field of view V1 is indicated by the radius of the arc, but in reality this distance may be infinite. When the position of the trainee moves, the portion included in the field of view V1 in the panorama image changes. Along with this, the display on the display unit 16 is changed.

In step S12, it is determined whether or not the sensor unit 18 has detected movement of the head as the action of the trainee during viewing of the content. If the trainee does not move his/her head, no movement of the trainee's head is detected, so the determination in step S12 is NO. In this case, the process of step S12 is repeated. On the other hand, as shown in FIG. 4, for example, when the trainee rotates the head counterclockwise in FIG. is detected. In this case, the determination in step S12 becomes YES, and the process proceeds to step S13.

In step S13, the identification unit 101 identifies the visual field of the trainee based on the movement of the head detected in step S12. As shown in FIG. 4, when the trainee rotates the head counterclockwise in FIG. 4 by α degrees, the visual field V1 is rotated counterclockwise by α degrees around the reference position P1. Then, the field of view V2 after rotational movement is identified.

In step S14, the direction of the camera viewpoint of the moving image is changed according to the field of view specified in step S13. As shown in FIG. 4, when the trainee rotates the head counterclockwise by α degrees in FIG. 4, the direction of the video camera viewpoint changes from the direction corresponding to the visual field V1 to the direction corresponding to the visual field V2. be done. When the direction of the camera viewpoint is changed in this way, a portion corresponding to the field of view V2 after rotational movement is extracted from the panorama image included in the moving image and displayed.

In step S15, it is determined whether or not the moving image has ended. If the moving image has not been reproduced to the end, the determination in step S15 is NO, and the process returns to step S12. On the other hand, for example, when the moving image has been reproduced to the end, the determination in step S15 becomes YES, and the process ends.

2.2. Evaluation Processing of Surrounding Confirmation FIG. 6 is a flowchart showing an example of evaluation processing of surrounding confirmation executed by the display device 10 . This evaluation process is started, for example, when a moving image is reproduced.

In step S21, it is determined whether or not the trainee is moving within the virtual space. For example, if the position of the trainee has not changed in the virtual space, the determination in step S21 is NO. In this case, the process of step S21 is repeated. On the other hand, if the position of the trainee has changed in the moving image, the determination in step S21 is YES. In this case, the process proceeds to step S22.

In step S22, a caution index value is calculated based on the amount of change in the line-of-sight direction D2 from the movement direction D1 of the trainee. In the example shown in FIG. 4, the movement direction D1 is the direction from the trainee's current position to the next position on the patrol route R, for example. The line-of-sight direction D2 is, for example, the direction from the trainee's current position toward the center of the visual field. The angle α formed by the movement direction D1 and the line-of-sight direction D2 is used as the amount of change in the line-of-sight direction D2 from the movement direction D1. Then, the first threshold, which is the criterion for determining whether or not the surroundings have been confirmed, and the second threshold, which is the criterion for determining whether the surroundings have been extensively confirmed, and the relationship between the angle α and the warning index Calculate the value. Here, it is assumed that the first threshold is 30 degrees and the second threshold is 45 degrees. If the angle α is greater than the first threshold of 30 degrees and less than the second threshold of 45 degrees, one point is added to the warning index value assuming that the surroundings are being checked. On the other hand, when the angle α is equal to or greater than the second threshold value of 45 degrees, 2 points are added to the caution index value assuming that the surroundings are widely checked. When the angle α is 30 degrees or less, the warning index value is not added because the surroundings are not being checked. Thus, the caution index value increases as the amount of change in the line-of-sight direction D2 from the movement direction D1 of the trainee increases.

At step S<b>23 , the warning index value calculated at step S<b>22 is stored in the storage 13 .

In step S24, similarly to step S15 described above, it is determined whether or not the moving image has ended. If the moving image has not been reproduced to the end, the determination in step S24 is NO, and the process returns to step S21. In this way, the attention index value stored in the storage 13 is added every time the trainee checks the surroundings while moving while the moving image is being played. On the other hand, for example, when the moving image has been reproduced to the end, the determination in step S24 becomes YES, and the process proceeds to step S25.

In step S25, the warning index value stored in the storage 13 is output by the output unit 104. FIG. For example, the alert index value is transmitted from the output unit 104 to the management device 20 together with the ID (identification) of the trainee.

2.3. Evaluation Processing for Object Confirmation FIG. 7 is a flowchart showing an example of evaluation processing for object confirmation executed by the display device 10 . This evaluation process is started, for example, when a moving image is reproduced.

In step S31, it is determined whether or not the attention target exists within a certain range from the current position of the trainee. This certain range is, for example, a range of distances in which the trainee can see the object of interest by looking around. In the example shown in FIG. 4, when none of the ashtray 41, the fire hydrant lamp 42, the emergency exit 43, the AED 44, and the suspicious object 45 exists within a certain range from the trainee's current position, the determination in step S31 is NO. Become. In this case, the process of step S31 is repeated. On the other hand, for example, when the trainee's current position is position P1, AED 44 exists within a certain range from position P1. In this case, the determination in step S31 is YES, and the process proceeds to step S32.

In step S32, the position of the attention target within a certain range from the trainee's current position is set as the attention position. In the example shown in FIG. 4, for example, when the trainee's current position is position P1, the position of AED 44 is set as the target position.

In step S33, the measurement unit 102 measures the distance from the trainee's current position to the target position. In the example shown in FIG. 4, for example, if the trainee's current position is position P1, the distance L1 from this position P1 to the position of the AED 44 is measured. This distance is measured each time the trainee's position changes. For example, when the trainee moves to position P2, the distance L2 from this position P2 to the position of AED 44 is measured.

In step S34, based on the distance measured in step S33, the presence or absence of overlap between the line-of-sight position of the trainee and the position of attention, and the time during which the line-of-sight position of the trainee and the position of attention continue to overlap. Then, the attention index value is calculated. Here, it is assumed that the predetermined distance is within 2 m and the predetermined range is 1 to 5 seconds.

FIG. 8 is a diagram showing an example of a moving image 50 displayed on the display unit 16. As shown in FIG. As mentioned above, the animation 50 represents the portion included in the trainee's field of view. In the example shown in FIG. 8, the line-of-sight position E1 is the center of the trainee's visual field. As described in the above display processing, when the trainee changes the orientation of the head, the part included in the trainee's visual field in the virtual space changes, and the direction of the camera viewpoint of the moving image 50 also changes.

FIG. 8(a) is a diagram showing an example of a moving image 50 displayed on the display unit 16 when the trainee moves to the position P1. In the example shown in FIG. 8A, the trainee's field of view includes the AED 44, which is the target of attention. However, the line-of-sight position E1 of the trainee and the area of the AED 44 do not overlap. In this case, the attention index value is not added.

FIG. 8(b) is a diagram showing another example of the moving image 50 displayed on the display unit 16 when the trainee moves to the position P1. In the example shown in FIG. 8(b), the trainee's visual field includes the AED 44, which is the target of attention. Further, the line-of-sight position E1 of the trainee and the area of the AED 44 overlap. However, the distance L1 from the trainee's position P1 to the AED 44 shown in FIG. 4 is 4 m, which is greater than the predetermined distance of 2 m. In this case, the attention index value is not added.

FIG. 8(c) is a diagram showing an example of the moving image 50 displayed on the display unit 16 when the trainee moves to the position P2. In the example shown in FIG. 8(c), the trainee's visual field includes the AED 44, which is the target of attention. Also, the line-of-sight position E1 and the area of the AED 44 overlap. Furthermore, the distance L2 from the trainee's position P2 to the AED 44 shown in FIG. 4 is 1 m, which is within 2 m of the predetermined distance. The time during which the line-of-sight position E1 and the area of the AED 44 continue to overlap is 3 seconds, which is included in the predetermined range of 1 to 5 seconds. In this case, 2 points are added to the attention index value.

At step S<b>35 , the attention index value calculated at step S<b>34 is stored in the storage 13 .

In step S36, similarly to step S15 described above, it is determined whether or not the moving image has ended. If the moving image has not been played to the end, the determination in step S36 is NO, and the process returns to step S31. In this way, the attention index value stored in the storage 13 is incremented each time the trainee performs an action to check the state of each attention target while the moving image is being played. Then, when the moving image has been reproduced to the end, the determination in step S36 becomes YES, and the process proceeds to step S37.

In step S<b>37 , the output unit 104 outputs the attention index value stored in the storage 13 . For example, the attention index value may be transmitted from the output unit 104 to the management device 20 together with the trainee's ID and the warning index value described above. When receiving these IDs, attention index values, and caution index values, the management device 20 displays the caution index values and attention index values for each trainee, for example. At this time, the sum of the attention index value and the caution index value may be displayed together with the caution index value and the attention index value.

In the above-described embodiment, the "sensor unit 18", the "storage unit 13", the "moving image data", the "attention index value", and the "warning index value" correspond to the "detection unit" and the "storage unit" according to the present invention, respectively. ”, “visual field image”, “index value”, and “other index value”.

According to the above-described embodiment, the attention index value is calculated according to the behavior of the trainee who confirms the state of the target. can do. In addition, since the attention index value is output, for example, the instructor of the trainee can objectively grasp the proficiency level of the trainee's behavior for confirming the target state. Further, when the line-of-sight position of the trainee overlaps the position of the target of interest, and the distance from the position of the trainee to the position of the target of interest is within a predetermined distance, the attention index value is added. A trainee who confirms the condition from an appropriate distance can be highly evaluated. Furthermore, if the trainee's line-of-sight position and the position of the target of attention overlap and the time for which the trainee's position and the position of the target of attention continue to overlap is within a predetermined range, the attention index value is added. Therefore, it is possible to highly evaluate the trainee who has spent an appropriate amount of time confirming the state of the object of interest. Furthermore, since the warning index value is calculated in addition to the attention index value, it is possible to evaluate not only the behavior of the trainee who checks the state of the target, but also the behavior of the trainee who checks the surroundings while moving in the virtual space. can.

3. Modifications The present invention is not limited to the embodiments described above. Various modifications may be made to the embodiments described above. Moreover, the following modifications may be combined and implemented.

In the embodiments described above, a priority may be set for each object included in the virtual space. Here, in the example shown in FIG. 4, it is assumed that the AED 44 is given a "low" priority and the suspicious object 45 is given a "high" priority. Note that this priority may be fixed or may change depending on the situation of the object. For example, even if "low" is set as the priority of the ashtray 41, the priority may become "high" when the ashtray 41 emits smoke. The calculation unit 103 calculates the amount of increase in the attention index value based on the priority determined for each of the plurality of attention targets and the order in which the line-of-sight position of the trainee overlaps with each of the plurality of attention targets. may be weighted. As a method of weighting the amount of increase in the index value of interest, for example, a value that is normally added to the index value of interest may be weighted using a weighting factor (>1). As a result, the increment of the index value becomes larger than in the normal case.

FIG. 9 is a diagram showing an example of a moving image 50 displayed on the display unit 16 when the trainee moves to the position P3 shown in FIG. 4 in this modified example. When the trainee faces the movement direction at position P3, both AED 44 and suspicious object 45 appear in moving image 50 . In the example shown in FIG. 9, when the trainee turns his or her head, the area of the AED 44 with "low" priority overlaps with the area of the suspicious object 45 with "high" priority and the line-of-sight position E1. and the line-of-sight position E1 overlap. That is, the line-of-sight position E1 and the regions of the plurality of attention targets overlap in order of priority of the attention targets. Also, the distance L3 from the trainee's position P3 to the AED 44 and the distance L4 from the trainee's position P3 to the suspicious object 45 shown in FIG. 4 are both within a predetermined distance. Furthermore, the time during which the line-of-sight position E1 and the area of the AED 44 continue to overlap and the time during which the line-of-sight position E1 and the area of the suspicious object 45 continue to overlap are both within a predetermined range. is. In this case, when the line-of-sight position E1 overlaps the area of the suspicious object 45 and the line-of-sight position E1 overlaps the area of the AED 44, the weighting factor is 2 points + 2 points = 4 points that are normally added to the attention index value. Weighting is performed using 1.5. As a result, 4 points×1.5=6 points are added to the target index value. According to this modification, if the states of a plurality of attention targets are checked in order according to the priority, the points added to the attention index value will increase. Trainees can be rated higher than other trainees.

In the above-described embodiments, the predetermined distance may differ according to the type of target of interest. This is because the distance at which the state of the target of interest can be visually recognized may differ depending on the type of the target of interest. For example, there may be objects of interest whose state can be visually recognized even from a distance. In this case, the storage 13 stores a correspondence table 131 as shown in FIG. The correspondence table 131 stores target types and predetermined distances in association with each other. For example, when the object of interest is the emergency exit 43, the predetermined distance "within 5 m" stored in association with the type of the emergency exit 43 in the correspondence table 131 is used. On the other hand, when the object of interest is the suspicious object 45, the predetermined distance “within 2 m” stored in association with the type of the suspicious object 45 in the correspondence table 131 is used. In this case, when the target of interest changes, a predetermined distance corresponding to the changed target of interest is selected, and the selected predetermined distance is used. That is, the predetermined distance is switched and used depending on the type of target of interest. According to this modification, it is possible to highly evaluate the trainee who has confirmed the state of the target from the distance corresponding to the type of the target.

In the above-described embodiments, the predetermined range may differ according to the type of target of interest. This is because the time required to visually recognize the state of the target of interest may differ depending on the type of the target of interest. In this case, the storage 13 stores a correspondence table 132 as shown in FIG. The correspondence table 132 stores target types and predetermined ranges in association with each other. For example, when the object of interest is the ashtray 41, the predetermined range "2 to 4 seconds" stored in association with the type of the ashtray 41 in the correspondence table 132 is used. On the other hand, when the object of interest is the suspicious object 45, the predetermined range “3 to 5 seconds” stored in association with the type of the suspicious object 45 in the correspondence table 132 is used. In this case, when the target of interest changes, a predetermined range corresponding to the changed target of interest is selected, and the selected predetermined range is used. That is, the predetermined range is switched according to the type of target of interest. Further, in this case, if the time period during which the line-of-sight position of the trainee and the position of the target of attention continue to overlap is longer than the predetermined range, a smaller value than when this time period is within the predetermined range is the attention point. May be added to the index value. For example, when the trainee's line-of-sight position E1 and the area of the ashtray 41 continue to overlap for 2 to 4 seconds, it is assumed that the status of the ashtray 41 has been properly checked. While 2 points are added to the attention index value, if this time is longer than 4 seconds, 1 point is added to the attention index value because the status of the ashtray 41 has been checked but it is taking time. good too. Also, if this time is shorter than 2 seconds, it is considered that the status of the ashtray 41 has not been sufficiently checked, and the attention index value does not need to be added.

In the above-described embodiment, the direction in which the target of interest is viewed when checking the state of the target of interest may be determined depending on the type of the target of interest. For example, if the ashtray 41 is viewed vertically, it is easier to check whether the cigarette is extinguished. In this case, the vertical direction may be determined as the direction for checking the state of the ashtray 41 . In this case, when the line-of-sight position of the trainee overlaps the area of the ashtray 41 and the line-of-sight direction of the trainee is along the vertical direction, the attention index value may be added. That is, even if the line-of-sight position of the trainee overlaps with the area of the ashtray 41, when the line-of-sight direction of the trainee is in a direction other than the vertical direction, the value is smaller than when the line-of-sight direction of the trainee is in the vertical direction. It may be added, or the attention index value may not be added. For this line-of-sight direction, for example, a direction from the position of the trainee's eyes toward the center of the visual field may be used.

In the above-described embodiment, when there is an obstacle between the position of the trainee and the position of the target of attention, the attention index value is It does not have to be added. The obstacle may be, for example, an object such as a pillar or a wall included in the virtual space, or a moving object that simulates a human being included in the virtual space. In addition, when the movement route of the trainee and the position of the object or the movement route of the moving object are predetermined, it is possible that there is an obstacle between the position of the trainee and the position of the target of attention depending on the playback time of the video. I understand. In this case, when there is an obstacle between the position of the trainee and the position of the target of attention, the attention index value is not added even when the line-of-sight position of the trainee overlaps the position of the target of attention. good too.

In the above-described embodiment, machine learning may be performed on the behavior of an expert who confirms the state of the target of interest, and the attention index value may be calculated using the result. In this case, a skilled person wears the display device 10 and performs an action to confirm the state of the attention target in the virtual space. The expert may be one person or may be multiple persons. Subsequently, machine learning is performed on the input data indicating the relationship between the line-of-sight position of the expert when checking the state of each target of interest and the position of the target of interest, thereby obtaining the position of the line-of-sight when checking the state of the target of interest. exemplar data indicating the positional relationship between the exemplar and the position of the target of interest is generated. This model data is stored in the storage 13 . When the trainee performs training in the virtual space, the degree of similarity between the positional relationship between the line-of-sight position of the trainee and the position of the attention target and the positional relationship of the model indicated by the model data stored in the storage 13 is calculated. This degree of similarity is a value indicating the degree of similarity. The attention index value may increase as the degree of similarity increases. In addition, in this modified example, the storage 13 is used as the "storage unit" according to the present invention. According to this modification, it is possible to evaluate a trainee who behaves like a skilled trainee in order to confirm the state of the target.

In the above-described embodiment, when the line-of-sight position of the trainee and the position of the target of interest overlap, the attention index value is added. If the target position does not overlap, the target index value may be subtracted. For example, when the line-of-sight position E1 and the area of the AED 44 do not overlap, 2 points may be subtracted from the attention index value.

In the above-described embodiments, the attention index value or caution index value may be weighted. For example, if the attention index value is to be emphasized in the evaluation, the attention index value may be weighted using a weighting factor (>1).

In the above-described embodiments, the condition regarding the distance from the position of the trainee to the position of the target of attention, or the condition regarding the time during which the line-of-sight position of the trainee and the position of the target of attention continue to overlap is not necessarily used. It doesn't have to be. For example, when the line-of-sight position of the trainee overlaps with the position of the target of attention, the attention index value may be added regardless of the distance and time. Further, when the line-of-sight position of the trainee and the position of the target of attention overlap and the distance from the position of the trainee to the position of the target of attention is within a predetermined distance, regardless of this time, the attention index value is may be added. Further, when the position of the line of sight of the trainee and the position of the target of attention overlap and the time during which the position of the line of sight of the trainee and the position of the target of attention continues to overlap is within a predetermined range, this The attention index value may be added regardless of the distance.

In the above-described embodiment, the alert index value is set to a plurality of may not be reflected. For example, in this case, the alert index value may not be added. In addition, when the amount of change in the line-of-sight direction from the moving direction of the trainee continues for a first predetermined time (for example, 10 seconds) or longer, the alert index value is may decrease. Furthermore, the warning index value may decrease if the state in which the amount of change in the line-of-sight direction from the movement direction of the trainee is equal to or less than the first threshold continues for a second predetermined time (eg, 30 seconds) or longer. good. Furthermore, when an event to be monitored occurs in the virtual space, the warning index value is the attention direction and the line-of-sight direction, which are the directions from the position of the trainee to another position where the event occurred when the event occurred. may be decreased if the difference is not less than a predetermined amount (eg, 15 degrees). Furthermore, when a specific work is performed in the virtual space, one time included in the period in which the amount of change in the line-of-sight direction from the moving direction of the trainee is greater than the first threshold, and the specific work is performed. The amount of increase in the alert index value may be weighted according to the relationship with other times. Also, the amount of change in the line-of-sight direction from the movement direction of the trainee is not limited to the angle formed by these directions, but is the length of the line connecting one point on the movement direction and another point on the line-of-sight direction. may be

In the above-described embodiments, the position of the target of interest refers to where the target of interest is. The position of the target of attention may be the entire area of the target of attention, a partial area of the target of attention, or a point included in the area of the target of attention. Also, the relationship between the line-of-sight position to which the attention index value is added and the position of the attention target is not limited to a state in which these positions overlap. For example, when the line-of-sight position moves within a certain range from the position of the target of interest, the attention index value may be added.

In the embodiments described above, the trainee's line-of-sight position is not limited to the center of the visual field. For example, the display device 10 may detect the line of sight of the trainee using a well-known eye tracking technique, and use the detected position ahead of the line of sight as the position of the line of sight. As a result, the trainee's behavior of paying attention to the target by changing the line of sight without changing the direction of the head is also evaluated, and more practical training can be performed.

In the embodiments described above, well-known motion capture techniques may be used to detect the trainee's movements. In this case, the trainee may move in the virtual space by performing a walking motion. That is, the trainee's position in the virtual space may change according to the trainee's walking motion. In another example, an operation using the operation unit 15 may change the position of the trainee in the virtual space.

In the embodiments described above, the display unit 16 may be a device that projects an image. For example, the display unit 16 may use a projection unit such as a projector and a display surface on which an image is displayed, or may use a projection unit that directly projects an image onto the retina. Moreover, the display device 10 is not limited to a head-mounted wearable terminal. For example, the display device 10 may be a glasses-type wearable terminal.

In the above-described embodiments, moving image data may be transmitted from the management device 20 to the display device 10 . In this case, for example, moving image data may be distributed from the management device 20 to a plurality of display devices 10 used in different locations via a wide area network such as the Internet.

In the above-described embodiments, the content of the video is not limited to that used for patrol work training. The content of the video may be any content as long as it is used for the training of the action for the trainee to confirm the state of the object.

In the embodiments described above, the management device 20 may not necessarily be provided. For example, the display device 10 may be used alone. In this case, the attention index value and the warning index value may be displayed on the display unit 16, for example.

In the above-described embodiment, the training system 1 may be used not only for training using VR, but also for training using AR (Augmented Reality) or MR (Mixed Reality).

In the above-described embodiments, the steps of processing performed in the training system 1 are not limited to the examples described in the above-described embodiments. The steps of this process may be interchanged unless inconsistent. The invention may be provided as a method comprising steps of processing performed in the training system 1 .

The present invention may be provided as a program executed by the display device 10 or the management device 20. FIG. This program may be downloaded via a communication line such as the Internet. In addition, this program is provided in a state recorded on computer-readable recording media such as magnetic recording media (magnetic tapes, magnetic discs, etc.), optical recording media (optical discs, etc.), magneto-optical recording media, semiconductor memories, etc. may

1: training system, 10: display device, 11: processor, 12: memory, 13: storage, 14: communication unit, 15: operation unit, 16: display unit, 17: sound output unit, 18: sensor unit, 101: Identification unit, 102: measurement unit, 103: calculation unit, 104: output unit

Claims (10)

A training system that evaluates a trainee's behavior in a virtual space,
a storage unit that stores a visual field image from the position of the trainee in the virtual space;
a detection unit that detects the visual field direction of the trainee in real space;
an identifying unit that identifies the visual field of the trainee in the virtual space based on the position and the visual field direction of the trainee in the virtual space;
a display unit that displays a portion included in the field of view in the field of view image;
Based on the positional relationship between the line-of-sight position included in the field of view and the position of the target included in the virtual space , and the distance from the position of the trainee moving in the virtual space to the position of the target, and an evaluator that evaluates the trainee's actions to ascertain conditions. The training system according to claim 1, wherein the evaluation unit calculates an index value as an evaluation of the behavior of the trainee, and increases the index value when the line-of-sight position and the target position overlap. When the line- of -sight position and the position of the target overlap and the distance from the position of the trainee in the virtual space at that time to the position of the target is within a predetermined distance, 3. The training system of claim 2, wherein the index value is increased. The training system according to claim 3, wherein the predetermined distance differs depending on the type of the target. When the line-of-sight position and the position of the object overlap and the time period during which the line-of-sight position and the position of the object overlap is within a predetermined range, the evaluation unit determines whether the index 5. A training system according to any one of claims 2 to 4, which increases the value. The training system according to claim 5, wherein the predetermined range differs depending on the type of the object. The virtual space includes a plurality of objects,
The evaluation unit weights the amount of increase in the index value based on the priority determined for each of the plurality of targets and the order in which the line-of-sight position overlaps the position of each of the plurality of targets. 7. The training system according to any one of claims 2-6 . The evaluation unit further evaluates the behavior of checking the surroundings of the trainee based on the amount of change in the line-of-sight direction included in the visual field from the movement direction of the position of the trainee. 8. Training system according to any one of clause 7. Data indicating a model positional relationship between a line-of-sight position and a position of the object when confirming the state of the object, which is generated based on actions performed by an expert to confirm the condition of the object in the virtual space. further comprising a storage unit for storing
The training system according to any one of claims 1 to 8, wherein the evaluation unit evaluates the behavior of the trainee based on the degree of similarity between the positional relationship and the positional relationship of the model. A computer included in a training system that evaluates the behavior of a trainee in a virtual space,
detecting the trainee's viewing direction in real space;
identifying a visual field of the trainee in the virtual space based on the position and the visual field direction of the trainee in the virtual space;
a step of displaying a portion included in the field of view of the field of view image stored in a storage unit and viewed from the position of the trainee in the virtual space;
Based on the positional relationship between the line-of-sight position included in the field of view and the position of the target included in the virtual space , and the distance from the position of the trainee moving in the virtual space to the position of the target, a step of evaluating the behavior of the trainee to confirm the condition;

Images