JP6955132B1 - Video display method, video display system, video display control device, and program - Google Patents

Abstract

An object of the present invention is to make it possible to suppress the occurrence of directional confusion after an instantaneous movement. The video display system identifies the user's current first position and current first line-of-sight direction. The video display system specifies the second position of the moving destination and the second line-of-sight direction at the moving destination based on the first operation of designating the moving destination. The image display system displays the first image corresponding to the first position and the first line-of-sight direction in the first area corresponding to the current visual field, and in the second area overlapping a part of the first area. The second image corresponding to the two positions and the second line-of-sight direction is displayed. The image display system updates the second image displayed in the second area based on the second operation of changing the second line-of-sight direction. When the second operation is completed, the image display system displays the second image corresponding to the specified second position and the changed second line-of-sight direction in the first area.

Description

The present invention relates to a method of displaying an image related to movement in space in virtual reality, augmented reality, or the like.

In recent years, research and development of systems that realize virtual reality and augmented reality have been active. A user who uses these systems wears, for example, a head-mounted display. An image of virtual space or real space is displayed on this head-mounted display. The displayed image is an image in the field of view according to the position and direction of the user in the space. The displayed image is updated according to the movement behavior and the change of the direction by the user. So-called teleportation is an example of a movement method under virtual reality or augmented reality. For example, when the user specifies the position of the movement destination, the current position of the user instantly changes to the specified position (for example, Patent Document 1).

JP-A-2018-175038

In the teleportation, a movement method is also conceivable in which the user can specify the direction in which the user faces after the movement as well as the position of the movement destination before the movement. However, this method has a problem that it is easy for the user to be confused as to which direction the user is facing after teleporting.

The present invention has been made in view of the above points, and one example of the problem is an image display method, an image display system, and an image display control that can suppress the occurrence of directional confusion after an instantaneous movement. To provide equipment and programs.

In order to solve the above-mentioned problems, the invention according to claim 1 is a first specific step of specifying a user's current first position and current first line-of-sight direction in a predetermined space to a computer. One position is a first specific step which is a position on the first surface of a plurality of surfaces existing in the predetermined space and which can be located by the user, and the predetermined position. A first operation for designating a movement destination in space , wherein a position on a second surface that is non-parallel to the first surface among the plurality of surfaces can be designated as a second position of the movement destination. based on the operation, the second specifying step of specifying a second viewing direction in the second position and the destination, the first image corresponding to the first position and the first viewing direction which is the specified, the current field of view The first display in which the second image corresponding to the specified second position and the second line-of-sight direction is displayed in the second area overlapping a part of the first area while being displayed in the first area corresponding to the above. In this step , the first image corresponding to the orientation of the first surface and the second image corresponding to the orientation of the second surface are displayed, and the orientation of the first surface and the second image are displayed. The second image displayed in the second area is updated based on the first display step of displaying the difference information indicating the difference from the orientation of the surface and the second operation of changing the second line-of-sight direction. In response to the completion of the update step and the second operation, the second image corresponding to the specified second position and the changed second line-of-sight direction is displayed in the first area. It is characterized by executing a display step and.

According to the present invention, the first image corresponding to the current field of view is displayed in the first area, and the second image corresponding to the moving field of view is displayed in the second area overlapping a part of the first area. Is displayed. After that, the second image in the second area is updated according to the operation of changing the line-of-sight direction at the moving destination. Therefore, the user can confirm the image seen at the moving destination while changing the line-of-sight direction at the moving destination before moving to the moving destination. Therefore, it is possible to suppress the occurrence of directional confusion after the movement is instantaneous.

Further , according to the present invention, the user can move on the second surface having a direction different from the direction of the first surface on which the user is currently located. Assuming that the user stands on the first surface and the second surface, respectively, the user's visual field range at the moving destination is rotated by the difference between the orientation of the first surface and the orientation of the second surface. Therefore, the second image is rotating. Here, since the information indicating the difference between the orientation of the first surface and the orientation of the second surface is displayed, the user can recognize that the visual field is rotating.

In the invention according to claim 2 , the second operation is the same as the operation for changing the first line-of-sight direction, and the update step is displayed in the first region based on the second operation. The first image and the second image displayed in the second area are updated.

According to the present invention, when the second image is displayed in the second region, the current line-of-sight direction is changed according to the operation of changing the line-of-sight direction, and the line-of-sight direction at the moving destination is changed. Will also be changed. The first video and the second video are updated according to this change. Therefore, the user can change the line-of-sight direction without discomfort.

According to the third aspect of the present invention, the first operation is a third operation for designating the second position and a fourth operation for changing the second line-of-sight direction before displaying the second image in the second region. The second specific step includes the operation, the position specifying step for specifying the second position based on the third operation, and the second line of sight according to the identification of the second position. The line-of-sight information display step for displaying the line-of-sight information indicating the direction, the change step for updating the display of the line-of-sight information based on the fourth operation, and the fourth operation according to the completion of the fourth operation. The second line-of-sight direction changed based on the above is included as a line-of-sight direction specifying step for specifying the second line-of-sight direction as the second line-of-sight direction at the start of displaying the second image in the second region.

According to the present invention, the user can specify the line-of-sight direction at the moving destination at the start of displaying the second image in the second region.

According to a fourth aspect of the present invention, the second specific step is a direction in which at least one predetermined object among a plurality of objects that can exist in the predetermined space is located from the specified second position, and the first aspect. The display mode determination step for determining the display mode of the line-of-sight information based on the two line-of-sight directions is further included, and the line-of-sight information display step is characterized in that the line-of-sight information is displayed in the determined display mode. do.

According to the present invention, the line-of-sight information is displayed in a display mode based on the direction in which the predetermined object is located and the line-of-sight direction at the moving destination. Therefore, the user can change or not change the line-of-sight direction after grasping the existence of the predetermined object in the line-of-sight direction at the moving destination.

The invention according to claim 5 is a first specific step for specifying a user's current first position and a current first line-of-sight direction in a predetermined space, and a first operation for designating a movement destination in the predetermined space. Based on the above, the second specific step of specifying the second position of the moving destination and the second line-of-sight direction at the moving destination, and the first image corresponding to the specified first position and the first line-of-sight direction are currently displayed. The second image corresponding to the specified second position and the second line-of-sight direction is displayed in the second area overlapping a part of the first area while being displayed in the first area corresponding to the field of view of. According to one display step, an update step of updating the second image displayed in the second area based on the second operation of changing the second line-of-sight direction, and the completion of the second operation. Then, the second display step of displaying the second image corresponding to the specified second position and the changed second line-of-sight direction in the first area is executed, and the first operation is performed. The second specific step includes the third operation of designating the second position and the fourth operation of changing the second line-of-sight direction before displaying the second image in the second region. A position specifying step for specifying the second position based on the three operations, and a direction in which at least one predetermined object among a plurality of objects that can exist in the predetermined space is located from the specified second position. A display mode determination step for determining a display mode of line-of-sight information indicating the second line-of-sight direction based on the second line-of-sight direction, and the determined display according to the identification of the second position. A line-of-sight information display step for displaying the line-of-sight information in an embodiment, a change step for updating the display of the line-of-sight information based on the fourth operation, and the fourth operation according to the completion of the fourth operation. At least one of the predetermined objects, including a line-of-sight direction specifying step for specifying the second line-of-sight direction changed based on One is a predetermined obstacle, and in the display mode determination step, the display mode is the display mode depending on whether the obstacle blocks or does not block the visual field according to the second position and the second line-of-sight direction. It is characterized by differentiating .

According to the present invention, the display mode of the line-of-sight information differs depending on whether the field of view at the moving destination is blocked by an obstacle or not. Therefore, the user who sees the line-of-sight information can deviate the line-of-sight direction at the moving destination from the obstacle, if necessary.

The invention according to claim 6 is a first specific step for specifying a user's current first position and a current first line-of-sight direction in a predetermined space, and a first operation for designating a movement destination in the predetermined space. Based on the above, the second specific step of specifying the second position of the moving destination and the second line-of-sight direction at the moving destination, and the first image corresponding to the specified first position and the first line-of-sight direction are currently displayed. The second image corresponding to the specified second position and the second line-of-sight direction is displayed in the second area overlapping a part of the first area while being displayed in the first area corresponding to the field of view of. According to one display step, an update step for updating the second image displayed in the second area based on the second operation for changing the second line-of-sight direction, and the completion of the second operation. Then, the second display step of displaying the second image corresponding to the specified second position and the changed second line-of-sight direction in the first area is executed, and the first operation is performed. The second specific step includes the third operation of designating the second position and the fourth operation of changing the second line-of-sight direction before displaying the second image in the second region. Based on the position specifying step for specifying the second position and the direction in which at least one predetermined object among the plurality of objects that can exist in the predetermined space is located from the specified second position based on the three operations. Then, the direction determination step for determining the initial line-of-sight direction and the line-of-sight information for displaying the line-of-sight information indicating the determined initial line-of-sight direction as the second line-of-sight direction according to the identification of the second position. The display step, the change step for updating the display of the line-of-sight information based on the fourth operation, and the second line-of-sight changed based on the fourth operation in response to the completion of the fourth operation. It is characterized by including a line-of-sight direction specifying step of specifying the direction as the second line-of-sight direction at the start of displaying the second image in the second region.

According to the present invention, the line-of-sight direction at the moving destination at the start of displaying the line-of-sight information is determined based on the direction in which the predetermined object is located from the position of the moving destination. Therefore, since an appropriate line-of-sight direction is determined in relation to the direction in which the predetermined object is located, the amount or number of changes in the line-of-sight direction by the user can be reduced.

In the invention according to claim 7 , at least one of the predetermined objects is a predetermined obstacle, and the direction determination step obtains a field of view according to the second position and the initial line-of-sight direction. It is characterized in that the initial line-of-sight direction is determined so that the object is not blocked.

According to the present invention, the initial line-of-sight direction is determined so that a predetermined obstacle does not block the field of view of the moving destination. Therefore, it is possible to reduce the operation of the user to deviate the line-of-sight direction from a predetermined obstacle.

In the invention according to claim 8 , at least one of the predetermined objects is an object that can be the target of the predetermined action by the user, and the direction determination step is an object that can be the target of the predetermined action from the second position. Is characterized in that the direction in which the object is located is determined in the initial line-of-sight direction.

According to the present invention, the direction in which an object that can be the target of a predetermined action by the user is located is determined as the initial line-of-sight direction. Therefore, it is possible to reduce the operation of the user looking at the object in preparation for taking a predetermined action.

The invention according to claim 11 is a first specific step for specifying a user's current first position and a current first line-of-sight direction in a predetermined space, and a first operation for designating a movement destination in the predetermined space. Based on the above, the second specific step of specifying the second position of the moving destination and the second line-of-sight direction at the moving destination, and the first image corresponding to the specified first position and the first line-of-sight direction are currently displayed. The second image corresponding to the specified second position and the second line-of-sight direction is displayed in the second area overlapping a part of the first area while being displayed in the first area corresponding to the field of view of. According to one display step, an update step of updating the second image displayed in the second area based on the second operation of changing the second line-of-sight direction, and the completion of the second operation. A second display step of displaying the second image corresponding to the specified second position and the changed second line-of-sight direction in the first area, and a plurality of objects that can exist in the predetermined space. Of these, a size determination step of determining the size of the second region based on at least one predetermined object included in the second image displayed in the second region is executed, and the first display step is performed. Is characterized in that the second image is displayed in the second region of the determined size.

According to the present invention, the size of the second region is determined based on a predetermined object. When the size of the second area changes, the area of the second area displayed in the second area changes. Further, since the second area overlaps with a part of the first area, when the size of the second area changes, the area of the first area displayed in the first area also changes. For example, as the second region becomes larger, the area of the second region becomes larger and the area of the first region becomes smaller. Therefore, the area of the first image and the area of the second image can be appropriately adjusted.

Further , according to the present invention, the size of the second region is determined based on the object included in the second video displayed in the second region. Therefore, the area of the first image and the area of the second image can be adjusted based on the importance of the second image.

The invention according to claim 10 is characterized in that the size determination step determines the size based on the number of the predetermined objects.

According to the present invention, the area of the first image and the area of the second image can be adjusted based on the number of objects included in the second image displayed in the second area.

The invention according to claim 11 is characterized in that the size determination step determines the size based on the distance between the second position and the predetermined object.

According to the present invention, the area of the first image and the area of the second image are based on the distance between the position at the moving destination and the position of the object included in the second image displayed in the second area. Can be adjusted.

The invention according to claim 12 is a first specific step of specifying a user's current first position and a current first line-of-sight direction in a predetermined space, and a first operation of designating a movement destination in the predetermined space. Based on the above, the second specific step of specifying the second position of the moving destination and the second line-of-sight direction at the moving destination, and the first image corresponding to the specified first position and the first line-of-sight direction are currently displayed. The second image corresponding to the specified second position and the second line-of-sight direction is displayed in the second area overlapping a part of the first area while being displayed in the first area corresponding to the field of view of. According to one display step, an update step of updating the second image displayed in the second area based on the second operation of changing the second line-of-sight direction, and the completion of the second operation. A second display step of displaying the second image corresponding to the specified second position and the changed second line-of-sight direction in the first area, and a plurality of objects that can exist in the predetermined space. In the size determination step of determining the size of the second region based on at least one predetermined object, the second image displayed in the second region is the target of the predetermined action by the user. A sizing step that determines the size based on whether or not an object that can be a potential object is included, and the first display step displays the second image in the second region of the determined size. It is characterized by displaying .

According to the present invention, the size of the second area is determined based on whether or not the second image displayed in the second area includes an object that can be a target of a predetermined action by the user. Therefore, the area of the first image and the area of the second image can be adjusted based on the possibility that the user takes a predetermined action at the moving destination.

The invention according to claim 13 is a first specific step for specifying a user's current first position and a current first line-of-sight direction in a predetermined space, and a first operation for designating a movement destination in the predetermined space. Based on the above, the second specific step of specifying the second position of the moving destination and the second line-of-sight direction at the moving destination, and the first image corresponding to the specified first position and the first line-of-sight direction are currently displayed. The second image corresponding to the specified second position and the second line-of-sight direction is displayed in the second area overlapping a part of the first area while being displayed in the first area corresponding to the field of view of. According to one display step, an update step of updating the second image displayed in the second area based on the second operation of changing the second line-of-sight direction, and the completion of the second operation. A second display step of displaying the second image corresponding to the specified second position and the changed second line-of-sight direction in the first area, and a plurality of objects that can exist in the predetermined space. In the first position, the size determination step of determining the size of the second region based on at least one predetermined object, which can pose a danger to the user in the predetermined space. A size determination step for determining the size based on the degree of danger approaching the user is provided, and the first display step displays the second image in the second region of the determined size. It is characterized by being displayed .

According to the present invention, a predetermined object determines the size of the second region based on the degree of danger approaching the user at the current position. Therefore, the area of the first image and the area of the second image can be adjusted according to the need for the user to take some action against the currently imminent danger.

In the invention according to claim 14 , the second display step further changes the first position and the first line-of-sight direction to the specified second position and the changed second line-of-sight direction, respectively. It is characterized by.
The invention according to claim 15 is in a video display system comprising at least one memory for storing a program and at least one processor operating according to the stored program, wherein the at least one processor according to the program. , A first specific step of specifying a user's current first position and current first line-of-sight direction in a predetermined space, wherein the first position is a plurality of surfaces existing in the predetermined space. The first specific step, which is a position on the first surface of a plurality of surfaces on which the user can be located, and the first operation of designating a movement destination in the predetermined space, which are the first operations of the plurality of surfaces. Of these, based on the first operation in which the position on the second surface that is non-parallel to the first surface can be designated as the second position of the movement destination, the second position and the second line-of-sight direction at the movement destination are set. While displaying the second specific step to be specified and the first image corresponding to the specified first position and the first line-of-sight direction in the first area corresponding to the current visual field, a part of the first area is displayed. This is the first display step of displaying the second image corresponding to the specified second position and the second line-of-sight direction in the overlapping second region, and is the first display step corresponding to the orientation of the first surface. The first display step of displaying the second image corresponding to the orientation of the second surface and displaying difference information indicating the difference between the orientation of the first surface and the orientation of the second surface. Based on the second operation of changing the second line-of-sight direction, the update step of updating the second image displayed in the second area and the identification according to the completion of the second operation. It is characterized in that the second display step of displaying the second image corresponding to the second position and the changed second line-of-sight direction in the first area is executed.
The invention according to claim 17 is in a video display system comprising at least one memory for storing a program and at least one processor operating according to the stored program, wherein the at least one processor according to the program. , The second position of the movement destination based on the first specific step of specifying the user's current first position and the current first line-of-sight direction in the predetermined space and the first operation of designating the movement destination in the predetermined space. And the second specific step of specifying the second line-of-sight direction at the moving destination, and the first image corresponding to the specified first position and the first line-of-sight direction are displayed in the first area corresponding to the current field of view. However, the first display step of displaying the second image corresponding to the specified second position and the second line-of-sight direction and the second line-of-sight direction in the second area overlapping a part of the first area. Based on the second operation to be changed, the update step for updating the second image displayed in the second area, and the specified second position and the said in response to the completion of the second operation. A second display step of displaying the second image according to the changed second line-of-sight direction in the first area, and a plurality of objects that can exist in the predetermined space are displayed in the second area. A size determination step of determining the size of the second region based on at least one predetermined object included in the second image is executed, and the first display step is the second display step of the determined size. The second image is displayed in the area .

The invention according to claim 18 is a first specific step for specifying a user's current first position and current first line-of-sight direction in a predetermined space in a video display method executed by a computer , wherein the first specific step is performed. The position is a first specific step which is a position on the first surface of a plurality of surfaces existing in the predetermined space and which can be located by the user, and the predetermined space. This is the first operation for designating the movement destination in the above, and the position on the second surface that is non-parallel to the first surface among the plurality of surfaces can be designated as the second position of the movement destination. Based on the above, the second specific step of specifying the second line-of-sight direction at the second position and the moving destination, and the first image corresponding to the specified first position and the first line-of-sight direction are set in the current field of view. The first display step of displaying the second image corresponding to the specified second position and the second line-of-sight direction in the second area overlapping a part of the first area while displaying in the corresponding first area. The first image corresponding to the orientation of the first surface and the second image corresponding to the orientation of the second surface are displayed, and the orientation of the first surface and the second surface are displayed. Update to update the second image displayed in the second area based on the first display step of displaying the difference information indicating the difference from the direction of the second and the second operation of changing the second line-of-sight direction. A second display for displaying the second image according to the specified second position and the changed second line-of-sight direction in the first area in response to the completion of the step and the second operation. It is characterized by including steps.

The invention according to claim 19 is in a video display control device including at least one memory for storing a program and at least one processor operating according to the stored program, wherein the at least one processor is the program. According to the first specific step of specifying the user's current first position and the current first line-of-sight direction in the predetermined space, the first position is a plurality of surfaces existing in the predetermined space. The first specific step, which is a position on the first surface of the plurality of surfaces on which the user can be located, and the first operation of designating a movement destination in the predetermined space , which are the plurality of surfaces. among the first surface and the position on the non-parallel second surface, based on the possible first operation as the second position of the movement destination, the second position and the second line-of-sight direction in the destination A part of the first area while displaying the second specific step for specifying the above and the first image corresponding to the specified first position and the first line-of-sight direction in the first area corresponding to the current field of view. This is the first display step of displaying the second image corresponding to the specified second position and the second line-of-sight direction in the second area overlapping with the above, and the first image corresponding to the orientation of the first surface. And the first display step of displaying the second image corresponding to the orientation of the second surface and displaying difference information indicating the difference between the orientation of the first surface and the orientation of the second surface. Based on the second operation of changing the second line-of-sight direction, the update step of updating the second image displayed in the second area, and the identification according to the completion of the second operation. It is characterized in that the second display step of displaying the second image corresponding to the changed second position and the changed second line-of-sight direction in the first area is executed.

According to the present invention, it is possible to suppress the occurrence of directional confusion after the movement is instantaneous.

It is a figure which shows an example of the outline structure of the VR system S which concerns on one Embodiment. It is a figure which shows an example of the functional block of the system control unit 11 in the control device 1 which concerns on one Embodiment. It is a figure which shows the screen example of HMD2. It is a figure which shows the screen example of HMD2. It is a figure which shows the screen example of HMD2. It is a figure which shows the screen example of HMD2. It is a figure which shows the example of the relationship between the direction of the surface of the moving destination, and the initial line-of-sight direction at the moving destination. It is a figure which shows the display example of the difference information. It is a figure which shows the display example of the line-of-sight information. It is a figure which shows the example of changing the display mode of the line-of-sight information. It is a figure which shows the determination example of the initial line-of-sight direction. It is a figure which shows the determination example of the size of the 2nd region. It is a flowchart which shows an example of application processing by the system control unit 11 of the control device 1 which concerns on one Embodiment. It is a flowchart which shows an example of the moving destination initial line-of-sight direction determination processing by the system control unit 11 of the control device 1 which concerns on one Embodiment. It is a flowchart which shows an example of the moving destination visual field image preview processing by the system control unit 11 of the control device 1 which concerns on one Embodiment.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. The embodiment described below is an embodiment when the present invention is applied to a system for realizing virtual reality that allows a user to experience a three-dimensional virtual space. The present invention may be applied to augmented reality for expanding the real space.

[1. VR system configuration]
First, the configuration and functional outline of the VR system S according to the present embodiment will be described with reference to FIG. FIG. 1 is a diagram showing an example of an outline configuration of a VR system S according to the present embodiment. As shown in FIG. 1, the VR system S includes a control device 1, an HMD (head-mounted display) 2, a controller 3, and a sensor 4. The control device 1 and each of the HMD2, the controller 3 and the sensor 4 are connected by wire or wirelessly.

The control device 1 is, for example, a terminal device that controls the entire VR system S. The control device 1 includes a system control unit 11, a stem bus 12, a storage unit 13, a communication unit 14, and an input / output interface 15. The system control unit 11, the storage unit 13, the communication unit 14, and the input / output interface 15 are each connected via the system bus 12.

The system control unit 11 is composed of a CPU (Central Processing Unit) 11a, a ROM (Read Only Memory) 11b, a RAM (Random Access Memory) 11c, and the like. The system control unit 11 performs processing according to a program stored in at least one of the ROM 11b, the RAM 11c, and the storage unit 13.

The storage unit 13 is a non-volatile storage device. The storage unit 13 is composed of, for example, an SSD (Solid State Drive), a hard disk drive, or the like. The storage unit 13 stores, for example, a database such as a terrain DB, an object DB, and a user DB. "DB" is an abbreviation for database. The terrain DB is a database that defines three-dimensional terrain in virtual space. The object DB is a database that defines each object that can exist in the virtual space. The object may include anything that the user (or player, etc.) can see in the virtual space. Examples of objects include humans, animals, imaginary creatures, robots, buildings, walls, floors, rocks, plants, automobiles, ships, aircraft, spaceships, and the like. Specifically, the object DB may define, for each object, the coordinates and orientation of the object, and parameters such as the shape, orientation, arrangement, and color or pattern of each part constituting the object. When a moving object exists, the parameters of the object are updated in real time in the object DB as needed. Information about a user who is currently experiencing virtual reality using the VR system S is stored in the user DB. For example, the user DB may store identification information for identifying the user, coordinates indicating the current position, the line-of-sight direction, and the like. The current position is the current position of the user in the virtual space. The line-of-sight direction is the direction of a straight line along the user's field of view in the virtual space. In other words, the line-of-sight direction is the direction in which the user's eyes are facing. The current position and the line-of-sight direction change in real time according to the user's operation or the user's operation.

The storage unit 13 further stores various programs such as an operating system and application software. The application software is a program that causes the system control unit 11 to execute a process for realizing virtual reality according to a specific purpose by using, for example, an API (Application Programming Interface) provided by an operating system. The application software may be, for example, a game or other types of software. The application software may be acquired from a predetermined server device via, for example, the communication unit 14, or may be recorded on a recording medium such as an optical disk or a memory card and read via the drive device. You may.

The communication unit 14 is composed of, for example, a network interface controller or the like. The communication unit 14 connects to a server device or the like via a network such as the Internet, and controls the communication state with the connected device.

The input / output interface 15 performs interface processing between the control device 1 and each of the HMD 2, the controller 3, and the sensor 4. Examples of the standard of the input / output interface 15 include USB (Universal Serial Bus), HDMI (registered trademark), Bluetooth (registered trademark) and the like.

The HMD2 is a display device worn on the user's head. The HMD 2 includes, for example, a display unit, headphones, and a sensor unit. The display unit mainly displays an image in the user's field of view according to the direction (line-of-sight direction) in which the user is facing at the place where the user is located in the virtual space. The displayed image may be a three-dimensional image or a two-dimensional image. Examples of the display unit include a liquid crystal display, an organic EL display, and the like. Headphones output sounds generated in virtual space and other sounds. The sensor unit detects the orientation and rotation of the user's head, and transmits the detection result to the control device 1. The sensor unit may be capable of detecting the yaw, pitch and roll of the user's head. In the local coordinate system based on the user, the axis in the direction from the user's foot to the head is defined as the z-axis. The axis in the direction from the left to the right of the user is defined as the x-axis. The axis in the line-of-sight direction of the user is defined as the y-axis. Yaw is a rotation about the z-axis. Pitch is a rotation about the x-axis. A roll is a rotation about the y-axis. The sensor unit may include, for example, at least one of a gyro sensor, a magnetic sensor, a gravity sensor, and an acceleration sensor. An operation such as rotation of the head by the user wearing the HMD2 may be regarded as one of the user's operations on the HMD2. The sensor unit may be able to detect the direction of the user's line of sight by eye tracking. In this case, the sensor unit may further include, for example, at least one of a digital camera and an infrared center. The HMD 2 and the control device 1 may be integrated.

The controller 3 is an input device that transmits the user's operation content as an operation signal to the control device 1. The controller 3 may include, for example, buttons, pads, sticks, and the like. The controller 3 may be, for example, a game controller.

The sensor 4 detects at least one of the user's position, orientation, and user's hand position in the space in which the user is actually present (for example, the user's room), and transmits the detection result to the control device 1. .. The sensor 4 is installed, for example, in a space where the user is actually present. The sensor 4 may include, for example, at least one of an infrared sensor and a digital camera. The sensor 4 may be incorporated in the HMD 2.

[2. Overview of system control functions]
Next, the outline of the function of the system control unit 11 in the control device 1 will be described with reference to FIGS. 2 to 12. FIG. 2 is a diagram showing an example of a functional block of the system control unit 11 in the control device 1 according to the present embodiment. As shown in FIG. 2, the system control unit 11 updates the first specific unit 111, the second specific unit 112, and the first display unit 113 by having the CPU 11a read and execute various program codes included in the application software. It functions as a unit 114, a second display unit 115, and the like. With these functions, the system control unit 11 realizes the teleportation of the user in the virtual space based on the operation by the user, and controls the display of the image for the teleportation.

[2-1. Basic function]
The first specifying unit 111 specifies the current position of the user and the current line-of-sight direction in a predetermined space. In the present embodiment, the predetermined space is a predetermined virtual space. For example, the first specific unit 111 may specify the line-of-sight direction based on the rotation of the user's head detected by the sensor unit provided in the HMD 2. Further, for example, the first specifying unit 111 may specify either the current position or the line-of-sight direction based on the operation of the controller 3 by the user. Further, for example, the first specific unit 111 may specify either the current position or the line-of-sight direction based on either the user's position or orientation detected by the sensor 4. The user may be able to be located on any of a plurality of faces existing in the virtual space. The surface existing in the virtual space may be a surface possessed by an object existing in the virtual space. Of the faces existing in the virtual space, the faces on which the user can be located may be predetermined. The surface on which the user can be located may be, for example, a floor, a road, the ground, or the like. The user may also be able to be located in the air.

The second specifying unit 112 specifies the position of the moving destination and the line-of-sight direction at the moving destination based on the operation of designating the moving destination in the predetermined space. For example, on the screen of the HMD2, the current visual field image corresponding to the visual field according to the user's current position and the line-of-sight direction is displayed in the virtual space. For example, the user may be able to specify a place in the user's field of view currently indicated by the field of view image as the position of the moving destination by operating the controller 3. Alternatively, by moving the user's head, the user can specify a location in the virtual space in the user's line-of-sight direction as the destination position based on the rotation of the head detected by the sensor unit of HDM2. There may be. Alternatively, based on the operation of the controller 3 by the user, the second specific unit 112 may superimpose the map on the current visual field image and display the map around the user's current position on the HMD2. The user may be able to specify the destination from the displayed map, for example, by operating the controller 3. The second specifying unit 112 may specify the position thus designated by the user as the moving destination position. Only a place within the user's field of view may be designated as the movement destination position, or a place outside the user's field of view may also be designated as the movement destination position. The second specifying unit 112 may specify the same direction as the current line-of-sight direction as the line-of-sight direction at the moving destination. The details of the method of specifying the line-of-sight direction at the moving destination will be described later.

The first display unit 113 displays the current visual field image corresponding to the current position and the current line-of-sight direction specified by the first specific unit 111 in the first region corresponding to the current visual field, and is one of the first regions. In the second region overlapping the unit, the moving destination visual field image corresponding to the position of the moving destination specified by the second specific unit 112 and the line-of-sight direction at the moving destination is displayed. As described above, the first region in which the current visual field image is displayed may be, for example, the entire screen of the HMD2 or a part of the screen. The second region in which the moving destination visual field image is displayed may be a part of the first region. Examples of the shape of the second region include a rectangle, a rhombus, a hexagon, a dodecagon, a circle, and the like. The second region may be displayed in a predetermined direction and / or at a predetermined position in a local coordinate system relative to the user. For example, the second region may be displayed in the positive direction of the Y axis in the local coordinate system, i.e. in front of the user. Alternatively, the second region may be displayed at a predetermined position in the first region. For example, the second region may be displayed in the center of the first region or at some other position. The first display unit 113 generates a current visual field image and a moving destination visual field image by three-dimensional rendering. For example, the first display unit 113 may generate a spherical image corresponding to the current position based on the current position, the terrain DB, the object DB, and the like. The spherical image may be, for example, an image that covers a field of view of 360 degrees vertically and horizontally from the position of the user. The first display unit 113 may cut and generate the current visual field image in the clipping region corresponding to the first region according to the current line-of-sight direction from the generated spherical image. Further, the first display unit 113 may generate a spherical image of the movement destination based on the position of the movement destination, the terrain DB, the object DB, and the like. The first display unit 113 may cut and generate a moving destination visual field image in a clipping region corresponding to a second region according to the line-of-sight direction at the moving destination from the generated spherical image. Since the second region is narrower than the first region, the clipping region of the moving destination visual field image may be narrower than the clipping region of the current visual field image. Before the user teleports to the movement destination specified by the user, the current visual field image is displayed and the moving destination visual field image is displayed overlapping with a part of the current visual field image, so that the user can see the state of the moving destination. Can be seen in advance.

The updating unit 114 updates the moving destination visual field image displayed in the second area based on the operation of changing the line-of-sight direction at the moving destination. For example, the user may be able to change the line-of-sight direction at the moving destination by operating the controller 3. Alternatively, the user may be able to change the line-of-sight direction at the destination based on the rotation of the head detected by the sensor unit of HDM2 by moving the user's head. Alternatively, the update unit 114 may use eye tracking to change the direction of the line of sight at the destination according to the movement of the user's own line of sight. For example, the user may be able to change the current line-of-sight direction by moving the user's head, and may be able to change the line-of-sight direction at the destination by moving the user's own line of sight. The updating unit 114 may cut and generate a new moving destination visual field image in a clipping region corresponding to the changed line-of-sight direction from, for example, the moving destination spherical image generated by the first display unit 113. Then, the updating unit 114 may update the moving destination visual field image displayed in the second region with the newly generated moving destination visual field image. The user can change the line-of-sight direction at the moving destination while watching the moving destination visual field image before teleporting.

The operation of changing the current line-of-sight direction and the operation of changing the line-of-sight direction at the moving destination while the moving-destination visual field image is displayed in the second region may be the same. The updating unit 114 may update both the current visual field image displayed in the first area and the moving destination visual field image displayed in the second area based on the operation of changing the line-of-sight direction. .. For example, the update unit 114 simultaneously changes both the current line-of-sight direction and the line-of-sight direction at the moving destination based on the user's operation. The updating unit 114 generates a current visual field image according to the changed current line-of-sight direction, and also generates a moving-destination visual field image according to the changed line-of-sight direction at the moving destination. The update unit 114 has a first local coordinate system whose z-axis is the axis perpendicular to the plane where the user is currently located, and a second local coordinate system whose z-axis is the axis perpendicular to the destination plane. In between, the current line-of-sight direction and the line-of-sight direction at the destination may be rotated in the same direction and at the same angle. Therefore, if the orientation of the surface on which the user is currently located and the orientation of the destination surface are different, in the world coordinate system of the virtual space, the rotation direction in the current line-of-sight direction and the rotation in the line-of-sight direction at the destination. The direction is different.

When the user changes the current line-of-sight direction, the update unit 114 does not move the direction or position of the second region in the local coordinate system with respect to the user, or the position of the second region within the first region. May be good. In this case, since the position of the second region in the first region is fixed, the second region does not deviate from the first region. Alternatively, the updating unit 114 may move the direction or position of the second region according to the change in the current line-of-sight direction. For example, the position of the second region may be fixed in the world coordinate system of the virtual space. For example, the updating unit 114 may move the second region to the left when the line-of-sight direction is rotated to the right, and may move the second region to the right when the line-of-sight direction is rotated to the left. In this case, the second region may deviate from the first region (may be out of frame). Therefore, the update unit 114 may be able to change at least one of the direction and the position of the second region by the operation of the user. For example, the user may be able to change the direction or position of the second region by operating the controller 3.

The second display unit 115 is changed based on the position of the movement destination specified by the second specific unit 112 and the operation of the user in response to the completion of the operation of changing the line-of-sight direction at the movement destination. The moving destination visual field image corresponding to the line-of-sight direction in is displayed in the first area. For example, the user operates the controller 3 to instruct the determination of the line-of-sight direction at the moving destination. In response to this, the second display unit 115 changes, for example, the current position and the current line-of-sight direction based on the position of the destination specified by the second specific unit 112 and the operation of the user. It may be changed to the line-of-sight direction in. The second display unit 115 may cut and generate a moving destination visual field image in a clipping region corresponding to the first region according to the changed line-of-sight direction from the moving destination spherical image. The second display unit 115 may display the generated destination visual field image as the current visual field image in the first region. The second display unit 115 may erase the second area from the screen. Further, the second display unit 115 may display only the moving destination visual field image among the current visual field image and the moving destination visual field image according to the current position before the change. The second display unit 115 may display the moving destination visual field image displayed in the second area so that the user can see it as if it were expanded to the first area. That is, the second display unit 115 continues to display the portion displayed in the second region of the moving destination spherical image, and is the current visual field image of the moving destination spherical image. The moving-destination field-of-view image may be displayed so that the hidden portion is displayed. Since at least a part of the current visual field image displayed immediately after the teleportation is displayed in the second area as the destination visual field image before the teleportation, it is possible to suppress the occurrence of directional confusion after the teleportation. ..

Next, a specific example of teleportation when the application software of the present embodiment is an FPS (First Person Shooter) will be described. FPS is an action game in which the space in the game can be confirmed from the user's point of view, for example. Various enemies that the user should defeat may appear in the space in the game. These enemies are objects. The enemy may attack the user.

3 (a) to 5 (b) are diagrams showing screen examples of HMD2. In FIG. 3A, for example, the field of view image 100 is currently displayed on the entire screen of the HMD2. Currently, in the field of view image 100, the blocks 310 and 320 and the enemy 210 are displayed. Each block is an object that can support the user, such as a wall, floor, pillar, rock, or building. The user stands on the surface 311 of the block 310. The enemy 210 stands on the surface 321 of the block 320 away from the block 310. The orientation of the surface 311 and the orientation of the surface 321 are the same. The orientation of the surface is a direction perpendicular to the surface and from the back side to the front side of the surface. The user and the enemy will stand on the surface along the orientation of the surface. The direction from the user's feet to the head when standing on the surface is called the standing direction. The standing direction coincides with the orientation of the surface. When the user operates, for example, the controller 3 to instruct the start of designating the position of the movement destination, the controller 110, the beam 120, and the pointer 130 are displayed on the current visual field image 100 as shown in FIG. 3A. Will be done. The controller 110 is an object that imitates the controller 3. The beam 120 is output from the tip of the controller 110. The beam 120 indicates the direction of the movement destination. The pointer 130 is displayed ahead of the beam 120. The pointer 130 indicates the position of the movement destination. For example, when the user moves the controller 3, the position of the pointer 130 currently displayed on the field of view image 100 also moves, and the movement destination can be changed. The movement of the controller 3 is detected by, for example, the sensor 4.

As shown in FIG. 3B, the pointer 130 on the current visual field image 100 moves in response to the user moving the controller 3. Then, the user moves the pointer 130 on the surface 322 of the block 320. The orientation of the surface 322 is the same as the orientation of the surfaces 311 and 321 respectively. The surface 322 is located higher than the surface 311.

Here, the user operates, for example, the controller 3 to instruct the determination of the position of the movement destination. In response to this, the specifying unit 2 specifies the position of the moving destination. Further, the specifying unit 2 may specify the same direction as the current line-of-sight direction as the line-of-sight direction at the moving destination. Then, as shown in FIG. 4A, the first display unit superimposes on a part of the current visual field image 100, displays the preview frame 400 as an example of the second region, and moves into the preview frame 400. The foreground image 410 is displayed. At this point, the horizon is displayed as the moving destination visual field image 410.

While displaying the preview frame 400, the user changes the line-of-sight direction at the moving destination, for example, by rotating the user's head. In the above-mentioned local coordinate system with reference to the user, it may be possible to rotate the line-of-sight direction left and right around the z-axis and rotate the line-of-sight direction up and down around the x-axis. The enemy 210 is located below the destination position on the surface 322. In addition, the enemy 210 is located on the right side of the initial line-of-sight direction at the destination. Therefore, the user first moves the line-of-sight direction to the right in order to shoot the enemy 210. In response to this, the updating unit 114 updates the current visual field image 100 and the moving destination visual field image 410 in the preview frame 400, as shown in FIG. 4B. Specifically, the current visual field image 100 and the moving destination visual field image 410 both flow to the left. Next, the user moves the line-of-sight direction downward. In response to this, the updating unit 114 updates the current visual field image 100 and the moving destination visual field image 410 in the preview frame 400, as shown in FIG. 5A. Specifically, the current visual field image 100 and the moving destination visual field image 410 both flow upward. Then, in the preview frame 400, a part of the image of the enemy 210 looking down from above the surface 311 is displayed as the moving destination visual field image 410. Here, the user operates, for example, the controller 3 to instruct the determination of the line-of-sight direction. In response to this, as shown in FIG. 5B, the second display unit 115 erases the preview frame 400, and displays the entire image of the enemy 210 looking down from above the surface 311 as the current field of view image 100, HMD2. Display on the entire screen of.

[2-2. Orientation of the destination surface]
In the examples shown in FIGS. 3A to 5B, the orientation of the surface of the movement source where the current user is located is the same as the orientation of the surface of the movement destination. However, the user may be able to specify a position on the surface having a direction different from the direction of the surface of the movement source as the position of the movement destination. As mentioned above, the user stands along the orientation of the face. Therefore, the standing direction of the user at the moving source and the standing direction of the user at the moving destination are different. In this case, gravity may be ignored in the virtual space. The surface of each block existing in the virtual space may have both floor and wall properties. The first display unit 113 may rotate the moving destination visual field image displayed in the second area according to the difference between the orientation of the moving source surface and the orientation of the moving destination surface. For example, the first display unit 113 may display the visual field image so that the orientation of the surface on which the user is located and the upward direction in the user's visual field match. Specifically, the first display unit 113 may display the current visual field image in the first region so that the orientation of the surface on which the user is currently located and the upward direction in the first region coincide with each other. Further, the first display unit 113 may display the moving destination visual field image in the second region so that the orientation of the moving destination surface and the upward direction in the second region coincide with each other. However, in the local coordinate system based on the user, when the user's head is tilted about the y-axis, the user's field of view may be further rotated by the tilt.

6 (a) and 6 (b) are diagrams showing screen examples of HMD2. As shown in FIG. 6A, the enemy 210, the blocks 310 and 320, and the bar 330 are currently displayed on the field of view image 100. Bar 330 is a bar that is long in the horizon direction. The user is currently standing on surface 311 of block 310. The enemy 210 stands on the surface 321 of the block 320. The user specifies the surface 323 of the block 320 with the pointer 130. The surface 323 is perpendicular to the surfaces 311 and 321 respectively. In the preview frame 400 displayed in response to the operation of confirming the position of the moving destination by the user, the moving destination visual field image 410 rotated by 90 degrees is displayed. For example, as shown in FIG. 6A, an image of a bar 330 long in the vertical direction is displayed as a moving destination visual field image 410. Here, the user makes the line-of-sight direction at the moving destination point toward the enemy 210. In response to this, as shown in FIG. 6B, the updating unit 114 displays the sideways enemy 210 as the moving destination visual field image 410 in the preview frame 400.

FIG. 7 is a diagram showing an example of the relationship between the orientation of the moving destination surface and the initial line-of-sight direction at the moving destination when displaying the moving destination visual field image in the second region. As shown in FIG. 7, in the virtual space, the user 10 stands on, for example, a surface 500. The orientation of the surface 500 is the positive direction of the z-axis. Therefore, the standing direction of the user 10 is also the positive direction of the z-axis. The line-of-sight direction 600 of the user 10 is the positive direction of the y-axis. The first display unit 113 may change the standing direction so that the standing direction of the user matches the direction of the moving destination surface. In response to this, the first display unit 113 determines the line-of-sight direction at the moving destination. At this time, the first display unit 113 may determine the line-of-sight direction so as to suppress the occurrence of confusion in the direction by the user. For example, suppose that the user 10 designates the surface 510 as the movement destination. The orientation of the surface 510 is the same as the orientation of the surface 500. Therefore, the line-of-sight direction 610 on the surface 510 may be the same as the line-of-sight direction 600. It is assumed that the user 10 designates the surface 520 as the movement destination. The orientation of the surface 520 is the negative direction of the y-axis. In this case, the line-of-sight direction 620 on the surface 520 may be the positive direction of the z-axis. For example, the first display unit 113 may determine the line-of-sight direction 620 by rotating the standing direction and the line-of-sight direction 600 upward by 90 degrees about the x-axis. It is assumed that the user 10 designates the surface 530 as the movement destination. The orientation of the surface 530 is the negative direction of the x-axis. In this case, the line-of-sight direction 630 on the surface 530 may be the positive direction of the z-axis. For example, the first display unit 113 determines the line-of-sight direction 630 by rotating the standing direction and the line-of-sight direction 600 upward by 90 degrees around the x-axis and rotating the line-of-sight direction 600 by 90 degrees to the right around the z-axis. You may. It is assumed that the user 10 designates the surface 540 as the movement destination. The orientation of the surface 540 is the positive direction of the x-axis. In this case, the line-of-sight direction 640 on the surface 540 may be the positive direction of the z-axis. For example, the first display unit 113 determines the line-of-sight direction 640 by rotating the standing direction and the line-of-sight direction 600 upward by 90 degrees around the x-axis and rotating the line-of-sight direction 600 by 90 degrees to the left around the z-axis. You may. It is assumed that the user 10 designates the surface 550 as the movement destination. The orientation of the surface 550 is the negative direction of the z-axis. In this case, the line-of-sight direction 650 on the surface 550 may be the positive direction of the y-axis. For example, the first display unit 113 may determine the line-of-sight direction 650 by rotating the standing direction and the line-of-sight direction 600 by 180 degrees around the y-axis. As a result, the line-of-sight direction does not change, but the standing direction changes. As the movement destination, it may be possible to specify a surface facing in the positive direction of the y-axis.

The first display unit 113 displays the current visual field image according to the orientation of the moving source surface on which the user is currently located and the moving destination visual field image according to the orientation of the moving destination surface, and displays the moving destination visual field image according to the orientation of the moving source surface. Difference information indicating the difference between the orientation and the orientation of the destination surface may be displayed. The first display unit 113 matches the upward direction in the clipping region (or the first region or the second region) with the direction of the surface, that is, the standing direction of the user, and the first display unit 113 projects the current visual field image onto the clipping region. Alternatively, a moving-destination field-of-view image is generated. In this way, a visual field image corresponding to the orientation of the surface is generated. The difference information may be information that allows the user to recognize the difference in the angle of orientation of the surface.

8 (a) and 8 (b) are diagrams showing a display example of the difference information. For example, as shown in FIG. 8A, the first display unit 113 may display the mark 421 and the mark 422 as difference information on the edge of the preview frame 400. The mark 421 indicates the orientation of the surface on which the user is currently located in the current visual field image. Therefore, the mark 421 usually indicates an upward direction in the first region. The mark 421 may be located, for example, from the center of the preview frame 400 in the same direction as the direction of the surface on which the user is currently located. The mark 422 indicates the orientation of the surface on which the user is currently located in the moving destination visual field image. For example, the mark 422 may be located in the same direction as the direction of the destination surface from the center of the preview frame 400. As shown in FIG. 6A, the difference information shown in FIG. 8A is a display example of the difference information when the surface 323 is designated as the movement destination. This corresponds to the case where the surface 530 is designated as the movement destination in FIG. 7. Therefore, the direction in which the mark 421 is located and the direction in which the mark 422 is located differ by 90 degrees. FIG. 8A shows an example of displaying difference information when the surface 550 is designated as the movement destination in FIG. 7. Therefore, the direction in which the mark 421 is located and the direction in which the mark 422 is located differ by 180 degrees.

When displaying the moving destination visual field image in the first area in response to the completion of the operation of changing the line-of-sight direction in the moving destination, the second display unit 115 has an effect on the current visual field image displayed in the first area. May be added. For example, the second display unit 115 rotates the current visual field image by an angle formed by the orientation of the surface where the user is currently located and the orientation of the destination surface, and immediately after that, the movement destination visual field image is displayed in the first region. It may be displayed in.

[2-3. Change the line-of-sight direction before preview]
The system control unit 11 may be configured so that the user can specify the initial line-of-sight direction at the moving destination after the user specifies the position of the moving destination and before displaying the moving destination visual field image. In this case, the operation of specifying the moving destination by the user may include an operation of specifying the position of the moving destination and an operation of changing the line-of-sight direction at the moving destination before displaying the moving destination visual field image in the second area. good. Here, the second specifying unit 112 specifies the position of the designated moving destination based on the operation of designating the position of the moving destination. The second specifying unit 112 may display the line-of-sight information indicating the line-of-sight direction at the moving destination in response to specifying the position of the moving destination. The line-of-sight information may be an arrow indicating the line-of-sight direction in, for example, a current visual field image or a map. The initial line-of-sight information may indicate the line-of-sight direction determined as shown in FIG. 7, for example. The second specific unit 112 may update the display of the line-of-sight information based on the operation of changing the line-of-sight direction at the moving destination. For example, the line-of-sight direction may be changed by the user operating the controller 3. In the local coordinate system when the direction of the destination surface is the positive direction of the z-axis, the arrow indicating the line-of-sight direction may be rotatable about the z-axis. Alternatively, the arrow indicating the line-of-sight direction may be rotatable about the z-axis and may be rotatable about the x-axis. The second specifying unit 112 may specify the line-of-sight direction changed based on the user's operation as the line-of-sight direction at the moving destination at the start of displaying the moving-destination visual field image in the second region. FIG. 9 is a diagram showing a display example of line-of-sight information. When the user performs an operation of, for example, determining the position of the moving destination, the second specific unit 112 may display the arrow 140 as the visual field information on the current visual field image 100. The arrow 140 may extend from, for example, the pointer 130. By user operation, the arrow 140 rotates around the pointer 130. When the user performs an operation of determining the direction of the arrow 140, the second specifying unit 112 specifies the direction of the arrow 140 as the line-of-sight direction at the moving destination. The first display unit 113 displays the preview frame 400 by superimposing it on the current visual field image 100, and displays the moving destination visual field image 410 according to the direction of the arrow 140 in the preview frame 400. Even while the preview frame 400 is being displayed, the user can change the line-of-sight direction at the moving destination.

The second specific unit 112 has a line of sight based on a direction in which at least one predetermined object among a plurality of objects that can exist in the virtual space is located from the position of the specified move destination and a line-of-sight direction at the move destination. The display mode of the information may be determined. Then, the second specific unit 112 may display the line-of-sight information in the determined display mode. Examples of display modes of line-of-sight information include transparency, color, pattern, shape, size, and the like. As a result, the user can direct the line-of-sight direction at the moving destination to the predetermined object or move it away from the predetermined object before displaying the moving destination visual field image in the second area. The second specific unit 112 may change the display mode of the displayed line-of-sight information in real time according to the change of the line-of-sight direction at the moving destination by the operation by the user.

The predetermined object may be, for example, a predetermined obstacle. The obstacle may be an object that blocks the user's line of sight or an object that blocks the user's movement. The predetermined obstacle may be, for example, a wall, a pillar, a building, a rock, a tree, or the like. Further, the predetermined obstacle may be the above-mentioned block. When a user stands on a block, the block usually does not interfere with the user. On the other hand, when a block stands in front of the user, the block can be an obstacle to the user. The second specific unit 112 may display the line-of-sight information differently depending on whether a predetermined obstacle blocks or does not block the user's field of view according to the position of the moving destination and the line-of-sight direction at the moving destination. good. As a result, the user knows the direction of the obstacle, and by shaving the line-of-sight direction from the obstacle, the user's field of view can be secured when the display of the moving-destination field-of-view image is started in the second region. The second area on the screen of the HMD2 corresponds to the user's field of view. When an obstacle blocks the visual field, the obstacle may be reflected in the entire visual field or in a region of a predetermined ratio or more of the visual field. The fact that the obstacle does not block the visual field may mean that the obstacle does not appear in the visual field at all, or that the obstacle appears in an area less than a predetermined ratio of the visual field. FIG. 10 is a diagram showing an example of changing the display mode of the line-of-sight information. Note that FIG. 10 shows a state in which the user and its surroundings are viewed from the sky. As shown in FIG. 10, the user 10 has designated the vicinity of the block 340 as the destination position. When the arrow 140 points in the direction of the block 340, the transparency of the arrow 140 increases, and when the arrow 140 deviates from the direction of the block 340, the transparency of the arrow 140 may decrease.

The second specific unit 112 determines the line-of-sight direction indicated by the initial line-of-sight information based on the direction in which at least one predetermined object among the plurality of objects that can exist in the virtual space is located from the specified position of the movement destination. You may decide. Then, the second specific unit 112 may display the line-of-sight information indicating the determined initial line-of-sight direction. As a result, an appropriate direction is determined as the initial line-of-sight direction of the moving destination, so that the amount of the line-of-sight direction changing operation by the user or the number of times of the changing operation can be reduced.

At least one of the predetermined objects may be a predetermined obstacle as described above. In this case, the second specific unit 112 may determine the initial line-of-sight direction so that the user's field of view according to the position of the moving destination and the line-of-sight direction at the moving destination is not blocked by a predetermined obstacle. FIG. 11A is a diagram showing an example of determining the initial line-of-sight direction. As shown in FIG. 11A, the user 10 points near the block 340 in the direction in which the block 340 is located according to the example of determining the line-of-sight direction at the moving destination shown in FIG. It will be. Therefore, the second specific unit 112 may determine the direction of the arrow 140 in a direction different from the direction in which the block 340 is located.

At least one of the predetermined objects may be an object that can be the target of a predetermined action by the user. Examples of predetermined actions include approaching an object, catching an object, defeating an object, talking to an object, observing an object, and the like. The object that can be the target of the predetermined action may be an object in which the user can obtain some benefit by taking the predetermined action. Examples of profits that a user can obtain include earning scores or experience points, earning money or items, earning information, and the situation in which the user is placed is better than the current situation. The second specific unit 112 may determine the direction in which the object that can be the target of the predetermined action is located from the position of the movement destination in the initial line-of-sight direction. FIG. 11B is a diagram showing an example of determining the initial line-of-sight direction. According to the example of determining the line-of-sight direction at the moving destination shown in FIG. 7, the arrow 140 does not face any object. However, there is an enemy 220 near the destination position. An enemy is an example of an object that can be the target of a given action. Generally, in FPS, one of the purposes of the game is for the user to defeat the enemy. The user may be able to earn scores or experience points, or earn money or items from the enemy by defeating the enemy. Therefore, the second specific unit 112 may determine the direction of the arrow 140 in the direction in which the enemy 220 is located. When a plurality of predetermined objects exist, the second specific unit 112 may determine the direction of the object having the smallest change in the line-of-sight direction or the object closest to the position of the movement destination as the initial line-of-sight direction. At this time, the second specific unit 112 may ignore the object hidden by the obstacle or the like.

[2-4. Second area size]
When displaying the moving destination visual field image in the second area, the first display unit 113 determines the size of the second area based on at least one predetermined object among the plurality of objects that can exist in the predetermined space. May be good. Then, the first display unit 113 may display the moving destination visual field image in the second region of the determined size. The area of the current visual field image and the area of the destination visual field image may be determined according to the size of the second region. Specifically, there may be a trade-off between the area of the current visual field image and the area of the destination visual field image. For example, as the size of the second region increases, the area of the destination visual field image increases, and the area of the current visual field image decreases. As the size of the second region becomes smaller, the area of the moving-destination visual field image becomes smaller, and the area of the current visual field image becomes larger. In this way, it is possible to adjust the amount of information that the user can obtain from the current visual field image and the amount of information that the user can obtain from the moving destination visual field image.

The predetermined object used for determining the size of the second region may be a predetermined object included in the moving destination visual field image displayed in the second region. In this case, the size of the second region is determined based on the importance of the moving destination visual field image. As described above, the predetermined object in this case may be an object that can be the target of a predetermined action by the user. Further, the predetermined object may be an object in which the user can obtain some benefit by taking a predetermined action.

The first display unit 113 may determine the size of the second area based on the number of predetermined objects included in the moving destination visual field image displayed in the second area. For example, in the first display unit 113, the larger the number of predetermined objects included in the moving destination visual field image, the larger the size of the second region, and the smaller the number of predetermined objects included in the moving destination visual field image, the second. The size of the area may be reduced. For example, a reference size may be predetermined for the second region. For example, when the number of predetermined objects included in the second region of the reference size is less than the predetermined number, the first display unit 113 may make the size of the second region smaller than the reference size. On the other hand, when the number of predetermined objects included in the second region of the reference size is equal to or larger than the predetermined number, the first display unit 113 may make the size of the second region larger than the reference size. The predetermined object in this case may be a specific type of object that is important to the user, such as an enemy. The greater the number of predetermined objects, the more important the destination visual field image becomes. Further, it is possible to adjust the area of the current visual field image and the area of the moving destination visual field image while including a certain ratio of objects in the moving destination visual field image.

The first display unit 113 may determine the size of the second area based on the distance between the position of the movement destination and the predetermined object included in the movement destination visual field image displayed in the second area. .. For example, the first display unit 113 may increase the size of the second region as the distance is shorter, and may decrease the size of the second region as the distance is longer. The first display unit 113 may determine the size of the second region so that the entire predetermined object or a predetermined ratio or more is displayed in the second region. When a plurality of predetermined objects exist, the first display unit 113 may determine the size of the second region based on the distance from the position of the movement destination to the nearest object. For example, the minimum size of the second region may be preset. The first display unit 113 is a distance between the position of the movement destination and the object when the movement destination visual field image displayed in the second area of the minimum size includes at least a part of a predetermined object. The size of the second region may be determined based on. The closer the user is to a predetermined object, the greater the need for the user to take some action in relation to the object at the destination, thus increasing the importance of the destination visual field image. Further, the closer a predetermined object is to the user, the larger the amount of movement of the object per unit time in the user's field of view tends to be. Even in that case, it is possible to increase the probability that the object is included in the field of view of the moving destination.

The first display unit 113 may determine the size of the second area based on whether or not the moving destination visual field image displayed in the second area includes an object that can be a target of a predetermined action by the user. .. For example, the first display unit 113 sets the size of the second region when the moving destination visual field image includes the object to the size of the second region when the moving destination visual field image does not include the object. May also be increased. For example, the first display unit 113 determines the size of the second region based on whether or not at least a part of a predetermined object is included in the moving destination visual field image displayed in the second region having the smallest size. You may. When a predetermined object exists, it becomes more necessary for the user to take some action in relation to the object at the moving destination, so that the moving destination visual field image becomes more important.

The predetermined object used for determining the size of the second region may be an object that can pose a danger to the user in the predetermined space. Examples of danger to the user include catching the user, defeating the user, impairing the physical strength or health of the user, making the situation of the user worse than the current situation, and the like. The enemies mentioned above are examples of objects that can pose a danger to the user. The first display unit 113 may determine the size of the second area based on the degree of danger approaching the user at the current position by an object that can pose a danger to the user in a predetermined space. For example, the first display unit 113 may increase the size of the second region as the degree of risk increases, and may decrease the size of the second region as the degree of danger decreases. The higher the degree of risk, the greater the amount of information currently obtained from the visual field image. And the danger can be dealt with. Alternatively, the user can complete the teleportation while currently viewing the visual field image before the user himself is at risk. The degree of risk may be determined, for example, based on the number of objects at risk. For example, the first display unit 113 may increase the risk as the number of objects visible from the current position or the number of objects within a predetermined distance from the current position increases. Alternatively, the degree of risk may be determined based on the distance between the current position and the object at risk. For example, the shorter the distance, the higher the risk of the first display unit 113 may be. Alternatively, the degree of risk may be determined based on the strength of the object at risk. For example, when the object seen from the current position is visible or the object is within a predetermined distance from the current position, the first display unit 113 may increase the risk as the object is stronger.

The updating unit 114 may determine and change the size of the second region in real time, for example, while the moving destination visual field image is displayed in the second region.

12 (a) and 12 (b) are diagrams showing an example of determining the size of the second region. 12 (a) and 12 (b) show an example in which the size of the second region is determined based on the distance from the destination position to the predetermined object. In FIG. 12A, the moving destination visual field image 410 displayed in the preview frame 400 includes the enemy 230. Since the distance from the destination position to the enemy 230 is relatively short, the size of the preview frame 400 is relatively large. In FIG. 12B, the moving destination visual field image 410 displayed in the preview frame 400 includes the enemy 240. The distance from the destination position to the enemy 240 is longer than the distance from the destination position to the enemy 230 in FIG. 12 (a). Therefore, the size of the preview frame 400 shown in FIG. 12B is smaller than the size of the preview frame 400 shown in FIG. 12A.

[3. Operation of VR system]
Next, the operation of the VR system S will be described with reference to FIGS. 13 to 15. The system control unit 11 executes the processes shown in FIGS. 13 to 15 according to various program codes stored in the application software stored in the storage unit 13.

FIG. 13 is a flowchart showing an example of application processing by the system control unit 11 of the control device 1 according to the present embodiment. For example, the user activates the application software by operating the controller 3. Then, the user logs in. In response to this, the system control unit 11 may execute the application process.

As shown in FIG. 13, the first specific unit 111 acquires the user's position and line-of-sight direction stored in the user DB as the current location and the current line-of-sight direction (step S101). Next, the first display unit 113 generates a spherical image according to the current position (step S102). For example, the first display unit 113 acquires the parameters of the terrain and the object around the current position based on the terrain DB and the object DB. The first display unit 113 may draw each object and terrain on a spherical surface centered on the current position, for example, by projective transformation based on the parameters. At this time, the first display unit 113 performs hidden surface erasing, shading, texture mapping, and the like. In this way, the first display unit 113 generates the spherical image. Next, the first display unit 1 displays the current visual field image according to the current line-of-sight direction (step S103). For example, the first display unit 113 cuts an image in the clipping region corresponding to the current line-of-sight direction from the spherical image as the current visual field image. At this time, the first display unit 113 rotates the clipping region as necessary so that the orientation of the surface on which the user is currently located coincides with the upward direction in the first region, for example, the upward direction on the screen. Then, the first display unit 113 displays the cut out current visual field image on the screen of the HMD2.

Next, the first specific unit 111 determines whether or not at least one of the movement operation such as walking and the current line-of-sight direction change operation is detected by at least one of the HMD2, the controller 3, and the sensor 4. Determine (step S104). When the operation is detected (step S104: YES), the first specific unit 111 changes the current position and the current line-of-sight direction based on the change operation (step S105). The first specific unit 111 updates the current position and the line-of-sight direction stored in the user DB, and the process proceeds to step S102. In this way, the current visual field image is updated in response to the movement of the user or the change of direction of the user. For convenience of explanation, the current visual field image displayed on the screen is updated only when the current position or the line-of-sight direction is changed. If there is a moving object in the virtual space, it is necessary to reflect the movement of that object in the current field of view image. In that case, the system control unit 11 changes the parameters of the object in real time. The first display unit 113 may update the spherical image and the visual field image based on the latest parameters at time intervals corresponding to the frame rate supported by the HMD2, for example.

On the other hand, when the operation is not detected (step S104: NO), the second specific unit 112 detects an operation of designating the position of the movement destination by teleportation by at least one of the HMD2, the controller 3 and the sensor 4. It is determined whether or not it is (step S106). When the operation is detected (step S106: YES), the system control unit 11 executes the movement destination initial line-of-sight direction determination process (step S107).

FIG. 14 is a flowchart showing an example of the movement destination initial line-of-sight direction determination process by the system control unit 11 of the control device 1 according to the present embodiment. As shown in FIG. 14, the second specifying unit 112 specifies the position of the moving destination (step S201). For example, the second specifying unit 112 may specify the position of the pointer 130 as the position of the moving destination.

Next, the second specific unit 112 determines the initial line-of-sight direction at the move destination, which is indicated by the line-of-sight information, based on the direction in which the predetermined object is located from the position of the move destination (step S202). For example, the second specific unit 112 first rotates the current line-of-sight direction based on the angle formed by the orientation of the surface where the user is currently located and the orientation of the destination surface, and temporarily rotates the current line-of-sight direction at the destination. Determine the line-of-sight direction. Next, the second specific unit 112 may change the line-of-sight direction when a predetermined obstacle blocks the field of view according to the temporary line-of-sight direction to determine the initial line-of-sight direction at the moving destination. .. For example, the second specific unit 112 may generate a spherical image according to the position of the movement destination. The method of generating the spherical image may be the same as in step S102 of the application process. The second specific unit 112 covers a region of a predetermined ratio or more of the visual field according to the tentative line-of-sight direction, for example, based on the information obtained by processing such as projective transformation and hidden surface erasure for generating spherical images. When an obstacle such as a block is included, the line-of-sight direction is determined so that the ratio of the obstacle is less than a predetermined value. Alternatively, the second specific unit 112 may determine the direction in which the object that can be the target of the predetermined action by the user is located from the position of the movement destination as the initial line-of-sight direction at the movement destination. For example, in the second specific unit 112, an object such as an enemy can be seen from the position of the movement destination based on the information obtained by the processing such as the projective transformation and the hidden surface erasure for generating the spherical image. It may be determined whether or not it can be done. When such an object is visible, the second specific unit 112 may calculate the direction of the object as the initial line-of-sight direction. When it is not necessary to change the temporary line-of-sight direction, the second specific unit 112 may determine the temporary line-of-sight direction as the initial line-of-sight direction.

Next, the second specific unit 112 determines the display mode of the line-of-sight information based on the direction in which the predetermined object is located from the position of the move destination and the line-of-sight direction at the move destination (step S203). For example, the second specific unit 112 may determine the transparency of the line-of-sight information to a predetermined transparency when a predetermined obstacle blocks the field of view according to the line-of-sight direction at the moving destination. The second specific unit 112 may determine the transparency of the line-of-sight information to be lower than the predetermined transparency when the field of view according to the line-of-sight direction at the moving destination is not blocked by a predetermined obstacle.

Next, the second specific unit 112 displays the line-of-sight information on the screen of the HMD2 in the determined display mode (step S204). For example, the second specific unit 112 may display an arrow 140 directed in the determined line-of-sight direction from the pointer 130. At this time, the second specific unit 112 may display the arrow 140 with the determined transparency.

Next, the second specific unit 112 determines whether or not the operation of changing the line-of-sight direction at the moving destination is detected by at least one of the HMD2, the controller 3, and the sensor 4 (step S205). When the operation is detected (step S205: YES), the second specific unit 112 updates the display of the line-of-sight information (step S206). For example, the second specific unit 112 may change the display of the arrow 140 so as to face the changed line-of-sight direction. After step S206, the process proceeds to step S203.

On the other hand, when the operation is not detected (step S205: NO), the second specific unit 112 determines whether or not the operation for confirming the line-of-sight direction at the moving destination is detected by at least one of the HMD2, the controller 3, and the sensor 4. Is determined (step S207). If no operation is detected (step S207: NO), the process proceeds to step S205. When the operation is detected (step S207: YES), the second specific unit determines the line-of-sight direction at the moving destination at the start of displaying the moving-destination visual field image to the line-of-sight direction indicated by the current line-of-sight information (step S207: YES). Step S208), the process of determining the initial line-of-sight direction of the moving destination is completed. Returning to FIG. 13, when the moving destination initial line-of-sight direction determination processing is completed, the system control unit 11 executes the moving destination visual field image preview processing (step S108).

FIG. 15 is a flowchart showing an example of the moving destination visual field image preview processing by the system control unit 11 of the control device 1 according to the present embodiment. As shown in FIG. 15, the first display unit 113 generates a spherical image according to the position of the movement destination (step S301). The method of generating the spherical image may be the same as that of step S102, except that the position of the user used for generation is the position of the movement destination.

Next, the first display unit 113 determines the size of the second region (step S302). The first display unit 113 may determine the size of the second area based on a predetermined object included in the moving destination visual field image displayed in the second area. For example, the first display unit 113 corresponds to the line-of-sight direction and the second region at the moving destination based on the information obtained by processing such as projective transformation and hidden surface erasing for generating the spherical image of the moving destination. You may calculate the number of predetermined objects in the field of view. The first display unit 113 may increase the size of the second region as the number of predetermined objects increases. Alternatively, the first display unit 113 may calculate the distance from the position of the movement destination to the line-of-sight direction at the movement destination and a predetermined object in the visual field corresponding to the second region. The first display unit 113 may increase the size of the second region as the calculated distance is shorter. Alternatively, the first display unit 113 may determine whether or not the predetermined object is included in the line-of-sight direction at the moving destination and the visual field corresponding to the second region. When the predetermined object is not included, the first display unit 113 may determine the size of the second region to the predetermined size. When a predetermined object is included, the first display unit 113 may determine the size of the second region to be larger than the predetermined size. Alternatively, the first display unit 113 may calculate the degree of danger approaching the user at the current position. For example, the first display unit 113 may calculate the number of objects that may pose a danger to the user among the objects existing within a predetermined distance from the current position based on the object DB. The first display unit 113 may increase the degree of danger as the number of objects that may pose a danger to the user increases. The size of the second region may be reduced as the risk of the first display unit 113 increases.

Next, the first display unit 113 superimposes and displays the second region on a part of the current visual field image displayed in the first region with the determined size (step S303). For example, the first display unit 113 may display the preview frame 400 as a part of the screen while continuing to display the current visual field image.

Next, the first display unit 113 displays the moving destination visual field image according to the determined line-of-sight direction at the moving destination in the second region (step S304). For example, the first display unit 113 cuts an image in the clipping region corresponding to the line-of-sight direction at the moving destination and the preview frame 400 of the determined size from the moving destination spherical image as the moving destination visual field image. At this time, the first display unit 113 rotates the clipping region as necessary so that the orientation of the destination surface and the upward direction in the preview frame 400 coincide with each other. Then, the first display unit 113 displays the cut-out destination visual field image in the preview frame 400.

Next, the update unit 114 determines whether or not the operation of changing the line-of-sight direction is detected by at least one of the HMD 2, the controller 3, and the sensor 4 (step S305). When the operation is detected (step S305), the update unit 114 changes the current line-of-sight direction and the line-of-sight direction at the moving destination (step S306). For example, the update unit 114 has a first local coordinate system according to the orientation of the surface on which the user is currently located, and a second local coordinate system according to the orientation of the surface to which the user is currently located, based on the user's operation. The current line-of-sight direction and the line-of-sight direction at the destination may be rotated in the same direction and at the same angle. Next, the updating unit 114 displays the changed current visual field image according to the current line-of-sight direction in the first display area (step S307), and the process proceeds to step S304. By repeating steps S304 to S307, the current visual field image displayed in the first area and the moving destination visual field image displayed in the first area are updated according to the operation of changing the line-of-sight direction. For the same reason as described above, the update unit 114 may update the current field of view image and the destination field of view image based on the latest parameters of the object at time intervals according to the frame rate supported by the HMD2. good.

On the other hand, when the operation is not detected (step S305: NO), the update unit 114 determines whether or not the operation for determining the line-of-sight direction is detected by at least one of the HMD 2, the controller 3, and the sensor 4 (step S305: NO). Step S308). If no operation is detected (step S308: NO), the process proceeds to step S305. On the other hand, when the operation is detected (step S308: YES), the moving destination visual field image preview process ends.

Returning to FIG. 13, when the moving destination visual field image preview processing is completed, the second display unit 115 changes the current position and the current line-of-sight direction to the position of the moving destination and the fixed line-of-sight direction at the moving destination (step S109). .. Next, the second display unit 115 erases the current visual field image and the second region from the screen, and displays the moving destination visual field image as a new current visual field image in the first region (step S110). After step S110, the process proceeds to step S102.

If the operation of designating the movement destination is not detected in step S106 (step S106: NO), the system control unit 11 determines whether or not to terminate the virtual reality (step S111). For example, the system control unit 11 may determine that the virtual reality is terminated when the user performs an operation to log out. Alternatively, the system control unit 11 may determine that the virtual reality is terminated when the game is over. If the virtual reality does not end (step S111: NO), the process proceeds to step S102. On the other hand, when the virtual reality ends (step S111: YES), the application process ends.

As described above, according to the present embodiment, the control device 1 specifies the current position of the user and the current line-of-sight direction in the predetermined space. Further, the control device 1 specifies the position of the moving destination and the line-of-sight direction at the moving destination based on the operation of designating the moving destination in the predetermined space. Further, the control device 1 displays the current visual field image corresponding to the specified current position and the current line-of-sight direction in the first region corresponding to the current visual field, and overlaps with a part of the first region. In the area, the moving destination visual field image corresponding to the specified position of the moving destination and the line-of-sight direction at the moving destination is displayed. Further, the control device 1 updates the moving destination visual field image displayed in the second area based on the operation of changing the line-of-sight direction at the moving destination. Further, when the control device 1 completes the operation of changing the line-of-sight direction, the control device 1 displays the moving-destination visual field image according to the specified position of the moving destination and the line-of-sight direction at the changed moving destination in the first region. Let me. Therefore, the user can confirm the image seen at the moving destination while changing the line-of-sight direction at the moving destination before moving to the moving destination. Therefore, it is possible to suppress the occurrence of directional confusion after the movement is instantaneous.

Here, the control device 1 displays the current visual field image according to the orientation of the surface on which the user is currently located and the moving destination visual field image according to the orientation of the moving destination surface, and the surface on which the user is currently located. Difference information indicating the difference between the orientation of and the orientation of the destination surface may be displayed. In this case, the user can move on a surface having a direction different from the direction of the surface on which the user is currently located. Assuming that the user stands on each surface, the user's field of view at the destination is rotated by the difference between the orientation of the surface on which the user is currently located and the orientation of the destination surface. Therefore, the destination visual field image is rotating. Here, since the information indicating the difference between the orientation of the surface on which the user is currently located and the orientation of the surface to which the user is moving is displayed, the user can recognize that the field of view is rotating.

The control device 1 may update the current visual field image displayed in the first region and the moving destination visual field image displayed in the second region based on the operation of changing the line-of-sight direction. In this case, when the moving destination visual field image is displayed in the second area, the current line-of-sight direction is changed according to the operation of changing the current line-of-sight direction, and the line-of-sight direction at the moving destination is changed. Will also be changed. In response to this change, the current visual field image and the moving destination visual field image are updated. Therefore, the user can change the line-of-sight direction without discomfort.

The control device 1 may specify the position of the movement destination based on the operation of designating the position of the movement destination. Further, the control device 1 may display the line-of-sight information indicating the line-of-sight direction at the move destination according to the position of the move destination being specified. Further, the control device 1 may update the display of the line-of-sight information based on the operation of changing the line-of-sight direction at the moving destination before displaying the moving-destination visual field image in the second region. Further, when the operation of changing the line-of-sight direction is completed, the line-of-sight direction at the moving destination changed based on this changing operation is set as the line-of-sight direction at the moving destination at the start of displaying the moving-destination visual field image in the second region. It may be specified. In this case, the user can specify the line-of-sight direction at the moving destination at the start of displaying the moving destination visual field image in the second region.

Here, the control device 1 is based on the direction in which at least one predetermined object among the plurality of objects that can exist in the predetermined space is located from the specified position of the movement destination and the line-of-sight direction at the movement destination. The display mode of the line-of-sight information may be determined. Further, the control device 1 may display the line-of-sight information in the determined display mode. In this case, the line-of-sight information is displayed in a display mode based on the direction in which the predetermined object is located and the line-of-sight direction at the moving destination. Therefore, the user can confirm the existence of the predetermined object in the line-of-sight direction at the moving destination and change or not change the line-of-sight direction.

Here, the control device 1 may display the line-of-sight information differently depending on whether the predetermined obstacle blocks or does not block the visual field according to the position of the moving destination and the line-of-sight direction at the moving destination. .. In this case, the display mode of the line-of-sight information differs depending on whether the field of view at the moving destination is blocked by an obstacle or not. Therefore, the user who sees the line-of-sight information can deviate the line-of-sight direction at the moving destination from the obstacle, if necessary.

Alternatively, the control device 1 may determine the initial line-of-sight direction from the specified movement destination position based on the direction in which at least one predetermined object among the plurality of objects that can exist in the predetermined space is located. .. Further, the control device 1 may display the line-of-sight information indicating the determined initial line-of-sight direction as the line-of-sight direction at the moving destination. In this case, since an appropriate line-of-sight direction is determined in relation to the direction in which the predetermined object is located, the amount or number of changes in the line-of-sight direction by the user can be reduced.

Here, the control device 1 may determine the initial line-of-sight direction so that the field of view corresponding to the position of the moving destination and the initial line-of-sight direction is not obstructed by obstacles. In this case, it is possible to reduce the operation by the user to deviate the line-of-sight direction from a predetermined obstacle.

Alternatively, the control device 1 may determine the direction in which the object that can be the target of the predetermined action is located from the position of the movement destination in the initial line-of-sight direction. In this case, the direction in which the object that can be the target of the predetermined action by the user is located is determined as the initial line-of-sight direction. Therefore, it is possible to reduce the operation of the user looking at the object in preparation for taking a predetermined action.

The control device 1 may determine the size of the second region based on at least one predetermined object among the plurality of objects that can exist in the predetermined space. Further, the control device 1 may display the moving destination visual field image in the second region of the determined size. In this case, when the size of the second area changes, the area of the second area displayed in the second area changes. Further, since the second area overlaps with a part of the first area, when the size of the second area changes, the area of the first area displayed in the first area also changes. For example, as the second region becomes larger, the area of the second region becomes larger and the area of the first region becomes smaller. Therefore, the area of the current visual field image and the area of the destination visual field image can be appropriately adjusted.

Here, the control device 1 may determine the size of the second region based on the object included in the moving destination visual field image displayed in the second region. In this case, the area of the current visual field image and the area of the moving destination visual field image can be adjusted based on the importance of the moving destination visual field image.

Here, the control device 1 may determine the size of the second region based on the number of predetermined objects. In this case, the area of the current visual field image and the area of the moving destination visual field image can be adjusted based on the number of objects included in the moving destination visual field image displayed in the second region.

Alternatively, the control device 1 may determine the size of the second region based on the distance between the destination position and the predetermined object. In this case, the area of the current visual field image and the area of the moving destination visual field image are set based on the distance between the position in the moving destination and the position of the object included in the moving destination visual field image displayed in the second area. Can be adjusted.

Alternatively, the control device 1 may determine the size of the second region based on whether or not the moving destination visual field image displayed in the second region includes an object that can be the target of a predetermined action by the user. good. In this case, the size of the second area is determined based on whether or not the moving destination visual field image displayed in the second area includes an object that can be a target of a predetermined action by the user. Therefore, the area of the current visual field image and the area of the moving destination visual field image can be adjusted based on the possibility that the user takes a predetermined action at the moving destination.

Alternatively, the control device 1 may determine the size of the second region based on the degree of danger approaching the user at the current position by an object capable of endangering the user in a predetermined space. In this case, the area of the current visual field image and the area of the moving destination visual field image can be adjusted according to the need for the user to take some action in response to the currently imminent danger.

The control device 1 may change the current position and the current line-of-sight direction to the specified position of the moving destination and the changed line-of-sight direction at the moving destination, respectively.

1 Control device 2 HMD
3 Controller 4 Sensor 11 System control unit 12 System bus 13 Storage unit 14 Communication unit 15 Input / output interface 111 1st specific unit 112 2nd specific unit 113 1st display unit 114 Update unit 115 2nd display unit S VR system

Claims (19)

On the computer
A first specific step of specifying a user's current first position and current first line-of-sight direction in a predetermined space, wherein the first position is a plurality of surfaces existing in the predetermined space, and the user. The first specific step, which is the position on the first surface of the plurality of surfaces on which the can be located, and
In the first operation of designating a movement destination in the predetermined space , a position on a second surface that is non-parallel to the first surface among the plurality of surfaces can be designated as a second position of the movement destination. based on the first operation, a second specifying step of specifying a second viewing direction in the second position and the destination,
The first image corresponding to the specified first position and the first line-of-sight direction is displayed in the first region corresponding to the current visual field, and in the second region overlapping a part of the first region. In the first display step of displaying the second image corresponding to the specified second position and the second line-of-sight direction, the first image corresponding to the orientation of the first surface and the orientation of the second surface A first display step of displaying the corresponding second video and displaying difference information indicating a difference between the orientation of the first surface and the orientation of the second surface.
An update step of updating the second image displayed in the second area based on the second operation of changing the second line-of-sight direction, and
A second display step of displaying the second image corresponding to the specified second position and the changed second line-of-sight direction in the first area in response to the completion of the second operation.
A program characterized by executing. The second operation is the same as the operation of changing the first line-of-sight direction.
The update step is characterized in that the first image displayed in the first area and the second image displayed in the second area are updated based on the second operation. The program according to claim 1. The first operation includes a third operation of designating the second position and a fourth operation of changing the second line-of-sight direction before displaying the second image in the second region.
The second specific step is
A position specifying step for specifying the second position based on the third operation, and
A line-of-sight information display step for displaying line-of-sight information indicating the second line-of-sight direction according to the identification of the second position, and
Based on the fourth operation, the change step of updating the display of the line-of-sight information and
The second line-of-sight direction changed based on the fourth operation according to the completion of the fourth operation is specified as the second line-of-sight direction at the start of displaying the second image in the second region. The step to specify the direction of the line of sight and
The program according to claim 1 or 2, wherein the program comprises. The second specific step is based on the direction in which at least one predetermined object among the plurality of objects that can exist in the predetermined space is located from the specified second position and the second line-of-sight direction. The display mode determination step for determining the display mode of the line-of-sight information is further included.
The program according to claim 3, wherein the line-of-sight information display step displays the line-of-sight information in the determined display mode. On the computer
The first specific step of specifying the user's current first position and the current first line-of-sight direction in a predetermined space, and
Based on the first operation of designating the movement destination in the predetermined space, the second specific step of specifying the second position of the movement destination and the second line-of-sight direction in the movement destination, and
The first image corresponding to the specified first position and the first line-of-sight direction is displayed in the first region corresponding to the current visual field, and in the second region overlapping a part of the first region. The first display step of displaying the second image corresponding to the specified second position and the second line-of-sight direction, and
An update step of updating the second image displayed in the second area based on the second operation of changing the second line-of-sight direction, and
A second display step of displaying the second image corresponding to the specified second position and the changed second line-of-sight direction in the first area in response to the completion of the second operation.
To run,
The first operation includes a third operation of designating the second position and a fourth operation of changing the second line-of-sight direction before displaying the second image in the second region.
The second specific step is
A position specifying step for specifying the second position based on the third operation, and
The second line-of-sight direction is indicated based on the direction in which at least one predetermined object among the plurality of objects that can exist in the predetermined space is located from the specified second position and the second line-of-sight direction. The display mode determination step for determining the display mode of the line-of-sight information, and
A line-of-sight information display step of displaying the line-of-sight information in the determined display mode according to the identification of the second position, and
Based on the fourth operation, the change step of updating the display of the line-of-sight information and
The second line-of-sight direction changed based on the fourth operation according to the completion of the fourth operation is specified as the second line-of-sight direction at the start of displaying the second image in the second region. The step to specify the direction of the line of sight and
Including
At least one of the predetermined objects is a predetermined obstacle and
The display mode determination step is a program characterized in that the display mode is different depending on whether the obstacle blocks or does not block the visual field according to the second position and the second line-of-sight direction. On the computer
The first specific step of specifying the user's current first position and the current first line-of-sight direction in a predetermined space, and
Based on the first operation of designating the movement destination in the predetermined space, the second specific step of specifying the second position of the movement destination and the second line-of-sight direction in the movement destination, and
The first image corresponding to the specified first position and the first line-of-sight direction is displayed in the first region corresponding to the current visual field, and in the second region overlapping a part of the first region. The first display step of displaying the second image corresponding to the specified second position and the second line-of-sight direction, and
An update step of updating the second image displayed in the second area based on the second operation of changing the second line-of-sight direction, and
A second display step of displaying the second image corresponding to the specified second position and the changed second line-of-sight direction in the first area in response to the completion of the second operation.
To run,
The first operation includes a third operation of designating the second position and a fourth operation of changing the second line-of-sight direction before displaying the second image in the second region.
The second specific step is
A position specifying step for specifying the second position based on the third operation, and
A direction determination step of determining an initial line-of-sight direction based on the direction in which at least one predetermined object among a plurality of objects that can exist in the predetermined space is located from the specified second position.
A line-of-sight information display step for displaying line-of-sight information indicating the determined initial line-of-sight direction as the second line-of-sight direction in response to the identification of the second position.
Based on the fourth operation, the change step of updating the display of the line-of-sight information and
The second line-of-sight direction changed based on the fourth operation according to the completion of the fourth operation is specified as the second line-of-sight direction at the start of displaying the second image in the second region. The step to specify the direction of the line of sight and
A program characterized by including. At least one of the predetermined objects is a predetermined obstacle and
The sixth aspect of claim 6, wherein the direction determination step determines the initial line-of-sight direction so that the obstacle does not block the visual field corresponding to the second position and the initial line-of-sight direction. Program. At least one of the predetermined objects is an object that can be a target of a predetermined action by the user.
The program according to claim 6 or 7, wherein the direction determination step determines a direction in which an object that can be a target of the predetermined action is located from the second position in the initial line-of-sight direction. On the computer
The first specific step of specifying the user's current first position and the current first line-of-sight direction in a predetermined space, and
Based on the first operation of designating the movement destination in the predetermined space, the second specific step of specifying the second position of the movement destination and the second line-of-sight direction in the movement destination, and
The first image corresponding to the specified first position and the first line-of-sight direction is displayed in the first region corresponding to the current visual field, and in the second region overlapping a part of the first region. The first display step of displaying the second image corresponding to the specified second position and the second line-of-sight direction, and
An update step of updating the second image displayed in the second area based on the second operation of changing the second line-of-sight direction, and
A second display step of displaying the second image corresponding to the specified second position and the changed second line-of-sight direction in the first area in response to the completion of the second operation.
A size determination step for determining the size of the second region based on at least one predetermined object included in the second image displayed in the second region among a plurality of objects that can exist in the predetermined space. ,
To run,
The first display step is a program characterized in that the second image is displayed in the second region of the determined size. The program according to claim 9, wherein the sizing step determines the size based on the number of the predetermined objects. The program according to claim 9 or 10, wherein the sizing step determines the size based on a distance between the second position and the predetermined object. On the computer
The first specific step of specifying the user's current first position and the current first line-of-sight direction in a predetermined space, and
Based on the first operation of designating the movement destination in the predetermined space, the second specific step of specifying the second position of the movement destination and the second line-of-sight direction in the movement destination, and
The first image corresponding to the specified first position and the first line-of-sight direction is displayed in the first region corresponding to the current visual field, and in the second region overlapping a part of the first region. The first display step of displaying the second image corresponding to the specified second position and the second line-of-sight direction, and
An update step of updating the second image displayed in the second area based on the second operation of changing the second line-of-sight direction, and
A second display step of displaying the second image corresponding to the specified second position and the changed second line-of-sight direction in the first area in response to the completion of the second operation.
A size determination step for determining the size of the second region based on at least one predetermined object among a plurality of objects that can exist in the predetermined space, and the second is displayed in the second region. A size determination step that determines the size based on whether or not the video contains an object that can be the target of a predetermined action by the user.
With
The first display step is a program characterized in that the second image is displayed in the second region of the determined size. On the computer
The first specific step of specifying the user's current first position and the current first line-of-sight direction in a predetermined space, and
Based on the first operation of designating the movement destination in the predetermined space, the second specific step of specifying the second position of the movement destination and the second line-of-sight direction in the movement destination, and
The first image corresponding to the specified first position and the first line-of-sight direction is displayed in the first region corresponding to the current visual field, and in the second region overlapping a part of the first region. The first display step of displaying the second image corresponding to the specified second position and the second line-of-sight direction, and
An update step of updating the second image displayed in the second area based on the second operation of changing the second line-of-sight direction, and
A second display step of displaying the second image corresponding to the specified second position and the changed second line-of-sight direction in the first area in response to the completion of the second operation.
A sizing step of determining the size of the second region based on at least one predetermined object among a plurality of objects that can exist in the predetermined space, which may pose a danger to the user in the predetermined space. A sizing step that determines the size based on the degree of danger approaching the user in the first position with possible objects.
With
The first display step is a program characterized in that the second image is displayed in the second region of the determined size. The second display step further changes the first position and the first line-of-sight direction to the specified second position and the changed second line-of-sight direction, respectively. The program described in any one of the above. At least one memory to store the program and
With at least one processor operating according to the stored program
In a video display system equipped with
The at least one processor according to the program
A first specific step of specifying a user's current first position and current first line-of-sight direction in a predetermined space, wherein the first position is a plurality of surfaces existing in the predetermined space, and the user. The first specific step, which is the position on the first surface of the plurality of surfaces on which the can be located, and
In the first operation of designating a movement destination in the predetermined space, a position on a second surface that is non-parallel to the first surface among the plurality of surfaces can be designated as a second position of the movement destination. Based on the first operation, the second specific step of specifying the second line-of-sight direction at the second position and the moving destination, and
The first image corresponding to the specified first position and the first line-of-sight direction is displayed in the first region corresponding to the current visual field, and in the second region overlapping a part of the first region. In the first display step of displaying the second image corresponding to the specified second position and the second line-of-sight direction, the first image corresponding to the orientation of the first surface and the orientation of the second surface A first display step of displaying the corresponding second video and displaying difference information indicating a difference between the orientation of the first surface and the orientation of the second surface.
An update step of updating the second image displayed in the second area based on the second operation of changing the second line-of-sight direction, and
A second display step of displaying the second image corresponding to the specified second position and the changed second line-of-sight direction in the first area in response to the completion of the second operation.
A video display system characterized by executing. At least one memory to store the program and
With at least one processor operating according to the stored program
In a video display system equipped with
The at least one processor according to the program
The first specific step of specifying the user's current first position and the current first line-of-sight direction in a predetermined space, and
Based on the first operation of designating the movement destination in the predetermined space, the second specific step of specifying the second position of the movement destination and the second line-of-sight direction in the movement destination, and
The first image corresponding to the specified first position and the first line-of-sight direction is displayed in the first region corresponding to the current visual field, and in the second region overlapping a part of the first region. The first display step of displaying the second image corresponding to the specified second position and the second line-of-sight direction, and
An update step of updating the second image displayed in the second area based on the second operation of changing the second line-of-sight direction, and
A second display step of displaying the second image corresponding to the specified second position and the changed second line-of-sight direction in the first area in response to the completion of the second operation.
And
The first operation includes a third operation of designating the second position and a fourth operation of changing the second line-of-sight direction before displaying the second image in the second region.
The second specific step is
A position specifying step for specifying the second position based on the third operation, and
A direction determination step of determining an initial line-of-sight direction based on the direction in which at least one predetermined object among a plurality of objects that can exist in the predetermined space is located from the specified second position.
A line-of-sight information display step for displaying line-of-sight information indicating the determined initial line-of-sight direction as the second line-of-sight direction in response to the identification of the second position.
Based on the fourth operation, the change step of updating the display of the line-of-sight information and
The second line-of-sight direction changed based on the fourth operation according to the completion of the fourth operation is specified as the second line-of-sight direction at the start of displaying the second image in the second region. The step to specify the direction of the line of sight and
A video display system characterized by including. At least one memory to store the program and
With at least one processor operating according to the stored program
In a video display system equipped with
The at least one processor according to the program
The first specific step of specifying the user's current first position and the current first line-of-sight direction in a predetermined space, and
Based on the first operation of designating the movement destination in the predetermined space, the second specific step of specifying the second position of the movement destination and the second line-of-sight direction in the movement destination, and
The first image corresponding to the specified first position and the first line-of-sight direction is displayed in the first region corresponding to the current visual field, and in the second region overlapping a part of the first region. The first display step of displaying the second image corresponding to the specified second position and the second line-of-sight direction, and
An update step of updating the second image displayed in the second area based on the second operation of changing the second line-of-sight direction, and
A second display step of displaying the second image corresponding to the specified second position and the changed second line-of-sight direction in the first area in response to the completion of the second operation.
A size determination step for determining the size of the second region based on at least one predetermined object included in the second image displayed in the second region among the plurality of objects that can exist in the predetermined space. ,
And
The first display step is a video display system characterized in that the second video is displayed in the second region of the determined size. In the video display method executed by the computer
A first specific step of specifying a user's current first position and current first line-of-sight direction in a predetermined space, wherein the first position is a plurality of surfaces existing in the predetermined space, and the user. The first specific step, which is the position on the first surface of the plurality of surfaces on which the can be located, and
In the first operation of designating a movement destination in the predetermined space, a position on a second surface that is non-parallel to the first surface among the plurality of surfaces can be designated as a second position of the movement destination. Based on the first operation, the second specific step of specifying the second line-of-sight direction at the second position and the moving destination, and
The first image corresponding to the specified first position and the first line-of-sight direction is displayed in the first region corresponding to the current visual field, and in the second region overlapping a part of the first region. In the first display step of displaying the second image corresponding to the specified second position and the second line-of-sight direction, the first image corresponding to the orientation of the first surface and the orientation of the second surface A first display step of displaying the corresponding second video and displaying difference information indicating a difference between the orientation of the first surface and the orientation of the second surface.
An update step of updating the second image displayed in the second area based on the second operation of changing the second line-of-sight direction, and
A second display step of displaying the second image corresponding to the specified second position and the changed second line-of-sight direction in the first area in response to the completion of the second operation.
A video display method comprising. At least one memory to store the program and
With at least one processor operating according to the stored program
In the video display control device equipped with
The at least one processor according to the program
A first specific step of specifying a user's current first position and current first line-of-sight direction in a predetermined space, wherein the first position is a plurality of surfaces existing in the predetermined space, and the user. The first specific step, which is the position on the first surface of the plurality of surfaces on which the can be located, and
In the first operation of designating a movement destination in the predetermined space , a position on a second surface that is non-parallel to the first surface among the plurality of surfaces can be designated as a second position of the movement destination. based on the first operation, a second specifying step of specifying a second viewing direction in the second position and the destination,
The first image corresponding to the specified first position and the first line-of-sight direction is displayed in the first region corresponding to the current visual field, and in the second region overlapping a part of the first region. In the first display step of displaying the second image corresponding to the specified second position and the second line-of-sight direction, the first image corresponding to the orientation of the first surface and the orientation of the second surface A first display step of displaying the corresponding second video and displaying difference information indicating a difference between the orientation of the first surface and the orientation of the second surface.
An update step of updating the second image displayed in the second area based on the second operation of changing the second line-of-sight direction, and
A second display step of displaying the second image corresponding to the specified second position and the changed second line-of-sight direction in the first area in response to the completion of the second operation.
A video display control device characterized by executing.

Images