JP2019061680A - INFORMATION PROCESSING METHOD, INFORMATION PROCESSING PROGRAM, INFORMATION PROCESSING SYSTEM, AND INFORMATION PROCESSING APPARATUS - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing method capable of providing a user with necessary information in a virtual space without discomfort. An information processing method in a system provided with a head-mounted device (HMD) includes a first object associated with a user of the HMD and a moving portion, and the moving portion performs a predetermined operation on the first object. A step of generating virtual space data that defines a virtual space including a second object to be executed, and a field image data indicating a field image V2 displayed on the HMD are generated based on the movement of the HMD and the virtual space data. When a predetermined motion is executed for the first object by the step and the motion portion, the display control of the first state is performed to narrow the field of view of the field image V2 and change the display mode of the field image V2. Including the steps to be taken. [Selection diagram] FIG. 6

Description

  The present disclosure relates to an information processing method, an information processing program, an information processing system, and an information processing apparatus.

  Patent Document 1 discloses a method of changing the field of view in a virtual space in conjunction with the attitude of a head mounted display.

JP, 2017-004357, A

  In order to convey information necessary for the user wearing the head mount device without discomfort, it is important how to transmit the information. For example, there is known a method in which a gauge is always displayed at the edge of a visual field (visual field), but such a display method may be awakening because it is far from reality.

  An object of the present disclosure is to provide an information processing method, an information processing program, an information processing system, and an information processing apparatus capable of providing information required in a virtual space to a user without discomfort.

According to one aspect of the present disclosure, there is provided an information processing method in a system comprising a head mount device, comprising:
The information processing method is
(A) a virtual space including a first object associated with a user of the head mounted device, and a second object having an action part, the action part performing a predetermined action on the first object Generating virtual space data to be defined;
(B) generating view image data indicating a view image to be displayed on the head mounted device based on the movement of the head mounted device and the virtual space data;
(C) When the predetermined action is performed on the first object by the action part, a process of narrowing the view of the view image and changing the display mode of the view image is performed. A step of performing display control;
including.

According to the present disclosure, it is possible to comfortably provide the user with necessary information in the virtual space.

FIG. 1 is a schematic view showing a virtual space distribution system according to an embodiment of the present invention (hereinafter simply referred to as the present embodiment). It is the schematic which shows a user terminal. It is a figure showing a user's head equipped with HMD. It is a figure which shows the hardware constitutions of a control apparatus. It is a figure which shows an example of a concrete structure of an external controller. It is a flowchart which shows the process which displays a visual field image on HMD. It is a xyz space figure which shows an example of virtual space. The state (a) is a yx plan view of the virtual space shown in FIG. The state (b) is a zx plan view of the virtual space shown in FIG. It is a figure which shows an example of the visual field image displayed on HMD. It is a figure which shows the hardware constitutions of the server shown in FIG. A state (a) is a view showing a real space in which a user wearing the HMD exists, and a state (b) is a view showing a virtual space including a virtual camera and various virtual objects. It is a flowchart for demonstrating the information processing method which concerns on this embodiment. It is a figure which shows the visual field image in the state (b) of FIG. It is a figure which shows the virtual space of the state which the virtual object touched the collision object. It is a figure which shows the visual field image in the state of FIG. It is a figure which shows the visual field image after the definite period of time passes the state of FIG. It is a figure which shows a visual field image in case a predetermined parameter of a user becomes below in a predetermined threshold.

[Description of Embodiments Presented by Present Disclosure]
An outline of an embodiment indicated by the present disclosure will be described.
(1) An information processing method in a system provided with a head mount device,
The information processing method is
(A) a virtual space including a first object associated with a user of the head mounted device, and a second object having an action part, the action part performing a predetermined action on the first object Generating virtual space data to be defined;
(B) generating view image data indicating a view image to be displayed on the head mounted device based on the movement of the head mounted device and the virtual space data;
(C) display control in a first state of narrowing the field of view of the view image and making the display mode of the view image different when the predetermined action is performed on the first object by the action portion The steps to be taken,
including.

  According to this method, necessary information in the virtual space can be provided to the user without discomfort. In this method, for example, when the user is attacked by an enemy object in a virtual space, the user is intuitively aware that the user is attacked by narrowing the user's view and making the display mode of the narrowed portion different. And can be recognized without discomfort.

  (2) The step (c) may include the step of hiding the moving part in the view image.

  When the motion part performs a predetermined motion on the first object, if the motion part is displayed in the view image, the inside of the motion part not supposed to be rendered appears in the view, which is not possible. It may be a natural view image. Therefore, in the step (c), it is preferable to hide the active part.

  (3) In the step (c), display control of the first state may be performed by displaying on the view image a first display related to a predetermined parameter associated with the predetermined operation.

  (4) In the step (c), the visual effect of the corner portion of the view image is reduced to narrow the field of view, and the first display is displayed on the center side of the view image rather than the corner portion. You may

  By these methods, it is possible to provide a new user interface for providing the user with necessary information.

  (5) (d) The method may further include the step of performing display control of a second state in which processing for narrowing the field of view is further performed after performing display control of the first state in the step (c).

  This method allows the user to recognize that the parameters (eg, hit points) associated with the user have decreased.

  (6) The display mode of the view image in the first state may be different from the display mode of the view image in the second state.

  This method allows the user to intuitively recognize, for example, a decrease in parameters without compromising the quality of the virtual space experience.

  (7) (e) When the predetermined parameter falls below the threshold, the view of the view image is displayed in a display mode different from the display mode of the view image in the first state and the second state. The method may further include the step of performing display control of a third state for applying narrowing processing.

  By this method, the user can recognize the reduction of the parameters in stages.

  (8) In the step (d), the visibility of the moving part may be higher than the visibility of the parts other than the moving part.

  Even when the field of view is narrowed, the game can be improved by securing the visibility of the moving part.

  (9) In the step (d), the active part may be displayed in color, and the part other than the active part may be displayed in grayscale.

  According to this method, even when the parameter decreases, the visibility of the moving part can be secured.

  (10) The information processing program is an information processing program for causing a computer to execute the information processing method according to any one of the items (1) to (9).

  It is possible to provide an information processing program capable of providing the user with information necessary in the virtual space without discomfort.

(11) An information processing system using a head mount device,
The information processing system is an information processing system configured to execute the information processing method according to any one of items (1) to (9).

  It is possible to provide an information processing system capable of providing the user with information necessary in the virtual space without discomfort.

(12) a processor,
An information processing apparatus comprising: a memory for storing computer readable instructions;
The information processing apparatus is an information processing apparatus that executes the information processing method according to any one of items (1) to (9) when the computer readable instruction is executed by the processor.

  According to the above, it is possible to provide an information processing apparatus capable of providing the user with information necessary in the virtual space without discomfort. It should be noted that the information processing apparatus is either a user terminal or a server.

[Details of Embodiment Presented by Present Disclosure]
Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. For the elements having the same reference numerals as the elements already described in the description of the present embodiment, the description thereof will not be repeated for convenience of the description.

  First, an outline of the configuration of the virtual space distribution system 100 (information processing system) will be described with reference to FIG. FIG. 1 is a schematic view of a virtual space distribution system 100 (hereinafter simply referred to as a distribution system 100). As shown in FIG. 1, the distribution system 100 includes a user terminal 1A (first user terminal) operated by a user A (first user) wearing a head mounted device (HMD) 110 (first HMD), and the HMD 110. A user terminal 1B (second user terminal) operated by a user B (second user) wearing the (second HMD) and the server 2 are provided. The user terminals 1A and 1B are communicably connected to the server 2 via a communication network 3 such as the Internet. Hereinafter, for convenience of description, each of the user terminals 1A and 1B may be simply referred to as the user terminal 1. The users A and B may be simply referred to as the user U. In the present embodiment, the user terminals 1A and 1B are assumed to have the same configuration. The virtual space includes a VR (Virtual Reality) space, an AR (Argumented Reality) space, and an MR (Mixed Reality) space.

  Next, the configuration of the user terminal 1 will be described with reference to FIG. FIG. 2 is a schematic view showing the user terminal 1. As shown in FIG. 2, the user terminal 1 includes a head mounted device (HMD) 110 mounted on the head of the user U, headphones 116, a position sensor 130, an external controller 320, and a control device 120. .

  The HMD 110 includes a display unit 112, an HMD sensor 114, and a gaze sensor 140. The display unit 112 includes a non-transmissive display device configured to completely cover the field of view (field of view) of the user U wearing the HMD 110. Accordingly, the user U can immerse in the virtual space by viewing only the view image displayed on the display unit 112. The display unit 112 may be configured of a display unit for the left eye configured to provide an image to the left eye of the user U, and a display unit for the right eye configured to provide an image to the right eye of the user U . The HMD 110 may include a transmissive display device. In this case, the transmissive display device may be temporarily configured as a non-transmissive display device by adjusting the transmittance thereof.

  The HMD sensor 114 is mounted near the display unit 112 of the HMD 110. The HMD sensor 114 includes at least one of a geomagnetic sensor, an acceleration sensor, and an inclination sensor (such as an angular velocity sensor or a gyro sensor), and detects various movements (such as inclination) of the HMD 110 mounted on the head of the user U be able to.

  The gaze sensor 140 has an eye tracking function of detecting the line of sight of the user U. The gaze sensor 140 may include, for example, a right eye gaze sensor and a left eye gaze sensor. The right eye gaze sensor emits infrared light, for example, to the right eye of the user U, and detects reflected light reflected from the right eye (in particular, the cornea and the iris) to obtain information on the rotation angle of the right eye You may On the other hand, the gaze sensor for the left eye irradiates, for example, infrared light to the left eye of the user U, and detects the reflected light reflected from the left eye (in particular, the cornea and the iris). You may get

  The headphones 116 (audio output unit) are attached to the left ear and the right ear of the user U, respectively. The headphones 116 are configured to receive audio data (electrical signals) from the control device 120, and to output audio based on the received audio data.

  The position sensor 130 is configured by, for example, a position tracking camera, and is configured to detect the position of the HMD 110 and the external controller 320. The position sensor 130 is communicably connected to the control device 120 wirelessly or by wire, and is configured to detect information on the position, inclination or light emission intensity of a plurality of detection points (not shown) provided on the HMD 110. . The position sensor 130 is configured to detect information on the position, the inclination, and / or the light emission intensity of a plurality of detection points (not shown) provided in the external controller 320. The detection point is, for example, a light emitting unit that emits infrared light or visible light. The position sensor 130 may include an infrared sensor and a plurality of optical cameras.

  The external controller 320 is used to control the movement of the hand object displayed in the virtual space by detecting the movement of the hand of the user U. The external controller 320 is an external controller 320R for the right operated by the user U's right hand (hereinafter simply referred to as the controller 320R) and an external controller 320L for the left hand operated by the left hand of the user U (hereinafter referred to simply as the controller 320L). And. The controller 320R is a device that indicates the position of the user U's right hand and the movement of the right hand's finger. The controller 320L is a device that indicates the position of the user U's left hand and the movement of fingers of the left hand. The left hand object and the right hand object existing in the virtual space move in response to the movement of the controllers 320R and 320L.

  Control device 120 is a computer configured to control HMD 110. Control device 120 specifies the position information of HMD 110 based on the information acquired from position sensor 130, and based on the specified position information, the position of the virtual camera in the virtual space (user U in the virtual space And the position of the user U wearing the HMD 110 in the real space can be accurately associated. The controller 120 identifies the operation of the external controller 320 based on the information acquired from the position sensor 130 and / or the sensor incorporated in the external controller 320, and based on the operation of the identified external controller 320, The motion of the hand object displayed in the virtual space can be accurately associated with the motion of the external controller 320 in the real space. In particular, the control device 120 specifies the operation of the controller 320L based on the information acquired from the sensor built in the position sensor 130 and / or the controller 320L, and based on the operation of the specified controller 320L, a virtual The motion of the left hand object displayed in the space can be accurately correlated with the motion of the controller 320L in the real space (the motion of the left hand of the user U). Similarly, the control device 120 specifies the operation of the controller 320R based on the information acquired from the position sensor and / or the sensor incorporated in the controller 320R, and based on the operation of the specified controller 320R, the virtual space The motion of the right hand object displayed in the inside and the motion of the controller 320R in the real space (the motion of the right hand of the user U) can be accurately correlated.

  The control device 120 can respectively identify the line of sight of the right eye and the line of sight of the left eye of the user U, and can specify a gaze point which is an intersection of the line of sight of the right eye and the line of sight of the left eye. The control device 120 can identify the line of sight of the user U (the line of sight of the user U) based on the identified fixation point. The line of sight of the user U is the line of sight of the user U and coincides with the direction of the straight line passing through the midpoint of the line connecting the right eye and the left eye of the user U and the point of gaze. Control device 120 is based on detection data (eye tracking data) transmitted from gaze sensor 140 (the gaze sensor for the left eye and the gaze sensor for the right eye), the center position of the right eye of user U and the left eye of user U And the center position of

  Next, with reference to FIG. 3, a method of acquiring information on the position and tilt of the HMD 110 will be described below. FIG. 3 is a view showing the head of the user U wearing the HMD 110. As shown in FIG. Information on the position and tilt of the HMD 110 interlocked with the movement of the head of the user U wearing the HMD 110 can be detected by the position sensor 130 and / or the HMD sensor 114 mounted on the HMD 110. As shown in FIG. 2, three-dimensional coordinates (uvw coordinates) are defined around the head of the user U wearing the HMD 110. A vertical direction in which the user U stands is defined as a v-axis, a direction perpendicular to the v-axis and passing through the center of the HMD 110 as a w-axis, and a direction orthogonal to the v-axis and the w-axis as a u-axis. The position sensor 130 and / or the HMD sensor 114 are angled about each uvw axis (ie yaw angle indicating rotation about v axis, pitch angle indicating rotation about u axis, w axis) The tilt determined by the roll angle indicating rotation is detected. The control device 120 determines angle information for controlling the visual axis of the virtual camera based on the detected angle change around each uvw axis.

  Next, the hardware configuration of the control device 120 will be described with reference to FIG. FIG. 4 is a diagram showing a hardware configuration of the control device 120. As shown in FIG. As shown in FIG. 4, the control device 120 includes a control unit 121, a storage unit 123, an I / O (input / output) interface 124, a communication interface 125, and a bus 126. The control unit 121, the storage unit 123, the I / O interface 124, and the communication interface 125 are communicably connected to one another via a bus 126.

  The control device 120 may be configured as a personal computer, a smartphone, a fablet, a tablet or a wearable device separately from the HMD 110, or may be built in the HMD 110. Moreover, while the one part function of the control apparatus 120 is mounted in HMD110, the remaining function of the control apparatus 120 may be mounted in another apparatus separate from HMD110.

  The control unit 121 includes a memory and a processor. The memory is configured of, for example, a ROM (Read Only Memory) storing various programs and the like, and a RAM (Random Access Memory) having a plurality of work areas storing various programs and the like executed by the processor. The processor is, for example, a central processing unit (CPU), a micro processing unit (MPU) and / or a graphics processing unit (GPU), and develops a program specified from various programs incorporated in the ROM on the RAM. The system is configured to execute various processes in cooperation with the

  In particular, the control unit 121 may control various operations of the control device 120 by the processor developing the control program on the RAM and executing the control program in cooperation with the RAM. The control unit 121 displays the view image on the display unit 112 of the HMD 110 based on the view image data. Thereby, the user U can immerse in the virtual space.

  The storage unit (storage) 123 is, for example, a storage device such as a hard disk drive (HDD), a solid state drive (SSD), or a USB flash memory, and is configured to store programs and various data. The storage unit 123 may store a control program for causing a computer to execute at least a part of the information processing method according to the present embodiment, or a control program for realizing sharing of a virtual space by a plurality of users. The storage unit 123 may store an authentication program of the user U and data regarding various images and objects (for example, hand objects and the like). In the storage unit 123, a database including tables for managing various data may be constructed.

  The I / O interface 124 is configured to communicably connect the HMD 110, the position sensor 130, the external controller 320, and the headphone 116 to the control device 120. For example, USB (Universal Serial Bus) A terminal, a DVI (Digital Visual Interface) terminal, an HDMI (registered trademark) (High-Definition Multimedia Interface) terminal, and the like are included. The control device 120 may be wirelessly connected to each of the HMD 110, the position sensor 130, the external controller 320, and the headphones 116.

  The communication interface 125 is configured to connect the control device 120 to a communication network 3 such as a Local Area Network (LAN), a Wide Area Network (WAN), or the Internet. The communication interface 125 includes various wired connection terminals for communicating with an external device such as the server 2 via the communication network 3 and various processing circuits for wireless connection, and communicates via the communication network 3. Are configured to meet the communication standards of

  Next, an example of a specific configuration of the external controller 320 will be described with reference to FIG. The external controller 320 detects the movement of a part of the user U's body (a part other than the head, and in this embodiment, the user U's hand) to detect the movement of the hand object displayed in the virtual space. Used to control the The external controller 320 is an external controller 320R for the right operated by the user U's right hand (hereinafter simply referred to as the controller 320R) and an external controller 320L for the left hand operated by the left hand of the user U (hereinafter referred to simply as the controller 320L). And. The controller 320R is a device that indicates the position of the user U's right hand and the movement of the right hand's finger. Further, the right hand object 400R (see FIG. 11) existing in the virtual space moves in accordance with the movement of the controller 320R. The controller 320L is a device that indicates the position of the user U's left hand and the movement of fingers of the left hand. Further, in accordance with the movement of the controller 320L, the left hand object 400L (see FIG. 11) existing in the virtual space is moved. The controller 320R and the controller 320L have substantially the same configuration, so only the specific configuration of the controller 320R will be described below with reference to FIG. In the following description, the controllers 320L and 320R may be simply referred to as the external controller 320 for convenience.

  As shown in FIG. 5, the controller 320R includes an operation button 302, a plurality of detection points 304, a sensor (not shown), and a transceiver (not shown). Only one of the detection point 304 and the sensor may be provided. The operation button 302 is configured of a plurality of button groups configured to receive an operation input from the user U. The operation button 302 includes a push button, a trigger button, and an analog stick. The push-type button is a button operated by an operation of pressing with the thumb. For example, on the top surface 322, two push-type buttons 302a and 302b are provided. The trigger button is a button operated by an operation such as pulling a trigger with the index finger or the middle finger. For example, the front portion of the grip 324 is provided with a trigger button 302e, and the side portion of the grip 324 is provided with a trigger button 302f. The trigger buttons 302 e and 302 f are assumed to be operated by the index finger and the middle finger, respectively. The analog stick is a stick-type button that can be operated by tilting it in any direction 360 degrees from a predetermined neutral position. For example, it is assumed that the analog stick 320i is provided on the top surface 322 and operated using the thumb.

  The controller 320R includes a frame 326 extending from both sides of the grip 324 in the direction opposite to the top surface 322 to form a semicircular ring. A plurality of sensing points 304 are embedded in the outer surface of the frame 326. The plurality of detection points 304 are, for example, a plurality of infrared LEDs aligned in a row along the circumferential direction of the frame 326. After the position sensor 130 detects information on the positions, inclinations, or light emission intensities of the plurality of detection points 304, the control device 120 detects the position or attitude of the controller 320R based on the information detected by the position sensor 130. Get information on

  The sensor of the controller 320R may be, for example, either a magnetic sensor, an angular velocity sensor, or an acceleration sensor, or a combination thereof. When the user U moves the controller 320R, the sensor outputs a signal (for example, a signal indicating information on magnetism, angular velocity, or acceleration) according to the direction or movement of the controller 320R. The control device 120 acquires information on the position and orientation of the controller 320R based on the signal output from the sensor.

  The transceivers of controller 320R are configured to transmit and receive data between controller 320R and controller 120. For example, the transceiver may transmit an operation signal corresponding to the operation input of the user U to the controller 120. The transceiver may also receive an instruction signal from the controller 120 instructing the controller 320R to emit light at the detection point 304. Additionally, the transceiver may transmit a signal to controller 120 indicating the value detected by the sensor.

  Next, processing for displaying a view image on the HMD 110 will be described with reference to FIGS. 6 to 9. FIG. 6 is a flowchart showing a process of displaying a view image on the HMD 110. FIG. 7 is an xyz space diagram showing an example of the virtual space 200. The state (a) of FIG. 8 is a yx plan view of the virtual space 200 shown in FIG. The state (b) of FIG. 8 is a zx plan view of the virtual space 200 shown in FIG. FIG. 9 is a view showing an example of the view image V displayed on the HMD 110. As shown in FIG.

  As shown in FIG. 6, in step S1, the control unit 121 (see FIG. 4) generates virtual space data indicating a virtual space 200 including the virtual camera 300 and various objects. As shown in FIG. 7, the virtual space 200 is defined as an omnidirectional sphere centered on the center position 210 (in FIG. 7, only the upper half celestial sphere is shown). In the virtual space 200, an xyz coordinate system having the center position 210 as an origin is set. The virtual camera 300 defines a viewing axis L for specifying a view image V (see FIG. 9) displayed on the HMD 110. The uvw coordinate system defining the field of view of the virtual camera 300 is determined to interlock with the uvw coordinate system defined around the head of the user U in the real space. The control unit 121 may move the virtual camera 300 in the virtual space 200 in conjunction with the movement of the user U wearing the HMD 110 in the real space.

  Next, in step S2, the control unit 121 specifies the field of view CV (see FIG. 8) of the virtual camera 300. Specifically, based on the data indicating the state of the HMD 110 transmitted from the position sensor 130 and / or the HMD sensor 114, the control unit 121 acquires information on the position and the inclination of the HMD 110. Next, the control unit 121 specifies the position and the orientation of the virtual camera 300 in the virtual space 200 based on the information on the position and the inclination of the HMD 110. Next, the control unit 121 determines the visual axis L of the virtual camera 300 from the position and the orientation of the virtual camera 300, and identifies the visual field CV of the virtual camera 300 from the determined visual axis L. The field of view CV of the virtual camera 300 corresponds to a partial area of the virtual space 200 visible to the user U wearing the HMD 110 (in other words, corresponds to a partial area of the virtual space 200 displayed on the HMD 110 ). The visual field CV is a first area CVa set as an angular range of the polar angle α centered on the visual axis L in the xy plane shown in the state (a) of FIG. 8 and xz shown in the state (b) in FIG. In the plane, it has a second region CVb set as an angular range of the azimuth angle β centered on the visual axis L. The control unit 121 identifies the line of sight of the user U based on the data indicating the line of sight of the user U transmitted from the gaze sensor 140, and based on the identified line of sight of the user U and information regarding the position and inclination of the HMD 110, The orientation of the camera 300 (the visual axis L of the virtual camera) may be determined.

  Thus, the control unit 121 can specify the field of view CV of the virtual camera 300 based on the data from the position sensor 130 and / or the HMD sensor 114. When the user U wearing the HMD 110 moves, the control unit 121 can update the field of view CV of the virtual camera 300 based on the data indicating the movement of the HMD 110 transmitted from the position sensor 130 and / or the HMD sensor 114. . That is, the control unit 121 can update the visual field CV according to the movement of the HMD 110. Similarly, when the line of sight of the user U changes, the control unit 121 may update the field of view CV of the virtual camera 300 based on the data indicating the line of sight of the user U transmitted from the gaze sensor 140. That is, the control unit 121 may change the visual field CV in accordance with the change in the line of sight of the user U.

  Next, in step S3, the control unit 121 generates view image data indicating the view image V displayed on the display unit 112 of the HMD 110. Specifically, the control unit 121 generates view image data based on virtual space data defining the virtual space 200 and the view CV of the virtual camera 300.

  Next, in step S4, the control unit 121 displays the view image V on the display unit 112 of the HMD 110 based on the view image data (see FIG. 8). Thus, according to the movement of the user U wearing the HMD 110, the visual field CV of the virtual camera 300 changes, and the visual field image V displayed on the display unit 112 of the HMD 110 changes. Can be immersed in 200.

  The virtual camera 300 may include a left-eye virtual camera and a right-eye virtual camera. In this case, the control unit 121 generates left-eye view image data indicating a left-eye view image based on the virtual space data and the view of the left-eye virtual camera. Further, the control unit 121 generates right-eye view image data indicating a right-eye view image based on the virtual space data and the view of the right-eye virtual camera. Thereafter, the control unit 121 displays the left-eye view image on the left-eye display unit based on the left-eye view image data, and the right-eye view image on the right-eye display unit based on the right-eye view image data. Display In this manner, the user U can visually recognize the view image three-dimensionally by the parallax between the left-eye view image and the right-eye view image.

  The processes in steps S1 to S4 shown in FIG. 6 may be performed for each frame. For example, when the frame rate of a moving image is 90 fps, the processes of steps S1 to S4 may be repeatedly performed at an interval of ΔT = 1/90 (seconds). As described above, since the processes of steps S1 to S4 are repeatedly executed at predetermined intervals, the visual field V of the virtual camera 300 is updated according to the operation of the HMD 110, and the visual field image V displayed on the display unit 112 of the HMD 110. Is updated.

  Next, the hardware configuration of the server 2 shown in FIG. 1 will be described with reference to FIG. FIG. 10 is a diagram showing a hardware configuration of the server 2. As shown in FIG. 10, the server 2 includes a control unit 23, a storage unit 22, a communication interface 21, and a bus 24. The control unit 23, the storage unit 22, and the communication interface 21 are communicably connected to each other via the bus 24. The control unit 23 includes a memory and a processor, and the memory includes, for example, a ROM and a RAM, and the processor includes, for example, a CPU, an MPU, and / or a GPU.

  The storage unit (storage) 22 is, for example, a large capacity HDD or the like. The storage unit 22 may store a control program for causing a computer to execute at least a part of the information processing method according to the present embodiment, or a control program for realizing sharing of a virtual space by a plurality of users. In addition, the storage unit 22 may store user management information for managing each user, data regarding various images and objects (for example, hand objects and the like). The communication interface 21 is configured to connect the server 2 to the communication network 3.

  Hereinafter, the flow of processing of the virtual space distribution system 100 according to the present embodiment will be described in detail with reference to FIGS. A state (a) of FIG. 11 is a view showing a real space in which the user U wearing the HMD 110 and the external controllers 320R and 320L is present. The state (b) of FIG. 11 is a diagram showing a virtual space 200 including the virtual camera 300 and various virtual objects. FIG. 12 is a flowchart for explaining the information processing method according to the present embodiment. FIG. 13 is a view showing the view image V1 in the state (b) of FIG.

  In the present embodiment, the virtual space 200 is configured as a game space provided by the virtual space distribution system 100. As shown in the state (b) of FIG. 11, the virtual space 200 includes the virtual camera 300, the collision object 350 (an example of the first object), the left hand object 400L, the right hand object 400R, and the enemy object 500 (second example). An example of an object). The control unit 121 generates virtual space data that defines a virtual space 200 including these objects. Further, the control unit 121 may update virtual space data for each frame. As described above, the virtual camera 300 interlocks with the movement of the HMD 110 worn by the user U shown in the state (a) of FIG. That is, the field of view of the virtual camera 300 is updated according to the movement of the HMD 110. The virtual camera 300 has a viewpoint associated with the first person viewpoint of the user U.

  The collision object 350 is an object associated with the user U wearing the HMD 110. The collision object 350 is subjected to collision determination (collision determination) between the user U and another object (for example, the enemy object 500) in the virtual space 200. For example, when the collision object 350 contacts the enemy object 500, it is determined that the user U contacts the enemy object 500. The collision object 350 may be defined by, for example, a sphere having a diameter R centered on the center position of the virtual camera 300. In the following description, the collision object 350 is formed in a spherical shape with a diameter R centered on the center position of the virtual camera 300. The collision object 350 is configured as a transparent body, and is not displayed in the view image V (V1 to V4).

  The left hand object 400L interlocks with the movement of the external controller 320L attached to the left hand of the user U. Similarly, the right hand object 400R interlocks with the movement of the external controller 320R attached to the right hand of the user U. These hand objects 400L and 400R may be provided with predetermined equipment (for example, a weapon object 450 shown in FIG. 13). Although the hand object 400 includes a gun as the weapon object 450 in the example shown in FIG. 13, the type of the weapon object 450 equipped by the hand object 400 can also be changed by the operation of the hand object 400. For example, the user U may equip the hand object 400 with a sword or a bow and arrow instead of a gun by performing a predetermined operation determined according to the weapon object 450 desired to be equipped while holding the controller 320. it can.

  The enemy object 500 is an object that exerts a predetermined influence on the user U, and includes, for example, a weapon object 600 (an example of an operation part). The control unit 121 controls the enemy object 500 so that the enemy object 500 attacks the user U using the weapon object 600, that is, the weapon object 600 performs a predetermined action on the collision object 350. Do. The enemy object 500 and the weapon object 600 each have a collision area. In the present embodiment, it is assumed that the collision areas of the enemy object 500 and the weapon object 600 correspond to the areas (outline areas of the enemy object 500 and the weapon object 600) constituting the enemy object 500 and the weapon object 600, respectively. In the examples shown in FIGS. 11 and 13, the enemy object 500 includes a sword as the weapon object 600. However, the weapon object 600 differs depending on the type and attribute of the enemy object 500. Sometimes they are equipped with guns.

  When the weapon object 600 included in the enemy object 500 contacts the collision object 350, a predetermined influence (collision effect) is given to the user U. Specifically, when the weapon object 600 collides with the collision object 350 due to the attack of the enemy object 500, for example, the user U receives a predetermined amount of damage, and a predetermined parameter associated with the user U, for example, the user U Hit points (hereinafter referred to as HP) decrease.

  The user U can give a predetermined influence to the enemy object 500 by operating the hand object 400. For example, the enemy object 500 can be attacked and damaged by using a weapon object 450 provided in the hand object 400 according to a predetermined action of the user U.

  Hereinafter, the flow of the game progressing process provided by the virtual space distribution system 100 according to the present embodiment will be described in detail with reference to FIGS. 12 to 17. The game according to the present embodiment is a battle game in which the user U wearing the HMD 110 plays a match against the enemy object 500 disposed in the virtual space 200. As described above, the user U can also attack the enemy object 500 using the hand object 400 or the weapon object 450 provided for the hand object 400, but in the present embodiment, the user U attacks from the enemy object 500. The processing in the case of receiving will be mainly described.

  When the battle game is started, in step S10 shown in FIG. 12, the control unit 121 (see FIG. 3) determines the view image data based on virtual space data defining the virtual space 200 and the visual field CV of the virtual camera 300. And display the view image on the display unit 112 of the HMD 110 based on the view image data. Step S10 is the same as step S1 shown in FIG. For example, when the virtual camera 300 is facing the direction of the enemy object 500, the enemy object 500 is displayed in the view image V1 as shown in FIG.

  Next, in step S12, the control unit 121 determines whether the attack from the enemy object 500 has hit the user U, that is, the weapon object 600 included in the enemy object 500 is in the collision area CA defined around the virtual camera 300. It is determined whether or not there is a collision. When it is determined that the weapon object 600 contacts the collision area CA in the virtual space 200 as shown in FIG. 14 (Yes in step S12), the control unit 121 darkens the view image V1 in step S14. . The process in step S14 may be a process of making the display mode of the view image different, for example, a process of reducing the visual effect given to the user U. For example, instead of darkening the view image V1, the view image V1 is blurred. You may process.

  Next, in step S16, the control unit 121 causes the display unit 112 to display the view image V2 shown in FIG. In the view image V2 shown in FIG. 15, an attack display 700 for indicating that the attack from the enemy object 500 has hit the user U is displayed in the center, and an HP display 720 regarding the HP of the user U ( An example of the first display is displayed. Thus, the user U can intuitively grasp that the enemy object 500 has been attacked and how much the user's HP has been reduced by the received attack, and the HP display 720 and the attack display 700 The view is narrowed.

  By the way, when the weapon object 600 collides with the collision area CA and slips through the collision area CA, if the weapon object 600 is kept displayed in the view image V2, it is not supposed to be rendered. The inside of the object 600 appears in the view of the user U, which may result in an unnatural view image. Therefore, in step S14, while the weapon object 600 collides with the collision area CA and slips through the collision area CA, the control unit 121 hides the weapon object 600 on the view image V2 in the view image V2. preferable. Here, as a method of “non-displaying the weapon object 600 on the view image V 2”, the control unit 121 performs a process of controlling the weapon object 600 itself disposed in the virtual space 200 so as not to be displayed. In rendering the view image V2 on the display unit 112 of the HMD 110, the weapon object 600 may not perform rendering.

  Further, as shown in FIG. 15, in step S16, the control unit 121 sets an area of the peripheral portion of the view image V2 (hereinafter referred to as a colored area R2) to the image of the virtual space 200 in the remaining portion R1 of the view. Apply image processing to color in different colors (for example, red). This makes it possible to further narrow the view when the enemy object 500 receives an attack.

  Next, in step S18, the control unit 121 determines whether a predetermined time has elapsed since the start of display of the view image V2 shown in FIG. If it is determined that a predetermined time (for example, 2 to 3 seconds) has elapsed (Yes in step S18), the control unit 121 displays the view image V3 illustrated in FIG. 16 on the display unit 112 in step S20. Do. In step S20, the enemy object 500 is displayed again according to the direction of the virtual camera 300, and the portion of the view image V3 excluding the weapon object 600 is displayed in gray scale, thereby narrowing the visual field (reducing the visual effect ). Note that blurring processing may be performed not on gray scale but on a portion of the view image V3 excluding the weapon object 600. Thus, after displaying the view image V2 of the state (first state) in step S16, the control unit 121 continues the state from the first state in step S20 to narrow the field of view of the user U (second ) Is displayed on the display unit 112. Thereby, the user U can easily recognize that his own HP has decreased.

  In addition, after receiving the attack from the enemy object 500, if the display of the view image V2 including the attack display 700 and the HP display 720 is continued, the user U can not visually recognize another attack from the enemy object 500. , There is a risk that the game nature will be lost. Therefore, in the present embodiment, after the view image V2 (the view image of the first state) is displayed for a predetermined time, the view image V3 (the view image of the second state) different from the display mode of the view image V2 is displayed on the display unit 112 It is displayed on. That is, in the view image V3 in the second state, the attack display 700 and the HP display 720 are not displayed, and the view image V3 is displayed in gray scale, for example, without impairing the game performance.

  In the view image V3, as with the view image V1, only the weapon object 600 preferably has a display mode not subjected to gray scale or blurring, that is, a color display. That is, it is preferable that the visibility of the weapon object 600 in the view image V3 be higher than the visibility of the portion other than the weapon object 600 by displaying only the weapon object 600 in color. As a result, even if the HP of the user U is reduced and the field of view is narrowed, the visibility of the weapon object 600 which is the attacking part is maintained, so that the game can be improved.

  Next, in step S22, the control unit 121 determines whether the HP of the user U has become equal to or less than a predetermined threshold. If it is determined that the user U's HP has become equal to or less than the predetermined threshold (Yes in step S22), the control unit 121 determines in step S24 the view image V4 (view image of the third state) shown in FIG. It is displayed on the display unit 112. In this step S24, in addition to displaying the view image V3 in grayscale, the coloring region R2 which is the peripheral portion of the view image V4 has a color different from that of the center of the view image V4, eg, gray in the center It is colored in a darker color (dark gray, black, etc.). Thereby, the view of the user U is further narrowed in the view image V4. Thus, when the HP of the user U falls below the predetermined threshold, the view of the user U is different from the display mode of the view image V2 (first state) or the view image V3 (second state). By being narrowed, the user U can recognize the decrease of HP in stages. Note that as the HP of the user U decreases, for example, the inner side of the colored area R2 gradually approaches the center side of the view image V4 (the range in which the enemy object 500 can be visually recognized gradually decreases), The view may be narrowed gradually.

  Next, in step S26, the control unit 121 determines whether the HP of the user U has become zero. If it is determined that the user U's HP has become zero (Yes in step S26), the control unit 121 ends the battle game in step S28, and the battle is over (the user U loses). The user U is notified (for example, displayed on the display unit 112), and the series of processing ends. If the HP held by the enemy object 500 becomes zero before the HP of the user U becomes zero, the control unit 121 ends the battle game, and the display unit 112 indicates that the user U has won. Can be displayed.

  As described above, when an attack is performed on the collision area CA of the virtual camera 300 by the weapon object 600 included in the enemy object 500, for example, the control unit 121 causes the HP display 720 or the attack to narrow the view. A process of displaying the display 700 in the view image V2 and changing the display mode of the view image V2 with respect to the remaining portion R1 of the narrowed view (including the colored region R2), for example, reducing the visual effect given to the user U Processing (for example, darkening processing, coloring / blurring processing) is performed. As described above, when the user U is attacked by the enemy object 500, the visual effect is reduced by making the view narrow by the HP display 720 or the like and making the display mode of the remaining portion R1 of the narrowed view different. The user U can intuitively and unambiguously recognize that an attack has been made. That is, according to this method, it is possible to comfortably provide the user U with necessary information in the virtual space 200.

  In the above embodiment, the enemy object 500 disposed in the virtual space 200 attacks the user U using the weapon object 600, but the present invention is not limited to this example. For example, the enemy object 500 may attack the user U with a part of the body of the enemy object 500 such as a bare hand or a foot without using the weapon object 600. In this case, the attacking part (the hand or foot of the enemy object 500) for attacking the user U is the moving part of the enemy object 500.

  In the above embodiment, after the field image V1 is darkened in step S14, the HP display 720 and the colored region R2 are displayed on the field image V2 in step S16 to narrow the field of view. It is not limited to the example. For example, when the weapon object 600 collides with the collision object 350, instead of darkening the view image V1, the visual effect of the HP display 720 of the view image V2 and the portion (remaining portion R1) other than the coloring region R2 is reduced After that, the HP display 720 or the colored area R2 may be displayed. That is, the display mode of the entire view image may be changed to narrow the view after the visual effect is reduced, and after narrowing the view, the display mode may be changed for the narrowed view portion. Effects may be reduced. Furthermore, when the weapon object 600 collides with the collision object 350, processing such as vibrating the field of view may be performed.

  Further, the colored region R2 displayed in step S16 or step S24 does not have to be provided at all the corners of the view image, and at least a part of the corners of the view image (for example, the left end portions of the view images V2 and V4) It may be provided only in Also, in step S16, for example, only the HP display 720 may be displayed (that is, the attack display 700 and the colored area R2 may or may not be displayed).

  In the above embodiment, the collision object 350 is defined on the basis of the virtual camera 300 by taking a first-person game as an example, but the present invention is not limited to this example. For example, in the case of a third person viewpoint game, an avatar object associated with the user U may be disposed in the virtual space 200 at a position where it can be viewed from the virtual camera 300. In this case, the collision object may coincide with the outer diameter area of the avatar object, or may be defined as a certain range surrounding the avatar object. When the weapon object 600 collides with the collision object associated with the avatar object due to the attack of the enemy object 500, the control unit 121 causes the process of step S12 and subsequent steps in FIG. 12 to proceed.

  Although the embodiments of the present disclosure have been described above, the technical scope of the present invention should not be construed as limited by the description of the present embodiments. It is understood by those skilled in the art that the present embodiment is an example, and that modifications of various embodiments are possible within the scope of the invention described in the claims. The technical scope of the present invention should be determined based on the scope of the invention described in the claims and the scope of equivalents thereof.

  Although the description of the present embodiment assumes that virtual space data indicating the virtual space 200 is updated on the user terminal 1 side, the virtual space data may be updated on the server 2 side. Furthermore, although it is premised that the view image data corresponding to the view image is updated on the user terminal 1 side, the view image data may be updated on the server 2 side. In this case, the user terminal 1 displays the view image on the HMD 110 based on the view image data transmitted from the server 2.

  Further, even if a control program for causing a computer (processor) to execute various processes is incorporated in advance in the storage unit 123 or the memory in order to realize various processes executed by the control unit 121 of the user terminal 1 by software. Good. Alternatively, the control program may be a magnetic disk (HDD, floppy disk), an optical disk (CD-ROM, DVD-ROM, Blu-ray disk, etc.), a magneto-optical disk (MO etc.), a flash memory (SD card, USB memory, SSD) Etc.) may be stored in a computer readable storage medium. In this case, by connecting the storage medium to the control device 120, the control program stored in the storage medium is incorporated in the storage unit 123. Then, the control program incorporated in the storage unit 123 is loaded onto the RAM, and the processor executes the loaded program, whereby the control unit 121 executes various processing.

  Also, the control program may be downloaded from a computer on the communication network 3 via the communication interface 125. Also in this case, the downloaded control program is incorporated in the storage unit 123.

  In this embodiment, the virtual space is used to provide the user with a virtual experience such as VR (Virtual Reality), AR (Argumented Reality) and MR (Mixed Reality). When the virtual space provides a VR, background data stored in memory is used as the background of the virtual space. When virtual space provides AR or MR, real space is used as the background. In this case, real space can be used as a background by providing the HMD 110 with a transmissive display (optical see-through or video see-through display). If a virtual space is applied to the MR, the object may be influenced by the real space. In this manner, the virtual space includes at least a virtual scene such as a background or a virtual object, thereby providing the user with a virtual experience capable of interacting with the virtual scene.

1, 1A, 1B: user terminal 2: server 3: communication network 21: communication interface 22: storage unit 23: control unit 24: bus 100: virtual space distribution system 110: head mounted device (HMD)
112: display unit 114: HMD sensor 116: headphones 118: microphone 120: control device 121: control unit 123: storage unit 124: I / O interface 125: communication interface 126: bus 130: position sensor 140: gaze sensor 200, 200A , 200B: virtual space 210: center position 300: virtual camera 320: external controller 320L: external controller (controller) for left hand
320R: Right-handed external controller (controller)
350: Collision object (an example of the first object)
400L, 400R: Hand object 450: Weapon object 500: Enemy object (an example of the second object)
600: Weapon object (an example of an action part)
700: Attack indication 720: HP indication (an example of the first indication)
R1: Remaining part R2: coloring area V, V1 to V4: view image

Claims (12)

An information processing method in a system comprising a head mounted device, comprising:
(A) a virtual space including a first object associated with a user of the head mounted device, and a second object having an action part, the action part performing a predetermined action on the first object Generating virtual space data to be defined;
(B) generating view image data indicating a view image to be displayed on the head mounted device based on the movement of the head mounted device and the virtual space data;
(C) When the predetermined action is performed on the first object by the action part, a process of narrowing the view of the view image and changing the display mode of the view image is performed. A step of performing display control;
Information processing methods, including:   The information processing method according to claim 1, wherein the step (c) includes the step of hiding the motion part in the view image.   The information according to claim 1 or 2, wherein in the step (c), display control of the first state is performed by displaying on the view image a first display regarding a predetermined parameter associated with the predetermined operation. Processing method.   In the step (c), the visual effect of the corner of the view image is reduced to narrow the field of view, and the first display is displayed on the center side of the view image rather than the corner. An information processing method according to Item 3.   (D) After the display control of the first state in the step (c) is performed, the step of performing the display control of the second state in which the process of narrowing the field of view is further performed is further included. The information processing method according to any one of the above.   The information processing method according to claim 5, wherein a display mode of the view image in the first state and a display mode of the view image in the second state are different.   (E) A process of narrowing the field of view of the visual field image in a display manner different from the display manner of the visual field image in the first state and the second state when the predetermined parameter falls below a threshold. 7. The information processing method according to claim 6, further comprising the step of performing display control of a third state of applying.   The information processing method according to any one of claims 5 to 7, wherein in the step (d), the visibility of the moving part is higher than the visibility of parts other than the moving part.   The information processing method according to claim 8, wherein in the step (d), the operation part is displayed in color, and the part other than the operation part is displayed in grayscale.   An information processing program for causing a computer to execute the information processing method according to any one of claims 1 to 9.   An information processing system using a head mounted device, wherein the information processing system is configured to execute the information processing method according to any one of claims 1 to 9. A processor,
An information processing apparatus comprising: a memory for storing computer readable instructions;
An information processing apparatus, wherein the information processing apparatus executes the information processing method according to any one of claims 1 to 9 when the computer readable instruction is executed by the processor.

Images