WO2026004381A1 - Information processing system and information processing method - Google Patents

Abstract

[Problem] To improve the review efficiency of a 3D virtual model using an HMD. [Solution] An information processing system comprising a control circuit for controlling review of a 3D virtual model on the basis of review information that records a predetermined positional relationship between a viewpoint of a user wearing an HMD and the 3D virtual model displayed by the HMD, wherein: the review information includes at least a positional relationship between a first viewpoint and the 3D virtual model and a positional relationship between a second viewpoint different from the first viewpoint and the 3D virtual model; and the control circuit starts the review at the first viewpoint on the basis of the review information, and transitions to the review at the second viewpoint when a trigger is detected.

Description

Information processing system and information processing method

This disclosure relates to an information processing system and an information processing method.

In recent years, technologies have been developed to support the review of designs such as 3D (three-dimensional) virtual models. For example, Patent Document 1 discloses technology that enables a walkthrough of a building using CG images.

Japanese Patent Application Laid-Open No. 2000-113212

However, the technology disclosed in Patent Document 1 may have difficulty providing sufficient support for changes in the user's viewpoint.

According to one aspect of the present disclosure, there is provided an information processing system including a control circuit that controls a review of a 3D virtual model based on review information that records a predetermined positional relationship between a viewpoint of a user wearing an HMD and a 3D virtual model displayed by the HMD, the review information including at least a positional relationship between a first viewpoint and the 3D virtual model, and a positional relationship between a second viewpoint different from the first viewpoint and the 3D virtual model, the control circuit starting the review from the first viewpoint based on the review information, and transitioning to the review from the second viewpoint when a trigger is detected.

Furthermore, according to another aspect of the present disclosure, there is provided an information processing method including a processor controlling a review of the 3D virtual model based on review information that records a predetermined positional relationship between a viewpoint of a user wearing an HMD and a 3D virtual model displayed by the HMD, the review information including at least a positional relationship between a first viewpoint and the 3D virtual model, and a positional relationship between a second viewpoint different from the first viewpoint and the 3D virtual model, and the controlling includes starting the review from the first viewpoint based on the review information, and transitioning to the review from the second viewpoint when a trigger is detected.

FIG. 1 is a diagram for explaining an overview of an embodiment of the present disclosure. FIG. 2 is a block diagram illustrating an example of a functional configuration of an information processing system 1 according to the embodiment. 10A and 10B are diagrams for explaining a transition of a review triggered by a user's operation on a 3D virtual model 50 according to the embodiment. 10A and 10B are diagrams for explaining an example of guidance control by a control unit 110 according to the embodiment. 10A and 10B are diagrams for explaining an example of guidance control by a control unit 110 according to the embodiment. 10A and 10B are diagrams illustrating saving and restoring of a viewpoint according to the embodiment. 10A and 10B are diagrams for explaining an editing operation that does not involve operation of the editing device 20 according to the embodiment. 10A and 10B are diagrams for explaining an editing operation that does not involve operation of the editing device 20 according to the embodiment. 10A and 10B are diagrams for explaining an editing operation that does not involve operation of the editing device 20 according to the embodiment. 10A and 10B are diagrams for explaining an editing operation that does not involve operation of the editing device 20 according to the embodiment. 10 is a sequence diagram showing an example of a flow of viewpoint adjustment based on acquired information according to the embodiment. FIG. FIG. 10 is a sequence diagram showing an example of a flow of review information update based on acquired information according to the embodiment. 10A and 10B are diagrams for explaining review control including interaction between a 3D virtual model 50 according to the embodiment and an object in the real world. 10 is a diagram for explaining a review in a case where the information processing system 1 according to the embodiment includes a plurality of HMDs 10 each worn by a plurality of users. FIG. FIG. 2 is a block diagram showing an example of a hardware configuration of an information processing device 90 according to the embodiment.

Preferred embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. Note that in this specification and drawings, components having substantially the same functional configuration will be assigned the same reference numerals, and redundant explanations will be omitted.

Furthermore, in this specification and drawings, when multiple identical components are described and differentiated, letters or other characters may be added to the end of the reference numeral. On the other hand, when there is no need to differentiate between multiple identical components, the letters or other characters may be omitted and a description common to all multiple identical components may be given.

The explanation will be given in the following order.
1. Embodiment 1.1. Overview 1.2. Functional Configuration Example 1.3. Functional Details 2. Hardware Configuration Example 3. Summary

1. Embodiment
<<1.1. Overview>>
First, an outline of one embodiment of the present disclosure will be described.

In recent years, reviews have been conducted of 3D virtual models designed using HMDs (Head Mounted Displays).

A user wearing an HMD can, for example, display a designed 3D virtual model on the HMD, review the 3D virtual model from various viewpoints, and edit (modify) the 3D virtual model.

However, design work for 3D virtual models, including editing, is generally carried out using a PC (Personal Computer) or similar device.

As a result, when repeatedly reviewing and editing a 3D virtual model, the user is required to alternate between the HMD and a PC, etc.

Furthermore, because reviews using an HMD can take a variety of viewpoints, the user is required to adjust the viewpoint each time to ensure the same viewpoint is maintained before and after editing.

Furthermore, even when users want to perform minor editing tasks, they may forgo such tasks to avoid the above-mentioned hassle, which reduces the efficiency of reviews.

The technical concept of one embodiment of the present disclosure was conceived with the above points in mind, and aims to improve the efficiency of reviewing 3D virtual models using an HMD.

For this reason, an information processing method according to one embodiment of the present disclosure includes a processor controlling the review of the 3D virtual model 50 based on review information that records a predetermined positional relationship between the viewpoint of the user wearing the HMD 10 (see FIG. 2) and the 3D virtual model 50 (see FIG. 1) displayed by the HMD 10.

Furthermore, the review information includes at least the positional relationship between a first viewpoint and the 3D virtual model 50, and the positional relationship between a second viewpoint different from the first viewpoint and the 3D virtual model 50.

Furthermore, the above control initiates a review from a first perspective based on the review information, and when a trigger is detected, transitions to a review from a second perspective.

Below, an overview of one embodiment of the present disclosure will be described with reference to Figure 1.

Figure 1 shows an example in which the review information records the positional relationships between viewpoint A, viewpoint B, and viewpoint C and the 3D virtual model 50.

Furthermore, in the example shown in Figure 1, the review information records at least the context of viewpoints A, B, and C.

Here, we assume that a series of review sequences for sequentially performing a review from viewpoint A, a review from viewpoint B, and a review from viewpoint C is recorded in the review information.

In this case, in the information processing method according to this embodiment, a review from viewpoint A is first started based on the review information, as shown in the first row of Figure 1.

The user wearing the HMD 10 can pause the review as needed, edit the 3D virtual model 50 using a PC or the like, and then put the HMD 10 back on and resume the review.

In this case, in the information processing method according to this embodiment, the review is resumed from the same viewpoint A as before the pause, as shown in the second row of Figure 1.

In this way, with the information processing method according to this embodiment, when a review is paused, the viewpoint at the time of the pause is saved, and when the review is resumed, the review is resumed from the saved viewpoint.

The above control eliminates the need for users to adjust the viewpoint each time after editing, greatly improving review efficiency.

Furthermore, in the information processing method according to this embodiment, when a trigger is detected in a review, a transition to a review from the next perspective is performed based on the review information.

For example, if a trigger is detected in the situation shown in the second row of Figure 1, a transition from review at viewpoint A to review at viewpoint B is performed based on the trigger information, as shown in the third row of Figure 1.

In this case, viewpoint A is an example of a first viewpoint, and viewpoint B is an example of a second viewpoint.

Furthermore, for example, if a trigger is detected in the situation shown in the third row of Figure 1, a transition from review at viewpoint B to review at viewpoint C is performed based on the trigger information, as shown in the fourth row of Figure 1.

In this case, viewpoint B is an example of a first viewpoint, and viewpoint C is an example of a second viewpoint.

The above control eliminates the need for users to adjust the viewpoint each time to match a specified scenario, greatly improving review efficiency.

The above provides an overview of the information processing method according to this embodiment. Below, we will explain an example of the functional configuration of an information processing system 1 that implements the information processing method according to this embodiment.

<<1.2. Example of functional configuration>>
FIG. 2 is a block diagram showing an example of the functional configuration of the information processing system 1 according to this embodiment.

As shown in FIG. 2, the information processing system 1 according to this embodiment includes, for example, an HMD 10 and an editing device 20.

The HMD 10 and editing device 20 communicate information via a wireless or wired network 30.

(HMD 10)
The HMD 10 according to this embodiment is an example of a control device that controls the review of the 3D virtual model 50 .

As shown in FIG. 1, the HMD 10 according to this embodiment may include a control unit 110, a display unit 120, an audio output unit 130, an imaging unit 140, a sensor unit 150, an audio input unit 160, a memory unit 170, and a communication unit 180.

(Control unit 110)
The control unit 110 according to this embodiment is an example of a control circuit that controls each component included in the HMD 10 .

In addition, the control unit 110 according to this embodiment controls the review of the 3D virtual model 50 based on review information that records a predetermined positional relationship between the viewpoint of the user wearing the HMD 10 and the 3D virtual model 50 displayed by the display unit 120.

In particular, the control unit 110 according to this embodiment starts a review from a first perspective based on the review information, and when a trigger is detected, performs control to transition to a review from a second perspective.

Note that in this embodiment, the viewpoint refers to the base point.

The functions of the control unit 110 in this embodiment are realized through cooperation between various processors and memories.

The functions of the control unit 110 according to this embodiment will be described in detail later.

(Display section 120)
The display unit 120 according to this embodiment displays information under the control of the control unit 110 .

The display unit 120 according to this embodiment displays various objects including, for example, a 3D virtual model 50.

For this reason, the display unit 120 according to this embodiment is equipped with various displays.

The above displays include see-through displays.

(Audio output unit 130)
The audio output unit 130 according to this embodiment outputs audio under the control of the control unit 110 .

For this reason, the audio output unit 130 in this embodiment is equipped with an amplifier, speaker, etc.

(Photographing unit 140)
The image capturing section 140 according to this embodiment captures an image from the user's viewpoint under the control of the control section 110 .

For this reason, the imaging unit 140 according to this embodiment is equipped with an imaging element positioned to correspond to the user's viewpoint.

(Sensor unit 150)
The sensor unit 150 according to this embodiment acquires various types of sensor information under the control of the control unit 110 .

For this reason, the sensor unit 150 according to this embodiment is equipped with sensors that correspond to the sensor information to be acquired.

Sensors included in the sensor unit 150 include, for example, an acceleration sensor, a gyro sensor, a geomagnetic sensor, an illuminance sensor, a temperature sensor, and radar.

(Audio Input Unit 160)
The voice input unit 160 according to this embodiment collects voices around the HMD 10 under the control of the control unit 110 .

For this reason, the audio input unit 160 according to this embodiment is equipped with at least one microphone.

(Storage unit 170)
The storage unit 170 according to this embodiment stores various types of information used in the HMD 10 .

In particular, the storage unit 170 stores review information.

(Communication unit 180)
The communication unit 180 according to this embodiment communicates information with other devices via the network 30 under the control of the control unit 110 .

The other devices include, for example, an editing device 20.

(Editing device 20)
The editing device 20 according to this embodiment is a computer such as a PC used to edit the 3D virtual model 50 .

Editing software 210 used to edit the 3D virtual model 50 is installed on the editing device 20 according to this embodiment.

The above describes an example of the functional configuration of the information processing system 1 according to this embodiment. Note that the functional configuration described above with reference to Figure 2 is merely an example, and the functional configuration of the information processing system 1 according to this embodiment is not limited to this example.

For example, the control unit that controls the review of the 3D virtual model 50 based on review information that records a predetermined positional relationship between the viewpoint of the user wearing the HMD 10 and the 3D virtual model 50 displayed by the display unit 120 does not necessarily have to be provided in the HMD 10.

The control unit may be provided, for example, in the editing device 20 or a server (not shown). In this case, the editing device 20 or a server (not shown) equipped with the control unit controls the review using the HMD 10 via the network 30.

Furthermore, for example, as described below, editing the 3D virtual model 50 does not necessarily require operations using an editing device. For this reason, the editing software 210 may be installed on the HMD 10.

Furthermore, for example, the information processing system 1 may further include a controller that is held or worn by the user. In this case, information acquired by the controller is transmitted to the HMD 10 via the network 30. Note that the controller may be, for example, a grip-type controller that is held by the user, or a ring-type controller that is worn on the user's finger or wrist.

The functional configuration of the information processing system 1 according to this embodiment can be flexibly modified depending on specifications, operations, etc.

<<1.3. Function details>>
Next, the functions of the information processing system 1 according to this embodiment will be described in detail.

First, we will describe the detection of a trigger in this embodiment and the transition to a review from a second perspective based on the detected trigger.

As described above, the control unit 110 according to this embodiment starts a review from a first perspective based on the review information, and when a trigger is detected, performs control to transition to a review from a second perspective.

The trigger according to this embodiment may include, for example, the passage of a predetermined amount of time.

In the example shown in Figure 1, the control unit 110 may detect that a predetermined time has elapsed after starting a review at viewpoint A, and control the transition to a review at viewpoint B.

Similarly, in the example shown in Figure 1, the control unit 110 may detect that a predetermined time has elapsed after the transition to review at viewpoint B, and control the transition to review at viewpoint C.

In the above case, the predetermined time for the transition from viewpoint A to viewpoint B may be different from the predetermined time for the transition from viewpoint B to viewpoint C. Each predetermined time is recorded in the review information.

Furthermore, the trigger according to this embodiment may include, for example, an explicit instruction from the user.

The user may be able to instruct the transition of the review by, for example, operating buttons or the like provided on the HMD 10, operating the controller described above, using gestures with their hands or the like, or by voice, etc.

The control unit 110 can detect gestures such as those described above based on images captured by the imaging unit 140.

The control unit 110 can also detect instructions given by the user's voice based on the results of voice recognition of the user's voice collected by the voice input unit 160. Note that voice recognition may be performed by the HMD 10, or by a device other than the HMD 10, such as a server.

Furthermore, triggers according to this embodiment include, for example, a predetermined interaction between the user and the 3D virtual model 50.

The above-mentioned predetermined interaction may be, for example, a user operation on the 3D virtual model 50.

Figure 3 is a diagram illustrating the review transition triggered by a user's operation on the 3D virtual model 50 according to this embodiment.

In the example shown in Figure 3, the user uses his or her hand H to perform operations on the 3D virtual model 50.

The above operation may be, for example, a rotation operation that rotates the 3D virtual model 50.

The control unit 110 according to this embodiment uses the rotation operation described above as a trigger to transition to review from the second viewpoint.

Generally, adjusting the amount of rotation, angle, etc., when rotating a 3D virtual model 50 is difficult. However, with the information processing method according to this embodiment, the first viewpoint and second viewpoint are determined in advance, making it possible to rotate the 3D virtual model 50 easily and smoothly without requiring delicate rotation operations.

The control unit 110 can detect operations on the 3D virtual model 50 based on images captured by the imaging unit 140.

Operations on the 3D virtual model 50 may be performed using a controller held by or attached to the user's hand H.

Furthermore, operations on the 3D virtual model 50 may be performed by voice.

Furthermore, the above-mentioned specified interaction may be, for example, the viewpoint of the user wearing the HMD 10 substantially matching the second viewpoint.

In this case, the control unit 110 according to this embodiment may perform control to guide the viewpoint of the user wearing the HMD 10 so that it approximately coincides with the second viewpoint.

Figures 4 and 5 are diagrams illustrating an example of guidance control by the control unit 110 according to this embodiment.

For example, the control unit 110 according to this embodiment may provide guidance by displaying an object 60, such as an arrow-shaped object, on the display unit 120, as shown in FIG. 4.

Furthermore, for example, the control unit 110 according to this embodiment may provide guidance by causing the audio output unit 130 to output audio 70 such as "Please look from the front," as shown in FIG. 4.

Furthermore, for example, the control unit 110 according to this embodiment may achieve guidance by displaying an object 61, such as an avatar, on the display unit 120 at a position corresponding to the second viewpoint, as shown in FIG. 5.

Furthermore, in addition to controlling the display of the object 61, the control unit 110 according to this embodiment may also provide guidance by having the audio output unit 130 output audio 71 such as "Please look from this location."

The above describes the review transition based on trigger detection in this embodiment. The control described above eliminates the need for the user to adjust the viewpoint each time to match a specified scenario, greatly improving review efficiency.

Furthermore, as described above, the control unit 110 according to this embodiment may perform control such that when a review is paused, the viewpoint at the time of the pause is saved, and when the review is resumed, the review is resumed from the saved viewpoint.

This type of control eliminates the need for users to adjust the viewpoint each time after editing, greatly improving review efficiency.

Furthermore, the control unit 110 according to this embodiment may store the viewpoint at the time when an instruction is detected based on a user instruction during review, and perform control to return to the stored viewpoint based on the user instruction.

Figure 6 is a diagram explaining how viewpoints are saved and restored in this embodiment.

In the example shown in Figure 6, the control unit 110 causes the display unit 120 to display an object 62 that allows the user to instruct the user to save the viewpoint.

For example, when the control unit 110 detects that the user's finger F has come into contact with the object 62, it saves the viewpoint at the time of detection.

Then, the control unit 110 causes the display unit 120 to display an object 63 that instructs the user to return to the saved viewpoint.

For example, when the control unit 110 detects that the user's finger F has come into contact with the object 63, it performs control to return to the viewpoint corresponding to the object 63.

The above control allows users to save their viewpoint at any time and then return to that saved viewpoint, greatly improving review efficiency.

In addition, a user may be able to save a viewpoint at any point in time along with any comment. Also, any comment may be able to be saved separately from the viewpoint.

Next, editing of the 3D virtual model 50 using the HMD 10 according to this embodiment will be described.

In the information processing system 1 according to this embodiment, the user can edit the 3D virtual model 50 without operating the editing device 20.

When a user performs an editing operation on the 3D virtual model 50 during review, the control unit 110 according to this embodiment displays the 3D virtual model 50 reflecting the editing operation on the display unit 120 while maintaining the review.

Figures 7 to 10 are diagrams illustrating editing operations that do not involve operating the editing device 20 according to this embodiment.

In the example shown in FIG. 7, the control unit 110 according to this embodiment causes the display unit 120 to display a simple editing screen 64 for easily editing various parameters related to the 3D virtual model 50.

On the other hand, Figure 8 shows an example of a standard editing screen 74 provided by the editing software 210 installed on the editing device 20.

The standard editing screen 74 is the main screen for designing and editing the 3D virtual model 50. As shown in FIG. 8, the standard editing screen 74 displays, for example, boxes for editing the style, color, scale, object candidates, etc. of the 3D virtual model 50 all together on one screen.

In contrast, the simplified editing screen 64 may be a user interface that includes only some of the boxes described above, or a simplified user interface that allows editing of some of the parameters that can be edited on the standard editing screen 74.

Note that examples of parameters that can be edited on the simple editing screen 64 include parameters related to the scale, style, color, texture, material, elasticity, etc. of each part of the 3D virtual model 50. In addition to the example shown in FIG. 8, the above styles include various other styles such as saturation, brightness, sharpness, and contrast.

The user may be able to edit any parameter related to the 3D virtual model 50, for example, by operating a slider included in the simple editing screen 64 with a finger F or the like.

In addition to the slider, the simple editing screen 64 may also include fields for inputting parameter values, buttons for increasing or decreasing parameter values, etc.

Furthermore, the control unit 110 may normally perform control such as not displaying the simple editing screen 64 or displaying it in a minimized size, and when a user instruction is detected, may perform control such as displaying the simple editing screen 64 or displaying it in an enlarged size.

On the other hand, the control unit 110 may also control a user interface that allows the user to perform direct editing operations on the 3D virtual model 50.

In the example shown in Figure 9, the user edits the height of the lens of the 3D virtual model 50 by pulling the lens of the 3D virtual model 50 using the hand H or the like.

On the other hand, the control unit 110 may also detect editing operations performed by the user's voice.

In the example shown in Figure 10, the control unit 110 detects an editing operation based on the results of voice recognition of the user's voice 72, such as "Increase the lens height by 15 mm."

In this way, the user interface that allows the user to perform editing operations may be voice-based.

As described above in the specific examples, in the information processing system 1 according to this embodiment, the user can use various user interfaces to perform editing operations without operating the editing device 20.

Furthermore, to achieve the above, the control unit 110 according to this embodiment transmits information relating to the acquired editing operations to the editing software 210 used to edit the 3D virtual model 50, and performs control to acquire from the editing software 210 the 3D virtual model 50 reflecting the editing operations.

In addition, the transmission of information regarding the acquired editing operations and the acquisition of the 3D virtual model 50 reflecting the editing operations may be achieved via the API (Application Programming Interface) of the editing software 210.

In addition, the information regarding the editing operation acquired above includes various parameters related to the 3D virtual model 50.

We have explained editing operations that do not involve operating the editing device 20 according to this embodiment.

The control described above eliminates the need for the user to alternate between using the HMD 10 and the editing device 20, greatly improving review efficiency.

Next, we will explain the control based on acquired information in this embodiment.

The acquired information includes sensor information acquired by the sensor unit 150, information related to editing operations, user information, etc.

For example, the control unit 110 according to this embodiment may adjust both or either the first viewpoint and the second viewpoint based on the acquired information.

FIG. 11 is a sequence diagram showing an example of the flow of viewpoint adjustment based on acquired information according to this embodiment.

In the example shown in Figure 11, the control unit 110 first acquires sensor information and information related to editing operations (S101).

The control unit 110 estimates the user's posture based on the sensor information and information related to the editing operation acquired in step S101, and the user information acquired in advance (S102).

The above user information includes, for example, information about the user's height.

Furthermore, the user's posture estimated in step S102 includes, for example, standing, sitting, crouching, etc.

The control unit 110 adjusts both or either of the first viewpoint and the second viewpoint based on the user's posture estimated in step S102 (S103).

The processing in steps S101 to S103 may be repeated from the start to the end of the review.

More specific examples are provided below.

For example, consider a case where the user is standing when the review from the first viewpoint begins, and then transitions to a sitting position midway through the review from the first viewpoint.

In this case, the control unit 110 may detect the change in the user's posture as described above and adjust the first viewpoint lower so that the 3D virtual model 50 is easier to observe when the user is sitting.

The control unit 110 may also adjust the second viewpoint to a lower value in a similar manner.

The above control allows for more efficient reviewing based on the user's posture.

Furthermore, for example, the control unit 110 according to this embodiment may update the review information based on the acquired information.

FIG. 12 is a sequence diagram showing an example of the flow of review information updating based on acquired information in this embodiment.

In the example shown in Figure 12, the control unit 110 first acquires sensor information and information related to editing operations (S201).

The control unit 110 performs analysis based on the sensor information and information related to editing operations acquired in step S201, as well as previously acquired user information (S202).

The control unit 110 updates the review information based on the analysis results in step S202 (S203).

The processing in steps S201 to S203 may be repeated from the start to the end of the review.

More specific examples are provided below.

For example, assume that the 3D virtual model 50 is a space including seats, such as an aircraft cabin, and that the space is to be reviewed.

In this case, the control unit 110 can analyze the acquired information to obtain, for example, an analysis result that the user made many edits to the seat size (height) during the review.

In the above case, the control unit 110 may update the review information so that a scene in which the user sits in the seat is added to the review scenario.

Note that the review information update described above may be achieved, for example, by learning based on acquired information.

For example, the control unit 110 may update the review information based on the learning result that tall users tend to edit seats more frequently.

The above control makes it possible to provide users with more efficient reviews.

Next, with reference to Figure 13, review control including interaction between the 3D virtual model 50 and objects in the real world according to this embodiment will be described.

In the example shown in Figure 13, the 3D virtual model 50 is a camera lens, and the object 55 in the real world is a mock-up of the camera housing.

In this case, the control unit 110 controls the review, which includes the interaction between the 3D virtual model 50 and the object 55 in the real world, based on review information corresponding to a review scenario that checks the attachability of the 3D virtual model to the object 55, for example.

In this way, the information processing system 1 according to this embodiment makes it possible to perform editing work while checking the interaction between the 3D virtual model 50 and the object 55 in the real world, greatly improving review efficiency.

Next, we will describe a case where the information processing system 1 according to this embodiment includes multiple HMDs 10 worn by multiple users.

The above mainly describes an example in which there is one user, but in this embodiment there may be multiple users.

FIG. 14 is a diagram illustrating a review when the information processing system 1 according to this embodiment includes multiple HMDs 10 worn by multiple users.

In the example shown in FIG. 14, the information processing system 1 includes a first HMD 10A worn by a first user 80A and a second HMD 10B worn by a second user 80B.

In this case, the first HMD 10A is equipped with a first control unit 110A that controls the review of the 3D virtual model 50 based on review information that records a predetermined positional relationship between the viewpoint of the first user 80A and the 3D virtual model 50 displayed by the first display unit 120A provided in the first HMD 10A.

Similarly, the second HMD 10B includes a second control unit 110B that controls the review of the 3D virtual model 50 based on review information that records a predetermined positional relationship between the viewpoint of the second user 80B and the 3D virtual model 50 displayed by the second display unit 120B included in the second HMD 10B.

Note that the review information stored in the first HMD 10A and the review information stored in the second HMD 10B may be the same or different from each other.

In the above configuration, in addition to the 3D virtual model 50, the field of view of the second user 80B may include the first user 80A (or an avatar corresponding to the first user 80A).

Similarly, in addition to the 3D virtual model 50, the field of view of the first user 80A may include the second user 80B (or an avatar corresponding to the second user 80B).

Therefore, with the above configuration, the second user 80B can confirm the interaction between the first user 80A and the 3D virtual model 50.

Similarly, the first user 80A can see the interaction between the second user 80B and the 3D virtual model 50.

Furthermore, in this respect, the first user 10A and the second user 10B can be considered to be objects in the real world.

As described above, the information processing system 1 according to this embodiment may include multiple HMDs 10 worn by multiple users.

With the above configuration, each user can check the interactions between other users and the 3D virtual model 50, greatly improving review efficiency, for example.

Next, we will explain the control related to limiting viewpoint movement in this embodiment.

Viewing images using the HMD 10A may cause what is known as 3D motion sickness.

For this reason, the control unit 110 according to this embodiment may impose restrictions on the movement of the viewpoint during review in order to prevent (reduce) motion sickness.

For example, the control unit 110 may set an upper limit on the speed and acceleration of movement when moving continuously from one viewpoint to the next.

Furthermore, for example, the control unit 110 may set an upper limit on the angular velocity and angular acceleration of the change in direction during the above movement.

Furthermore, for example, the control unit 110 may limit the directional changes that can be made at one time to only one of the yaw, pitch, or roll directions, and achieve movement by combining these directions continuously in a time series.

Furthermore, for example, the control unit 110 may represent the above movement discretely rather than continuously.

The above describes the main functions of the information processing system 1 according to this embodiment.

Note that, although the above describes a case where the positional relationship between the viewpoint and the 3D virtual model 50 is defined in the review information, various conditions other than this positional relationship can also be defined in the review information.

Other conditions defined in the review information include, for example, the size, texture, and surrounding environment (e.g., background, lighting environment, etc.) of the 3D virtual model 50.

In addition, the control unit 110 may perform various other controls in addition to those described above.

For example, the control unit 110 may perform control so that an explanation of the perspectives to be reviewed is displayed before the review begins, immediately after the review begins, etc.

Furthermore, for example, when resuming review after editing using the editing device 20, the control unit 110 may change the viewpoint or the posture of the 3D virtual model 50 so that the editing target is easier to see, and then resume review.

Furthermore, for example, the control unit 110 may automatically generate preset review information from the 3D virtual model 50.

Furthermore, for example, the control unit 110 may update the preset series of review information by learning the review history (for example, updating it to a collection of frequently used perspectives).

<2. Hardware configuration example>
Next, an example of a hardware configuration common to the HMD 10 and the editing device 20 according to an embodiment of the present disclosure will be described.

FIG. 15 is a block diagram showing an example hardware configuration of an information processing device 90 according to an embodiment of the present disclosure. The information processing device 90 may be a device having a hardware configuration equivalent to that of the HMD 10 and the editing device 20. However, the HMD 10 and the editing device 20 may not have some of the configuration shown in FIG. 15, or may have configurations other than those shown in FIG. 15.

As shown in FIG. 15, the information processing device 90 includes, for example, a processor 871, a ROM 872, a RAM 873, a host bus 874, a bridge 875, an external bus 876, an interface 877, an input device 878, an output device 879, storage 880, a drive 881, a connection port 882, and a communication device 883. Note that the hardware configuration shown here is an example, and some of the components may be omitted. Furthermore, the information processing device 90 may include further components in addition to the components shown here.

(Processor 871)
The processor 871 functions, for example, as an arithmetic processing device or control device, and controls the overall operation or part of the operation of each component based on various programs recorded in the ROM 872, the RAM 873, the storage 880, or the removable storage medium 901.

(ROM872, RAM873)
The ROM 872 is a means for storing programs to be read into the processor 871, data to be used for calculations, etc. The RAM 873 temporarily or permanently stores, for example, the programs to be read into the processor 871 and various parameters that change as appropriate when the programs are executed.

(Host bus 874, bridge 875, external bus 876, interface 877)
The processor 871, ROM 872, and RAM 873 are connected to one another via, for example, a host bus 874 that is capable of high-speed data transmission. On the other hand, the host bus 874 is connected to an external bus 876 that has a relatively low data transmission speed via, for example, a bridge 875. Furthermore, the external bus 876 is connected to various components via an interface 877.

(Input device 878)
The input device 878 may be, for example, a mouse, keyboard, touch panel, button, switch, lever, etc. Furthermore, a remote controller (hereinafter referred to as a remote control) capable of transmitting control signals using infrared rays or other radio waves may also be used as the input device 878. The input device 878 may also include an audio input device such as a microphone.

(output device 879)
The output device 879 is a device capable of visually or audibly notifying the user of acquired information, such as a display device such as a CRT (Cathode Ray Tube), LCD, or organic EL, an audio output device such as a speaker or headphones, a printer, a mobile phone, a facsimile, etc. The output device 879 according to the present disclosure also includes various vibration devices capable of outputting tactile stimuli.

(Storage 880)
The storage 880 is a device for storing various types of data. For example, a magnetic storage device such as a hard disk drive (HDD), a semiconductor storage device, an optical storage device, or a magneto-optical storage device may be used as the storage 880.

(Drive 881)
The drive 881 is a device that reads information recorded on a removable storage medium 901 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory, or writes information to the removable storage medium 901 .

(Removable storage medium 901)
The removable storage medium 901 is, for example, a DVD medium, a Blu-ray (registered trademark) medium, an HD DVD medium, various semiconductor storage media, etc. Of course, the removable storage medium 901 may also be, for example, an IC card equipped with a contactless IC chip, an electronic device, etc.

(Connection port 882)
The connection port 882 is a port for connecting an external device 902, such as a USB (Universal Serial Bus) port, an IEEE 1394 port, a SCSI (Small Computer System Interface), an RS-232C port, or an optical audio terminal.

(Externally connected device 902)
The externally connected device 902 is, for example, a printer, a portable music player, a digital camera, a digital video camera, or an IC recorder.

(Communication device 883)
The communication device 883 is a communication device for connecting to a network, such as a communication card for wired or wireless LAN, Bluetooth (registered trademark), or WUSB (Wireless USB), a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), or a modem for various types of communication.

<3. Summary>
As described above, the information processing method implemented by the information processing system 1 according to one embodiment of the present disclosure includes a processor controlling the review of the 3D virtual model 50 based on review information that records a predetermined positional relationship between the viewpoint of the user wearing the HMD 10 and the 3D virtual model 50 displayed by the HMD 10.

Furthermore, the review information includes at least the positional relationship between a first viewpoint and the 3D virtual model 50, and the positional relationship between a second viewpoint different from the first viewpoint and the 3D virtual model 50.

Furthermore, the above control initiates a review from a first perspective based on the review information, and when a trigger is detected, transitions to a review from a second perspective.

The above configuration makes it possible to improve the efficiency of reviewing 3D virtual models using an HMD.

The above describes in detail preferred embodiments of the present disclosure with reference to the accompanying drawings, but the technical scope of the present disclosure is not limited to such examples. It is clear that a person with ordinary skill in the technical field of the present disclosure would be able to conceive of various modified or revised examples within the scope of the technical ideas set forth in the claims, and it is understood that these also naturally fall within the technical scope of the present disclosure.

Furthermore, the steps involved in the processing described in this disclosure do not necessarily have to be processed chronologically in the order depicted in the flowcharts or sequence diagrams. For example, the steps involved in the processing of each device may be processed in an order different from that depicted, or may be processed in parallel.

Furthermore, the series of processes performed by each device described in this disclosure may be realized by a program stored on a non-transitory computer-readable storage medium. Each program is, for example, loaded into RAM when executed by a computer, and executed by a processor such as a CPU. The storage medium may be, for example, a magnetic disk, optical disk, magneto-optical disk, flash memory, etc. Furthermore, the program may be distributed, for example, via a network, without using a storage medium.

Furthermore, the effects described in this specification are merely descriptive or exemplary and are not limiting. In other words, the technology disclosed herein may achieve other effects in addition to or in place of the above-mentioned effects that would be apparent to those skilled in the art from the description herein.

The following configurations also fall within the technical scope of the present disclosure.
(1)
a control circuit for controlling a review of the 3D virtual model based on review information that records a predetermined positional relationship between a viewpoint of a user wearing an HMD and the 3D virtual model displayed by the HMD;
Equipped with
the review information includes at least a positional relationship between a first viewpoint and the 3D virtual model, and a positional relationship between a second viewpoint different from the first viewpoint and the 3D virtual model;
the control circuit starts the review from the first perspective based on the review information, and transitions to the review from the second perspective when a trigger is detected.
Information processing system.
(2)
the trigger comprises a predetermined interaction between the user and the 3D virtual model;
The information processing system according to (1) above.
(3)
the predetermined interaction includes an operation of the user on the 3D virtual model;
The information processing system according to (2) above.
(4)
the predetermined interaction includes the user's viewpoint substantially matching the second viewpoint;
The information processing system according to (2) above.
(5)
the control circuit performs control to guide the user's viewpoint so that it substantially coincides with the second viewpoint.
The information processing system according to (4) above.
(6)
and when the user performs an editing operation on the 3D virtual model during the review, the control circuit causes the HMD to display the 3D virtual model reflecting the editing operation while maintaining the review.
The information processing system according to any one of (1) to (5).
(7)
the control circuit controls a user interface for the user to perform the editing operation;
The information processing system according to (6) above.
(8)
the control circuit causes the HMD to display an object for the user to edit parameters of the 3D virtual model;
The information processing system according to (7) above.
(9)
the control circuit transmits the acquired information about the editing operation to software used for editing the 3D virtual model, and performs control to acquire the 3D virtual model reflecting the editing operation from the software.
The information processing system according to any one of (6) to (8).
(10)
the control circuit adjusts at least one of the first viewpoint and the second viewpoint based on information acquired by the HMD during the review;
The information processing system according to any one of (1) to (9).
(11)
the control circuit updates the review information based on information acquired by the HMD in the review.
The information processing system according to any one of (1) to (10).
(12)
The information acquired by the HMD includes sensor information and information regarding an editing operation of the 3D virtual model.
The information processing system according to any one of (10) and (11).
(13)
the control circuitry controls the review, including interactions between the 3D virtual model and objects in the real world.
The information processing system according to any one of (1) to (12).
(14)
the control circuit, when the review is paused, saves a viewpoint at the time of the pause, and when the review is resumed, resumes the review from the saved viewpoint;
The information processing system according to any one of (1) to (13).
(15)
a first control circuit that controls a review of the 3D virtual model based on review information that records a predetermined positional relationship between a viewpoint of a first user wearing a first HMD and a 3D virtual model displayed by the first HMD;
a second control circuit that controls the review of the 3D virtual model based on review information that records a predetermined positional relationship between a viewpoint of a second user wearing a second HMD and the 3D virtual model displayed by the second HMD;
Prepare,
The information processing system according to any one of (1) to (14).
(16)
a processor controlling the review of the 3D virtual model based on review information that records a predetermined positional relationship between a viewpoint of a user wearing an HMD and a 3D virtual model displayed by the HMD;
Including,
the review information includes at least a positional relationship between a first viewpoint and the 3D virtual model, and a positional relationship between a second viewpoint different from the first viewpoint and the 3D virtual model;
The controlling includes starting the review in the first perspective based on the review information, and transitioning to the review in the second perspective when a trigger is detected.
Information processing methods.
(17)
the trigger comprises a predetermined interaction between the user and the 3D virtual model;
The information processing method according to (16) above.
(18)
the predetermined interaction includes an operation of the user on the 3D virtual model;
The information processing method according to (17) above.
(19)
the predetermined interaction includes the user's viewpoint substantially matching the second viewpoint;
The information processing method according to (17) above.
(20)
The controlling includes performing control to guide the user's viewpoint so that the viewpoint substantially coincides with the second viewpoint.
The information processing method according to (19) above.
(21)
the controlling includes, when the user performs an editing operation on the 3D virtual model during the review, displaying the 3D virtual model reflecting the editing operation on the HMD while maintaining the review.
The information processing method according to any one of (16) to (20).
(22)
said controlling controlling a user interface for said user to perform said editing operation;
The information processing method according to (21) above.
(23)
the controlling causes the HMD to display an object for the user to edit parameters of the 3D virtual model;
The information processing method according to (22) above.
(24)
The controlling includes transmitting the acquired information about the editing operation to software used for editing the 3D virtual model, and performing control to acquire the 3D virtual model reflecting the editing operation from the software.
The information processing method according to any one of (16) to (23).
(25)
and the controlling includes adjusting at least one of the first viewpoint and the second viewpoint based on information acquired by the HMD during the review.
The information processing method according to any one of (16) to (24).
(26)
The controlling updates the review information based on information acquired by the HMD in the review.
The information processing method according to any one of (16) to (25).
(27)
The information acquired by the HMD includes sensor information and information regarding an editing operation of the 3D virtual model.
The information processing method according to any one of (25) and (26).
(28)
and controlling the review includes an interaction between the 3D virtual model and an object in the real world.
The information processing method according to any one of (16) to (27).
(29)
The controlling step includes, when the review is paused, saving a viewpoint at the time of the pause, and when the review is resumed, resuming the review from the saved viewpoint.
The information processing method according to any one of (16) to (28).
(30)
Controlling a review of the 3D virtual model based on review information that records a predetermined positional relationship between a viewpoint of a first user wearing a first HMD and the 3D virtual model displayed by the first HMD;
controlling a review of the 3D virtual model based on review information that records a predetermined positional relationship between a viewpoint of a second user wearing a second HMD and the 3D virtual model displayed by the second HMD;
include,
The information processing method according to any one of (16) to (29).

1 Information processing system 10 HMD
110 Control unit 120 Display unit 130 Audio output unit 140 Photography unit 150 Sensor unit 160 Audio input unit 170 Storage unit 180 Communication unit 20 Editing device 210 Editing software 50 3D virtual model

Claims (20)

a control circuit for controlling a review of the 3D virtual model based on review information that records a predetermined positional relationship between a viewpoint of a user wearing an HMD and the 3D virtual model displayed by the HMD;
Equipped with
the review information includes at least a positional relationship between a first viewpoint and the 3D virtual model, and a positional relationship between a second viewpoint different from the first viewpoint and the 3D virtual model;
the control circuit starts the review from the first perspective based on the review information, and transitions to the review from the second perspective when a trigger is detected.
Information processing system. the trigger comprises a predetermined interaction between the user and the 3D virtual model;
The information processing system according to claim 1 . the predetermined interaction includes an operation of the user on the 3D virtual model;
The information processing system according to claim 2 . the predetermined interaction includes the user's viewpoint substantially matching the second viewpoint;
The information processing system according to claim 2 . the control circuit performs control to guide the user's viewpoint so that it substantially coincides with the second viewpoint.
The information processing system according to claim 4 . and when the user performs an editing operation on the 3D virtual model during the review, the control circuit causes the HMD to display the 3D virtual model reflecting the editing operation while maintaining the review.
The information processing system according to claim 1 . the control circuit controls a user interface for the user to perform the editing operation;
The information processing system according to claim 6. the control circuit causes the HMD to display an object for the user to edit parameters of the 3D virtual model;
The information processing system according to claim 7 . the control circuit transmits the acquired information about the editing operation to software used for editing the 3D virtual model, and performs control to acquire the 3D virtual model reflecting the editing operation from the software.
The information processing system according to claim 6. the control circuit adjusts at least one of the first viewpoint and the second viewpoint based on information acquired by the HMD during the review;
The information processing system according to claim 1 . the control circuit updates the review information based on information acquired by the HMD in the review.
The information processing system according to claim 1 . The information acquired by the HMD includes sensor information and information regarding an editing operation of the 3D virtual model.
The information processing system according to claim 10. the control circuitry controls the review, including interactions between the 3D virtual model and objects in the real world.
The information processing system according to claim 1 . the control circuit, when the review is paused, stores a viewpoint at the time of the pause, and when the review is resumed, resumes the review from the stored viewpoint;
The information processing system according to claim 1 . a first control circuit that controls a review of the 3D virtual model based on review information that records a predetermined positional relationship between a viewpoint of a first user wearing a first HMD and a 3D virtual model displayed by the first HMD;
a second control circuit that controls the review of the 3D virtual model based on review information that records a predetermined positional relationship between a viewpoint of a second user wearing a second HMD and the 3D virtual model displayed by the second HMD;
Prepare,
The information processing system according to claim 1 . a processor controlling the review of the 3D virtual model based on review information that records a predetermined positional relationship between a viewpoint of a user wearing an HMD and a 3D virtual model displayed by the HMD;
Including,
the review information includes at least a positional relationship between a first viewpoint and the 3D virtual model, and a positional relationship between a second viewpoint different from the first viewpoint and the 3D virtual model;
The controlling includes starting the review in the first perspective based on the review information, and transitioning to the review in the second perspective when a trigger is detected.
Information processing methods. the trigger comprises a predetermined interaction between the user and the 3D virtual model;
17. The information processing method according to claim 16. the controlling includes, when the user performs an editing operation on the 3D virtual model during the review, displaying the 3D virtual model reflecting the editing operation on the HMD while maintaining the review.
17. The information processing method according to claim 16. and controlling the review by adjusting at least one of the first viewpoint and the second viewpoint based on information acquired by the HMD during the review.
17. The information processing method according to claim 16. and controlling the review includes an interaction between the 3D virtual model and an object in the real world.
17. The information processing method according to claim 16.