JP7208549B2 - VIRTUAL SPACE CONTROL DEVICE, CONTROL METHOD THEREOF, AND PROGRAM - Google Patents

Description

The present invention relates to a virtual space control device, a control method therefor, and a program that enable viewing of part data in a virtual space without manipulating a three-dimensional model composed of part data whose design has been changed in the virtual space.

Conventionally, front-loading of operations has been carried out in the assembly manufacturing industry. Front-loading is an approach that aims to improve quality by examining problems that may occur in later processes in the initial process. For example, CAD data being created in the design department is displayed by three-dimensional CAD software, and downstream departments such as the assembly department and the quality assurance department simulate interference checks of parts.

However, the CAD data obtained from the design department is incomplete. Since the design department and the downstream department cooperate to proceed with product design, CAD data is exchanged many times between the design department and the downstream department. In other words, the downstream department needs to acquire the CAD data improved by the design department many times and perform simulation each time.

In doing so, it is important for the design department to properly communicate the details of the design change to the downstream departments. Therefore, the downstream department must confirm what has been changed from the previously obtained CAD data by comparing the CAD data before the design change and the CAD data after the design change.

Therefore, in Patent Document 1, the CAD data before the design change and the CAD data after the design change are each divided into a plurality of areas with a certain shape, and the volume of each area is calculated and compared before and after the design change. , a mechanism for identifying a portion where the shape has been changed is disclosed.

JP-A-11-345256

Since there is a mechanism for automatically identifying the design-changed part by the mechanism of Patent Document 1 and the conventional function of the three-dimensional CAD software, workers in the downstream department can easily confirm the design-changed part. can.

However, in the case of an assembly that consists of a large number of parts, the part where the design change was made may be incorporated inside the assembly. It takes time and effort to confirm as a model. This is because the three-dimensional model of the CAD data must be displayed by three-dimensional CAD software, and settings must be made so that only the design change points are displayed.

In addition, an image including a three-dimensional model is superimposed on an image of the real space and presented on the observer's head-mounted display, which is called mixed reality (MR) or augmented reality. There is a mechanism that is Using this mechanism to display the 3D model of the assembly and check the design changes inside the assembly, the 3D model that exists in the virtual space must be manipulated and disassembled into parts. Therefore, it is very troublesome for the observer.

SUMMARY OF THE INVENTION An object of the present invention is to provide a mechanism that enables viewing of part data in a virtual space without manipulating a three-dimensional model configured by the part data whose design has been changed in the virtual space.

In order to achieve the above object, the information processing device of the present invention includes:
identifying means for identifying a portion where there is a difference between the first three-dimensional model and the second three-dimensional model;
The first three-dimensional model and the second three-dimensional model are independent positions in the virtual space, and the three-dimensional part where the difference exists in the position corresponding to the movable target in the real space display control means for controlling display of a third three-dimensional model, which is a model;
characterized by comprising

According to the present invention, the part data can be browsed in the virtual space without manipulating the three-dimensional model configured by the part data whose design has been changed in the virtual space.

It is a figure showing an example of the system configuration of the information processing system in the embodiment of the present invention. 1 is a diagram showing an example of hardware configurations of an information processing apparatus 101 and an HMD 102; FIG. 2 is a diagram illustrating an example of a functional configuration of an information processing apparatus 101; FIG. 4 is a flowchart showing an example of an overall processing flow; 9 is a flowchart showing an example of a detailed processing flow of difference identification processing; 6 is a diagram showing a reference structure of old assembly data and new assembly data, and an example of a data structure of a comparison table 610; FIG. 7 is a diagram illustrating an example of a table configuration of a difference list 700; FIG. 8 is a diagram showing an example of a screen configuration of an operation screen 800; FIG. 9 is a flowchart illustrating an example of a detailed processing flow of differential data arrangement processing; FIG. 9B is a flow chart following FIG. 9A showing an example of a detailed processing flow of difference data arrangement processing. FIG. 8 is a diagram showing a state in which an operation screen 800 is being operated by a target marker; FIG. 11 is a diagram showing a state in which selection difference data whose shape has been changed is displayed; FIG. 11 is a flowchart showing an example of a detailed processing flow of display setting reflection processing; FIG. It is a figure which shows a mode that the selection difference data before design change is displayed. FIG. 10 is a diagram showing how newly added selection difference data is displayed; FIG. 10 is a diagram showing a state in which selection difference data whose position or orientation has been changed is identified and displayed; FIG. 10 is a diagram showing a state in which deleted selected difference data is identified and displayed;

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

FIG. 1 is a diagram showing an example of the system configuration of an information processing system according to this embodiment. The information processing system shown in FIG. 1 is a system for allowing a user to browse a virtual space using mixed reality technology (hereinafter referred to as MR technology). In the information processing system, an HMD 102 is connected to an information processing apparatus 101 so as to be able to communicate data with each other. The connection between the information processing apparatus 101 and the HMD 102 may be wired connection or wireless connection. Note that the configuration of various terminals connected to the system in FIG. 1 is an example, and there are various configuration examples depending on the application and purpose.

The information processing device 101 (virtual space control device) is a general-purpose computer. The information processing apparatus 101 generates an image of a physical space captured (captured) by the HMD 102 (hereinafter referred to as a physical space image) and an image of a virtual space generated by the information processing apparatus 101 (hereinafter referred to as a virtual space image). and superimposed. An image thus generated (hereinafter referred to as a mixed reality image) is transmitted to the HMD 102 . As for the MR technique, a conventional technique is used, so a detailed explanation is omitted. Also, the information processing apparatus 101 may be a personal computer or a large computer such as a server. Furthermore, it may be a portable terminal such as a mobile phone or a tablet terminal. The type of computer is not particularly limited.

The HMD 102 is a head mounted display. The HMD 102 is a device worn on the user's head, and includes video cameras for right and left eyes and displays for right and left eyes. The HMD 102 transmits to the information processing apparatus 101 a physical space image captured by the video camera of the HMD 102 . Then, the mixed reality image is received from the information processing apparatus 101 and displayed on the display. Since the HMD 102 is provided with displays for the right eye and the left eye, a stereoscopic effect can be obtained by parallax. The physical space image captured by the HMD 102 and the mixed reality image displayed by the HMD 102 are preferably moving images (video), but may be still images captured at predetermined intervals. Also, a device such as a smartphone in which a display and a video camera are installed on the front and back of the hardware may be substituted for the head-mounted display.

In this embodiment, a system including the information processing apparatus 101 and the HMD 102 is described, but a device (virtual space control apparatus) in which the information processing apparatus 101 and the HMD 102 are integrated may be used. That is, the various hardware of the information processing apparatus 101 may be installed in the HMD 102 so that the HMD 102 alone may operate.

An infrared camera 104 is also connected to the information processing apparatus 101 . The infrared camera 104 is an optical sensor using infrared rays. The infrared camera 104 irradiates the real space with infrared rays and photographs the infrared rays reflected by the objects in the real space, thereby specifying the position and orientation of the objects in the real space in the coordinate system defined by the infrared camera 104 . The infrared camera 104 is used to specify the position and orientation (orientation, inclination, line-of-sight direction, etc.; hereinafter the same) of the HMD 102 (that is, the user) in the physical space. The HMD 102 is equipped with an optical marker 103 , an object that reflects infrared rays, and an infrared camera 104 captures the infrared rays reflected by the optical marker 103 to identify the position and orientation of the HMD 102 . In the information processing system, it is desirable to install a plurality of infrared cameras 104 so that the optical markers 103 of the HMD 102 worn by the user can be photographed or detected regardless of the position or posture of the user. Also, the HMD 102 whose position and orientation can be specified is assumed to exist within the imaging range of the infrared camera 104 .

In the present embodiment, the infrared camera 104 is used to identify the position and orientation of the HMD 102 in the physical space, but the camera is not limited to this as long as the position and orientation in the physical space can be identified. For example, a magnetic sensor may be used, or an image captured by the HMD 102 may be analyzed to specify the position and orientation.

In this embodiment, the user wearing the HMD 102 uses the target marker 105 to perform various operations. The target marker 105 is composed of a two-dimensional marker. The information processing device 101 identifies the position and orientation of the target marker 105 in the physical space in the virtual space by identifying the inclination of the two-dimensional marker and the distance from the HMD 102 by image recognition. be able to. Although the type of the two-dimensional marker is not particularly limited, it is preferable that the two-dimensional marker is of a type that allows easy acquisition of the shape of the two-dimensional marker by image recognition.

FIG. 2 is a diagram showing an example of each hardware configuration of the information processing apparatus 101 and the HMD 102. As shown in FIG.

The CPU 201 comprehensively controls each device and controller connected to the system bus 204 .

The ROM 202 or the external memory 211 also stores a BIOS (Basic Input/Output System), which is a control program for the CPU 201, and an operating system. Further, various programs, etc., which are necessary for realizing the functions executed by the information processing apparatus 101 and which will be described later, are stored. A RAM 203 functions as a main memory, a work area, and the like for the CPU 201 .

The CPU 201 implements various operations by loading programs and the like necessary for execution of processing into the RAM 203 and executing the programs.

An input controller (input C) 205 controls input from a pointing device (input device 210) such as a keyboard or mouse.

A video controller (VC) 206 of the information processing apparatus 101 controls display on displays such as the right-eye/left-eye display 222 and the display 212 provided in the HMD 102 . Output to the right-eye/left-eye display 222 is performed using, for example, an external output terminal (for example, Digital Visual Interface). The right-eye/left-eye display 222 is composed of a right-eye display and a left-eye display. The display 212 is a liquid crystal display or the like, and displays the same display as the right-eye/left-eye display 222 or a GUI (Graphical User Interface) for operating the virtual space.

A memory controller (MC) 207 controls access to an external memory 211 (storage means) that stores a boot program, browser software, various applications, font data, user files, edit files, various data, and the like. The external memory 211 is, for example, a hard disk (HD), a flexible disk (FD), or a card-type memory connected to a PCMCIA card slot via an adapter.

A communication I/F controller (communication I/FC) 208 connects and communicates with an external device via a network, and executes communication control processing in the network. For example, Internet communication using TCP/IP is possible. In particular, the communication I/F controller 208 of the information processing apparatus 101 also controls communication with the infrared camera 104 .

The general-purpose bus 209 of the information processing apparatus 101 is used to capture images captured by the right-eye/left-eye video cameras 221 of the HMD 102 connected to the information processing apparatus 101 . Input from the right-eye/left-eye video camera 221 is performed using an external input terminal (for example, an IEEE1394 terminal). The right-eye/left-eye video camera 221 is composed of a right-eye video camera and a left-eye video camera.

Note that the CPU 201 enables display on the display by, for example, rasterizing an outline font to a display information area in the RAM 203 . The CPU 201 also allows the user to issue instructions using a mouse cursor (not shown) on the display.

Various programs and the like used by the information processing apparatus 101 of the present invention to execute various processes to be described later are recorded in the external memory 211 and are executed by the CPU 201 by being loaded into the RAM 203 as necessary. be. Furthermore, the definition files and various information tables used by the program according to the present invention are stored in the external memory 211 .

FIG. 3 is a functional configuration diagram showing the functional configuration of the information processing apparatus 101. As shown in FIG. It should be noted that the functional configuration of the information processing apparatus 101 in FIG. 3 is an example, and there are various configuration examples depending on the application and purpose.

The information processing apparatus 101 includes, as functional units, a communication control unit 301, a position/orientation identification unit 302, a real space image acquisition unit 303, a virtual space generation unit 304, a 3D model control unit 305, a virtual space image acquisition unit 306, a mixed reality An image generator 307 is provided. Further, the information processing apparatus 101 includes a display control unit 308, a data management unit 309, a variable control unit 310, a data comparison unit 311, a screen generation unit 312, an operation reception unit 313, and a distance calculation unit 314 as functional units.

A communication control unit 301 is a functional unit that performs transmission and reception of various types of information between the HMD 102 and the infrared camera 104 that can communicate with the information processing apparatus 101 . The communication control unit 301 transmits and receives information to and from these devices through the aforementioned video controller 206, communication I/F controller 208, general-purpose bus 209, and the like.

The position/orientation identification unit 302 is a functional unit for identifying the position and orientation in the virtual space based on information indicating the position and orientation (orientation) of the HMD 102 in the real space acquired from the infrared camera 104 . In this embodiment, since the optical marker 103 is attached to the HMD 102, the position and orientation of the optical marker 103 in the physical space are acquired first. Since the coordinate system of the real space (the coordinate system of the infrared camera 104) and the coordinate system of the virtual space are calibrated (aligned) in advance, a virtual image corresponding to the position and orientation in the real space is based on this calibration. Identify position and orientation in space. Using the right-eye real-space image and the left-eye real-space image acquired from the right-eye/left-eye video camera 221 by the real-space image acquisition unit 303, the position and Posture may be specified.

Further, the position/orientation identification unit 302 analyzes the physical space image acquired by the physical space image acquisition unit 303, identifies the position and shape of the two-dimensional marker appearing in the physical space image, and determines the position of the target marker in the physical space. And it is also possible to specify the posture. The position/orientation identifying unit 302 also identifies the position and orientation in the virtual space from the position and orientation of the target marker in the physical space image.

The physical space image acquisition unit 303 is a functional unit that acquires physical space images captured by the right-eye and left-eye video cameras 221 of the HMD 102 .

A virtual space generation unit 304 is a functional unit that generates a virtual space based on information stored in the external memory 211 of the information processing apparatus 101 . Since the virtual space is a virtual space generated inside the information processing apparatus 101, information regarding the shape and size of the space is stored in the external memory 211, based on which the virtual space is generated. A three-dimensional model can be placed in the virtual space. The 3D model is arranged by the 3D model control unit 305 .

A three-dimensional model control unit 305 is a functional unit for placing a three-dimensional model in a virtual space. When the virtual space is generated, the 3D model is placed in the virtual space at a predefined position and orientation. If necessary, the arranged three-dimensional model is rearranged (moved) in a different position and orientation. Also, the 3D model control unit 305 can place the 3D model in the virtual space based on the position and orientation of the target marker in the virtual space specified by the position/orientation specifying unit 302 . That is, it is possible to arrange the three-dimensional model so as to be superimposed on the target marker 105 when the mixed reality image is generated. Furthermore, the 3D model control unit 305 can change the display form of the 3D model placed in the virtual space. For example, the color of the 3D model can be changed, or the shape can be changed.

A virtual space image acquisition unit 306 is a functional unit that acquires an image of the virtual space generated by the virtual space generation unit 304 . When the virtual space image acquisition unit 306 acquires an image of the virtual space, the viewpoint is set in the virtual space based on the position and orientation of the HMD 102 in the virtual space specified by the position/orientation specifying unit 302 . Then, a virtual space image viewed from the viewpoint is generated and acquired. A virtual camera may be installed at this viewpoint to capture an image of the virtual space. i.e. rendering

The mixed reality image generation unit 307 is a functional unit that generates a mixed reality image by superimposing the virtual space image acquired by the virtual space image acquisition unit 306 on the physical space image acquired by the physical space image acquisition unit 303 .

The display control unit 308 is a functional unit that controls display of various types of information on the right-eye/left-eye display 222 of the HMD 102 connected to the information processing apparatus 101 and the display 212 connected to the information processing apparatus 101 . In particular, it has a function of displaying the mixed reality image generated by the mixed reality image generation unit 307 on the right-eye/left-eye display 222 of the HMD 102 .

A data management unit 309 is a functional unit for managing data stored in a storage unit such as the RAM 203 of the information processing apparatus 101 or the external memory 211 . Data is added, updated, deleted, etc., as necessary.

The variable control unit 310 is a functional unit for controlling variables used in programs executed by the information processing apparatus 101 . Execute processing such as declaration and assignment of variables.

The data comparison unit 311 is a functional unit that compares a plurality of three-dimensional models stored in the external memory 211 or the like and specifies the difference between them. Since the three-dimensional model has information on its own polygon count and position/orientation, whether or not there is a difference is specified by comparing using this information. Further, as in the conventional technology, comparison based on volume or area, or comparison based on image processing may be used.

The screen generation unit 312 is a functional unit for generating an operation screen to be displayed in the virtual space. In this embodiment, since the operation screen is displayed in the virtual space, the generated screen is a three-dimensional model.

The operation reception unit 313 is a functional unit that receives selection operations on the operation screen generated by the screen generation unit 312 . An operation on the operation screen displayed in the virtual space is performed using the target marker 105 . Since the position and orientation of the target marker 105 in the virtual space are identified by the position/orientation identification unit 302, the cursor object is placed above the target marker 105 based on this position and orientation. When the cursor object and the button on the operation screen are in contact for a certain period of time, it is determined that the button has been pressed (selected). The operation method for the operation screen is not limited to this, and it may be operated by the finger of the user wearing the HMD 102, or the operation screen may be displayed on the display 212 of the information processing apparatus 101 and an operation may be accepted by the input device 210. .

A distance calculation unit 314 is a functional unit for calculating the distance between two points in the virtual space. Using the Pythagorean theorem or the like, the distance between two points in the coordinate system on the virtual space is calculated.

Next, an example of the overall processing flow in the embodiment of the present invention will be described using the flowchart shown in FIG. Each step of S401 to S412 described below is a process executed by the CPU 201 of the information processing apparatus 101 operating each functional unit.

In step S401, the information processing apparatus 101 executes processing for identifying the difference between the assembly before the design change and the assembly after the design change. FIG. 5 shows a detailed processing flow of the difference identification processing.

FIG. 5 is a flowchart illustrating an example of a detailed processing flow of difference identification processing. Each step from S501 to S509 described below is a process executed by the CPU 201 of the information processing apparatus 101 operating each functional unit.

In step S501, the data management unit 309 of the information processing apparatus 101 stores the assembly data before the design change (hereinafter referred to as old assembly data) stored in the external memory 211 and the assembly data after the design change (hereinafter referred to as "old assembly data"). hereinafter referred to as new assembly data) is acquired (acquisition means). That is, the first three-dimensional model before the design change and the second three-dimensional model after the design change are acquired. The user stores these data in the external memory 211 in advance. Then, the data management unit 309 acquires the old assembly data and the new assembly data specified by the user. Assembly data in this embodiment is assembly data of CAD data.

In FIG. 6A, examples of reference structures of old assembly data and new assembly data (hereinafter referred to as old and new assembly data) are shown at 601 and 602, respectively. Since the old and new assembly data is the top assembly, it consists of multiple parts and subassemblies. For example, old assembly data A consists of parts data B, parts data C, and assembly data D, and assembly data D, which is a sub-assembly, consists of parts data E, F, and G. Parts and subassemblies to which assembly data refers are hereinafter referred to as reference data.

In step S502, the data management unit 309 of the information processing apparatus 101 acquires information about the reference destination data that is the reference destination of the old and new assembly data acquired in step S501, and compares the comparison table 610 as shown in FIG. to create The comparison table 610 has an old reference destination data name 611 and a new reference destination data name 612, which indicate the names of the reference destination data of the comparison source and the comparison destination, respectively. If there is reference data of the new assembly data corresponding to the reference data of the old assembly data, it is stored side by side in the same row. Leave the product data item blank. FIG. 6B is the result of creating a comparison table 610 of the old assembly data A and the new assembly data A shown in FIG. 6A in this way.

At step S503, the data management unit 309 of the information processing apparatus 101 generates a difference list 700 as shown in FIG. In the process of step S503, an empty difference list 700 is generated.

A difference list 700 in FIG. 7 is a data table for indicating what kind of difference exists before and after the design change. The difference list 700 includes a difference data name 701 , an addition flag 702 , a shape change flag 703 , a position/orientation change flag 704 and a deletion flag 705 . The table configuration of the data table is not limited to this.

The difference data name 701 is an item that stores the name of the referenced data that has a difference before and after the design change. The term "difference" as used in this embodiment refers to the difference between before and after the design change. For example, it refers to adding new reference data to old assembly data, changing the shape of reference data, changing the position or posture of reference data, and deleting reference data from old assembly data. An addition flag 702, a shape change flag 703, a position/orientation change flag 704, and a deletion flag 705 are items for which flags are set when there is a difference between them.

Returning to the description of FIG. In step S504, the variable control unit 310 of the information processing apparatus 101 declares the variable i of the counter and substitutes 0 as the initial value. Then, in step S505, the variable control unit 310 of the information processing apparatus 101 increments the variable i.

In step S<b>506 , the data management unit 309 of the information processing apparatus 101 acquires the i-th record reference destination data among the records of the comparison table 610 from the external memory 211 or the like, and refers to it. What is acquired here is the reference destination data constituting the old assembly data indicated by the old reference destination data name 611 and the reference destination data constituting the new assembly data indicated by the new reference destination data name 612 . If either the old referenced data name 611 or the new referenced data name 612 is blank, only the referenced data whose name is entered is acquired.

In step S507, the data comparison unit 311 of the information processing apparatus 101 compares the reference destination data constituting the old assembly data and the reference destination data constituting the new assembly data acquired in step S506. The comparison here refers to whether new reference destination data has been added, whether the shape of the reference destination data has been changed, whether the position or orientation of the reference destination data has been changed, whether the reference destination data has been deleted, and so on. is identified by comparing the reference data obtained in step (identifying means).

For example, in the case of reference destination data that does not exist in the old assembly data but exists in the new assembly data (when the reference destination data constituting the old assembly data could not be acquired in step S506), a new Identifies it as added referenced data. If the number of polygons included in the reference data of the old assembly data obtained in step S506 does not match the number of polygons included in the reference data of the new assembly data, it is determined that the reference data has been changed in shape. do. Match/mismatch determination is not limited to the number of polygons, and comparison using other factors such as volume and area may also be used. Also, if the information indicating the position or orientation indicated by the assembly data that is the parent of the referenced data of the old assembly data and the assembly data that is the parent of the referenced data of the new assembly data do not match, the position or orientation is the changed referenced data. In the case of reference destination data that exists in the old assembly data but does not exist in the new assembly data (when the reference destination data constituting the new assembly data could not be acquired in step S506), the deleted reference Identifies as prior data. If none of these conditions apply, it is determined that the acquired referenced data has not undergone a design change.

As a result of this comparison, it is determined whether or not there is a design change in the reference destination data in the old and new assembly data referred to in step S506, that is, whether or not there is a difference. If it is determined that there is a difference (addition, shape change, position or posture change, or deletion), the process proceeds to step S508. If it is determined that there is no difference, the process proceeds to step S509.

In step S508, the data management unit 309 of the information processing apparatus 101 adds a new record to the difference list 700 generated in step S503, and adds the name of the referenced data referenced in step S506 to the difference data name 701. . Then, "1" is stored in the flag corresponding to the content of the difference. In other words, flag it. "0" is stored in other flags. By doing so, the information about the reference destination data whose design has been changed is added to the difference list 700 . The difference list 700 shown in FIG. 7 is the result of executing step S508 on all the reference destination data. Hereinafter, the reference destination data whose design has been changed will be referred to as difference data.

In step S509, the data management unit 309 of the information processing apparatus 101 determines whether or not the i-th row of the comparison table 610 is the last row (last record). If it is determined that the line is the last line, the difference specifying process is terminated, and the process proceeds to the next step after step S401. If it is determined that the line is not the last line, the process returns to step S505.

In this way, a plurality of referenced data that constitutes the old assembly data and a plurality of referenced data that constitutes the new assembly data are compared to identify the referenced data whose design has been changed and the content of the change. do.

Returning to FIG. When step S401 is completed, the virtual space generation unit 304 of the information processing apparatus 101 acquires virtual space information stored in the external memory 211 in step S402. Virtual space information is various information for generating a virtual space. Specifically, it is the information necessary to form the virtual space, such as the shape and size of the virtual space, and the three-dimensional model to be placed and its placement location. Then, the virtual space generation unit 304 generates a virtual space using the acquired virtual space information. The generated virtual space is a virtual space having the shape and size indicated by the virtual space information, and a three-dimensional model indicated by the virtual space information is arranged in the virtual space. In this embodiment, the virtual space generation unit 304 arranges the old assembly data and the new assembly data acquired in step S501 in the virtual space. Since the old and new assembly data are assembly data as described above, the referenced data is displayed as a three-dimensional model and arranged in the position and orientation defined in the assembly data. As a result, a three-dimensional model showing new and old assembly data is arranged in the virtual space.

In step S<b>403 , the physical space image acquisition unit 303 of the information processing apparatus 101 acquires physical space images from the right-eye and left-eye video cameras 221 of the HMD 102 and stores them in the RAM 203 .

In step S404, the position/orientation identification unit 302 of the information processing apparatus 101 acquires information indicating the position and orientation of the HMD 102 in the physical space. As described above, the infrared camera 104 acquires information indicating the position and orientation specified by detecting the optical marker 103 included in the HMD 102 from the infrared camera 104 . Then, the position/orientation identifying unit 302 identifies the position and orientation in the virtual space from the acquired information indicating the position and orientation. The specified position and orientation are stored in the RAM 203 as HMD position/orientation information. Since this information is the position and orientation of the HMD 102 in the current frame, this information is updated each time the frame is switched.

In step S405, the position/orientation identification unit 302 of the information processing apparatus 101 identifies information indicating the position and orientation of the target marker 105 in the physical space (position identification means). Specifically, as described above, the target marker 105 is detected from the physical space image captured by the HMD 102 . Then, the position and orientation of the target marker 105 in the physical space are estimated based on the shape of the target marker 105 captured in the physical space image and the parallax specified from the two physical space images acquired by the right-eye and left-eye video cameras 221 . Then, the position and orientation in the virtual space are specified from the position and orientation in the real space. In other words, the position in the virtual space corresponding to the position specified by the user wearing the HMD 102 in the real space is specified. The specified position and orientation are stored in the RAM 203 as target marker position/orientation information. Since this information is the position and orientation of the HMD 102 in the current frame, this information is updated each time the frame is switched.

In step S<b>406 , the screen generation unit 312 of the information processing apparatus 101 acquires the difference list 700 to which the reference data is added in step S<b>508 , and generates an operation screen including a list of difference data names 701 . An example of the operation screen is shown in FIG.

The operation screen 800 includes a target button 801, a display hierarchy button 802, and a display object button 803 as display settings. A target button 801 is a button for selecting whether to superimpose part data as difference data on the target marker 105 or to superimpose assembly data including the part data. A display hierarchy button 802 is a button for selecting how many upper layers of assembly data are to be displayed in a superimposed manner when the target button 801 selects to display the assembly data in a superimposed manner. For example, if "1" is selected, the assembly data that is the parent of the difference data is superimposed on the target marker 105, and if "2" is selected, the assembly data that is the parent of the difference data is superimposed on the target marker 105. FIG. The display target button 803 is a button for selecting whether the difference data in the old assembly data is superimposed on the target marker 105 or the difference data in the new assembly data is superimposed on the target marker 105 .

The operation screen 800 also has a differential data list 804 . A difference data list 804 is an area in which the names of the difference data shown in the difference data name 701 of the difference list 700 are listed. The name of each difference data is displayed in the form of a button. Difference data corresponding to the selected button is superimposed and displayed on the target marker 105 according to display settings.

Since this operation screen 800 is arranged in the virtual space generated in step S402, it is generated as a three-dimensional model. Specifically, the screen generation unit 312 generates image data representing the operation screen 800, and pastes it as a texture on the surface of the three-dimensional model generated by the three-dimensional model control unit 305, thereby arranging it in the virtual space. A three-dimensional model of screen 800 is generated. The position at which the operation screen 800 is arranged may be a predetermined position, or it may be arranged at a position a predetermined distance and direction away from the viewpoint of the HMD 102 set in the virtual space, and the arrangement position may follow the viewpoint. may be moved.

In step S407, the information processing apparatus 101 executes processing for arranging the difference data specified by the user in the virtual space. 9A and 9B show the detailed processing flow of the differential data arrangement processing.

9A and 9B are flowcharts showing an example of a detailed processing flow of differential data arrangement processing. Each step from S901 to S917 described below is a process executed by the CPU 201 of the information processing apparatus 101 operating each functional unit.

In step S901, the operation reception unit 313 of the information processing apparatus 101 identifies the operation content for the operation screen 800 arranged in the virtual space. In this embodiment, the operation on the operation screen 800 is performed using the target marker 105 . A cursor object is superimposed and displayed in the direction indicated by the target marker 105, and when this cursor object and a button on the operation screen 800 are in contact for a certain period of time, it is determined that the button has been pressed.

FIG. 10 is a diagram showing how the operation screen 800 is operated by the target marker 105. As shown in FIG. In the virtual space, the old assembly data 1001 and the new assembly data 1002 acquired in step S501 and the operation screen 800 generated in step S406 are arranged. A virtual space image representing such a virtual space is superimposed on the physical space image and displayed on the HMD 102 in a process described later. Furthermore, the virtual space generator 304 places the cursor object 1003 in the virtual space in the direction indicated by the target marker 105 based on the position and orientation of the target marker 105 in the virtual space. This cursor object 1003 follows the direction indicated by the target marker 105 even if the user wearing the HMD 102 moves the target marker 105 in the real space. When the user wants to superimpose the target marker 105 on the difference data displayed in the difference data list 804, the user can set the cursor object 1003 of the target marker 105 to the name of the difference data to be displayed. time contact. As a result, the button is pressed.

In step S902, the operation reception unit 313 of the information processing apparatus 101 determines whether a button corresponding to difference data whose operation content specified in step S901 is different from the difference data currently arranged in the virtual space has been selected. determine whether or not For example, if the difference data currently arranged in the virtual space is "parts data B" and the newly selected difference data is "parts data C", or if any difference data is arranged in the virtual space. This is a case where the button corresponding to the difference data is pressed in a state in which the In such a case, the process proceeds to step S903. Otherwise, the difference data arrangement process is terminated, and the process proceeds to the next step after step S407.

In step S903, the data management unit 309 of the information processing apparatus 101 refers to various flags (addition flag 702 to deletion flag 705) of difference data corresponding to the button pressed by the cursor object 1003. FIG. Hereinafter, the difference data corresponding to the button pressed by the cursor object 1003 will be referred to as selected difference data.

In step S904, the data management unit 309 of the information processing apparatus 101 determines whether or not the addition flag 702 of the selected difference data is "1", that is, whether or not the addition flag 702 is set. If it is determined that the addition flag 702 of the selected difference data is "1", the process proceeds to step S905. If it is determined that the addition flag 702 of the selected difference data is "0", the process proceeds to step S908.

In step S<b>905 , the data management unit 309 of the information processing apparatus 101 acquires from the external memory 211 the selection difference data constituting the new assembly data. That is, the selection difference data newly added to the old assembly data is obtained from the external memory 211 .

In step S906, the 3D model control unit 305 of the information processing apparatus 101 colors the selected difference data portion of the new assembly data arranged in the virtual space. In other words, the setting is made so that the selected difference data can be displayed in an identifiable form in the new assembly data arranged in the virtual space. Therefore, the display mode is not limited to coloring, and the selected difference data may be blinked, or may be identified by an arrow or the like.

In step S907, the screen generation unit 312 of the information processing apparatus 101 regenerates a screen in which the display target button 803 of the operation screen 800 cannot be selected, and rearranges the screen in the virtual space. The newly added difference data means that it did not exist before the design change, so even if "OLD" is selected with the display object button 803, there is nothing to be displayed. Therefore, the display target button 803 is made incapable of being pressed. After step S907 is completed, the process proceeds to step S911.

On the other hand, in step S908, the data management unit 309 of the information processing apparatus 101 determines whether the shape change flag 703 of the selected difference data is "1", that is, whether the shape change flag 703 is set. If it is determined that the shape change flag 703 of the selected difference data is "1", the process proceeds to step S909. If it is determined that the shape change flag 703 of the selected difference data is "0", the process proceeds to step S913 in FIG. 9B.

In step S<b>909 , the data management unit 309 of the information processing apparatus 101 acquires from the external memory 211 the selection difference data constituting the new assembly data. That is, the selection difference data after shape change is obtained from the external memory 211 . Different from the newly added difference data, the selection difference data before the design change also exists, but the user wants to check the shape after the design change, so the selection difference data after the design change is acquired. It should be noted that it may be possible to set which selection difference data is to be displayed first. That is, if the setting is to initially display the selected difference data before the design change, step S909 acquires the selected difference data before the design change.

In step S910, the 3D model control unit 305 of the information processing apparatus 101 colors the selection difference data portion of the old and new assembly data arranged in the virtual space (display form changing means). In other words, the setting is made so that the selected difference data can be identified in the new and old assembly data arranged in the virtual space. Therefore, the display mode is not limited to coloring, and the selected difference data may be blinked, or may be identified by an arrow or the like. Unlike step S907, step S910 changes the display form of the selection difference data also in the old assembly data.

In step S911, the distance calculation unit 314 of the information processing apparatus 101 calculates the position of the 3D model of the selected difference data included in the 3D model of the new assembly data arranged in the virtual space from the user's position in the virtual space. Calculate the distance to (distance acquisition means). The position of the selected differential data is the position of the central point of the three-dimensional model indicating the selected differential data, but is not limited to this. Then, it is determined whether or not the calculated distance is equal to or greater than a predetermined distance (or larger than the predetermined distance). That is, it is determined whether or not the calculated distance satisfies a predetermined condition. If the user exists near the three-dimensional model of the new assembly data placed in the virtual space, the user can directly confirm the three-dimensional model of the selected difference data included in the new assembly data. However, if the user is at a position away from the three-dimensional model of the new assembly data placed in the virtual space, the user must approach the new assembly data and check it. Therefore, in such a case, by displaying the selected difference data superimposed on the target marker 105, it is possible to reduce the trouble of approaching. Therefore, in step S911, determination regarding these distances is performed. If it is determined that the distance is greater than or equal to the predetermined distance, the process proceeds to step S912. Otherwise, the difference data arrangement process is terminated, and the process proceeds to the next step after step S407.

In this embodiment, step S911 is always executed, but it may be set so that it is not executed. That is, regardless of the distance calculated in step S911, the three-dimensional model of the acquired selection difference data may be superimposed on the target marker 105 and displayed.

In step S912, the virtual space generation unit 304 of the information processing apparatus 101 arranges the 3D model of the selection difference data acquired in S905 or S909 at the position and orientation in the virtual space of the target marker 105 specified in step S405 ( placement means). By doing so, when the virtual space image is superimposed on the physical space image, the three-dimensional model of the selected difference data is superimposed on the position of the target marker 105 . In other words, the user wearing the HMD 102 looks as if the three-dimensional model of the selected difference data is attached to the target marker 105 in the real space. When the process of step S912 is completed, the difference data arrangement process is terminated, and the process proceeds to the next step after step S407.

FIG. 11 is a diagram showing how the selected difference data whose shape has been changed is displayed on the target marker 105. As shown in FIG. As shown in FIG. 10, when "parts data B" is selected as difference data to be displayed, the flag indicating the changed contents of "parts data B" is checked. Then, as shown in FIG. 7, since the shape change flag 703 of "component data B" is flagged, it can be understood that it is the selection difference data whose shape has been changed. Therefore, in step S909, the selected difference data after the design change is acquired from the external memory 211, and the three-dimensional model of this acquired selected difference data is placed in the virtual space at the position and orientation of the target marker 105. FIG. In addition, in the three-dimensional model of the new and old assembly data, the display form of the three-dimensional model of the selected difference data is changed so that the selected difference data can be distinguished from other reference destination data.

FIG. 11 shows the result of generating a virtual space image representing this virtual space and superimposing it on the physical space image. As shown in FIG. 11, a three-dimensional model (a car wheel in FIG. 11) of the selection difference data after the design change is displayed superimposed on the position of the target marker 105 . A three-dimensional model of the new and old assembly data is also displayed, and the portion corresponding to the selected difference data is identified and displayed. In FIG. 11, 1101 is the selection difference data before the design change, and 1102 is the selection difference data after the design change. Both are 3D models of selected differential data that constitute the 3D model of old and new assembly data. In this way, by displaying a three-dimensional model of the selected difference data on the target marker 105 held by the user, separately from the selected difference data incorporated in the new and old assembly data, it is possible to easily view the selected difference data whose design has been changed. can be verified.

The description moves to FIG. 9B. In step S913, the screen generation unit 312 of the information processing apparatus 101 regenerates a screen in which the target button 801, the display hierarchy button 802, and the display target button 803 of the operation screen 800 cannot be selected, and rearranges them in the virtual space. . With regard to selection difference data whose position or orientation has been changed and deleted selection difference data, which will be described later, there is little need to check the state after the design change with the target marker 105 at hand. Therefore, in the processing to be described later, since the three-dimensional model of these selected difference data is not superimposed on the target marker 105, various buttons on the operation screen 800 are also controlled so as not to be selectable.

In step S914, the data management unit 309 of the information processing apparatus 101 determines whether the position/orientation change flag 704 of the selection difference data is "1", that is, whether or not the position/orientation change flag 704 is set. If it is determined that the position/orientation change flag 704 of the selected difference data is "1", the process proceeds to step S915. If it is determined that the position/orientation change flag 704 of the selected difference data is "0", the process proceeds to step S916.

In step S915, the 3D model control unit 305 of the information processing apparatus 101 colors the selection difference data portion of the old and new assembly data arranged in the virtual space. In other words, the setting is made so that the selected difference data can be identified in the new and old assembly data arranged in the virtual space. Therefore, the display mode is not limited to coloring, and the selected difference data may be blinked, or may be identified by an arrow or the like. Then, the difference data placement process is terminated without superimposing the three-dimensional model of the selected difference data whose position or orientation has been changed on the target marker 105, and the process proceeds to the next step after step S407. .

On the other hand, in step S916, the data management unit 309 of the information processing apparatus 101 determines whether the deletion flag 705 of the selected difference data is "1", that is, whether the deletion flag 705 is set. If it is determined that the deletion flag 705 of the selected difference data is "1", the process proceeds to step S917. If it is determined that the deletion flag 705 of the selected difference data is "0", the difference data arrangement processing is terminated, and the processing proceeds to the next step after step S407.

In step S917, the 3D model control unit 305 of the information processing apparatus 101 colors the selection difference data portion of the old assembly data arranged in the virtual space. In other words, the setting is made so that the selected difference data can be identified in the old assembly data arranged in the virtual space. Therefore, the display mode is not limited to coloring, and the selected difference data may be blinked, or may be identified by an arrow or the like. Then, the difference data placement process is terminated without superimposing and displaying the 3D model of the deleted selected difference data on the target marker 105, and the process proceeds to the next step after step S407.

In this embodiment, the three-dimensional model of the selected difference data whose position or orientation has been changed is not displayed superimposed on the target marker 105, but these may also be displayed superimposed.

Returning to FIG. When the process of step S407 is completed, in step S408, the information processing apparatus 101 executes a process of reflecting the content of the display settings on the operation screen 800. FIG. FIG. 12 shows a detailed processing flow of display setting reflection processing.

FIG. 12 is a flowchart illustrating an example of a detailed processing flow of display setting reflection processing. Each step from S1201 to S1213 described below is a process executed by the CPU 201 of the information processing apparatus 101 operating each functional unit.

In step S1201, the operation reception unit 313 of the information processing apparatus 101 identifies the operation content for the operation screen 800 arranged in the virtual space. The identification method is the same as in step S901.

In step S<b>1202 , the operation reception unit 313 of the information processing apparatus 101 determines whether or not pressing of the display hierarchy button 802 has been received. If it is determined that the display layer button 802 has been pressed, the process advances to step S1203. Otherwise, the process advances to step S1204.

In step S<b>1203 , the data management unit 309 of the information processing apparatus 101 saves in the RAM 203 information about the display hierarchy determined by pressing the display hierarchy button 802 . The display hierarchy button 802 can increase/decrease the display hierarchy by pressing the left and right arrows. This information is used to determine up to which hierarchy of assembly data should be displayed. After the processing of step S1203 is completed, the processing advances to step S1209.

In step S1204, the operation accepting unit 313 of the information processing apparatus 101 determines whether or not pressing of the display target button 803 has been accepted. If it is determined that the display target button 803 has been pressed, the process advances to step S1205. Otherwise, the process advances to step S1208.

In step S1205, the operation reception unit 313 of the information processing apparatus 101 determines whether the "NEW" button or the "OLD" button among the display target buttons 803 has been pressed. If it is determined that the "NEW" button has been pressed, the process advances to step S1206. If it is determined that the "OLD" button has been pressed, the process advances to step S1207.

In step S<b>1206 , the data management unit 309 of the information processing apparatus 101 acquires from the external memory 211 the selection difference data forming the new assembly data. On the other hand, in step S<b>1207 , the data management unit 309 of the information processing apparatus 101 acquires from the external memory 211 selection difference data forming the old assembly data. In this embodiment, control is performed so that the display target button 803 cannot be operated except for the selected differential data whose shape has been changed. cannot be obtained. Therefore, the display setting reflecting process may be ended when the information cannot be acquired.

In step S1208, the operation accepting unit 313 of the information processing apparatus 101 determines whether or not pressing of the target button 801 has been accepted. If it is determined that pressing of the target button 801 has been accepted, the process advances to step S1209. Otherwise, the display setting reflecting process is ended, and the process proceeds to the next step after step S408.

In step S<b>1209 , the distance calculation unit 314 of the information processing apparatus 101 calculates the three-dimensional distance of the selected difference data included in the three-dimensional model of the assembly data arranged in the virtual space from the position in the virtual space of the user wearing the HMD 102 . Calculate the distance to the position of the model. The selection difference data referred to here is the selection difference data in the assembly data in the virtual space corresponding to the selection difference data superimposed and displayed on the target marker 105 . For example, when "OLD" is selected by the new display object button 803, the position of the acquired three-dimensional model of the selected difference data before the design change, which is included in the three-dimensional model of the old assembly data, is used as the reference. Become. Calculation of the distance is as described above. Then, it is determined whether or not the calculated distance is equal to or greater than a predetermined distance (or larger than the predetermined distance). If it is determined that the distance is greater than or equal to the predetermined distance, the process proceeds to step S1210. Otherwise, the display setting reflecting process is ended, and the process proceeds to the next step after step S408.

In this embodiment, step S1209 is always executed, but it may be set so that it is not executed. That is, regardless of the distance calculated in step S1209, the three-dimensional model of the acquired selection difference data may be superimposed on the target marker 105 and displayed.

In step S1210, the operation reception unit 313 of the information processing apparatus 101 determines whether the "assembly" button or the "component" button among the target buttons 801 has been pressed. If it is determined that the "Assembly" button has been pressed, the process proceeds to step S1211. If it is determined that the "component" button has been pressed, the process proceeds to step S1213.

In step S1211, the data management unit 309 of the information processing apparatus 101 acquires the information of the hierarchy saved in step S1203, and obtains the assembly data obtained by going back from the selection difference data superimposed on the target marker 105 by the amount of the hierarchy indicated by the information. identify. For example, the reference structure of FIG. 6A will be described. If the selected difference data superimposed on the target marker 105 is "parts data E" and the layer information is "1", the assembly data one level above "parts data E" is identified. will specify "assembly data D".

In step S1212, the virtual space generation unit 304 of the information processing apparatus 101 arranges the three-dimensional model of the assembly data acquired in step S1211 at the position and orientation in the virtual space of the target marker 105 specified in step S405. That is, the reference destination data constituting the assembly data is acquired, and the 3D model of the reference data is arranged in the position and orientation set in the assembly data to generate the 3D model of the assembly. Then, this three-dimensional model can be arranged at the position and orientation of the target marker 105 .

On the other hand, in step S1213, the virtual space generation unit 304 of the information processing apparatus 101 creates a three-dimensional model of the selected difference data to be superimposed and displayed on the target marker 105 at the position and orientation in the virtual space of the target marker 105 specified in step S405. Deploy.

By doing so, when the virtual space image is superimposed on the physical space image, the selected difference data or the three-dimensional model of the assembly that is the parent of the selected difference data is superimposed on the position of the target marker 105 . When the process of step S1212 or S1213 is completed, the display setting reflection process is terminated, and the process proceeds to the next step after step S408.

Returning to FIG. In step S409, the virtual space image acquisition unit 306 of the information processing apparatus 101 acquires a virtual space image by capturing an image of the virtual space from a viewpoint set in the virtual space, and stores it in the RAM 203 or the like (virtual space image acquisition unit 306). image generating means). Two virtual space images for the right eye and left eye are obtained to be displayed on the right-eye and left-eye displays 222 of the HMD 102, respectively.

In step S410, the mixed reality image generation unit 307 of the information processing apparatus 101 reads the physical space image acquired in step S403 and the virtual space image acquired in step S409 from the RAM 203 or the like. Then, the virtual space image is superimposed on the real space image to generate a mixed reality image. The generated mixed reality image is stored in the RAM 203 or the like. As described above, two physical space images and two virtual space images for the right eye and two virtual space images are stored in the RAM 203 or the like. , the virtual space image for the left eye is superimposed on the real space image for the left eye.

In step S<b>411 , the display control unit 308 of the information processing apparatus 101 reads the mixed reality image generated in step S<b>410 from the RAM 203 or the like, and displays it on the right-eye/left-eye display 222 of the HMD 102 via the video controller 206 . There are two mixed reality images stored in the RAM 203 or the like, one for the right eye and one for the left eye. Therefore, the mixed reality image for the right eye is controlled to be displayed on the right eye display of the right/left eye display 222, and the mixed reality image for the left eye is controlled to be displayed on the left eye display of the right/left eye display 222.例文帳に追加

In step S412, the operation reception unit 313 of the information processing apparatus 101 determines whether or not an instruction to end the process of presenting mixed reality to the user wearing the HMD 102 has been issued. For example, it is determined whether or not there is an instruction to stop or terminate the application of the information processing apparatus 101 that executes the processes of steps S403 to S411 described above. If it is determined that there is an end instruction, this series of processes is terminated. If it is not determined that there is an end instruction, that is, if there is no end instruction, the process returns to step S403, and the processes from step S403 to step S412 are repeated until an end instruction is given.

FIG. 13 is a diagram showing a state in which the selection difference data before shape change is superimposed and displayed on the target marker 105. As shown in FIG. When "OLD" of the display object button 803 is selected by the target marker 105, the selection difference data before the shape change is obtained and superimposed on the target marker 105 and displayed. 1301 shows how the selection difference data after the shape change of 1103 shown in FIG. 11 is replaced with the selection difference data after the shape change. In this way, the selected difference data before and after the design change can be switched and displayed.

FIGS. 14 to 16 show the selection difference data after design changes other than the shape change. FIG. 14 shows a case where newly added differential data is selected. As shown in 1402 of the new assembly data, it can be seen that new difference data (car wing in FIG. 14) has been added. Therefore, this selection difference data is identified and displayed in the new assembly data. In addition, as indicated by 1401, the three-dimensional model of the selected difference data is superimposed on the target marker 105 and displayed.

FIG. 15 shows a case where difference data with changed position or orientation is selected. The three-dimensional model of the selected difference data (the door mirror of the car in FIG. 15) has been relocated as indicated by 1501 in the old assembly data and 1502 in the new assembly data. Therefore, this selected difference data is identified and displayed in the new and old assembly data. Also, since the three-dimensional model of the selected difference data whose position or orientation has been changed is not superimposed on the target marker 105, it can be seen that it is not superimposed on the target marker 105 either.

FIG. 16 shows a case where deleted differential data is selected. As indicated by 1601 in the old assembly data, the selection difference data (car muffler in FIG. 16) existed in the three-dimensional model of the old assembly data, but has been deleted from the new assembly data. Therefore, the selected difference data is identified and displayed in the old assembly data. Also, since the three-dimensional model of the deleted selection difference data is not superimposed on the target marker 105, it can be seen that it is not superimposed on the target marker 105 either.

As described above, the part data can be browsed in the virtual space without operating the three-dimensional model configured by the part data whose design has been changed in the virtual space.

The present invention can also be embodied as, for example, a system, device, method, program, storage medium, etc. Specifically, it may be applied to a system composed of a plurality of devices, or It may be applied to an apparatus consisting of one device.

It should be noted that the present invention includes those that directly or remotely supply a software program that implements the functions of the above-described embodiments to a system or apparatus. The present invention also includes a case where the computer of the system or apparatus reads and executes the supplied program code.

Therefore, the program code itself installed in the computer to implement (execute) the functional processing of the present invention also implements the present invention. That is, the present invention also includes the computer program itself for realizing the functional processing of the present invention.

In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, or the like.

Recording media for supplying programs include, for example, flexible disks, hard disks, optical disks, magneto-optical disks, MOs, CD-ROMs, CD-Rs, and CD-RWs. There are also magnetic tapes, non-volatile memory cards, ROMs, and DVDs (DVD-ROM, DVD-R).

Another method of supplying the program is to connect to a home page on the Internet using a browser on the client computer. It can also be supplied by downloading the computer program itself of the present invention or a compressed file including an automatic installation function from the home page to a recording medium such as a hard disk.

It is also possible to divide the program code constituting the program of the present invention into a plurality of files and download each file from a different home page. In other words, the present invention also includes a WWW server that allows a plurality of users to download program files for implementing the functional processing of the present invention on a computer.

In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and key information for decryption is downloaded from a homepage via the Internet to users who have cleared predetermined conditions. Let Then, by using the downloaded key information, the encrypted program can be executed and installed in the computer.

Also, the functions of the above-described embodiments are realized by the computer executing the read program. In addition, based on the instructions of the program, the OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments can also be realized by the processing.

Further, the program read from the recording medium is written into a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. After that, based on the instruction of the program, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing, and the processing also realizes the functions of the above-described embodiments.

It should be noted that the above-described embodiments merely show specific examples for carrying out the present invention, and the technical scope of the present invention should not be construed to be limited by these. That is, the present invention can be embodied in various forms without departing from its technical concept or main features.

101 information processing device 102 HMD
103 optical marker 104 infrared camera 105 target marker

Claims (13)

identifying means for identifying a portion where there is a difference between the first three-dimensional model and the second three-dimensional model;
The first three-dimensional model and the second three-dimensional model are independent positions in the virtual space, and the three-dimensional part where the difference exists in the position corresponding to the movable target in the real space display control means for controlling display of a third three-dimensional model, which is a model;
An information processing device comprising: The display control means is
Displaying the third three-dimensional model,
2. The information processing apparatus according to claim 1, wherein control is performed to display at least one of the first three-dimensional model and the second three-dimensional model. The display control means is
The third three-dimensional model is controlled to be displayed at the independent position in the virtual space where at least one of the first three-dimensional model and the second three-dimensional model is displayed. Item 3. The information processing apparatus according to item 2. The display control means is
2. Control is performed to display the third three-dimensional model at the independent position in the virtual space where both the first three-dimensional model and the second three-dimensional model are displayed. The information processing device according to . The display control means is
Displaying the third three-dimensional model,
the first three-dimensional model that identifies and displays the location with the difference identified by the identifying means;
the second three-dimensional model that identifies and displays the location with the difference identified by the identifying means;
5. The information processing apparatus according to any one of claims 1 to 4, wherein control is performed so as to display at least one of . The first three-dimensional model and the second three-dimensional model are three-dimensional models composed of a plurality of parts, and the identifying means comprises the first three-dimensional model and the second three-dimensional model. Identify the parts that differ between
2. The display control means performs control to display the three-dimensional model of the part having the difference specified by the specifying means as the third three-dimensional model in the virtual space. 3. The information processing device according to 2. 7. The portion having the difference is a portion where a design change has been made between the first three-dimensional model and the second three-dimensional model. The information processing device according to . 8. The portion according to any one of claims 1 to 7, wherein the portion having the difference is a portion that exists only in one of the first three-dimensional model and the second three-dimensional model. information processing equipment. 8. The method according to any one of claims 1 to 7, wherein the location having the difference is a portion having a different position and orientation between the first three-dimensional model and the second three-dimensional model. Information processing equipment. 2. The information processing apparatus according to claim 1, wherein the position corresponding to the target is associated with the position of a physical object that the user can hold. 11. The information processing apparatus according to claim 10, wherein the position corresponding to said target is indicated by a marker. A control method for an information processing device,
an identifying step of identifying a portion where there is a difference between the first three-dimensional model and the second three-dimensional model;
The first three-dimensional model and the second three-dimensional model are independent positions in the virtual space, and the three-dimensional part where the difference exists in the position corresponding to the movable target in the real space a display control step for controlling to display a third three-dimensional model, which is a model;
A control method for an information processing device including A program that causes a computer to function as each means of the information processing apparatus according to any one of claims 1 to 11.

Images