JP2005148844A - Display system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a display system which can perform operation for changing an object itself currently displayed in virtual reality. <P>SOLUTION: A host 1 has a display change reception means 14 to receive change of a display form, and an operator in the virtual reality has an input information output means 31 to indicate change of display form and is provided with a portable terminal 3 which transmits the indication inputted in this input information output means 31. The operator in the virtual reality performs intended input using a portable terminal 3, the input information output means 31 transmits input information corresponding to this input to the display change reception means 14, the display change reception means 14 requests processing corresponding to the received input information to a display processing means 11, and the display processing means 11 performs processing based on this request. The operator in the virtual reality can perform these operations without help of others staying in the virtual reality by adding only other operations to the portable terminal 3 to which operations can be easily added versatilely. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a display system for displaying an object, and more particularly to a display system in which a user can experience various operations with a display system and perform various operations while in a remote place.

  As a background art, there is a display system, which will be described below. The display system includes a host 101 having a display processing unit 111 that outputs image information of an object to be displayed as display information, and a display device 102 that displays the display information output by the display processing unit 111. Reference numeral 101 denotes a configuration having display change input means 112 for the operator to instruct a change in display form. According to this configuration, the display processing unit 111 converts the object to be displayed into display information and outputs the display information to the display device 102. The display device 102 displays the input display information, and the operator uses an input device such as a keyboard. If necessary, the display change input means 112 is instructed to change the display form. Based on the change of the display form from the display change input means 112, the display processing means 111 outputs the display information, and the display device is input. If the operator is in the vicinity of the computer on which the host display change input means 112 is mounted, the display change input means 112 can be changed via an input device such as a keyboard as necessary. The display form on the display device 102 can be changed accordingly.

  Here, in the display system, there are operators in virtual reality who operate while experiencing virtual reality, and operators outside virtual reality who operate without experiencing virtual reality. Although it depends on the virtual reality display system, in general, the operator in the virtual reality can handle the displayed object by hand, move the body to navigate and change the display form, The non-virtual reality operator can perform operations such as changing the displayed object and changing the background color. Further, an operator outside the virtual reality cannot experience the virtual reality because it is not in the virtual reality, but can change the display form by handling or navigation by directly changing the parameters. Non-virtual reality operations that can be performed by an operator outside virtual reality include operations within virtual reality that can be performed by an operator within virtual reality. The host is mounted on a computer such as a PC (Personal Computer), and a non-virtual reality operator operates using a keyboard while viewing the PC screen. The display change input unit 112 converts the input from the keyboard into display control information for the display processing unit and outputs the display control information to the display processing unit 111. Various sensors such as a data glove carried by the operator in the virtual reality, when operated by the operator in the virtual reality, output information associated in advance to the operation accepting unit 113 in the virtual reality. The virtual reality operation accepting unit 113 converts the input from various sensors into display control information for the display processing unit 111 and outputs the display control information to the display processing unit 111.

  The background art is configured as described above, and an operator in the virtual reality moves and rotates an object displayed by handling and navigation using various sensors in the virtual reality, and the virtual object outside the virtual reality. Although the non-real operator can change the displayed object, if the virtual reality operator wants to change the object, the virtual reality is notified to the non-virtual operator. It must be performed by an outside operator, or the operator in virtual reality must move to the host display change input means and operate it. It is necessary to interact with other people in conversation and to operate the host in consideration of the real space, which has the problem of impairing the virtual reality experience.

  In addition, an operator in virtual reality can operate an object displayed in virtual reality using an input device such as a data glove, feedback mouse, or Wanda, but tries to add a new operation. In some cases, there is no input device that can be used for general purposes, and thus it is necessary to manufacture or remodel the input device, which entails a problem of enormous development time and cost. Moreover, it cannot be said that the current input device is excellent in the user interface, and it is necessary to get used to using it.

  Therefore, in order to solve the above-described problem, the present invention introduces an input device that can be used quickly and easily in the virtual reality by the operator in the virtual reality, and is displayed using this input device for general purposes. It is an object of the present invention to provide a display system capable of performing an operation on an object or an operation of changing a displayed object itself.

  The display system according to the present invention is a display system comprising: a host having a display processing unit that outputs an object to be displayed as display information; and a display device that displays the display information output by the display processing unit. The host has a display change accepting means for accepting a change in display form, and the operator has an input information output means for instructing a change in the display form, and inputs an instruction input to the input information output means A portable terminal that transmits as information is provided.

  As described above, in the present invention, the host has a display change accepting unit that accepts a change in the display form, and an operator in virtual reality has an input information output means for instructing the change in the display form. And a portable terminal that transmits the instruction input to the input information output means as input information. Therefore, the operator in virtual reality makes an intended input using the portable terminal, and the input information output means corresponds to the input. The information is transmitted to the display change accepting means, the processing corresponding to the input information received by the display change accepting means is requested to the display processing means, and the display processing means performs processing based on this request. In addition to operations that can be performed by sensors such as data globes that are carried by the operator, just by adding other operations to the mobile terminal that can be easily operated in general, Who can perform an operation that takes without borrow the hands of other people while are in virtual reality, it is possible to quickly carry out an effective operation by the loss WAS the experience of virtual reality.

  In some cases, all operations can be performed in the virtual reality even if the operator in the virtual reality is away from the input device of the host. The portable terminal may be any portable terminal, and a PDA is a representative example, but a mobile phone, a sub-note, and the like are also applicable. The display device may be any device, and may be a large display or a three-dimensional display. Depending on the display device, a pair of devices for carrying the VR in order for the operator in virtual reality to experience VR is required. For example, CAVE is stereoscopic shutter glasses.

  The display system according to the present invention is an additional information output unit that is one of the input information output units for adding the additional information to the display form displayed on the display device by the portable terminal, if necessary. Additional information input to the additional information output means is transmitted as display additional information, and the host has additional input information reception means which is one of display change reception means for receiving display additional information. is there. Thus, in the present invention, the portable terminal has additional information output means that is one of the input information output means for adding additional information to the display form displayed on the display device. Additional information input to the information output means is transmitted as display additional information, and the host has additional information reception means that is one of display change reception means for receiving display additional information. Additional information is input using a terminal, and this additional information is displayed as additional display information by the additional information output unit and output to the additional information receiving unit. The additional information receiving unit outputs the display additional information as display control information to the display processing unit. The display processing means performs display processing and the display information is displayed on the display device, and the additional information for the target object etc. that is currently displayed by the operator in virtual reality is actively displayed. It is possible to add a VR with higher added value. Further, when there are a plurality of operators in virtual reality, the operator in virtual reality can display additional information by displaying other information. It is possible to share and experience a value-added target object on which additional information is displayed in common with an operator in virtual reality.

  In addition, the display system according to the present invention uses at least one memo, mark, underline, tag, or link as the additional information as necessary. As described above, in the present invention, the additional information is specifically shown as memo, mark, underline, tag, link, etc., but if it is a memo, mark, or underline, it can be referred to when experiencing the same object VR again. And can be shared with multiple users. The sticky note stores the position where the memo is attached. For example, by selecting a certain sticky note, the memo is displayed and can be viewed from the viewpoint of the position and direction where the sticky note is attached. Even familiar ones cannot move in the same way as in realization, even for beginners, and it is useful to be able to change the viewpoint position without moving / operating on sticky notes in such cases. The link can move to another VR, display an object other than the currently displayed object, call another virtual reality operator, and embed an operation. Further, not only the sticky note but also a memo, a mark, and an underline can be touched to change to an optimal viewpoint and / or position.

  Moreover, the display system according to the present invention handles the additional information in a hierarchical manner as necessary. As described above, in the present invention, since the objects in the virtual reality and the display in the virtual reality related to the additional information are handled hierarchically, the display without additional information and the display with additional information can be quickly switched, Additional information can be deleted all at once, and the display within the virtual reality related to additional information can be handled hierarchically for each user, making it more convenient and improving operability and display. In addition, additional information can be managed individually, and information that is not desirable to be displayed can be hidden.

  Moreover, the display system according to the present invention includes a plurality of the hosts, the display device corresponding to the hosts, and at least one portable terminal as necessary. Thus, in the present invention, a plurality of hosts, a display device corresponding to the host, and at least one portable terminal are provided, and the portable terminal transmits input information to each host. It is possible to operate with one mobile terminal, it is not necessary to go close to each host, it can be managed quickly, and the virtual reality operator who is experiencing virtual reality with other hosts and display devices Even we can respond quickly. Specifically, a case where this display system is applied corresponds to a case where a lecturer who is a doctor conducts a nationwide class to display and teach a patient's brain three-dimensionally in a VR. . Another example is a case where one foreign instructor conducts classes with multiple Japanese students when conducting an English conversation at home. For details, refer to foreign instructors and multiple Japanese. Each student is assigned a host and a display device, and only a foreign instructor can actually perform a management operation. Japanese students can take classes without any operational burden. Here, a Japanese student can also have a portable terminal that cannot be managed, so that the target object can be operated.

(First embodiment of the present invention)
A display system according to a first embodiment of the present invention will be described with reference to FIG. FIG. 1 is an overall block diagram of a display system according to this embodiment.

  In FIG. 1, the display system according to the present embodiment includes a display processing unit 11 that outputs image information of an object to be displayed as display information and a display for an operator to instruct a change in display form. A host 1 having a change input unit 12 and a display device 2 for displaying display information output from the display processing unit 11 are provided. In addition to this, in this display system, the host 1 displays a display for accepting a change in display form. A mobile phone which has a change receiving unit 14 and has an input information output unit 31 for an operator in virtual reality to instruct a change in display form, and transmits an instruction input to the input information output unit as input information The terminal 3 is provided.

  In order to make the following description easy to understand, VR image information used in the display system according to the present embodiment will be described. However, the image information described here is only an example, and a number of methods for realizing VR have already been proposed, and image information other than the image information described here can be used in the present display system. The creator creates an object by combining primitives such as a sphere, a rectangular parallelepiped, and a cylinder using a modeling tool or the like. In addition to combining them, operations such as rotation, parallel movement, and enlargement / reduction are added to create a desired object. Furthermore, the color, material, and light source of each primitive are set. That is, the image information of the object includes image information of at least one primitive, and the primitive image information includes primitive type information, position information, enlargement / reduction information, rotation information, parallel movement information, color information, It consists of material information and light source information. Specifically, the image information of an object includes a VRML (Virtual Reality Modeling Language) format and a DXF (Drawing Interchange File) format, and various other formats are prepared.

  Here, VRML is a specification for describing 3DCG handled on the Internet, and DXF is a de facto standard format for CAD and 3D data. Only the image information of this object cannot be displayed on the display device 2, and further, viewpoint information is required. This is because the displayed object must change according to the position and orientation of the viewpoint. A space for modeling an object is called a modeling space, and coordinates used in this modeling space are called object coordinates. A space for determining the position and orientation of the viewpoint is called a virtual space, and the coordinates used in this virtual space are called view coordinates.

  Since the position information in the image information of the object is on the object coordinates, it is necessary to perform coordinate conversion to those on the view coordinates. However, it is only necessary to perform coordinate conversion on the image information of the object in the viewing volume that is the range visible through the display screen. This is because it is useless even if coordinate conversion is performed on image information of an object that is not displayed on the display device 2. The image information of the object after coordinate conversion is used as real image information. Next, if it is a normal display device, it is converted into coordinates on the two-dimensional plane of the display device 2, and if it is a special display device, it is converted into coordinates according to the display device 2. Information converted into coordinates corresponding to the display device 2 is used as display information. Objects are displayed on the display device 2 by this display information, but since all the wire frames are visible, hidden line elimination and surface hiding are performed so as not to draw hidden lines in the wire frames. Processes such as shading to prevent the drawing of the shaded area, shading to add color and shading, etc. are performed.

  The operations that can be performed by the non-virtual reality operator (hereinafter referred to as non-virtual reality operations) include changing the viewpoint, moving the object, transforming the object, scaling the object, changing the color of the object, and changing the origin of the view coordinates. The background color, which is a color other than the object, can be changed, the object can be added / deleted, and the virtual reality space can be switched. The non-virtual reality operator can directly operate the host 1 and can perform all operations.

  On the other hand, since the operator in virtual reality cannot directly operate the host 1 unless he / she goes to the host 1, only a part of the operation outside virtual reality (operation in virtual reality) can be performed. Although the internal operation depends on the sensor carried by the operator in the virtual reality, it is about the movement of the object, the deformation of the object, the enlargement / reduction of the object. Sensors include magnetic sensors, ultrasonic sensors, optical sensors, and data gloves. The reason why an operator in virtual reality can only perform operations in virtual reality and cannot perform operations outside virtual reality except for operations in virtual reality is that the operations of operators in virtual reality are limited in order to make virtual reality closer to reality. Conceivable.

  That is, in reality, the object cannot be scaled or the color of the object cannot be changed. However, there are cases where it is effective to be able to perform operations in virtual reality that cannot be performed in actuality. In such a case, an operation can be associated with an input of the sensor. For example, a data glove can be used, and the data can be changed to the next color of the object by firmly grasping the data glove. However, even if an operator in virtual reality actually performs a data glove in the virtual reality for a purpose other than changing the object to the next color, the object is changed to the next color, and the operator in virtual reality This situation may be contrary to the intention, and this situation can be considered as the number of operations to be associated increases. After all, it is preferable to use the data glove within the range originally intended by the data glove, and it is not preferable to additionally associate the operations with each other, and it is necessary to provide a separate input means. Further, as the input means, an input means specialized for a certain operation may be used, but such an input means is expensive to manufacture, is expensive to purchase, and is scalable to accommodate other operations. Since it lacks, the portable terminal 3 was selected.

  The portable terminal 3 corresponds to, for example, a PDA (Personal Digital Assistant) or a cellular phone. Some PDAs can be held with one hand and operated with one hand and have high operability (some types can be operated with one hand), and some have built-in communication means. Even a PDA that does not include a communication means can communicate by connecting with a communication device through an interface. The PDA is equipped with several switches and can be operated on a touch screen. As a merit when using a PDA as compared with a mobile phone, not only in terms of visibility and operability, but also in the case of a PDA, using various communication devices is an advantage and excellent extensibility.

  In other words, communication can be made peer-to-peer with an infrared port, and when a wireless LAN is used, the communication range is further expanded and the movement range of the operator in the virtual reality is also widened. In the case of using a mobile phone, it is necessary to use a mobile communication means, and the communication cost becomes enormous due to packet charging. In particular, in the case of multimedia, the amount of data handled becomes enormous and the packet fee becomes enormous. By using PHS, it is possible to perform packet communication for a fixed amount, but communication costs are still higher compared to communication not via a communication carrier such as a wireless LAN (wireless LAN using an ISM band). Further, the communication may be wireless communication in another band (for example, Bluetooth may be used) other than the infrared communication and the wireless communication in the band used in the wireless LAN. Although wireless communication was explained, this was made to improve mobility so that the communication cable can move freely without causing any adverse effects when the operator in virtual reality moves. Even if it is wired communication, it can be used if it does not cause adverse effects on mobility.

  The input information output means 31 of the portable terminal 3 generates input information according to the input of the operator in virtual reality from a switch or a touch screen, and outputs it to the host display change accepting means using a communication means (not shown). To do. The input information generated by the input information output means 31 can be considered in several formats, and the processing performed by the display change receiving means 14 varies depending on this format. The input information corresponds to input information in an operation format indicating what operation has been performed on the input device on the portable terminal 3 such as a switch or a touch screen. For example, which switch is pressed by how many times This is information indicating which part of the touch screen has been pressed. The input information in the other format corresponds to the input information in the control format, and this input information can be, for example, display control information. In addition, input information in a display format is applicable, and this input information can be, for example, display information. Depending on the format, not only the processing of the input information output means but also the processing of the display change receiving means 14 and the processing of the display processing means 11 are different.

  In the case of operation type input information, the input information output unit 31 only outputs the input information indicating what operation has been performed to the display change receiving unit 14. The display change accepting unit 14 is a display control that is a table that knows in advance what kind of display control information should be generated if any switch or touch screen is operated when the mobile terminal 3 is in any display state. It has an information conversion table (may be realized by logic), and generates display control information with reference to this table. In this case, the display processing means 11 only operates in the conventional manner. In this embodiment, input information in this operation format is used.

  In the case of the input information in the control format, the input information output means 31 has a display control information conversion table, generates display control information using this display control information conversion table, and outputs it to the display change accepting means 14. . The display change accepting unit 14 simply outputs the display control information output from the input information output unit 31 to the display processing unit 11, and the display processing unit 11 also operates as usual.

  In the case of display format input information, the input information output means 31 has a display control information conversion table, which is used as display control information using the display control information conversion table. Is performed based on the display control information, and is output to the display device 2 via the display change receiving means 14 and the display processing means 11.

  As long as the host 1 has the display processing means 11 and the display change accepting means 14, there are various devices that can be specifically realized. Generally, the host 1 is realized by creating a program on a computer such as a PC. The There are no particular restrictions on the type of computer, and it may be a workstation or a supercomputer, or may be a graphic-dedicated computer that specializes in graphic processing and realizes high-speed processing by partially implementing hardware. Further, it is not necessary to consist of one computer, and a cluster configuration consisting of a plurality of computers may be used.

  Examples of the special display device 2 include a large display CAVE (immersive 3D visualization device), an HMD (Head Mounted Display), and a stereoscopic display. This CAVE system is composed of a stereo projector, a reflecting mirror that reflects the light emitted from the projector, and a projection screen. The sensors carried by the operator in virtual reality include stereoscopic shutter glasses and head-linked display. There are magnetic sensors for. When CAVE is used, the display information output from the display processing unit 11 is display information corresponding to CAVE. Specifically, in CAVE, specifically, the stereoscopic shutter glasses perform alternate shuttering of the left lens and the right lens several tens of times per second, and the image for the left lens and the image for the right lens are accordingly received from the stereo projector. As a display information, such a frame for each lens must be output several tens of times per second.

  Next, the operation of the display system according to this embodiment will be described. The host 1 and the display device 2 are activated by the operation of the non-virtual reality operator. Next, the non-virtual reality operator operates to select the target object and display it on the display device 2 using the keyboard of the PC on which the host 1 is mounted. This operation is input to the display change input unit 12, display control information corresponding to the operation is generated, and output to the display processing unit 11. The display processing unit 11 converts the image information of the target object into display information based on the input display control information and outputs the display information to the display device 2. The display device 2 performs display based on the display information. The virtual reality operator can experience VR by performing the display.

  As the operator in the virtual space moves in the virtual space, the position and orientation of the viewpoint changes, and this information is input to the virtual reality operation accepting means 13 to obtain the displacement of the viewpoint and display processing information as display control information. 11 is output. Based on the input display control information, the display processing means 11 converts the image information of the target object as display information and outputs it to the display device 2, and the display device 2 displays the display information. Thus, the virtual reality operation is performed.

  Assume that an operator in virtual reality wants to perform an operation outside virtual reality, for example, an attempt is made to display an object other than the current target object in virtual reality in the virtual space. In this case, the operator in virtual reality operates to select and display an object to be displayed using an input device such as a touch screen of the mobile terminal 3. The input information output unit 31 outputs input information indicating what operation has been performed to the display change receiving unit 14. The display change receiving unit 14 generates display control information with reference to the display control information conversion table. Based on the input display control information, the display processing means 11 converts the image information of the target object into display information and outputs it to the display device.

  As described above, according to the display system according to the present embodiment, the host 1 has the display change accepting unit 14 that accepts the change of the display form, and the input for the operator in virtual reality to instruct the change of the display form. And a portable terminal 3 having the information output means 31 and transmitting the instruction input to the input information output means 31 as input information, so that an operator in virtual reality makes an intended input using the portable terminal 3. The input information output means 31 transmits the input information corresponding to this input to the display change accepting means 14, and requests the display processing means 11 to perform the process corresponding to the input information received by the display change accepting means 14. The display processing means 11 performs processing based on the above, and in addition to operations that can be conventionally performed by a sensor such as a data glove that is carried by an operator in virtual reality, operations can be easily performed on a general basis. By simply performing another operation on the portable terminal 3 that can be operated, the operator in the virtual reality can perform the operation without taking the help of another person while in the virtual reality, and the virtual reality experience is impaired. Therefore, more effective operation can be performed quickly.

  Here, in addition, when the non-virtual reality operator and the virtual reality operator are located at a distance from each other, characters and voices are transmitted between the portable terminal 3 and the computer on which the host is mounted. You can also exchange. Further, when almost all the operations outside the virtual reality can be performed from the mobile terminal 3, the operations can be limited for each operator in the virtual reality, the mobile terminal 3, or the target object. For example, when a certain important object is displayed on the display device 2, only a limited virtual reality operator can perform the deletion operation.

  In the display system according to the present embodiment, the activation part is performed by operating the host 1 by the non-virtual reality operator. However, the activation part can also be implemented so as to be operated by the mobile terminal 3. VR can be realized without moving in the virtual reality operator, and until now, VR has been realized jointly by a plurality of humans, but it can be started quickly without requiring any manpower. it can.

(Second embodiment of the present invention)
A display system according to a second embodiment of the present invention will be described with reference to FIG. FIG. 2 is an overall block diagram of the display system according to the present embodiment.

  Similar to the display system according to the first embodiment, the display system according to the present embodiment is a display processing unit 11 that outputs image information of an object to be displayed as display information. Host 1 having display change input means 12 for instructing change and display change accepting means 14 for accepting a change in display form, display device 2 for displaying display information output by display processing section 11, and virtual reality The mobile terminal 3 includes an input information output unit 31 for an operator to instruct a change in display form, and transmits an instruction input to the input information output unit as input information. In addition, The portable terminal 3 has additional information output means 31a which is one of input information output means 31 for adding additional information to the display form displayed on the display device. Sending the additional information input to the output unit 31a as the display additional information, the host 1 is the configuration with is one additional information receiving means 14a of the display change receiving unit 14 for receiving the display additional information.

  The additional information is something other than an object or the like necessary for the virtual reality displayed by the display device 2 and is not necessarily required for the virtual reality only for experiencing, but makes it stand out from other parts other than experiencing it. Or a mark, or a description, and specifically, a memo, a mark, an underline, a sticky note, or the like.

  The additional information output means 31a converts the additional information input using an input device such as a touch screen of the portable terminal 3 to the additional information receiving means 14a as display additional information that is one of the input information.

  The additional information receiving unit 14a refers to the display control information conversion table based on the input display additional information and outputs the display control information to the display processing unit 11.

  Since the additional information output unit 31a is called by an operation in the virtual reality operator, there is a problem even when the input information output unit 31 adds a unit corresponding to another operation in addition to the additional information output unit 31a. Absent. However, the additional information receiving unit 14a is not called by the operation of the operator in the virtual reality, and there is no problem if the display change receiving unit 14 is not associated with any operation other than displaying the additional information. If the operation is associated, a mechanism for identifying the display additional information in a plurality of pieces of input information is necessary, and information that can be identified is embedded in the display additional information to be transmitted, or by communication. Need to identify. The identification by communication means that a session is established exclusively for the additional information receiving means 14a.

  Next, the operation of the display system according to this embodiment will be described. The operation until the virtual reality operator can experience VR in the first embodiment and the operation based on the virtual reality operation have already been described. Here, the operation for displaying additional information is performed. Only will be described.

  The operator in the virtual reality inputs the additional information by using the input device of the portable terminal 3 to add the additional information to the certain part of the target object displayed in the virtual reality for display. To do. The additional information output unit 31 converts the input into display additional information and outputs it to the additional information receiving unit 14a. Here, since it can be said that additional information is added to a certain part of the target object, the display additional information includes coordinate information and additional information of the part. The additional information receiving unit 14a refers to the display control information conversion table based on the input display additional information and outputs the display control information to the display processing unit 11. The display processing unit 11 converts the image information of the target object into display information together with additional information based on the input display control information, and outputs the display information to the display device 2. The display device 2 performs display based on the display information. The operator in virtual reality can experience VR by displaying additional information along with the target object in such a portion.

  As described above, according to the display system according to the present embodiment, the mobile terminal 3 is one of the input information output means 31 for adding the additional information to the display form displayed on the display device. Additional information received as one of display change receiving means 14 that has output means 31a, transmits the additional information input to the additional information output means 31a as display additional information, and the host 1 receives the display additional information. Since the means 14a is provided, the operator in the virtual reality inputs additional information using the portable terminal 3, and this additional information is displayed as additional display information by the additional information output means 31a and is output to the additional information receiving means 14a. The display additional information is output as display control information to the display processing means 11 by the means 14a, the display processing means 11 performs display processing, and the display information is displayed on the display device 2. The virtual reality operator can positively add additional information to the currently displayed target object, etc., and can realize a VR with higher added value. When there are a plurality of virtual reality operators, additional information is displayed by the operator within the virtual reality so that a shared object with a value-added target object in which additional information is displayed can be shared and experienced. be able to.

  Further, in the display system according to the present embodiment, the additional information is specifically shown as memo, mark, underline, sticky note, etc., but if it is a memo, mark, or underline, it is referred to when experiencing the same object again in VR. And can be shared with multiple users. The sticky note stores the position where the memo is attached, and for example, by selecting a certain sticky note, the memo is displayed and can be viewed from the viewpoint of the position and direction where the sticky note is attached.

  In the display system according to the present embodiment, the additional information is added in the virtual reality, but the objects in the virtual reality and the display in the virtual reality related to the additional information can be handled hierarchically. The display without additional information and the display with additional information can be quickly switched, and the additional information can be deleted at once. In addition, it is possible to improve the convenience by handling the display in the virtual reality related to the additional information for each user in a hierarchical manner, and not only the operability and the display are improved, but also the additional information can be managed individually. It is also possible to cover information that is not desirable to display.

(Third embodiment of the present invention)
A display system according to a third embodiment of the present invention will be described with reference to FIG. FIG. 3 is a system configuration diagram of the display system according to the present embodiment.
The display system according to this embodiment includes a plurality of hosts 1 and display devices 2 of the display system according to the first embodiment, and one mobile terminal 3 transmits input information to each host 1. is there.

  In the display system according to each of the embodiments, the host 1 and the mobile terminal 3 have a one-to-one or one-to-many configuration. However, in the display system according to the present embodiment, the host 1 and the mobile terminal 3 have a multi-pair. 1 configuration. When the input information is transmitted from the portable terminal 3 to the host 1, the input information may be transmitted to a plurality of hosts 1 when the input information is transmitted to only one host 1. The former is single transmission and the latter is broadcast transmission.

  An operation of the display system according to the present embodiment will be described. Since the operation until the virtual reality operator can experience VR and the operation based on the virtual reality operation have already been described in the first embodiment, single transmission and broadcast transmission will be described. .

  It is assumed that the operator in virtual reality is in the virtual reality displayed by a certain host 1 and display device 2. Since the operations of the host 1 (referred to as the own host) and the display device 2 (referred to as the own display device) displaying the virtual reality where the user is present are the operations described in the display system according to the first embodiment, the description thereof will be omitted. In the case of operating on the host 1 other than the own host 1 and the own display device 2 and in the case of single transmission, the host 1 that the user wants to give an operation is first selected and the intended operation is performed via the portable terminal 3. In the case of operating on the host 1 other than the own host 1 and the own display device 2, in the case of broadcast transmission, the broadcast transmission is designated and the intended operation is performed via the portable terminal 3.

  As described above, according to the display system according to the present embodiment, a plurality of hosts 1 are provided, and one mobile terminal 3 transmits input information to each host 1. It is possible to operate, it is not necessary to go to the vicinity of each host 1, it can be managed quickly, and an operator in virtual reality who is experiencing virtual reality with another host 1 and display device 2 Can also respond quickly.

  In the display system according to the present embodiment, input information is transmitted to each host 1, but in the case of single transmission, the virtual reality displayed by the selected host 1 is further displayed on the display device 2 itself. It becomes easier to perform the operation by displaying. In the case of broadcast transmission, the virtual reality can be switched and displayed on the own display device 2 for each host, or can be easily operated by split display.

  Hereinafter, embodiments of the present display system will be described based on FIGS. 4 to 10 together with experiments. 4 is an external perspective view of the display device and the mobile terminal according to the present embodiment (the host is a general PC and is omitted), FIG. 5 is an external perspective view of the mobile terminal according to the present embodiment, and FIG. 7 is an external perspective view of the stereoscopic shutter glasses for use according to the embodiment, FIG. 7 is a GUI mounted on the PDA, FIG. 8 is a configuration diagram in which a menu is displayed in the VR according to the embodiment, and FIG. 9 is an evaluation result regarding the GUI. FIG. 10 shows a plurality of tables that are evaluations related to drawing.

The host computer on which the host 1 is mounted has an Intel Pentium III (1 [GHx]) in the CPU.
), And a PC using ATI FireGL4 on the video board. This host computer used RedHat 7.1 for the OS, VTK for the visualization library, OpenGL Performer for the graphic library, and CAVE library for the VR library, the OS was installed on the PC, and various libraries were arranged at appropriate positions.

  VTK is an open source class library that runs on Unix, Windows, and Linux to visualize data. VTK provides various visualization methods such as isosurfaces, volume rendering, and vector field display. The core part is implemented in C ++, but processing by Java, Tck / TK or Python is also possible. VTK originally does not support display with multiple View Ports. Therefore, Performer is used to link the CAVE library and VTK.

  Performer is widely used in 3D graphics, and the CAVE library has a Performer version (pfCAVE library). Performer is good at controlling multiple View Ports or multiple channels in multiple processes. The geometry generated by VTK is stored in a data structure called vtkPolyData. vtkPolyData is converted to pfGeode, which is one of Performer nodes, using vtkActToPF and incorporated as part of the Performer scene graph.

  Since the conversion is executed only when the VTK pipeline part is re-executed, the VTK data can be displayed by the Performer without stress when the viewpoint is changed or moved by the VR. Once converted, Performer and VTK can manipulate objects with each other. Performer explicitly checks for VTK pipeline updates. When the VTK pipeline is updated, a new pfGeode is generated by the vtkActorToPF-Translator object of vtkActorToPF again and the VR is updated.

  As the display device 2, CAVE ImmesaDesk was used. ImmesaDesk includes a stereo projector (not shown), a reflecting mirror (not shown) that reflects light emitted from the projector, and a screen 2a for projection. The screen 2a is tiltably held. With this ImmesaDesk, stereoscopic shutter glasses 2b for an operator in virtual reality to experience VR are used, and a spatial sensor 2c called Fastrak (Polhemus) is disposed on the arm portion of the stereoscopic shutter glasses 2b. In order to widen the measurement range by the space sensor 2c, the LongRanger 2d is disposed in the forward direction of ImmesaDesk. The space sensor and the host computer are connected by an RS-232 cable 2e.

  The PDA 3a (Toshiba Genio e550GX OS: PocketPC2002) was used for the portable terminal 3. A wireless LAN adapter 3b is attached to the PDA 3a, and communicates with a host computer via an access point (not shown) and a switching hub (not shown). TCP / IP is used as the communication protocol. As with the space sensor 2c, Fastrac (Polhemus) was used as the space sensor 3c and attached to the back end of the separation plate 3d attached to the back surface of the PDA 3a. The spatial range of the spatial sensor 3c is widened by the LongRanger 2d similarly to the spatial sensor 2c.

  The separation plate 3d to which the space sensor 3c is attached is attached with velcro so as to be detachable from the PDA 3d. The separation plate 3d is attached to the PDA 3a and has a tip about 10 [cm] ahead of the tip of the PDA 3a so that the PDA 3a and the space sensor 3c are kept separated. This is because the space sensor 3c is affected by magnetism, and information for the current position (the position and inclination of the space sensor) cannot be measured correctly. The influence of magnetism is considered to be from a metal plate forming the PDA 3a, a power supply unit (not shown) and a transformer (not shown) disposed inside the PDA 3a.

  As described above, if the separation plate is detachable from the PDA 3a, the RS-232 cable 3e connected to the space sensor 3c does not cause any adverse effects when used alone with the PDA 3a. The PDA 3a has a 4.0 type liquid crystal screen 3f at the top, a plurality of program buttons 3g at the bottom, and a cursor button 3h at the bottom center. The liquid crystal screen 3f is a touch screen, and a stylus pen 3i is provided so that the touch screen operation can be performed more easily.

  With the configuration as described above, the correspondence between the display system according to each embodiment and the examples will be described. The display processing means 11 for mounting the host 1 is realized by a function call in a general library, VTK and OpenGL Performer. The display change input means 12, the virtual reality operation receiving means 13 and the display change receiving means 14 are generally used. This is realized by calling a function in an API, VTK, and OpenGL Performer prepared for a simple library and OS. The virtual reality operation accepting means 13 also uses a CAVE library function call. However, mounting in this way is an example, and mounting using other libraries is also possible.

  Next, the input information output means 31 of the portable terminal 3 is realized by API prepared by PocketPC2002, which is the OS of the PDA 3a, and a general library function call. The portable terminal 3 has a GUI shown in FIG. 7 so that an operator in virtual reality can use it quickly and smoothly. Through this GUI, the input information output means outputs the input from the operator in virtual reality as input information to the host 1. Switching pages from FIG. 7A to FIG. 7D is performed by switching tabs arranged at the top of the screen. Each page is called “Button page”, “Slider page”, “Drawing page”, “File I / O page”, and the corresponding command is sent to the host 1 when the button is selected on “Button page” By adjusting the slide button left and right on the “Slider page”, the parameter change corresponding to the corresponding slide is transmitted to the host 1, and the additional information input in the drawing window is transmitted to the host 1 on the “Drawing page”. This additional information can be saved or read on the “File I / O page”.

  In this embodiment, two types of drawing methods (methods for displaying additional information in the VR) are mounted on the PDA 3a. One is a method of drawing by moving the PDA 3a while pressing the program button 3g of the PDA 3a (referred to as a first drawing method), and a method of drawing after drawing on the touch screen of the PDA 3a and sending to the host 1 collectively (drawing) A second drawing method).

  In the first drawing method, drawing is performed directly in the VR using the virtual pen displayed on the VR by moving the PDA while pressing the program button 3g of the PDA 3a. The pen tip of the virtual pen is displayed at a depth of 15 [cm] and an upward direction of 7 [cm] from the tip of the PDA 3a. If it is wrong, the data drawn one by one is erased by pressing another program button 3g of the PDA 3a. Alternatively, all data drawn by the program button 3g disposed on the side surface of the PDA 3a is erased at once. In this first drawing method, drawing is performed using a virtual pen in a three-dimensional space, so that the drawn object is displayed three-dimensionally.

  On the other hand, in the second drawing method, “Drawing page” is selected on the tab, and drawing is performed using the stylus pen 3 i in the window in “Drawing page”. “POST IT” button to send drawing data to the host 1, “OK” button to decide to paste the drawing data at an arbitrary position in VR, drawing in the window when drawing is wrong on “Drawing page” The drawing is performed using the “CANSEL” button for initializing “CLEAR” and the “CLEAR” button for deleting drawing data on the “Drawing page”. In other words, the virtual reality operator draws in the “Drawing page” window, sends the drawing data drawn by the “POST IT” button to the host 1, and the additional information is displayed at the tip of the virtual pen and can be moved freely. You can paste it by pressing the “OK” button at the desired position.

  Next, the GUI is evaluated. The evaluation related to the GUI is for evaluating the usability of the GUI, and is performed by comparing the conventional method with the method described in the present embodiment. The target work is to create the same object in the wire frame of the VR space as the green tilted cube displayed on the right side of the VR.

  In the conventional method, a menu displayed in the VR is selected by using various sensors such as a data glove carried by the operator in the virtual reality (see FIG. 8, such a VR is used). The menu appears inside, but it is 3D rather than 2D, so it is difficult to select the menu.) Here, the spatial sensor 3c of the PDA 3a is used instead of the data glove. The method described in the present embodiment is specifically a method performed using “Button page” and “Slider page” (in this case, the menu is not displayed in the VR as shown in FIG. 7 (a) and FIG. 7 (b) using the PDA GUI).

  In the conventional method, first, the program button 3g of the PDA 3a is pressed to display a menu, the pointer displayed on the VR is selected by moving the space sensor 3c, and the parameters such as movement and rotation of the object are changed by menu selection. Move the slider button with the pointer in the same way. In the method described in this embodiment, the button of “Button page” corresponds to the selection item of the menu of the conventional method, and the command is transmitted to the host 1 by selecting the button, and the slide button of “Slider page” is used. Change parameters such as moving and rotating objects.

  The subjects were seven male students who had never used either method, who had practiced about three times until they understood how to use it before actually performing the work. Each of the operations described above was performed in the manner described in the examples. FIG. 9 shows the results of the experiment. “Using 3DMenu” is a result obtained by the conventional method, and “Using PDA” is a result obtained by the method described in this embodiment.

  In the bar graph of FIG. 9, the vertical axis represents time taken for work, and the horizontal axis represents each subject. What can be derived from this bar graph is that it is possible to perform in approximately 60 [%] time performed by the method described in the present embodiment for all subjects in the time performed by the conventional method. The usefulness of the methods described in the examples can be seen.

  Finally, the drawing is evaluated. The evaluation related to drawing is performed by comparing two types of the first drawing method and the second drawing method described in this embodiment. There are two target tasks. The first task is to draw the surname of the subject in a wire frame cube of 40 [cm] on one side in VR, and the second task is 45 [on one side in VR. cm] cubes are circled from three directions. The subjects are 22 students who have no drawing experience with PDAs and practice with each drawing method only once before the evaluation experiment.

  The evaluation was performed based on subjective evaluation and drawing time in five stages (5: easy to use, 1: difficult to use). The drawing time is the time from when the button is first pressed in the first drawing method until all drawing is completed, and is pasted from the time when the stylus pen 3i touches the touch screen in the second drawing method. The time until the “OK” button for determining the attachment position is pressed is targeted. FIG. 10 shows the result of the experiment.

  FIG. 10A shows the result of the first work, and shows the subjective evaluation, the average drawing time, and the median. When attention is paid to the value of the subjective evaluation of usability, it can be seen that the second drawing method is more useful. FIG. 10B shows the result regarding the second work. From the drawing time, it can be seen that the first drawing method is quick and useful for drawing characters. FIG. 10C shows the average drawing time of experienced and inexperienced people using the second method. In the evaluation experiment, “Towa” and drawing were performed in the same manner as the second drawing method. Subjects were divided into groups of four experienced and inexperienced people. Experienced persons averaged 5.61 [seconds], while inexperienced persons averaged 15.57 [seconds].

1 is an overall block diagram of a display system according to a first embodiment of the present invention. It is a whole block diagram of the display system concerning a 2nd embodiment of the present invention. It is a system configuration figure of a display system concerning a 3rd embodiment of the present invention. It is an external appearance perspective view of the display apparatus and portable terminal which concern on an Example. It is an external appearance perspective view of the portable terminal which concerns on an Example. It is an external appearance perspective view of the stereoscopic shutter glasses for use concerning an example. It is GUI mounted on PDA which concerns on an Example. It is the form figure by which the menu was displayed within VR which concerns on an Example. It is a bar graph which is an evaluation result about GUI concerning an example. It is a some table | surface which is evaluation regarding the drawing which concerns on an Example. It is a whole block diagram of the conventional display system.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Host 11 Display processing means 12 Display change input means 13 Virtual reality operation acceptance means 14 Display change acceptance means 14a Additional information acceptance means 2 Display device 2a Screen 2b Stereoscopic shutter glasses 2c Spatial sensor 2d LongRanger
2e RS-232 cable 3 Mobile terminal 3a PDA
3b Wireless LAN adapter 3c Spatial sensor 3d Spacer plate 3e RS-232 cable 3f LCD screen 3g Program button 3h Cursor button 3i Stylus pen 31 Input information output means 31a Additional information output means 101 Host 111 Display processing means 112 Display change input means 113 Virtual Real operation reception means 102 Display device

Claims (5)

In a display system comprising a host having display processing means for outputting an object to be displayed as display information, and a display device for displaying display information output by the display processing unit,
The host has a display change accepting means for accepting a change in display form;
A display system comprising an input information output unit for an operator to instruct a change in display form, and transmitting an instruction input to the input information output unit as input information. The display system according to claim 1,
The portable terminal has additional information output means that is one of input information output means for adding additional information to the display form displayed on the display device, and the additional information input to the additional information output means Send information as additional display information,
The display system, wherein the host has additional input information receiving means which is one of display change receiving means for receiving display additional information. In the display system according to claim 1 or 2,
A display system comprising at least one memo, mark, underline, tag, and link as the additional information. The display system according to claim 3, wherein
A display system characterized in that the additional information is handled hierarchically. A display system comprising a plurality of hosts according to claim 1, a display device corresponding to the host, and at least one portable terminal.
A display system, wherein the portable terminal transmits input information to each host.