JP2002082751A - Interaction device with virtual space and virtual space system using it - Google Patents

Abstract

(57) [Summary] [Problem] It is possible to provide a variety of contents using virtual reality technology so that it can be used by many subjects at facilities and hospitals without any special restrictions on the background and use environment, Provided is a virtual space system that is optimal for a body and body activating device that can further enhance the effect of inducing a user's body and mind activities. SOLUTION: This is a virtual space system to which an interaction device 100 with a virtual space having a user portable unit 1 having a shape that can be easily formed by a user is applied. A three-dimensional visual presentation of the virtual object to the user, and a predefined relationship between the three-dimensional position of the virtual object perceived by the user and the three-dimensional position of the user portable unit 1 obtained by the interaction device with the virtual space. , It is determined that contact with the virtual object has occurred, the visual shape of the virtual object is changed, and
And a virtual space generation control device 3 that generates an interaction control signal for causing the computer to present vibration and sound.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION The present invention relates to the use of virtual reality technology, for example, for the elderly who are in a facility for the elderly, receive a day care service, or a patient who has been in a hospital for a long time. The present invention relates to an apparatus for interacting with a virtual space for the purpose of activating an activity and accompanying mental activity, and a virtual space system to which the apparatus is applied.

[0002]

2. Description of the Related Art In a virtual space system having the same object as the present invention, a virtual reality system is applied, in which a video of a user is captured by a camera, a mirror image is superimposed on the virtual space, and a mirror image is presented. In some cases, contact with a virtual space is determined on a two-dimensional view, and a game-like action is performed by interacting with the virtual space. No patent application has been filed for this type of device, but a paper by Wakamatsu et al. (A theater-type virtual space entertainment environment created with a demented elderly and a computer, Health Science, Vo
l.13, No.3, pp.150-156, 1997),
"Virtual play" STUDIO KOKORO, which is being developed by a certain company with the subsidy of NEDO, is a commercial system, Mandara of V company
There is the Virtual Reality System.

FIG. 5 shows the configuration of a system "theater type virtual space entertainment environment" in the above-mentioned paper by Wakamatsu et al. In the figure, 9
0 is a user, 91 is a blue back screen, 92 is a mirror, 93 is a video camera, 94 is a video board, 95 is a computer, 96 is a projector, and 97 is a presentation screen. Next, the operation will be described. The computer 95 provides a game-like content that constructs a planar virtual space and activates physical exercise. For example, like a paper balloon.

On the other hand, an image of a user 90 in front of a blue back screen 91 reflected on a mirror 92 is captured by a video camera 93 and is captured by a computer 95 via a video board 94. The computer 95 cuts out the person from the captured image by removing the blue portion of the background, and superimposes the cut out person and the visual image of the virtual space created in the computer on the video board 94. Via the projector 96, the image is projected on a display screen 97.

At the same time, the hands and feet of the cut person,
It is determined that an object in the virtual space (paper balloon in the above example) has touched due to overlap on a plane, and changes are made to the virtual object (such as a balloon that has fallen and rises again). The user 90 visually determines the positional relationship between the mirror image of the user on the display screen 97 and the object in the virtual space, and tries to contact the virtual object. In addition, as a real system with the same purpose (without using virtual reality),
Alligator crocodile panic (registered trademark: registered No. 2233)
No. 273), there is a system in which the operating speed of an existing game for amusement is changed, and the system is diverted for rehabilitation.

[0006]

In the first example using virtual reality (that is, the system shown in the above-mentioned paper by Wakamatsu et al.), One's own movement is represented by a plane mirror image, Because the relationship between positions must be understood only by visual information on a plane, and only a visual burden is imposed, and since it is not possible to obtain a real feeling of touching an object in a virtual space, for elderly people and patients, There was a difficulty that it was hard to understand. Further, there is a troublesome restriction in use such as a need to set a blue background for extracting only a human image. On the other hand, in the second example using an actual system (such as crocodile crocodile), one system cannot provide a large amount of content (for example, a training program), and the content is adjusted to the function level of the user. Nor could it be controlled in accordance with the state of service and the state at that time.

[0007] The present invention has been made to solve the above problems, and appeals not only to visual sense but also to a plurality of senses, and allows interaction with a virtual space that can be easily understood by elderly people and patients. Operation) It is intended to provide a device. In addition, by applying such an interaction device, various contents using virtual reality technology can be used at many facilities and hospitals without any special restrictions on the background and usage environment. (Training program) It is an object of the present invention to provide a virtual space system that is optimal for a mental and physical activation device and the like that can further enhance the effect of inducing a user's physical and mental activities.

[0008]

An interaction device according to the present invention is a three-dimensional position measuring marker for measuring a three-dimensional position in a virtual space, a vibration presenting device, a sound presenting device, and transmits and receives signals to and from these markers. A signal transmitting and receiving device is integrated, and a signal from a three-dimensional position measurement marker is input via a user portable unit having a shape that can be carried by a user and a signal transmitting and receiving device of the user portable unit. The position of the portable part of the person is measured and the position measurement information is transmitted to the virtual space generation control device, and the vibration presentation signal is generated according to the interaction control signal generated by the virtual space generation control device based on the transmitted position measurement information. And a control unit for transmitting a sound presentation signal to the vibration presentation device and the sound presentation device of the user's portable unit via the signal transmission / reception device.

Further, a three-dimensional position measuring device for a user's portable unit used in the interaction device according to the present invention comprises:
A three-dimensional position measurement marker using an LED arranged on the user's portable part side, a device arranged on the control part side for controlling light emission timing of the three-dimensional position measurement marker, and light emission of the three-dimensional position measurement marker From an image pickup device for picking up images.

Further, the signal transmitting / receiving device of the interaction device according to the present invention is a cordless wireless transmitting / receiving device.

Further, a virtual space system according to the present invention is a virtual space system to which the interaction apparatus with a virtual space according to any one of claims 1 to 3 is applied. 3D visual presentation of the virtual object to the user, and 3
When the three-dimensional position and the three-dimensional position of the user's portable unit obtained by the interaction device with the virtual space have a predefined relationship, it is determined that contact with the virtual object has occurred, and the visual A virtual space generation control device that changes the shape and generates an interaction control signal for causing the user's portable unit to present vibration and sound is provided.

Further, the three-dimensional visual presenting apparatus of the virtual space system according to the present invention selects two liquid crystal projectors each equipped with a polarizing filter, a screen for superimposing and projecting the images, and images of the respective projectors for the left and right eyes. The present invention is characterized in that eyeglasses equipped with a polarizing filter for presenting information are used.

Further, the virtual space generation control device of the virtual space system according to the present invention obtains at least one of the moving speed or the acceleration in addition to the three-dimensional position measurement information of the user portable unit, and obtains the three-dimensional position. It is configured to change the manner of changing the visual shape of the virtual object after contact determination and the manner of presenting vibration / sound based on the result of combination with the measurement information.

The virtual space system according to the present invention further includes at least one of a performance evaluation device for evaluating the performance of the user interacting with the virtual object and a physiological condition measuring device for measuring the physiological condition of the user. It is characterized in that the definition of contact determination or the number or movement of virtual objects to be presented is changed in real time according to the performance evaluation result of the user or the physiological state of the user.

Further, a virtual space system according to the present invention is a virtual space system to which the interaction device with a virtual space according to any one of claims 1 to 3 is applied, wherein the interaction device with the virtual space is provided. When the change in the three-dimensional position of the user's portable unit determined by the above shows a preset pattern, the background music generation control device that changes the tempo or strength of the background music is provided.

Further, the virtual space system according to the present invention further comprises at least one of a performance evaluation device for evaluating the performance of the rhythm exercise of the user and a physiological condition measuring device for measuring the physiological condition of the user.
The present invention is characterized in that the type of the background music to be presented, the tempo, or the change in strength is changed in real time in accordance with a change in either the performance of the rhythm exercise or the physiological state of the user.

[0017]

DESCRIPTION OF THE PREFERRED EMBODIMENTS One embodiment of the present invention will be described with reference to the drawings. Note that the same reference numerals in the drawings indicate the same or corresponding ones. Embodiment 1 FIG. FIG. 1 is a block diagram illustrating a configuration of an interaction device 100 with a virtual space according to the first embodiment. In FIG. 1, 1 is an interaction device 10
Reference numeral 11 denotes a marker for three-dimensional position measurement, reference numeral 12 denotes a vibration presenting unit (e.g., a vibrator), and reference numeral 13 denotes a sound presenting unit (apparatus). [For example, a speaker] and 14 are wireless transmission / reception means (devices) [for example, an infrared communication device] with the control unit. Reference numeral 2 denotes a control unit of the interaction apparatus 100. In the control unit 2, reference numeral 21 denotes a wireless transmission / reception unit (device) of the control unit, reference numeral 22 denotes an imaging unit of the three-dimensional position measurement unit (device), and reference numeral 23 denotes three-dimensional position measurement. The control means (device), 24 is a vibration presentation control means (device), and 25 is a sound presentation control means (device). Reference numeral 3 denotes virtual space generation control means (apparatus).

In FIG. 1, the user portable unit 1 has a shape that can be held by the user's hand, but has a shape that can be attached to other parts of the user's body (for example, the arm or back of the hand). It may be a thing. FIG. 2 shows an example of the shape of a part to be worn by a user in this case, which is used by being wrapped around an arm or the back of a hand. In FIG.
11 is a marker for three-dimensional position measurement, 12 is a vibration presenting unit (e.g., a vibrator), 13 is a sound presenting unit (e.g., a speaker), 14 is a wireless transmitting / receiving unit (e.g., a speaker) with the control unit 2. Device). FIG. 3 shows an embodiment of a virtual space system to which the interaction device 100 shown in FIG. 1 is applied, in which a three-dimensionally presented virtual object is contacted using the interaction device 100. This is an example of activating physical exercise by trying.

In FIG. 3, reference numeral 1 denotes a user portable unit which is a part held by a user of the interaction apparatus 100;
Denotes a control unit of the interaction device 100, and 3 denotes a virtual space generation control unit (device). In the virtual space generation control means (device) 3, 30 to 35 are blocks representing internal processing, 30 is virtual object description data, 31 is a three-dimensional visual information generation unit, 32 is an interaction determination unit, and 33 is an interaction control signal generation And 34, an interaction rule changing unit and 35 is a virtual object description data changing unit. Reference numeral 4 denotes a virtual space generation control unit (device) 3.
It is a two-dimensional visual presentation means (apparatus), and includes a liquid crystal projector 41, a polarizing filter 42, a screen 43, and polarizing glasses 44 worn by a user. Reference numeral 6 denotes performance evaluation means (apparatus) for evaluating the performance of the user, and reference numeral 7 denotes a physiological state measuring means (apparatus) for measuring the physiological state of the user.

Next, the operation will be described. Three-dimensional visual information generation unit 31 in virtual space generation control means (device) 3
Creates virtual object visual information with parallax corresponding to the left and right eyes of the background world and the virtual object, and sends it to the liquid crystal projector 41 of the three-dimensional visual presentation means (device) 4. At this time, the virtual object has attributes (such as the type, color, and size of the object) described in the virtual object description data 30, an appearance timing,
It is generated based on appearance position and trajectory data. The liquid crystal projector 41 is provided with different polarizing filters 42 on the left and right, and an image projected superimposed on the screen 43 is separated into right and left eyes by polarizing glasses 44 worn by the user, thereby realizing stereoscopic vision. I do.

The three-dimensional position measurement marker 11 in the user portable unit 1 is sent from the three-dimensional position measurement control unit (device) 23 in the control unit 2 to the wireless transmission / reception unit 21
Then, information on the light emission timing is obtained via the wireless transmission / reception means device 14 on the user's portable unit side, and light emission for a predetermined duration is performed at a predetermined timing. The imaging unit 22 of the three-dimensional position measurement means (device) similarly obtains information on the light emission timing from the three-dimensional position measurement control means (device) 23,
The light emission of the three-dimensional position measurement marker 11 is captured, the captured two-dimensional luminance data is transmitted to the three-dimensional position measurement control means (device), and the transmitted two-dimensional luminance is transmitted to the three-dimensional position measurement control means (device) 23. The three-dimensional position of the three-dimensional position measurement marker 11 is obtained from the data set and the preset position of the imaging device.

For this purpose, a generally used stereo vision algorithm (a method for obtaining a three-dimensional position from two-dimensional positions obtained by two imaging devices) may be used. As an imaging device, a CCD camera, a CMOS imaging device, a PSD
(PositionSensitive Detector) is used.
The obtained three-dimensional position information is sent to the interaction determination unit 32 of the virtual space generation control device 3. The interaction determination unit 32 determines whether or not there is contact with the virtual object according to a preset definition (interaction rule). For example, when the distance between the three-dimensional position of the virtual object obtained from the virtual object description data 30 and the three-dimensional position of the user portable unit 1 mapped on the virtual space is equal to or less than a predetermined threshold, It is determined that contact with the virtual object has occurred.

In the case of this interaction rule,
A sense of "interaction (interaction) with the virtual space" that the user directly touches (hits) the virtual object with the user portable unit 1 or a body part to which the user portable unit 1 is attached is given. As another example, if the distance between the position of the user portable unit 1 and the virtual object enters a certain positive range, it is determined that the contact has occurred, and the object is stabbed or stabbed into “interaction with the virtual space”. Is given. On the other hand, if a negative range is set, an object is emitted (or illuminated) from the device, giving a sense that the object is a virtual object. The vibration sensation given at this time is a sensation of "interaction (interaction) with virtual space", which is a recoil of firing (or irradiation).

However, even in this case, the three-dimensional visual information generating unit 31 and the three-dimensional visual presenting means (device) 4 use a stick-like device extending from the tip of the device held in hand (ie, the user's portable unit 1). The visual presentation of the virtual object gives the sensation of touching the virtual object with the tip of a stick. Also, depending on the body part on which the user portable unit 1 is worn and the device shape of the user portable unit 1, there may be a slight shift between the position where the three-dimensional position measurement is performed and the vibration presentation position. It is desirable to make a contact determination after correcting this deviation.

When the interaction determination unit 32 determines that contact with the virtual space has occurred, a signal is sent to the interaction control signal generation unit 33, and a visual information change signal,
Control signals (ie, interaction control signals) necessary for the vibration presentation and the sound presentation are generated. The visual information change signal is sent to the three-dimensional visual information generating unit 31, and three-dimensional visual information is generated by adding a change defined for each virtual object to the virtual object description data 30. For example, specifically, in a case where the virtual object is a balloon, when the contact occurs, the balloon ruptures to generate visual information in which fragments are scattered. The vibration and acoustic control signals generated by the interaction control signal generation unit are transmitted to a vibration presentation control unit (device) 2.
4. It is sent to the sound presentation control means (device) 25.

Vibration presentation control means (device) 2 in control unit 2
Reference numeral 4 denotes a vibration presentation unit (device) [for example, a vibrator] 12 of the user portable unit 1 and a sound presentation control unit (device) in the control unit 2.
Reference numeral 25 denotes a wireless transmission / reception device 21 or 14 for a sound presentation means (device) [for example, a speaker] 13 of the user portable unit 1.
, A vibration control signal and a sound control signal are transmitted, respectively. The vibration presenting means (device) 12 generates vibration in accordance with the transmitted vibration control signal. The vibration control signal relates to the strength, frequency, duration, time pattern, and the like of the vibration, and a signal described in the virtual object description data 30 for each virtual object is used.

The sound presenting means (device) 13 generates sound according to the transmitted control signal. Here, the sound control signal is the sound signal itself, and the virtual object description data 30
Sound data specified for each virtual object and means (device) for reproducing the sound data.
For example, a WAVE file may be used as the sound data, and a PC sound card may be used as the generation device. In this way, the interaction with the virtual space with respect to the movement of the user portable unit 1 is performed.

The "contact" means the three-dimensional position of the user's portable unit 1 (or the body part of the user wearing it) in the virtual space (calculated from the value measured in the real space).
Enters the three-dimensional space occupied by the virtual object (or when the distance from the boundary falls below the set value)
State. Also, "interaction"
Is an interaction between the virtual space (including the virtual object therein) and the user, and includes any operation of the user (for example, hitting, touching, or touching the virtual object with the user portable unit 1). Shaking, bending and stretching with the user's portable unit 1 causes a change in the virtual space (for example, deformation of a virtual object, sounding, transmission of vibration, change in rhythm of background music, etc.), and the change. Is presented by visual sense, auditory sense, tactile sense (vibration) or the like so as to be perceived by the user.

By setting, in the virtual object description data 30, settings relating to visual information change, vibration, and sound at the time of interaction, not only for each virtual object, but also for each type of operation that causes an interaction. Changes in visual information, vibration, and sound can be changed for each type. In this case, the interaction determination unit 32 obtains not only the three-dimensional position of the user portable unit 1 but also its change (speed) and the time difference value (acceleration) of the change. Instead, the type of operation is determined. For example, when the three-dimensional position of the user's portable unit 1 changes from up to down and the speed and the acceleration are larger than predetermined threshold values, it is determined that the contact is made by the operation of “fast and strong swing down”.

In the specific example described above, only when the ball is hit strongly (pulled down), the balloon is set to be broken cleanly, strong vibration is transmitted, and a violent popping sound is generated, and guidance is given to perform such a movement. Thereby, powerful exercise can be emphasized and promoted. Conversely, the balloon is shaken when a slow contact is made, so that quiet vibrations and gentle sounds are presented, and it is also used for training to extend the range of motion by slowly reaching out. In these trainings, instead of forcing the movements themselves, replacing them with movements with the purpose of breaking or touching the balloons and giving them a game-like nature, it is possible to promote physical activity happily.

Further, at the same time, a stimulus is applied to visual, tactile, and auditory senses, and by appropriately combining them, mental activation can be expected. In the performance evaluation means (apparatus) 6, the occurrence frequency and rate (for the number of virtual objects to be presented) for each interaction or for each type of operation causing the interaction
Is evaluated, and the interaction rules and virtual object description data are changed in real time so that an appropriate interaction occurs according to the purpose, and the user's interest is maintained and the enjoyment is increased.

For example, when it is impossible to reach the virtual object accurately and cannot make contact with the virtual object, the interaction rule changing unit 34 may increase the threshold value of the distance for contact determination. Further, in the case where the setting is such that all the balloons are cracked, if the appearance frequency is too large and many unbreakable ones remain or many unreachable balloons, the virtual object description data changing unit 35 The appearance frequency, appearance location, and trajectory (moving speed) of the virtual object may be changed.

The physiological condition measuring means (device) 7 measures a physiological index of the user. For example, a physiological index such as a heart rate, a pulse, a respiration, a pulse wave transmission speed, a skin temperature, and an electromyogram which can be easily measured without disturbing the operation and reflects the physical and mental state of the user is desirable. The physiological condition of the user is evaluated based on the physiological index, and the interaction rule and the virtual object description data are changed so that the desired physiological condition is obtained according to the purpose. For example, if the heart rate is too high, the determination of the interaction is loosened, or the frequency of occurrence of the virtual object is reduced.

In the first embodiment, an active (light emitting) marker using an LED is described as a three-dimensional position measuring marker.
(A marker using a magnetic field generator and a coil) or a position measurement method using a gyro and an acceleration sensor may be used. Also,
It may be realized by a passive marker, for example, a color marker type (a position of a marker of a specific color is determined by a camera) or an infrared reflection marker. In these cases, the communication related to the three-dimensional position measurement part is only transmission of the measurement result from the marker (or sensor) side.

Further, in the first embodiment, an example has been described in which the communication between the control unit 2 and the part that the user wears or holds on the body (ie, the user portable unit 1) is wireless. In some cases, wired communication may be used. In addition, as a method of wireless communication, a method using infrared communication has been described, but wireless communication may be realized using low-power radio waves, ultrasonic waves, or the like. Further, in the first embodiment, the three-dimensional visual presenting means (apparatus) using the liquid crystal projector and the polarized glasses is described. However, in some cases, the three-dimensional visual presenting means (apparatus) using the time-division method using the liquid crystal shutter glasses. May be used. Further, in the first embodiment, as the user portable section of the interaction apparatus, the one used by the user by hand is used, but the one shown in FIG. 2 is used by the user's body (for example, the wrist). Or the back of the hand). In this case, it gives a sense of tapping and touching with bare hands, rather than using a tool.

As described above, according to the apparatus for interacting with a virtual space according to the present embodiment, it is possible to realize an interaction that activates a plurality of senses by stimulating not only sight but also touch and hearing. Can be. Also, 3
The marker of the three-dimensional position measuring means, the vibration presenting means, the sound presenting means, and the communication means with these control means have been integrated into a shape that can be used by the user with his / her hand or attached to a part of the body. So, the user's sense of space (3D position)
Where vibration is given and where sound is heard are matched, a more realistic interaction is realized.

Further, by using an LED marker capable of controlling the timing of light emission and an imaging device for imaging the light emission of the marker as the three-dimensional position measuring means, the three-dimensional position at a plurality of positions can be determined without being limited by the background. Measurement can be performed easily. Furthermore, since the communication between the user portable unit including the three-dimensional position measurement marker, the three-dimensional position measurement control unit, the vibration presenting unit and the sound presenting unit and the control unit on the main body side is made wireless, the communication between the user portable unit and the The cordlessness between the control units can be achieved, and the operation of the user is not hindered.

In the virtual space system according to the present embodiment, the virtual object is presented to the user three-dimensionally, and the three-dimensional position obtained by the interaction between the three-dimensional position of the virtual object perceived by the user and the virtual space is obtained. When the dimensional position has a predetermined relationship, it is determined that contact with the virtual object has occurred, and the visual shape of the virtual object is changed, and
Since the sound is presented, the assumed position of the object in the virtual space can be matched with the position that is actually seen by the user. , Which makes it easier to interact with objects in a virtual space that appeal more intuitively.

This has the effect of maintaining the user's interest, increasing pleasure, and further enhancing the effect of inducing mental and physical activities. Further, in this system, as a three-dimensional visual presentation means, images of the left and right eyes are projected onto two liquid crystal projectors equipped with polarizing filters, a screen on which the images are superimposed, and images of the respective projectors are projected on the left and right eyes. Since a pair of glasses equipped with a polarizing filter for selective presentation is used, a system that does not cause flickering of images and reduces eye fatigue is realized as compared with a time-division stereoscopic system.

Further, in a developed form of the virtual space system according to the present embodiment, not only the three-dimensional position obtained by the interaction device but also its change amount (moving speed) and its time difference value (acceleration) are obtained. By combining these values, it is possible to change the visual shape of the virtual object after contact determination and to change the presentation style of vibration and sound, so it is defined by the speed and acceleration of a specific motion of the user. Possible features can be differentiated and induced.

In a further development of the virtual space system according to the present embodiment, at least one of the user's performance and the physiological state is evaluated.
According to the evaluation result, a function to change the definition of contact judgment in real time or a function to change the number or movement of virtual objects to be presented in real time is provided, depending on the user's current state Training programs can be provided, and higher mental and physical effects can be expected.

Embodiment 2 FIG. 4 shows a configuration of the virtual space system according to the second embodiment, which is another example in which the interaction device 100 shown in FIG. 1 is applied. This is an embodiment aiming to control the tempo and the size of the background music by moving rhythmically, thereby activating the rhythm movement. In FIG. 4, reference numeral 1 denotes a user portable unit of the interaction device, 2 denotes a control unit of the interaction device, and 50 denotes a virtual space generation control unit (device) in the present embodiment. Reference numeral 8 denotes a background music generation control unit (device) in the virtual space generation control unit (device) 50. In the background music generation control unit (device) 8, reference numeral 80 denotes music data, 81 denotes a music data control unit, and 82 denotes music. It is a data reproducing unit. Reference numeral 83 denotes a music data control rule changing unit.

Reference numeral 30 is virtual object description data, 31 is a three-dimensional visual information generation unit, 32 is an interaction determination unit, 33 is an interaction control signal generation unit, and 34 is an interaction rule change unit. Reference numeral 4 denotes a three-dimensional visual presentation means (apparatus), similar to the first embodiment, and includes a liquid crystal projector 41, a polarizing filter 42, not shown.
It comprises a screen 43 and polarized glasses 44 worn by the user. Reference numeral 6 denotes performance evaluation means (apparatus) for evaluating the performance of the user, and reference numeral 7 denotes a physiological state measuring means (apparatus) for measuring the physiological state of the user.
The virtual space system according to the present embodiment is characterized in that the virtual space generation control means (device) 50 includes a background music generation control means (device) 8, a music data control rule change unit 83, and the like.

Next, the operation will be described. The same parts as in the first embodiment are omitted. The visual presentation of the virtual space and the virtual object is the same as in the first embodiment. However, here,
A part of the virtual object description data 30 has a structure similar to the music data 80. That is, a virtual object that appears periodically is defined, and its appearance timing, appearance position, moving speed, and the like are modified by the music data 80. Thereby, presentation of the virtual object synchronized with the background music can be performed.

The interaction determination unit 32 determines the speed and acceleration from the three-dimensional position data of the user portable unit 1 sent from the control unit 2 of the interaction device 100, and determines a predetermined phase of the rhythm movement by a combination thereof. To detect. For example, assuming a rhythmic movement in which the user's portable unit 1 shakes like a maraca, it detects when the downward velocity exceeds a set threshold and the acceleration becomes zero, and the user's portable unit 1 and the virtual object are detected. Is determined to have occurred, the maximum speed from the previous detection is held, and the swing strength is used. The music data control unit 81 determines the tempo and strength of the background music based on the detected timing and strength of the contact in the interaction determination unit 32, changes the music data 80, and sends it to the music data reproduction unit 82.

The music data reproducing section 82 reproduces the music data of which the tempo and the intensity have been modulated and sends them to the sound presentation control means (device) 25 of the control section 2 having the configuration shown in FIG. Occurs. The music data control unit 81
At the same time, the virtual object description data 30 is rewritten to synchronize the background music with the appearance and movement of a part of the virtual object.
The interaction control signal generation unit 33 transmits a vibration control signal and predefined sound data to the interaction device 2 when the interaction is determined. In the above-mentioned example of maracas, the vibration gives the feeling of shaking the maracas, and the acoustic data gives the sound of the maracas. Depending on the hardware to be realized, the music data reproducing unit 82
You need a mixer to synthesize the sound from the sound.

In the performance evaluation means (apparatus) 6,
Evaluate the magnitude and intensity of the rhythmic motion from the rhythmicity of the detection data of the interaction determination unit 32 (whether there is no rhythm loss or disturbance) or the speed and acceleration data of the user's portable unit 1 in the previous stage. Then, an interaction rule (a criterion for detecting a rhythmic movement) is changed, and a control rule for music data is changed. In the former example, the speed threshold is changed for a user who can make only a small slow motion. In the latter example, if the user cannot perform a stable rhythm movement, the detected movement is not directly reflected in the music tempo, but a positive fluctuation exceeding the value set for the rhythm movement detection interval occurs. In this case, the detection is compensated for, and in the case of a negative fluctuation, the detection is ignored, so that relatively stable background music is generated.

As described above, in the virtual space system according to the present embodiment, the background music generation control is performed when the change in the three-dimensional position obtained by the interaction device with the virtual space indicates a predetermined pattern. A signal is sent to the means (device) to provide the function of changing the tempo and strength of the background music to be presented. , And motivation to perform larger movements and movements, thereby obtaining the effect of inducing rhythmic movements.

Further, in the development of the virtual space system according to the present embodiment, the type of background music to be presented, the tempo, and the strength or weakness may be determined depending on at least one of the performance of the user's rhythmic movement and the physiological state. A function to change the change method in real time is provided, so that a training program can be provided according to the actual situation of the user's exercise and movement at that time, and rhythmic movement can be better induced and higher mental and physical activation can be achieved. The effect can be expected.

[0050]

The interaction apparatus according to the present invention is a three-dimensional position measuring marker for measuring a three-dimensional position in a virtual space, a vibration presenting apparatus, a sound presenting apparatus, and a signal transmitting / receiving apparatus for transmitting / receiving signals to / from these. Are integrated, and a signal from a three-dimensional position measurement marker is input via a user portable unit having a shape that can be carried by the user, and a signal transmitting / receiving device of the user portable unit. The position measurement information is transmitted to the virtual space generation control device by measuring the position, and the vibration presentation signal and the sound presentation signal are generated according to the interaction control signal generated by the virtual space generation control device based on the transmitted position measurement information. And a control unit for transmitting the signal to the vibration presenting device and the sound presenting device of the user's portable unit via the signal transmitting / receiving device. Stuck, it is possible to use by mounting the part of the body, the space feeling of the user (3D position)
Thus, there is an effect that a place where vibration is given and a place where sound is heard coincide with each other, and a more realistic interaction is realized. In other words, it is possible to appeal not only to visual sense but also to a plurality of senses, to match the visual and tactile / auditory perceived positions, and to provide interaction with a virtual space that is easily understood by elderly people and patients.

Further, the three-dimensional position measuring device of the portable user unit used in the interaction device according to the present invention is:
A three-dimensional position measurement marker using an LED arranged on the user's portable part side, a device arranged on the control part side for controlling light emission timing of the three-dimensional position measurement marker, and light emission of the three-dimensional position measurement marker Since the imaging device is configured to capture an image of a subject, there is an effect that three-dimensional measurement at a plurality of positions can be easily performed without being restricted by the background.

Further, the signal transmitting / receiving device of the interaction device according to the present invention is characterized by being a cordless wireless transmitting / receiving device, so that the moving operation of the user portable unit can be smoothly performed without being hindered by the wiring. This has the effect.

Further, a virtual space system according to the present invention is a virtual space system to which the interaction device with a virtual space according to any one of claims 1 to 3 is applied. 3D visual presentation of the virtual object to the user, and 3
When the three-dimensional position and the three-dimensional position of the user's portable unit obtained by the interaction device with the virtual space have a predefined relationship, it is determined that contact with the virtual object has occurred, and the visual It has a virtual space generation control device that generates an interaction control signal that changes the shape and causes the user's portable unit to present vibration and sound. It is possible to match the position seen by the person with not only vision but also auditory and tactile sense of position, and it is easier to interact with objects in a virtual space that appeals more intuitively than a system based on a simple mirror image .

Further, the three-dimensional visual presentation device of the virtual space system according to the present invention selects two liquid crystal projectors equipped with a polarizing filter, a screen on which the images are superimposed and projected, and images of the respective projectors for the left and right eyes. It is characterized by the use of eyeglasses equipped with a polarizing filter for presenting information in a virtual manner, and realizes a virtual space system with less flickering of images and less eye fatigue compared to a time-division stereoscopic system. be able to.

Further, the virtual space generation control device of the virtual space system according to the present invention obtains at least one of the moving speed and the acceleration in addition to the three-dimensional position measurement information of the user portable unit, and obtains the three-dimensional position. Based on the result of the combination with the measurement information, it is configured to change the visual form of the virtual object after contact determination and the presentation style of vibration and sound, so grasp the characteristics of the user's movement, Interaction corresponding to this is made possible, and a more realistic virtual space system can be constructed. In particular, in the application to physical rehabilitation, it is possible to individually set necessary exercise training for each patient.

The virtual space system according to the present invention further comprises at least one of a performance evaluation device for evaluating the performance of the user interacting with the virtual object and a physiological condition measuring device for measuring the physiological condition of the user. The feature is that the definition of contact judgment or the number and movement of virtual objects to be presented are changed in real time according to the user's performance evaluation result or the user's physiological state, so that the characteristics of the user's movement are grasped In addition, an interaction corresponding to this can be performed, and a training program according to the performance and physiological condition of the user at that time can be provided, so that a higher effect of mental and physical activation can be expected.

Further, a virtual space system according to the present invention is a virtual space system to which the interaction device with a virtual space according to any one of claims 1 to 3 is applied, wherein the interaction device with a virtual space is provided. When the change in the three-dimensional position of the user's portable unit determined by the above shows a preset pattern, the background music generation control device that changes the tempo or strength of the background music is provided.
The sensation of controlling the sound is obtained by the movement and movement of the user, and the motivation to continue the movement and movement and to perform a larger movement and movement is obtained, thereby obtaining the effect of inducing the rhythm movement.

Further, the virtual space system according to the present invention further comprises at least one of a performance evaluation device for evaluating the performance of the rhythm exercise of the user and a physiological condition measuring device for measuring the physiological condition of the user.
The feature is to change the type, tempo or strength of the background music to be presented in real time in accordance with either the performance of the user's rhythm exercise or the change in the physiological state. In addition to providing the feeling of controlling the sound, it can also provide a training program according to the user's realization of the movement and movement at that time, can better induce rhythmic movements, and have a higher mental and physical activation effect Can be expected.

[Brief description of the drawings]

FIG. 1 is a diagram showing a configuration of an interaction device with a virtual space according to a first embodiment.

FIG. 2 is a diagram showing another example of the user portable unit.

FIG. 3 is a diagram showing a configuration of a virtual space system according to the first embodiment.

FIG. 4 is a diagram showing a configuration of a virtual space system according to a second embodiment.

FIG. 5 is a diagram showing a configuration of a conventional virtual space system.

[Explanation of symbols]

DESCRIPTION OF SYMBOLS 1 User portable part 2 Control part 3 Virtual space generation control device 4 Three-dimensional visual presentation means (device) 6 Performance evaluation means (device) 7 Physiological condition measuring means (device) 8 Background music generation control means (device) 11 3D Marker of position measuring means (apparatus) 12 Vibration presenting means (apparatus) 13 Sound presenting means (apparatus) 14 Wireless transmitting / receiving means (apparatus) 21 Wireless transmitting / receiving means (apparatus) 22 Imaging section of 3D position measuring means (apparatus) 23 3 Dimensional position measurement control means (device) 24 Vibration presentation control means (device) 25 Sound presentation control means (device) 30 Virtual object description data 31 Three-dimensional visual information generation unit 32 Interaction determination unit 33 Interaction control signal generation unit 34 Interaction rule change Unit 35 virtual object description data changing unit 41 liquid crystal projector 42 polarizing filter 43 Clean 44 Polarized glasses 50 Virtual space generation control device 80 Music data 81 Music data control unit 82 Music data reproduction unit 83 Music data control rule change unit 100 Interaction device with virtual space

Continuation of the front page (72) Inventor Shotaro Miwa 2-3-2 Marunouchi, Chiyoda-ku, Tokyo F-term in Mitsubishi Electric Corporation 5B050 AA10 BA09 CA07 DA03 FA02 FA06 FA10 5E501 AA25 AB30 BA11 BA17 CA02 CA06 CA10 CB14 FA36 5L096 AA09 CA05 DA05 FA69 HA03

Claims (9)

[Claims] 1. A three-dimensional position measurement marker for measuring a three-dimensional position in a virtual space, a vibration presenting device, a sound presenting device, and a signal transmitting and receiving device for transmitting and receiving signals to and from the three-dimensional position measuring marker are integrated. A signal from the three-dimensional position measurement marker is input through a user portable unit having a portable shape and a signal transmitting / receiving device of the user portable unit, and the position of the user portable unit is measured. The position measurement information is transmitted to the virtual space generation control device, and a vibration presentation signal and a sound presentation signal are transmitted to the virtual space generation control device according to an interaction control signal generated by the virtual space generation control device based on the transmitted position measurement information. An interaction with a virtual space, comprising: a control unit configured to transmit a vibration presenting device and a sound presenting device of the user portable unit via a transmitting / receiving device. Apparatus. 2. A three-dimensional position measuring device for a user portable unit,
A three-dimensional position measurement marker using an LED arranged on the user portable part side, an apparatus for controlling the light emission timing of the three-dimensional position measurement marker arranged on the control part side, and the three-dimensional position measurement The apparatus for interacting with a virtual space according to claim 1, further comprising an imaging device that captures light emitted from the marker. 3. The apparatus according to claim 1, wherein the signal transmitting / receiving apparatus is a cordless wireless transmitting / receiving apparatus. 4. A virtual space system to which the device for interacting with a virtual space according to claim 1 is applied, comprising a three-dimensional visual presentation device, and providing a virtual object to a user. When the three-dimensional position of the virtual object presented by the three-dimensional visual perception and perceived by the user and the three-dimensional position of the user's portable unit determined by the interaction device with the virtual space have a predetermined relationship, the virtual A virtual space generation control device that determines that contact with an object has occurred, changes the visual shape of the virtual object, and generates an interaction control signal for causing the user's portable unit to present vibration and sound. Virtual space system. 5. A three-dimensional visual presentation device, comprising: two liquid crystal projectors each equipped with a polarization filter; a screen for superimposing and projecting the images; and a polarization filter for selectively presenting images of the respective projectors to left and right eyes. 5. The virtual space system according to claim 4, wherein glasses for wearing are used. 6. The virtual space generation control device obtains at least one of the moving speed and acceleration in addition to the three-dimensional position measurement information of the user's portable unit, and based on the result of combination with the three-dimensional position measurement information. The virtual space system according to claim 4, wherein the virtual space system is configured to change a visual form of the virtual object after the contact determination and a vibration / sound presentation style. 7. The apparatus further comprises at least one of a performance evaluation device for evaluating the performance of the user's interaction with the virtual object and a physiological condition measuring device for measuring the physiological condition of the user, wherein the performance evaluation result of the user or the user is provided. The virtual space system according to claim 4, wherein the definition of the contact determination or the number or movement of the virtual object to be presented is changed in real time according to the physiological state of the virtual space. 8. A virtual space system to which the device for interacting with a virtual space according to claim 1 is applied, wherein the user's portable unit determined by the device for interacting with the virtual space is provided. A virtual space system comprising a background music generation control device that changes the tempo or strength of background music when the change in three-dimensional position indicates a preset pattern. 9. The apparatus further includes at least one of a performance evaluation device for evaluating the performance of the user's rhythmic exercise and a physiological condition measuring device for measuring the user's physiological condition, wherein the performance of the user's rhythmic exercise or a change in the physiological condition is provided. Depending on the type of background music to present,
9. The virtual space system according to claim 8, wherein a change in tempo or strength is changed in real time.