CN102769764B - Be applied to method and the relevant apparatus of three dimensional display - Google Patents

Abstract

The invention relates to a method and a related device applied to a three-dimensional display, which displays a three-dimensional image and senses a user's response to the three-dimensional image to obtain positioning information corresponding to the three-dimensional image.

Description

Method and related device applied to three-dimensional display

technical field

The present invention relates to a method and a related device applied to a 3D display, and in particular to a method and a related device combining 3D image and sensing positioning.

Background technique

3D display technology can display 3D images and bring users a more vivid and rich sensory experience, and has become the research and development focus of modern information manufacturers.

When a three-dimensional image of an object is to be displayed, the three-dimensional display displays the left image and the right image of the object to the left and right eyes of the user (viewer) respectively; , the user will see a three-dimensional image of the object.

However, there are many factors that affect the position of the 3D image viewed by the user; the position of the 3D image to be conveyed by the parallax offset in the content of the 3D image data will be different from the actual position of the 3D image viewed by the user. This will not only affect the playback effect of the 3D image, but also the user cannot interact with the 3D image correctly.

Contents of the invention

Therefore, the present invention provides a technology capable of combining the imaging position of the 3D image and the position of the viewer, so that the user can correctly interact with the virtual environment of the 3D image structure.

One of the objects of the present invention is to provide a method applied to a three-dimensional display, including: displaying one or more three-dimensional images of test objects with a three-dimensional display, each three-dimensional image of the test object is related to a preset parallax offset; sensing The user responds to the response signal of the three-dimensional image of the test object, and senses the user's test positioning information (including the distance between the three-dimensional image and the screen); wherein, there is a specific relationship between the preset parallax offset and the test positioning information, so The positioning information corresponding to different parallax offsets can be described by a functional relationship.

The present invention can utilize the functional relationship between the parallax offset and the positioning information to enable the user to interact with the three-dimensional image. For example, a 3D display can be used to display 3D images of a preset number of interactive objects, and each 3D image of an interactive object is associated with a preset program and corresponds to a second parallax offset; and each second parallax offset can be substituted into Functional relationship to provide a corresponding reference positioning information. The sensor senses the interactive positioning information of the user responding to the 3D image of the interactive object, and then compares the interactive positioning information with the reference positioning information corresponding to the 3D image of each interactive object. If the interactive positioning information matches the reference positioning information corresponding to an interactive object, the 3D display can perform the preset program associated with the matching interactive object.

That is to say, the 3D display can present a plurality of three-dimensional virtual interactive objects with 3D images, and predict the 3D imaging position felt by the user (ie, refer to positioning information) according to the correction of the functional relationship. When the user interacts with each interactive object, the sensor detects an interactive positioning information according to the user's action; if the detected interactive positioning information matches the reference positioning information of an interactive object, the 3D display can execute the interactive object associated program. In this way, the user can perform three-dimensional interaction with the three-dimensional display.

In one embodiment, when the comparison result shows that the interactive positioning information does not match the reference positioning information, a reminder can be issued to the user to adjust the user position and sense the user's interactive positioning information again.

In another embodiment, when the comparison result shows that the interactive positioning information does not match the reference positioning information, an interactive parallax offset can be provided according to the user interactive positioning information and the specific relationship, and the parallax of the 3D image of the interactive object can be biased. The offset is equal to the interactive parallax offset, so that the corresponding reference positioning information matches the user's interactive positioning information.

In one embodiment, the sensor is provided with multiple lenses; the multiple lenses capture user images to generate multiple sensing images, and provide sensing positioning information based on differences between the sensing images. When generating the sensing images, the difference between the sensing images can also be utilized for focusing.

In one embodiment, the sensor includes a transmitter and a receiver. The transmitter sends positioning waves to the user, and the receiver receives the reflected waves of the positioning waves; according to the reflected waves, the sensed positioning information can be provided. Positioning waves can be electromagnetic waves, infrared light waves or sound waves, ultrasonic waves, shock waves, etc.

The present invention can use the functional relationship between the parallax offset and the positioning information to predict the position of the 3D image felt by the user, and then adjust the 3D image so that the user can experience the 3D image at the expected position. When displaying a display object, if users want to feel its 3D imaging at a given display position, the display positioning information corresponding to the display position can be substituted into the functional relationship to obtain a corresponding display parallax offset . The 3D display displays the 3D image of the displayed object according to the display parallax offset, and the user can correctly see the 3D image of the displayed object at the display position.

The 3D display can include multiple speakers. The present invention can also adjust the playback parameters of each speaker according to the positioning information sensed by the sensor, and correct the stereo sound field of the speakers so that the user can experience the proper stereo sound effect at the expected position.

Yet another object of the present invention is to provide a method applied to a 3D display, comprising: displaying a preset number of test 3D images with the 3D display, each test 3D image corresponding to a preset parallax offset; and, according to the usage The tester obtains a corresponding test positioning information from the response signal of each test 3D image. According to each preset parallax offset and each test positioning information, a functional relationship can be calculated to correlate different parallax offsets with corresponding positioning information.

The function relationship can be used to adjust/correct the 3D image, including: obtaining a display image data, the display image data has a second preset number of 3D display images, and each 3D display image corresponds to a piece of display positioning information. Each display positioning information is substituted into the functional relationship to provide a corresponding display parallax offset; according to each display parallax offset, each 3D display image in the display image data can be adjusted, so that the user can display the corresponding display positioning information View the individual 3D display images at the placement of the .

The invention can also use the stereo sound field to correct the three-dimensional image. When displaying the test 3D image, the 3D display is used to play a second preset number of test sound data, and each sound test data corresponds to a preset sound effect offset information. A functional relationship is calculated according to each preset sound effect offset information and each test location information to describe the location information corresponding to different sound effect offset information. This functional relationship can be used to correct/adjust the sound data, so that the stereo sound effect corresponding to the sound data conforms to the virtual environment constructed by the 3D image. The display sound data to be adjusted/played is obtained, and the display positioning information corresponding to the display sound data is substituted into the functional relationship to provide a corresponding display sound effect offset information. According to the display sound offset information, the display sound data can be adjusted.

Another object of the present invention is to provide a device for three-dimensional playback, including an image processing module, an image correction module, an image adjustment module, an audio processing module, an optional audio correction module and an optional Set the sound adjustment module. The image processing module receives a preset number of test 3D images and obtains a preset number of preset parallax offsets. The image correction module receives the parallax offset, and receives a corresponding test positioning information for each preset parallax offset. Each preset parallax offset is associated with a test 3D image, and each test positioning information is provided by the sensor, corresponding to the user's response to each test 3D image. The image correction module further calculates a functional relationship according to each preset parallax offset and each test positioning information, so as to correlate different visual offsets with corresponding positioning information.

The image adjustment module receives display image data, and the display image data corresponds to a second preset number of display positioning information. The image correction module substitutes each display positioning information into the functional relationship, and provides a corresponding display parallax offset for each display positioning information, which is received by the image adjustment module. The image adjustment module also adjusts the display image data according to each display parallax offset.

The sound processing module receives a preset number of preset sound effect offset information, which is related to a preset number of test sound data and a test three-dimensional image; the sound correction module receives each test positioning information and a second preset number of preset sound effect offset information , each preset sound effect offset information is associated with a test sound data, and is associated with a test 3D image. The sound correction module calculates a second functional relationship according to each preset sound effect offset information and each test location information, so as to correlate different sound effect offset information with corresponding location information.

The display image data is also associated with a display sound data, and the display sound data corresponds to a second display positioning information. The sound correction module substitutes the second display positioning information into the second functional relationship to provide associated display audio offset information, and the image adjustment module adjusts the display image data according to each display parallax offset and display audio offset information.

The sound adjustment module receives display sound data, and receives display sound offset information provided by the sound correction module. The sound adjustment module adjusts the display sound data according to each display sound effect offset information and each display parallax offset.

In order to have a better understanding of the above-mentioned and other aspects of the present invention, the preferred embodiments are specifically cited below, together with the accompanying drawings, and are described in detail as follows:

Description of drawings

1A to 1C illustrate the relationship between the parallax offset and the three-dimensional imaging position.

2A to 2B and FIGS. 3A to 3B illustrate various factors affecting the three-dimensional imaging position.

4A to 4C are schematic diagrams of sensing and positioning according to an embodiment of the present invention.

FIG. 5 is a flow chart of correcting a three-dimensional imaging position by sensory positioning according to an embodiment of the present invention.

FIG. 6 is a flowchart of implementing 3D interaction according to an embodiment of the present invention.

FIG. 7 is a schematic diagram of establishing a stereo sound field according to an embodiment of the present invention.

FIG. 8 is a flow chart of correcting stereo sound field by sensory positioning according to an embodiment of the present invention.

FIG. 9 is a schematic diagram of a three-dimensional playback application device according to an embodiment of the present invention.

Description of main component symbols

10: device

12: Control module

13: Image processing module

14: Image correction module

16: Image adjustment module

17: Sound processing module

18: Sound correction module

20: Sound adjustment module

100, 200, 300: process

102-110, 202-208, 302-310: Steps

IR, IR1-IR2, IRt, IRc(.): right image

IL, IL1-IL2, ILt, ILc(.): left image

Xa-Xc, Xs, Xt, Xc(.): Offset

Ya-Yc, Ys, Yt, Yc(.), Yi: distance value

Pt: test positioning information

fcn(.): program

Pc(.), Pi: positioning information

SC, SC2: screen

OB: object

Iob, Iobt, Iob(.): Imaging

MS: sensor

PR: right picture

PL: left picture

CR: right camera

CL: left lens

SK1-SK5: Speakers

Detailed ways

The offset between the left image and the right image can be called 3D depth, which is related to the position of the 3D image perceived by the viewer. Please refer to FIG. 1A to FIG. 1C , which illustrate the three-dimensional imaging positions corresponding to different offsets. As shown in Fig. 1A, if the left image IL of an object on the screen SC is to the right, and the right image IR is to the left, the offset Xa between the two will make the imaging Iob of the object in front of the screen SC, and SC distance - distance value Ya. Assuming that the position of the viewer remains unchanged, if the left image IL and the right image IR move to the middle respectively to reduce the size of the offset Xa, the size of the distance value Ya will also be reduced jointly, and the 3D imaging Iob of the object will gradually approach the screen SC. As shown in FIG. 1B , if the offset Xb between the left image IL and the right image IR is 0, the formed 3D image Iob just lands on the screen SC, and the distance Yb from the screen SC is 0. As shown in FIG. 1C , if the left image IL is to the left and the right image IR is to the right, the offset Xc between the two will make the image Iob behind the screen SC, which is separated by a distance Yc from the screen SC.

From the examples in FIGS. 1A to 1C , it can be seen that by changing the offset between the left and right images, the three-dimensional imaging position (relative to the screen or the user) can be adjusted. However, there are other factors that affect the imaging position of the 3D image; FIGS. 2A to 2B and FIGS. 3A to 3B illustrate other factors that affect the position of the 3D image.

One of the factors affecting the imaging position of the 3D image is the size of the display screen. In Figure 2A, the left and right images IL and IR are displayed on a relatively small screen SC; in Figure 2B, the left and right images IL and IR are displayed in equal proportions on a larger screen SC2, and the offset between them The volume is also scaled up proportionally. Although the distance between the viewer and the screen is the same, for the larger screen SC2, the three-dimensional imaging Iob presented will be closer to the viewer.

Another factor affecting the imaging position of the 3D image is the distance between the viewer and the screen. In FIGS. 3A and 3B , both the left and right images IL and IR are displayed on the screen SC with the same size, and the offset between the left and right images remains unchanged. It can be seen from FIG. 3B that the distance between the viewer and the screen is longer than that in FIG. 3A , and accordingly, the position of the three-dimensional imaging Iob also changes accordingly. In addition, factors such as the distance between the eyes of the viewer will also affect the position (distance) of the three-dimensional image viewed by the viewer.

From the above discussion, it can be seen that there are many factors that affect the position of the 3D image viewed by the user; the position of the 3D image to be conveyed by the parallax offset in the content of the 3D image data will be different from the actual position of the 3D image viewed by the user. This will not only affect the playback effect of the 3D image, but also the user cannot interact with the 3D image correctly.

The 3D display of one embodiment of the present invention uses the display technology of 3D display to make the viewer feel the 3D depth of the image, and at the same time, uses the sensor to detect the position of the viewer relative to the screen, and then combines the position of image formation and viewing The position of the viewer can be adjusted to allow the viewer to interact with the images displayed on the 3D display. Please refer to FIGS. 4A to 4C , which are schematic diagrams of sensing and positioning according to an embodiment of the present invention. To realize the present invention, a three-dimensional display screen SC includes or cooperates with a sensor MS, such as a camera (image) machine. In the embodiments of FIGS. 4A to 4C , the sensor MS is provided with a left lens CL and a right lens CR. As shown in FIG. 4A , for an object OB which is a distance value Ys away from the sensor MS, for example, a viewer being photographed in front of a screen, the left image IL1 produced by the left lens CL shooting the object OB is located in the left frame PL, and the right lens CR The right image IR1 generated by photographing the object OB is located in the right frame PR. Due to the positional difference between the left and right lenses CL and CR, there is an offset Xs between the left and right images IL1 and IR1 on the left and right frames PL and PR.

As shown in Figure 4B, the two lenses of the sensor are like the eyes of human beings. If the distance Ys between the object OB and the sensor MS becomes larger, when the left and right lenses CL and CR capture the left and right images IL2 and IR2 of the object OB respectively , the offset Xs between the left and right images IL2 and IR2 will become smaller. It can be seen from this that the position of the object OB (such as the distance value Ys) is related to the difference between its left and right images (such as the offset Xs). As shown in Figure 4C, a larger offset Xs(1) corresponds to a shorter distance Ys(1), a smaller offset Xs(2) corresponds to a longer distance Ys(2), and the offset There is a functional relationship between the quantity and the distance value. This functional relationship is related to the size of a three-dimensional display screen SC, that is, the functional relationship obtained by different screens may be different. According to the offset Xs between the left and right images, the sensor MS can determine the relative position of the object OB (for example, the distance from the sensor MS), and provide corresponding positioning information for the object OB, including the distance value Ys . Furthermore, in one embodiment, the sensor MS can also focus according to the offset between the left and right images; is correctly focused on the object.

In another embodiment of the present invention (not shown), the sensor MS can be one or more transmitters and one or more receivers instead of a camera (image) camera. The transmitter sends a positioning wave to the object, and the receiver receives the reflected wave of the positioning wave; based on the reflected wave (for example, comparing the difference between the positioning wave and the reflected wave, or comparing the reflected waves received by different receivers), the sensing positioning information can be provided . Positioning waves can be electromagnetic waves, infrared light waves or sound waves, ultrasonic waves, shock waves, etc.

The three-dimensional display system provided by the present invention uses the position of the user (viewer) obtained by the sensor, and considers the previously described three-dimensional imaging position actually felt by the user, by adjusting the position of the three-dimensional imaging and/or adjusting With the position of the user, the user (by eyes or body movements) can correctly experience the three-dimensional imaging, and then interact with the three-dimensional imaging to achieve a sense of presence in a virtual reality.

Please refer to FIG. 5 , which shows a process 100 according to an embodiment of the present invention for combining three-dimensional display and sensing positioning; its main steps can be described as follows:

Step 102: Display the test 3D image. For example, the left and right images ILt and IRt of an object can be generated on the screen SC of the 3D display; there is a parallax offset Xt between the left and right images ILt and IRt, so that the user can feel the imaging Iobt, that is, the imaging Iobt is used as a test 3D image.

Step 104: The 3D display can prompt the user to make some specific response signals according to the clear 3D image Iobt observed, such as some response actions: special gestures, touching or tapping the 3D image observed; or, In another example, when the three-dimensional imaging can be clearly perceived, the user actively provides a response signal. In one embodiment, the sensor MS detects the test location information Pt; the test location information Pt includes the distance value Yt between the sensed user and the sensor. In one embodiment, the sensor MS detects a specific response action of the user and senses its position, so as to obtain the positioning information Pt. In another embodiment, the user's specific response signal can be transmitted to the 3D display in other ways, for example, the user presses a button on the remote control to transmit the response.

In detail, the sensor MS can use the edges and/or positions of various reference points (such as body, shoulder and/or arm, finger) of the user to sense the test location information Pt. For example, the user images can be captured by the lenses CL and CR in FIG. 4C to generate left and right images, and test positioning information Pt is provided according to the difference between the left and right images.

Step 106: Establish a functional relationship between the parallax offset Xt of the three-dimensional test image and the distance value Yt in the test positioning information Pt, so that this functional relationship can describe the positioning information corresponding to various parallax offsets (including distance value). In fact, steps 102 and 104 are performed recursively. For example, in step 102 each time, 3D images with different parallax offsets Xt can be used to form imaging Iobt with different positions; in step 104, according to the use According to the actual experience of the user, different positioning information Pt and distance value Yt corresponding to each parallax offset Xt are sensed, so as to accumulate multiple sets of specific relationships corresponding to "parallax offset Xt and distance value Yt" in step 106, and then The general functional relationship between the two is deduced.

Step 108: Determine whether to continue accumulating the corresponding relationship between the "parallax offset Xt and the distance value Yt", if not, continue to recursively perform steps 102 and 104. If the functional relationship between the parallax offset and the distance value/positioning information has been established, go to step 110 .

Step 110: End the process 100.

Using the functional relationship obtained in the process 100, the present invention enables the user to correctly interact with the virtual object/virtual environment of the 3D image. Please refer to FIG. 6 , which illustrates a process 200 for performing 3D virtual environment interaction according to an embodiment of the present invention. The main steps of the process 200 can be described as follows.

Step 202: Use the screen SC of the 3D display to display 3D images of one or more interactive objects, such as imaging Iob(n) and Iob(n'). Each image Iob(n) is formed by left and right images ILc(n) and IRc(n), and the left and right images correspond to a parallax offset Xc(n). Each parallax offset Xc(n) can be substituted into the functional relationship in step 106 to provide a corresponding distance value Yc(n) and reference positioning information Pc(n). Each image Iob(n) is associated with a preset program fcn(n). The preset program fcn(n) can be various operations that the three-dimensional display can perform, such as adjusting the brightness of the display, contrast, pausing playback, continuing playing, selecting a chapter to play, etc.; moreover, the three-dimensional display can also be used for The content provided by a host is played, and the preset program fcn(n) is an operation that the host can execute. For example, the host can be a game console. When the preset program is executed, the display content provided by it can be changed, so that the three-dimensional virtual scene displayed on the three-dimensional display can be changed, so as to respond to the trigger of the user. Interact with users.

Step 204: Sensing the interactive location information Pi of the user.

Step 206: Compare the interactive positioning information Pi with the reference positioning information Pc(n) of each imaging Iob(n). If the comparison result shows that the interactive positioning information Pi matches the reference positioning information Pc(n0) corresponding to an interactive object, proceed to step 208; otherwise, return to step 204 to continuously sense the user's subsequent actions. The purpose of step 206 in this embodiment is to obtain the specific object (a specific 3D image) that the user wants to touch among the many objects in the 3D interactive scene.

Step 208: Since the interactive positioning information Pi matches the reference positioning information Pc(n0), for example, the error between the two is within a preset error range, the 3D display can execute its associated preset program fcn(n0). This step means that if Pi matches the reference positioning information Pc(n0), it means that the user responds to the imaged 3D object corresponding to the position Pc(n0).

In one embodiment, when the user's interaction positioning information does not match the reference positioning information of any interactive object, it may mean that the user does not want to perform any interaction. For example, when the user triggers the interaction, it may just be to adjust the sitting posture, answer the phone, drink a drink, etc.; through the comparison of the interaction location information, it can be known that the user does not intend to trigger any preset program. Furthermore, if there are multiple interactive objects (interactive imaging), the user can only interact with one (or some) of them.

In one embodiment, when the comparison result shows that the interactive positioning information does not match the reference positioning information, a reminder can be issued to the user to adjust the user position and sense the user's interactive positioning information again.

In another implementation, when the comparison result shows that the interactive positioning information does not match the reference positioning information, an interactive parallax offset can be provided according to the user interactive positioning information and the specific relationship, and the parallax of the 3D image of the interactive object can be biased. The offset is equal to the interactive parallax offset, so that the corresponding reference positioning information matches the user's interactive positioning information.

After the process 100 is performed, a functional relationship can be established between the parallax offset X of the 3D image and the distance Y actually felt by the user. When the image Iob(n) is displayed in step 202, the position and distance at which the user will observe the image Iob(n) can be known according to the parallax offset Xc(n). If the user interacts with the imaging Iob(n) at this position and at this distance value, it means that the user wants to trigger the program fcn(n) corresponding to the imaging Iob(n). Accordingly, the user can perform three-dimensional interaction with the three-dimensional image played by the three-dimensional display.

As discussed in Figures 2A to 2B and Figures 3A to 3B, only based on the parallax offset of the left and right images, it is not possible to accurately know at which position and at which distance the user will see the 3D image, and it is impossible to correctly Let the user interact with the 3D scene. Because the parallax offsets of the 3D images produced by screens of different sizes are different, the established distance values are also different; moreover, the actual distance between the user and the screen will also affect the distance value. This embodiment of the present invention can actually sense the distance value and position of the imaging observed by the user in the process 100, and accordingly correct the functional relationship between the parallax offset and the distance value/imaging position, thereby enabling the three-dimensional interaction of the process 200 be achieved.

For example, if the parallax offset Xc(n) of the imaging Iob(n) should be imaged at a certain distance value Y_expect, but the actual imaging distance value Y_actual is different due to factors such as FIGS. 2A to 2B and FIGS. 3A to 3B It is different from the distance value Y_expect. However, through the functional relationship established in the present invention, the distance value Y_actual can be calculated from the parallax offset Xc(n). When the user interacts (sends an interaction signal) at the distance Y_actual, it can be known that the user intends to interact with the image Iob(n). Therefore, the expected distance value Y_expect will not affect the correctness of the interaction; even if the distance value Y_expect is different from the distance value Y_actual actually felt by the user, the present invention can still interact with the user correctly.

In addition to combining sensory positioning with three-dimensional images, the present invention can further combine sensory positioning with a stereo sound field. Please refer to FIG. 7 , which is a schematic diagram of establishing a stereo sound field with multiple speakers SK1 to SK5 according to an embodiment of the present invention. In addition to the screen SC and the sensor MS, the 3D display can also be matched with speakers SK1 to SK5 at different positions to play stereo sound effects in conjunction with the 3D display. The image data displayed in 3D will create a 3D image with the visual parallax offset, and the sound data of the stereo sound field will be shaped with the acoustic offset information (including frequency, phase, delay and/or volume offset) The stereo sound effect in the three-dimensional virtual space allows the user to hear the position of the virtual sound source. However, the 3D imaging distance value established by the parallax offset will be different from the distance value actually observed by the user, and the sound source position to be established by the sound effect offset information will also be different from the sound source position actually heard by the user. However, the present invention can correct the functional relationship between the sound effect offset information and the sound source position according to the position sensing of the sensor MS.

For example, step 106 of the process 100 has established a functional relationship between the parallax offset and the imaging distance value, and the present invention can directly use this functional relationship to adjust/correct the sound effect offset information of the sound data, so that the sound effect offset information The created sound source location conforms to the sound source location actually positioned by the user. Or, similar to steps 102 and 104, the present invention can also play test sound data to the user, and the user responds or indicates the location of the sound source it hears, (even the sensor MS can sense the user's action to obtain information ) to obtain test location information for the user to locate the location of the sound source. According to the sound effect offset information and the test location information of the test sound data, a functional relationship between the sound effect offset information and the sound source location information can be established, similar to step 106 .

Just like the process 200, the present invention can also use the functional relationship between the sound effect offset information and the sound source positioning information in the stereo sound field to interact with the user. For example, if the sound data of an interactive sound source corresponds to a sound effect offset information, the corresponding reference positioning information can be obtained from the functional relationship. When the sensor MS detects the user's action and provides corresponding interactive positioning information, if the interactive positioning information matches the reference positioning information, the sound data of the interactive sound source can be played. The interaction of the stereo sound field can be combined with the interaction of the three-dimensional image, which increases the fun of use. For example, a virtual bell can be displayed with a three-dimensional image. When the user touches the virtual bell, the ringtone sound data of the virtual bell can be played, and the sound source position of the sound will be consistent with the imaging position of the virtual bell.

Using the functional relationship between the parallax offset in the three-dimensional image data and the sensory positioning information, and/or the functional relationship between the sound effect offset information and the sensory positioning information in the stereo sound field sound data, the present invention can adjust the image data and the sound data , so that the 3D imaging distance value established by the parallax offset can conform to the distance value observed by the user, and the sound source position established by the sound offset information can also conform to the sound source position heard by the user.

Please refer to FIG. 8 , which illustrates a sound data adjustment process 300 according to an embodiment of the present invention. The main steps of the process 300 can be described as follows:

Step 302: Play test sound data. The test sound data is based on a corresponding test sound offset information to create a stereo sound effect.

Step 304: The user locates the sound source of the test sound data, and the user's positioning action can be sensed or the user responds with a positioning message as the test positioning data.

Step 306: Compare whether the sound source position positioned by the user in the test positioning data matches the sound source position to be established by the test sound effect offset information. If the two do not match (the error between the two exceeds a tolerance value), go to step 308 . If both match, proceed to step 310 .

Step 308 : Adjust the test sound effect offset information of the test sound data to change the sound source position established by the test sound effect offset information, and repeat step 302 . The functional relationship between the sound effect offset information and the sound source positioning information can be used to adjust the test sound effect offset information, so that the sound source position established by the test sound effect offset information can match the sound source position positioned by the user in the test positioning data. Equivalently speaking, adjusting the test sound effect offset information is to change the playing parameters of each speaker, such as volume, delay, phase and/or frequency.

Step 310: End the process 300.

Please refer to FIG. 9 , which shows a device 10 according to an embodiment of the present invention; the device 10 is applied to three-dimensional display to adjust/correct the imaging/sound effect of three-dimensional image data and sound data according to the sensed user positioning information. Location. The device 10 is provided with a control module 12 , an image processing module 13 , an image correction module 14 , an image adjustment module 16 , an audio processing module 17 , an audio correction module 18 and an audio adjustment module 20 . The control module 12 controls the operation of other modules. Like steps 102 and 104 in the process 100, the image processing module 13 receives the test 3D image and obtains the relevant preset parallax offset; the image correction module 14 receives the parallax offset of the test 3D image, and receives it from the sensor MS The test positioning information corresponding to each parallax offset is the user's positioning information on the test 3D image. Accordingly, the image correction module 14 can calculate a functional relationship according to each preset parallax offset and the corresponding test positioning information (such as step 106 ), so as to correlate different parallax offsets with the corresponding positioning information.

When a three-dimensional display image data is to be displayed, the image adjustment module 16 can adjust the offset of the display image data according to the operation of the image correction module 14 . The image adjustment module 16 receives display image data, the display image data corresponds to display positioning information of one or more 3D images, and each display positioning information represents a position to be imaged of an object to be displayed. The image correction module 14 substitutes each display positioning information into the functional relationship to provide a corresponding display parallax offset, which is received by the image adjustment module 16 . That is to say, in order for the user to view the 3D image at the position represented by the display positioning information, the parallax offset between the left and right images of the 3D image should match the display parallax offset calculated by the image correction module 14 . The image adjustment module 16 is to adjust each parallax offset in the displayed image data according to each displayed parallax offset. Since the size of each screen is fixed, there is a maximum limit to the amount of offset that can be adjusted when generating a three-dimensional image. Therefore, in another embodiment, the display may also send instructions or information to make the user slightly adjust its positioning so that the positioning information The corresponding offset is within the maximum limit that the screen can take.

For example, the 3D display image data displayed on the 3D display can be obtained from the real-time rendering of the 3D virtual model, so the focus, angle, etc. of the rendering can be changed according to the offset when rendering the 3D virtual model parameter, so that the 3D imaging can be located at the position specified in the display positioning information. Alternatively, the 3D display can adjust the optical parameters (such as angle, direction, etc.) of playing the left and right images to adjust the parallax offset between the left and right images, thereby correcting the distance value of the 3D imaging.

Similar to the operating principle of the image correction module 14, the sound processing module 17 receives preset sound effect offset information, which is respectively associated with the test sound data; The shift information receives test location information for the user to locate the sound source. The sound correction module 18 calculates a second functional relationship according to each piece of sound effect offset information and the corresponding test location information, so as to correlate different sound effect offset information with the corresponding location information.

When the display audio data associated with the display image data is to be played, the sound correction module 18 substitutes the display positioning information corresponding to the display audio data into the second functional relationship to provide an associated display sound offset information. The sound adjustment module 20 receives the display sound data, and receives the display sound effect offset information provided by the sound correction module 18, to adjust according to the display sound effect offset information and/or the display parallax offset and/or the functional relationship of the image correction module 14 Displays the sound effect offset of the sound data. The sound data adjusted by the sound adjustment module 20 can be output to the speakers of the 3D display (such as the speakers SK1 to SK5 in FIG. 7 ), so that the position of the sound source to be established for displaying the positioning information can match the position of the sound source heard by the user. Similarly, the image adjustment module 16 can also adjust the display image data according to each display parallax offset and/or display sound effect offset information and/or the second functional relationship of the sound correction module 18, and the adjusted image The data is output to the 3D display, so that the 3D imaging position (distance value) to be established for displaying the positioning information conforms to the position actually seen by the user.

The device 10 can be implemented in a control chip of a 3D display, and the image correction module 14 , the image adjustment module 16 , the sound correction module 18 and the sound adjustment module 20 can be implemented by software, firmware and/or hardware. The sound correction module 18 and/or the sound adjustment module 20 can also be omitted.

When the interactive image Iob(n) is displayed in the embodiment of FIG. 6 , the expected distance value Y_expect of the parallax offset Xc(n) may be different from the actual imaging distance value Y_actual; even if the two are different, the present invention 3D interactions are still implemented correctly. In the embodiment in FIG. 9 , the present invention further adjusts the parallax offset to change the actual imaging distance value Y_actual, so that the imaging expected distance value Y_expect can be equal to the distance value Y_actual actually felt by the user. Of course, the embodiments shown in FIG. 6 and FIG. 9 can be combined with each other.

To sum up, the present invention combines sensory positioning with 3D image display/sound playback, which can make 3D playback conform to the user's feeling, and also allow users to correctly interact with the virtual environment created by 3D playback.

To sum up, although the present invention has been disclosed above with preferred embodiments, it is not intended to limit the present invention. Those skilled in the art of the present invention can make various changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention should be defined by the claims.

Claims (18)

1. A method applied to a three-dimensional display, comprising: generating a three-dimensional image of a test object by using the three-dimensional display, and the three-dimensional image is related to a preset parallax offset; sensing a response signal from a user, the response signal being made with respect to the sharp imaging of the 3D image; and In response to the response signal, sensing a test location information related to the user, wherein the preset parallax offset has a specific relationship with the test location information, Wherein said method also comprises: Using the three-dimensional display to play a plurality of test sound data, each of the sound test data corresponds to a preset sound effect offset information; calculating a functional relationship according to each of the preset sound effect offset information and each of the test location information to describe multiple location information corresponding to different sound effect offset information; Acquiring display sound data, the display sound data corresponding to display positioning information; Substituting the display positioning information into the functional relationship to provide a corresponding display sound effect offset information; and The display sound data is adjusted according to the display sound effect offset information. 2. The method of claim 1, further comprising: generating multiple 3D images having different multiple preset disparity offsets; sensing a user's multi-response signal; sensing multi-test location information about the user; and The specific relationship is obtained according to the multiple preset parallax offsets and the multiple test positioning information for describing different positioning information corresponding to different parallax offsets. 3. The method of claim 1, further comprising: Using the 3D display to generate 3D images of multiple interactive objects, each of the 3D images corresponds to a second parallax offset; providing a corresponding reference positioning information according to each of the second parallax offsets and the specific relationship; sensing at least one interactive positioning information of the user; The at least one interactive positioning information is compared with each of the reference positioning information. 4. The method of claim 3, further comprising: sensing at least one response signal from a user, the at least one response signal being related to the three-dimensional image of the multiple interactive objects; and After responding to the at least one response signal, the at least one interactive positioning information of the user is sensed. 5. The method of claim 3, further comprising: When a comparison result shows that the interactive positioning information does not match the reference positioning information, send a reminder to the user to adjust a user location; and An interactive positioning information of the user is sensed again. 6. The method of claim 3, further comprising: When a comparison result shows that the interactive positioning information does not match the reference positioning information, an interactive parallax offset is provided according to the user's interactive positioning information and the specific relationship; and A second parallax offset of at least one of the 3D images of the multiple interactive objects is adjusted to become the interactive parallax offset, so that a corresponding reference positioning information conforms to the interactive positioning information of the user. 7. The method of claim 3, further comprising: associating each of the multiple interactive objects with a respective preset program; and If a comparison result shows that the interactive positioning information matches one of the multi-reference positioning information, the 3D display is made to perform the preset program associated with the matching interactive object. 8. The method of claim 1, further comprising: capturing user images to generate multiple sensing images; The test positioning information is provided according to the difference between the sensing images. 9. The method of claim 8, further comprising: When the sensing images are generated, the difference between the sensing images is utilized for focusing. 10. The method of claim 1, wherein sensing the test location information of the user comprises: emit a positioning wave to the user; receiving a reflected wave of the positioning wave; and The test positioning information is provided according to the reflected wave. 11. The method of claim 1, further comprising: When displaying a display object, substituting display positioning information into the specific relationship to provide a corresponding display parallax offset; and According to the displayed parallax offset, the 3D display is used to display the 3D image of the displayed object. 12. The method of claim 1, wherein the three-dimensional display comprises a plurality of speakers, and the method further comprises: Adjust the playback parameters of each speaker according to the display positioning information. 13. A method applied to a three-dimensional display, comprising: Using the 3D display to display multiple test 3D images, each of the test 3D images corresponds to a preset parallax offset; and Acquiring a corresponding test positioning information according to a response signal of the user to each of the test 3D images, the response signal is made with respect to the clear imaging of each of the test 3D images; The method also includes: When displaying the multi-test three-dimensional image, play multi-test sound data with the three-dimensional display, and each sound test data corresponds to a preset sound effect offset information; calculating a functional relationship according to each of the preset sound effect offset information and each of the test location information to describe multiple location information corresponding to different sound effect offset information; Acquiring display sound data, the display sound data corresponding to display positioning information; Substituting the display positioning information into the functional relationship to provide a corresponding display sound effect offset information; and The display sound data is adjusted according to the display sound effect offset information. 14. The method of claim 13, further comprising: A functional relationship is calculated according to each of the preset parallax offsets and each of the test positioning information to associate different parallax offsets with corresponding positioning information. 15. The method of claim 14, further comprising: Acquiring display image data corresponding to the second display positioning information; Substituting each second display positioning information into the functional relationship to provide a corresponding display parallax offset; and The display image data is adjusted according to each display parallax offset. 16. The method of claim 13, further comprising: A sensor is used to sense the user's response to each of the test three-dimensional images to obtain each of the test positioning information. 17. A device for three-dimensional playback, comprising: An image processing module, receiving a preset number of test 3D images and obtaining a related preset number of preset parallax offsets; An image correction module, receiving the preset number of preset parallax offsets, and receiving the preset number of test positioning information, the response of a user to the preset number of test 3D images, and according to each of the preset calculating a functional relationship between the parallax offset and each of the test positioning information, the response is made by the user regarding the clear imaging of the predetermined number of test 3D images; and An image adjustment module, receiving a second preset number of display positioning information corresponding to a display image data and a corresponding second preset number of display parallax offsets, and adjusting the display image data according to each of the display parallax offsets, Wherein, described device also comprises: A sound processing module, receiving a preset number of preset sound effect offset information, which is related to a preset number of test sound data and the preset number of test 3D images; A sound correction module, receiving each of the test location information, calculating a second functional relationship according to each of the preset sound effect offset information and each of the test location information, so as to associate different sound effect offset information with the corresponding location information, The display image data is also associated with a display sound data, and the display sound data corresponds to a second display positioning information; the sound correction module also substitutes the second display positioning information into the second functional relationship to provide an associated display sound effect offset information, and the image adjustment module further adjusts the display image data according to the display parallax offset and the display sound effect offset information, The device also includes: A sound adjustment module receives the display sound data, and adjusts the display sound data according to the display sound offset information. 18 . The device according to claim 17 , wherein the sound adjustment module further adjusts the display sound data according to each of the display sound offset information and each of the display parallax offsets.