DE102024002109A1 - Active multiview 3D display with observer position tracking - Google Patents

Abstract

An active multiview 3D (AM3D) display is considered, in which detectors are used to recognize the observer's position both parallel and perpendicular to the display surface. Using various modes for shifting at least one of the objects of the AM3D display, such as parallax barriers, 2D displays, or the 3D image itself, the proposed method allows the viewing area of the stereo pairs to be extended and prevents the observer's gaze from entering the 3D image breakup area. Furthermore, the proposed approach allows the parameters of a 3D image to be changed depending on the distance between the observer and the AM3D display, in order to make the perception of 3D objects on the AM3D display as realistic as possible.

Description

The subject matter of the following patent is a method for demonstrating an observational 3D representation dependent on the viewer's position, using an active AM3D display and detectors that track the viewer's position both longitudinally and transversely relative to the AM3D surface of the display. In the absence of the detectors determining the viewer's position, the AM3D display functions like a conventional 3D autostereoscopic display (ASD) because it does not require 3D glasses for viewing the 3D representations and is characterized by the ability to view the representation in volume from different perspectives and simultaneously by multiple viewers. Furthermore, the viewer can move parallel to the AM3D display and perceive 3D objects by viewing them from N different perspectives. One of the disadvantages of using ASD (Automatic Display Detection) is that as the viewer moves along the 3D display, the viewing areas of a high-quality 3D image alternate periodically with those of a fragmented 3D image. The size of the viewing area of a high-quality 3D image is determined by the number N of views and the size of the viewing area of each view, and the boundary of this viewing area is defined by the appearance of pseudoscopic views. In practice, this means that at the optimal viewing distance of the 3D image, which consists of N > 2 views, the area of 3D image fragmentation appears where the absolute difference between the numbers of the observing pseudoscopic views is equal to |N - 1|.

Within the observed 3D image interruption area, it is not possible to see a high-quality 3D image, and the viewer will strive to leave this area as quickly as possible, either by moving along the AM3D display or backwards. The size of the 3D image interruption area depends on the structure of the parallax barrier and the 3D image itself. It should be noted that as the length of the 3D image interruption area increases, the transition to a high-quality, uninterrupted 3D image becomes smoother; conversely, as the size of the observed 3D image interruption area decreases, the transition to a high-quality, uninterrupted image becomes sharper, leading to greater visual discomfort.

In the following, we will call such an uninterrupted area of 3D image viewing in which, during the movement of the viewer along the AM3D display, no stereo pair is seen that contains pseudoscopic views with numbers whose difference is |N - 1|.

In this patent, a parallax barrier is considered as a specific example of an ordered and directional optical structure that is an integral part of an AM3D display, although a lens grid, an arrangement of prisms, lenses, or an optical holographic element may be used.

Furthermore, instead of the term "AM3D display with detectors" that track the position of the observer relative to the surface of the AM3D display, as well as devices that enable a shift of AM3D display objects, such as 2D display, parallax barrier and 3D image shift, we will use the term AM3D system in the following.

The term “active” in the name of our AM3D display with observer position tracking means that, according to the change in the observer’s position, at least one of the above-mentioned objects can be moved, such as moving the 2D display, moving the parallax barrier, or moving the 3D image.

A number of patents present various methods for enlarging the viewing area of a high-quality 3D image between two adjacent areas of an observing 3D image interruption while the viewer is continuously moving in any direction along the display. TW200950501A It is proposed to use detectors to record the viewer's position during movement along the display and to generate new views instead of previously seen ones only when the viewer approaches the boundary of the observing 3D image interruption area.

In most practical cases, the viewer's distance from the 3D display surface deviates from the optimal distance, reducing the size of the viewing area for each view. This means that the frequency of occurrences where a 3D image break is observed increases, leading to significant visual discomfort. Therefore, even with relatively small deviations of the viewer from the initial position, it may become necessary to generate a large number of new views, potentially requiring additional time, hardware, and financial resources. Furthermore, with small display sizes... The viewing areas of the views will be in TW200950501A The computer system used with a detector may not be able to prepare the views necessary for the demonstration on the surface of the 3D display in time.

Furthermore, even at an optimal viewing distance, the viewer's eye may enter the 3D image interruption zone if the criterion v _{_a}>n_e / -τ _{_a} is met, where v _a is the average speed of the observer's movement along the 3D display surface with interval n × e, n is the number of views the viewer sees before approaching the 3D image interruption boundary, e is the distance between the viewer's pupils, and τ _{_a} is the time the 3D system needs to prepare a stereo pair for demonstration purposes that contains no pseudoscopic views. This stereo pair should be located in the areas of the 3D display visible to a viewer positioned at the boundary of interval n × e.

It should be noted that in (ASD) and TW200950501A In contrast to our method, the size of the viewing area for each view of a 3D representation is fixed for every distance from the viewer to the 3D display and does not exceed a certain value. In our method, the size of the viewing area of a stereo pair can be controlled by the AM3D system and adjusted to a size that provides the viewer with the best viewing experience of the 3D image.

From the above, it becomes clear that one of the most important parameters of a 3D image is the size of the viewing area R _{_i} of the stereo pair S(i - 1, i), where i denotes the number of views seen by one eye. Since our approach allows us to control and modify the size of the viewing area R _{_i} , this size can be designed such that R _{_i}> e. Therefore, even if the criterion v _{_a} <e_n / τ _{_a} applies to TW200950501A If this condition is not met, it is possible to fulfill the criterion v _a < ∑ _i R _i / τ _a with the same number of observed views as in [previous approach] by increasing the observation range R _i in our approach. TW200950501A This achieves the uninterrupted observation of the 3D image by generating additional stereo pair views. It should be noted that the index i in the summation discussed above can change from N - n + 1 to N or from n to 1, depending on the viewer's direction of movement.

Furthermore, in most cases n can be chosen such that it is in the vicinity of the interval midpoint, i.e. next to the view which is calculated using the IntegerPart [N / 2] operation.

In the approach proposed here, we should speak of a stereo pair rather than a separate view, since we are moving stereo pairs synchronously across the AM3D display surface, not individual views. If one of the moving views does not correspond to or belong to the same stereo pair as its neighbor, we will obtain a low-quality 3D image.

It should be noted that when we speak of the size of the observation area of a particular view n in (ASD) during the observer's movement, we actually mean the size of the observation areas in which a transition occurs, for example, from viewing view n to viewing view n + 1. The transition from observing one view to observing another can be defined in different ways, but the size of the observation area of the transition will be the same in all cases. Let us determine the size of the observation area of the transition between adjacent views such that, during such a transition, the contribution of any given view to the subpixel value changes from its maximum to its minimum value, for example, from 1 to 0.

Similar to the above, when discussing the size of the observation area R _{_i} of the stereo pair S(i - 1, i) in our AM3D display, we are actually referring to the size of the transition area from observing the stereo pair S(i - 1, i) to observing the stereo pair S(i, i + 1), or, if the observer moves with decreasing numbers of observed views, then to observing the stereo pair S(i - 2, i - 1). Let us define the size of the observation area of the transition between adjacent stereo pairs such that, during such a transition, the contribution of a particular stereo pair to the value of the observed elements of the AM3D display surface changes from a maximum to a minimum value, for example, from 1 to 0. Therefore, for the sake of brevity, we will henceforth refer to the observation of a stereo pair, rather than constantly emphasizing that it is the observation of a mixture of adjacent stereo pairs.

Some glasses-free 2-view 3D systems also use detectors to determine the observer's position. It should be noted that 3D displays that include detectors are not... The system can react immediately to the observer's movement. This means that in two-view systems, the observer can very quickly enter the observed 3D image interruption area for a time e/V. To prevent this, the average movement speed of the observer V on the interval e and the reaction time of the 3D system t _r must satisfy the criterion V × t _r << e.

In our case, an AM3D display with observer position tracking is used, so that before the AM3D system determines the observer's position, the observer has the opportunity to view several stereo pairs in autostereoscopic mode. In autostereoscopic mode, the observer can move at any speed without losing the perception of high-quality 3D representation. Thus, if the observer begins their movement from view n towards view N, the AM3D system we propose has the opportunity, during the observation time of views n to N in autostereoscopic mode, to prepare the necessary view for demonstration and continue demonstrating a high-quality, uninterrupted 3D image according to their position relative to the AM3D display. As shown below, depending on the selected demonstration mode, after view number N, views numbered N - 1, N, or N + 1 can be demonstrated.

Furthermore, unlike 3D systems with 2 views, the proposed AM3D system with position tracking of the observer allows additional visual information about different views of the object to be obtained as the observer moves along the AM3D display.

It is important to emphasize that 3D image interruption viewing areas are present in every ASD (Automatic Display Device) and prevent continuous, comfortable viewing of high-quality 3D objects as the viewer moves across the display. We will not discuss specific calculations of the size of image viewing areas between adjacent interruption points in a 3D image, as there are numerous parallax barriers with structures of varying complexity, requiring separate calculations for each.

Our method makes it possible to apply different modes of 3D image display using detectors to determine the position of the observer when at least one of the following displacements is used, for example, mechanical displacement of the 2D display, mechanical displacement of the parallax barrier, as well as displacement of the 3D image using computer simulation, which prevents the moving observer from entering the observation area of the 3D image interruption.

Using at least one of the 3D image display modes listed below can prevent the viewer from entering the 3D image interruption area. However, the viewer will receive a significantly better 3D viewing experience when a combination of different 3D image display modes is used. We will examine these 3D image display modes in more detail below.

To illustrate different demonstration modes of a 3D representation, we consider ( 1 (a, b, c, d)). On ( 1 (a, b, c, d)) Line 1 represents the screen surface of a 2D display, and the numbers on this line represent the numbers of the views in stereo pairs that the viewer 11 sees when positioned at optimal distance 5 and moving parallel to the screen surface of the AM3D display to the left along line 7 from point _x0 to point _x1 or _x2 . The numbers on line 1 or line 10 represent the numbers of the stereo pair views, with the stereo pairs corresponding to those numbers being available to the viewer when the numbers are located within a lighter rectangle. To illustrate the methods used, the spatial points _x0 , _x1 , _x2 are considered at times _t0 , _t1 , _t2 . On the contrary, if the numbers are located within a gray rectangle, then the views containing these numbers at the points in space _x0 , _x1 , _x2 and at the times _t0 , _t1 , _t2 are hidden from the viewer. For the sake of simplicity, the demonstration of the applied methods in ( 1 (a, b, c, d)) only shows one opening of the parallax barrier 2. In reality, there is a parallax barrier 2 with multiple openings in front of the 2D display 1.

On ( 1(a) ) Stereo pairs are shown on line 1, which the viewer 11 can see while moving left from point x ₀ to point x _1. Without detector measurements to determine the observer's position, our AM3D display functions like a conventional (ASD) display, where the sizes of the viewing areas of stereo pairs have a strictly defined shape in space and the distance between adjacent views at the optimal distance is equal to e. Therefore, the observer sees in ( 1 (a) ) at point x ₀ at time t ₀ he sees the stereo pair S(3, 4), while at point x ₁ at time t ₁ he sees the stereo pair S(7, 8).

The proposed main modes for demonstrating stereo pairs, including their combination, are in ( 1 (b, c, d)) shown. In ( 1 (b, c, d)) are numbers on line 1 for comparison with the numbers on line 10 and show which numbers of views an observer on the surface of the AM3D display at point x ₁ or x ₂ at time t ₁ or t ₂ would see if he was at point x ₀ at the initial time t ₀ .

On ( 1 (b) One of the proposed 3D image demonstration modes is considered, where the viewing areas of a stereo pair are enlarged by shifting a 3D image. In this mode, detector 4 transmits a signal to the AM3D display about the position of the viewer 11 relative to the AM3D display. According to this signal, the 3D image is shifted to the right, that is, in the direction of arrow 12, opposite to the movement of the viewer 11. Thus, at point _x0 at time _t0, the viewer sees the stereo pair S(3, 4) as depicted on line 10, and at point _x1 at time _t1, the viewer sees the stereo pair S(5, 6) as depicted on line 1. Therefore, due to the shift of the 3D image onto ( 1(b) ) the viewer was shown the stereo pair S(5, 6) and not the stereo pair S(7, 8), which is on ( 1(a) ) is depicted.

When comparing ( 1(a) ) with ( 1(b) ) you can see that on ( 1(b) ) when the observer was shifted by the same distance x ₁ - x _0, fewer stereo pairs were considered than on ( 1(a) This effectively increases the size of the viewing area of the stereo pairs when the viewer moves parallel to the surface of the AM3D display. For brevity, we will call this mode the viewing area expansion mode.

This means that by moving the entire 3D image or the observing stereo pair, the size of the observation area and the viewing time of the views can be increased in a controlled manner, while slowing down the observer's approach to the 3D image break point.

It should be noted that it is possible to shift the 3D image according to the observer's movement, so that the observer sees the same stereo pair throughout the entire movement. Let us call such a shift of the 3D image "corresponding" if the viewer can see the same stereo pair during their parallel movement to the 3D display surface. As shown in ( 1(b) As shown, a 3D image moves a smaller distance in the same time period than the "corresponding" displacement. In other words, the closer the displacement length of the observed stereo pair is to the length of the "corresponding" displacement, the slower the observing stereo pair is viewed, and the larger the size of the observed stereo pair's viewing area will be.

On ( 1 (c) Another way of demonstrating a seamless 3D image is considered, in which a 3D image moves over a much greater distance in the same time period compared to the "corresponding" displacement. The observer sees the displacement of the 3D representation in reverse order, so that the 3D representation repeatedly shows the observer the stereo pairs just seen, in reverse order.

At point _x0 at time _t0, the observer sees the stereo pair S(4, 5), which contains views numbered 4 and 5, as shown on line 10. If there were no 3D image shift, the observer at point _x1 at time _t1 would see a stereo pair S(7, 8), which contains views numbered 7 and 8. However, due to the significant shift of the 3D image in the opposite direction to the observer's movement, the observer sees the stereo pair S(2, 3), which contains views numbered 2 and 3, as shown on line 1. While moving the observer without 3D image shift would result in an increase in the view numbers of the observed stereo pairs, this 3D image demonstration mode leads to a decrease in the view numbers of the observed stereo pairs due to the 3D image shift, and vice versa. For brevity, we will refer to this mode as the reverse lookup mode. Generally, each time the observer approaches the breakpoint due to the increasing (decreasing) number of the observed views, the number of the observed view decreases (increases) until it reaches the number located between two adjacent breakpoints of the 3D image, due to the movement of at least one of the three parts of the AM3D display, such as the ordered and directional optical structure, the 2D displays, or the 3D image. In this case, the observer can continue moving in the original direction, allowing them to view the increasing (decreasing) number of views again.

It should be noted that no movement of the observer is necessary when using this mode. The mode also works if the observer remains stationary. This is an important finding that makes it possible to use this mode even This mode should be used when the observer is at the holding point. It allows the viewer's gaze to be directed towards the observation area of stereo pairs that are further away from the area where the 3D image break is observed.

Furthermore, the generation of new views is offered as a third mode to demonstrate the seamless 3D image ( 1(d) ) is used. For example, when the observer has finished viewing stereo pair S(7, 8), a new stereo pair S(8, 9) is displayed instead of stereo pair S(8, 1). This mode prevents the observer's gaze from falling into the area where the 3D image break is being viewed. In contrast to the ones in ( 1(b) ) and ( 1(c) In the modes shown, this mode enhances the look-around effect, but requires a significant increase in material and computational costs. From a practical point of view, it is necessary to use this 3D image demonstration mode in combination with the modes shown ( 1(b) ) and ( 1(c) ) to use the modes shown so that the AM3D system has enough time to generate new views in a timely manner. In fact, the modes shown ( 1 (b) ) and ( 1(c) The modes shown provide additional time that the AM3D system can use to generate the new stereo pairs whose views are numbered outside the range of the currently displayed N views. For example, if we increase the size of the viewing area containing views from 2 to N+1, we give the AM3D system time to prepare another demonstration of the stereo pair S(N+1, N+2), and so on.

Thus, the mode of generating new views can be implemented highly effectively through the observation area expansion mode, or reverse lookup mode, or through a combination of both.

In one embodiment, when the moving observer, while viewing stereo pairs, reaches the boundary of the seamless viewing area corresponding to the stereo pair S(N - 1, N), the 3D image can shift in the opposite direction to n < N views. After the 3D representation shifts in the opposite direction, the observer will see the previously viewed stereo pairs in reverse order, remaining on the S(N - 1 - n, N - n) stereo pair. The observer can then move again in the same direction along the surface of the AM3D display to the stereo pair S(N - 1, N) until reaching the boundary of the beginning of the seamless viewing area, thereby seeing a high-quality, seamless 3D image. This demonstration mode can be repeated if necessary.

In TW200950501A The overall size of the observation area is increased by increasing the total number of observation views. With the AM3D display proposed here, the size of the observation areas of the stereo pairs can be increased using the observation area expansion mode and/or the reverse lookup mode. The number of stereo pairs viewed cannot exceed N - 1, even when the observer is significantly displaced parallel to the AM3D display surface.

As on ( 1 (d) As shown in Figure 10, if the observer is initially located at point _x0 , they can observe the stereo pair S(7, 8). If it were an ordinary stereo image (ASD), then at point _x2 the stereo pair S(2, 3) would be observed, and the observer would have to traverse the observation area of the 3D image break. If the image in the TW200950501A If the proposed method were used, a stereo pair S(10, 11) containing views numbered 10 and 11 would be observed at point _x2 . In our case, however, a stereo pair S(8, 9) containing views numbered 8 and 9 is observed, as shown on line 1. This increases the size of the observation area of the stereo pair S(8, 9). Thus, the mode we propose has the advantage that, by extending the demonstration time of the S(8, 9) stereo pair, it gives the 3D system additional time to prepare for the demonstration of the next newly generated stereo pairs, for example, the generation of new S(9, 10) and S(10, 11) stereo pairs.

All three modes discussed—the reverse lookup mode, the new view generation mode, and the observation area expansion mode—prevent the observer from falling into the 3D image breakup area. Although each mode can be used independently, in practical applications it is useful to use a combination of the three suggested modes.

It should be noted that the one on ( 1 (b , c)) discussed observation area extension mode and/or reverse lookup mode can be performed not only when shifting the 3D image, but also when shifting any part of the AM3D display in the form of 2D display and/or parallax barrier.

Thus, by shifting to a specific distance, which is determined by the position of the moving observer, at least one of the parts of the AM3D display, such as the 2D display, ordered and directional optical structure, as well as one The area expansion mode and/or the reverse lookup mode discussed above can be implemented for part or all of the 3D image. Using at least one of these modes gains the additional time needed to demonstrate a stereo pair S(N + m, N + m + 1) instead of the pseudoscopic stereo pair S(N + m, m + 1), where m is an integer. In this case, for the sake of certainty, we assume that the observer moves in the direction from view N + m - 1 to view N + m and beyond.

From a mathematical point of view, this means that in order for the system to generate view N + m + 1 instead of view m + 1 at the moment the observer approaches the 3D image break area, it is necessary that the criterion L _gen > v _gen τ _gen is satisfied, where τ _ger is the time interval that the AM3D system needs to generate a new stereoscopic view instead of the pseudoscopic view, where L _gen is the distance from the current position of the observer to the 3D image break area where the m + 1 pseudoscopic view would be observed if the new view N + m + 1 had not been generated in its place, and v _gen is the average speed of movement of the observer on the interval L _gen .

In this case, the choice of using 3D stereo pair shifting or parts of the AM3D display in the form of a 2D display and/or parallax barrier depends on the technical characteristics of the AM3D system used and the type of movement of the observer. Generally, our method allows for any combination of shifting parts of the AM3D display and the entire 3D image on the surface of the AM3D display relative to the observer. For example, to increase the viewing area of high-quality 3D images, it is possible to shift not only the 3D stereo pairs but also the parallax barrier simultaneously, according to the observer's movement.

( 2(a) Figure 1 is a schematic representation of the general scheme of the AM3D display, showing one of the options for the order of sizes of the stereo pair observation areas when the observer moves horizontally parallel to the screen surface of the AM3D display. The dimensions of the arrows on lines 8 and 9 indicate the dimensions of the stereo pair observation areas, and the direction of the arrows indicates the direction of movement of the observer, who moves along line 7. In ( 2(a) ) the observer first moves from the holding point O _i to the right to the next holding point O _i+1 and successively passes through the stereo pair observation areas from R _n+1 to R _n+4 , where n is a specific integer within the interval between the numbers 0 and N - 4. After the observer has reached the holding point O _i+1 , he can, for example, move to the left to the holding point O _i+2 , as in ( 2(a) ) is shown. In the case under consideration, the sizes of the observation areas R _n+3 , R _n+4 and R _n+1 have different values when the observer moves in opposite directions, as shown on lines 8 and 9.

As opposed to ( 2(a) ), where the main elements of the AM3D display scheme are shown, on ( 2 (b, c, d)) For brevity, only lines 7 and 9 are shown, which indicate the order of the sizes of the stereo pair observation areas when the observer moves horizontally to the right parallel to the screen surface of the AM3D display.

As on ( 2(a) As shown, when traversing the observation areas R _{_n+1} on line 9 and R _{_n+3} , R_{_n+4} on line 8 in autostereoscopic mode, the AM3D system successfully prepares the sequence of observation areas R _{_n+2} , R _{_n+3} , and R_{_n+4} on line 9 and R_{_n+2} , R_{_n+1} , and R _{_n} on line 8 in the mode of magnifying the observation area for demonstration purposes. In this case, the sizes of the observation areas R _{_n+4} on line 9 and R _{_n} on line 8 are limited by the stopping point O _{_i+1} and O _{_i+2}, respectively.

( 2(b) ) shows one of the options for the distribution of the sizes of the stereo pair observation areas along line 7 of the observer movement from stopping point O _i to the next stopping point O _i+1 to the right and further from stopping point O _i+1 to stopping point O _i+2 . ( 2(b) Figure 9 shows a combination of modes such as the stereo pair's observation area extension mode and the reverse lookup mode to prevent the observer from falling into the observation area of the 3D image break. In this case, we consider a special case where the period of the 3D image structure is 8. The sizes of the observation areas _R4 and _R5 from the holding point _Oi to the holding point Oi ₊₁ , and the observation area _R5 from the holding point Oi ₊₁ to the holding point Oi ₊₂ shown on line 9, are exactly the same as in normal (ASD). Since the AM3D system does not react instantly to the observer's movement due to its inertia, the system is given time to determine the observer's position as it traverses the observation areas _R4 and _R5 . In this case, the 3D image remains stationary, and the sizes of the observation areas are _R4 ≤ e and _R5 ≤ e.

It should be noted that each of the ( 2 (a, b, c, d)) depicted 3D image observation area which can be linked to the demonstration modes discussed above, enabling the observation of a seamless 3D image. To implement any of the modes discussed above, the criterion τ _{_m}<L_{_m} / v_{_m} must be met, where τ _{_m} denotes the time the AM3D system needs to prepare a stereo pair for demonstration according to one of the selected modes. This stereo pair should be displayed in those areas of the AM3D display that are visible to a viewer located at the boundary of the interval L _{_m} . In this case, L_{_m} is the distance the viewer travels from their current position to the position where, without using demonstration modes, a 3D image break would be observed, and v _{_m} denotes the average movement speed within this interval L_{_m} .

It should be noted that the size of the observation area _R8 is determined by the mode used to magnify the observation area. Therefore, depending on the measurement, the size of _R8 can exceed the size of the observation areas of the previous stereo pairs and is limited by the stop point Oi ₊₁ . It should be noted that at stop point Oi ₊₁ , the reverse lookup mode of the currently viewed stereo pairs is activated, so the viewer's view is in the observation area _R5 . If there were no shift of the 3D image, the viewer would end up in the _R1 observation area, where they would observe the break in the 3D image. Due to the effect of the reverse lookup mode, the observed stereo pair is moved away from the boundary of the 3D image break. Subsequently, the viewer's movement can continue to the next stop point Oi ₊₂ .

( 2(c) ) shows a combination of such 3D demonstration modes as the observation area expansion mode, the reverse lookup mode, and the new view generation mode. In ( 2(c) ) we, unlike ( 2(b) ) supposes that the period of the image structure has a period equal to N. The dimensions of the observation areas of stereo pairs R _n+1 and R _n+2 are the same as in an ordinary (ASD). This means that their values do not exceed e. If the observer is located within these observation areas, the 3D image does not move. This means that the observer can move within these areas at any speed, and the AM3D system with observer position tracking uses these observed areas to determine the observer's position and prepare the views R _n+3 for demonstration.

The sizes of the observation ranges from R _{_n+3} to R_{_N} are determined by the observation range expansion mode. In practice, the values of these observation ranges can be such that they provide the observer with a high-quality, realistic 3D representation. As soon as the observer reaches the boundary of the R_{_N} observation range during their movement, AM3D Display shows a new stereo pair S(N, N+1) corresponding to the R_{_N+1} observation range. For simplicity, we consider only one observation range, which corresponds to the generation of the stereo pair with views whose numbers lie outside the range of views currently present on the display surface; however, there can be multiple such observation ranges.

At observation point O _i+1, the reverse lookup mode is activated, so the observer lands in the observation area R _n+1 . This reverse shift of the 3D image prevents the viewer's gaze from entering the 3D image interruption area and ensures that the currently viewed stereo pairs are seen again. The size of the observation area R _n+1 corresponds to the size of the usual (ASD) and is not exceeded. The size of the observation area R _n+2 is determined by the observation area extension mode. The size of this observation area is limited by the stop point O _i+2 .

( 2(d) ) shows one of the options for the order of the stereo pair observation areas based on the observation area expansion mode. In this case, the sizes of all observation areas can be approximately equal to or greater than e, with the possible exception of the start and end areas in the interval between the breakpoints O _{_i} and O _{_i+1} .

It should be noted that the number of possible options for the order of the observation areas does not correspond to the number in ( 2 The options shown (a, b, c, d)) are limited. In reality, the number of options used may be much larger, but in any case, each option uses the demonstration modes discussed above in different combinations.

It should be noted that the selection of the mode for displaying the image to the moving observer can be automated. In this case, the parameters of the detector that determines the observer's position, the speed of the computer's processor and graphics card, the speed of the device that shifts the 2D display and/or parallax barrier, and the type of movement should first be determined. The viewing angle of the 3D image must be taken into account. Based on this data, the computer can determine not only the size of the observation area, but also the order of the sizes of the observation areas, which are determined by the aforementioned 3D image demonstration modes.

If in (ASD) or 3D display in TW200950501A Since the maximum size of the viewing area of a view in the stereo pair is achieved at the optimal distance, in our case any viewing distance from the observer to the display can be optimized, which can be very important for applications. Furthermore, when using our approach, the concept of the optimal distance between adjacent stereo pairs changes. This distance can now be greater than e and can be determined by the high quality of the 3D image observed from different views, as well as by the perceived proximity of the observed virtual 3D objects to reality, as if real objects were being observed instead of virtual 3D objects.

In a conventional (ASD) display and 3D display in TW200950501A is the matrix $M_{i,j}^n$ The contribution coefficient of each view n to the value of each subpixel with coordinates i and j is an important element for constructing a 3D image. The size value of the elements in this matrix can vary from 0 to 1. This matrix for (ASD) or TW200950501A It is calculated once for each viewer position and does not change depending on the viewer's position relative to the screen surface of the 3D display, i.e. $M_{i,j}^n=\text{const}\text{.}$ An example of how to calculate this matrix can be found in US2011109623A1 This means that the dimensions of the viewing areas in stereo pairs are fixed and have a strictly defined shape in space. As mentioned above, the size of the viewing areas unfortunately decreases when the distance between the viewer and the display surface changes. This leads to discomfort when viewing a 3D image.

With the method proposed here, the matrix can be changed when the 2D display and/or the parallax barrier is shifted. $M_{i,j}^n$ The coefficient matrix must also be calculated once for each position of the observer relative to the AM3D display. If the displacement of any part of the 3D image relative to the display surface is used, the coefficient matrix must be adjusted. $M_{i,j}^n$ individually calculated for each position of the part of a 3D image to be moved, so that $M_{i,j}^n=M_{i,j}^n\left(\overrightarrow{r_{3Dlm}}\left(t\right)\right),$ where $\overrightarrow{r_{3Dlm}}\left(t\right)$ denotes a two-component vector of the image displacement in the plane of the AM3D display during time t. Each of the coefficients $M_{i,j}^n$ is a rational number that varies within the interval from 0 to 1 inclusive. It is also easy to write analytical formulas to calculate the coefficient matrix that determines the displacement of a 3D image depending on the viewer's position.

It should be noted that the matrix P _{_i,j} determines which part of the subpixel area with coordinates i and j is seen by a viewer moving relative to the AM3D display. In this case, the value of the matrix element P _{_i,j} changes from 0 to 1 inclusive. The time dependence of such a matrix can be represented in a more general form as: $P_{i,j}=P_{i,j}\left({\overrightarrow{r}}_0\left(t\right),{\overrightarrow{r}}_{2D}\left(t\right),{\overrightarrow{n}}_{2D}\left(\alpha \left(t\right),\beta \left(t\right),\gamma \left(t\right)\right),{\overrightarrow{r}}_{\text{str}}\left(t\right)\right),$ where the three-dimensional vector ${\overrightarrow{r}}_0\left(t\right)$ denotes the position of the viewer relative to the 2D display surface in space at time t. The three-dimensional vector ${\overrightarrow{r}}_{2D}\left(t\right)$ denotes the position of the surface center of a 2D display at the moment of time t relative to the selected coordinate system, the three-component unit vector ${\overrightarrow{n}}_{2D}\left(\alpha \left(t\right),\beta \left(t\right),\gamma \left(t\right)\right),$ perpendicular to the 2D display surface and lying in the middle of the 2D display, its beginning being defined by the end of the vector ${\overrightarrow{r}}_{2D}\left(t\right)$ The direction of this three-component unit vector is determined by three Euler angles α(t), β(t), γ(t). It should be noted that the surface of the parallax barrier remains parallel to the screen surface of the 2D display at every time t; therefore, its normal direction in space is also determined by the vector. ${\overrightarrow{n}}_{2D}\left(\alpha \left(t\right),\beta \left(t\right),\gamma \left(t\right)\right)$ determined. Since, moreover, the distance between the surface of the 2D display and the surface of the parallax barrier remains constant at every time t, the position of the center of the surface of the parallax barrier is determined by the two-dimensional vector ${\overrightarrow{r}}_{\text{str}}\left(t\right)$ The beginning of this vector is determined. ${\overrightarrow{r}}_{\text{str}}\left(t\right)$ determined by the intersection point of the surface of the parallax barrier with a straight line defined by the vector ${\overrightarrow{n}}_{2D}\left(\alpha \left(t\right),\beta \left(t\right),\gamma \left(t\right)\right)$ The vector runs along a path and its end is determined by the position of the center of the parallax barrier's surface.

Thus, the matrices P _i,j and $M_{i,j}^n,$ that at least one of the shifts of such AM3D display objects, such as 2D displays, the parallax barrier or the 3D image, gives us the possibility to adapt the shape of the viewing area of a stereo pair to the predetermined dimensions. This makes it possible to create a more realistic and seamless 3D image for observation.

Our invention can be applied to large AM3D displays and for multiple viewers simultaneously. Each viewer must have their own detectors to determine their position relative to the screen surface of the AM3D display. As the viewer approaches each other, it is reasonable to assume that the number of stereo pairs they observe should coincide. This can be achieved by shifting the stereo pair they are observing. If, in this case, one viewer remains stationary and the other approaches, it is reasonable to assume that the stereo pairs observed by the stationary viewer remain unchanged, while for the moving viewer, the scene is rotated based on the movement of the observed stereo pair so that, as the viewers approach each other, the numbers of the stereo pairs they observe would coincide or become very similar in size.

It should be noted that the approach proposed here can also be used for a single viewer. Instead of moving the entire 3D image according to the viewer's movement, only a portion of the 3D image corresponding to the observing stereo pair or a mixture of stereo pairs can be moved. This saves considerable computing resources and makes the proposed AM3D display fast in its operation.

As mentioned above, only at an optimal viewing distance is the size of the viewing area during the transition from one view to another in (ASD) equal to e (the distance between the viewer's pupils). If the distance from the 3D display surface to the viewer decreases or increases, the size of the viewing area for each view decreases. This leads to discomfort when viewing the 3D image with (ASD), as it does not correspond to observing the same 3D objects in reality. With the approach proposed here, it is also possible to change the size of the viewing area of the observed views of a 3D representation according to changes in the distance between the viewer and the 3D display surface of an AM3D display, which could more closely resemble reality. This is a major difference between the proposed AM3D display and (ASD), where there is no control over the viewer. This also distinguishes the proposed 3D display from the one in TW200950501A The proposed 3D display adds additional observation areas of the same size to N observation areas, and the size of the observation areas decreases when the distance between the viewer and the 3D display surface deviates from the optimal distance, leading to visual discomfort.

With the AM3D display we have proposed, the size of the viewing area of the stereo pair can be changed depending on the distance of the viewer to the AM3D display surface by adjusting this size to the predetermined shape.

In order to demonstrate higher quality 3D images in the proposed AM3D system, the detector(s) must therefore determine not only the displacement of the observer parallel to the surface of the AM3D display, but also the distance from the observer to the surface of the AM3D display. 3 Figure 1 shows an approximate diagram of an AM3D display with detectors 4 that determine changes in the observer's position along and perpendicular to the surface of the AM3D display. This data from the detectors 4 is processed by the computer 3, which uses this data to create a high-quality 3D image that closely approximates reality and corresponds to the observer's position at any given time. This means that when viewing 3D objects on our AM3D display, the viewer experiences visual sensations similar to those experienced when observing similar objects in reality. If required, the 2D display and/or parallax barrier can also be shifted using attachment 13, in conjunction with the 3D image shift.

In general, the dependence of the size R_{_n} of the observation areas of a stereo pair S(n - 1, n) on the distance z between the observer and the AM3D display surface can be represented as R _{_n} = R _{_n}(z). It should be noted that the approach proposed here allows the size of the observation area to be changed as a function of the distance between the observer and the surface of the AM3D display using a predefined mathematical formula.

Furthermore, our approach even makes it possible to provide a predefined shape for the observation area in three-dimensional space. This means that an observer at any point in space sees the stereo pair determined by the given function. From a practical standpoint, this spatial observation area of a specific stereo pair is realized when the criterion _τf < _Lf / _vf is satisfied, where _τf is the time interval the AM3D system requires to prepare a given stereo pair for demonstration, as seen by an observer located at the boundary of the interval _Lf. is located where L _f is the distance traveled by the observer from the current position to the position corresponding to the observation of a particular stereo pair, and v _f denotes the average speed of movement of the observer in the interval L _f .

It should be noted that the time τ_{_m} can differ for each of the three stereo pair demonstration modes under consideration, where τ_{_m} is the time the AM3D system requires to demonstrate the stereo pairs according to a given mode. It should be noted that the feasibility of the criterion τ _{_m}<L_{_m} / v_{_m} depends not only on the speed of the system but also on the motion characteristics of the observer v _{_m}, where v _{_m} is the average speed of the observer over the interval L _{_m} , and L _{_m} is the distance the observer travels from their current position to the position where a stereo pair is observed according to the selected mode. For example, the slower the observer moves, the easier it is to satisfy the criterion. This selects the mode whose characteristic time τ _{_m} satisfies the above criterion τ _{_m}<L_{_m} / v_{_m} . If several modes satisfy the criterion τ _{_m}< L _{_m} /v_{_m} , the mode that provides the best perception of the 3D image is selected as the result. Depending on the application, an algorithm can always be specified according to which the most suitable mode is selected from several modes.

It should be noted that changing the distance between the observer and the surface of the AM3D display can affect not only the size of the stereo pair's viewing area but also other 3D image parameters. Of particular interest is the change in the dimensions of the parts of the observed 3D image depending on the distance between the observer and the screen surface of the AM3D display, or the change in the viewing angle. For example, if the observer moves closer to the display, the size of the observed 3D object may increase.

The zoom operation, already familiar from computer graphics, can be used for the observed 3D objects. This allows the approach and distance of the 3D objects displayed on the AM3D display to be simulated by enlarging or reducing the observed objects in a scene. This is a very important addition to the capabilities of the AM3D display presented here. Very often, when viewing the image on the 3D display, the observer instinctively tends to move closer to the display to examine the 3D image more closely. However, when approaching the surface of a conventional 3D display (ASD), the observer is disappointed to find that the details or the size of the image do not change, which leads to discomfort and weakens the sense of reality when perceiving the displayed 3D images.

Additionally, the strength of the 3D out-screen and in-screen effects can change as the viewer moves closer to or further away from the AM3D display. Even a small change in these effects can enhance the perception of a 3D image when the distance between the observer and the screen surface of the AM3D display changes.

( 4(a , b)) shows the change in the 3D image parameters depending on the distance to the viewer. As in ( 4(b) ) shown, the size of the displayed object and the outscreen effect decrease with decreasing distance 5 from the screen surface of the AM3D display to the viewer compared to the in ( 4(a) ) used distance to.

Using the above-mentioned methods to display a 3D image with our process makes it possible to increase the perception of reality of the 3D images observed on our AM3D display.

1 (a, b, c, d)) show different demonstration modes of the stereo pairs. On ( 1 (b, c, d)) demonstrates in detail modes and their combinations that arise due to shifts and the generation of stereo pairs of the fracturing-free 3D image, which occur according to the movement of the observer. On ( 1(a) ) an autostereoscopic mode is shown for comparison when the 3D image is not moving.

2 (a, b, c, d) demonstrates the order of the sizes of the stereo pair observation areas under different demonstration modes of the stereo pairs.

3 shows a schematic of an AM3D system.

4(a , b) shows the change in 3D image parameters depending on the distance to the viewer.

Reference symbol list

1

2D display

2

Parallax barrier

3

computer

4

Detector for determining the observer position

5

Distance from the viewer to the parallax barrier

6

AM3D Display

7

The line along which the viewer moves

8

The line showing the sizes of the observation ranges of stereo pairs seen by the observer moving to the left along line 7.

9

The line showing the sizes of the observation areas of stereo pairs seen by the observer moving to the right along line 7.

10

Line showing the stereo pair observed on the 2D display when the observer is at point x = x ₀ at time t = t ₀

11

Viewer

12

Direction of displacement of the 3D image when the observer moves from point x ₀ to point x ₁ or x ₂ .

13

System for the horizontal movement of the parallax barrier and/or the 2D display

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions.

Cited patent literature

• TW 200950501A [0007, 0008, 0010, 0011, 0035, 0036, 0054, 0055, 0061]
• US 2011109623A1 [0055]

Claims (10)

A method for displaying a 3D image on an active 3D multiview display (AM3D display) is presented, comprising: a 2D display; 3D images displayed on the 2D display, having a periodic structure comprising N > 2 views; an ordered and directional optical structure corresponding to the 3D image structure and arranged parallel to the surface of the 2D display, wherein the ordered and directional optical structure enables the viewing of the N > 2 views from different positions of an observer relative to the surface of the AM3D display without any additional optical devices; - and a position detector system by means of which the eye position of the observer in three-dimensional space relative to the display surface is determined, characterized in that at least one of the objects of the AM3D display, such as an ordered and directional optical structure, 2D display, a 3D image, can be moved relative to the observer and according to the displacement of the observer, wherein 3D image parameters can vary depending on the distance between the observer and the AM3D display. Procedure according to Claim 1 , characterized in that in the proposed methods for visualizing 3D images a parallax barrier, lenticular lenses, prisms, spherical lenses and optical holographic elements can be used as an ordered and directional optical structure of an AM3D display. The procedure according Claim 1 or 2 is characterized in that by moving at least one of the objects of the AM3D display, such as ordered and directional optical structure, 2D displays or the 3D image, the size of the observation area and the viewing time of the views can be increased in a controlled manner, while slowing down the approach of the observer to the 3D image break area. The procedure according Claim 1 or 2 is characterized in that each time the observer approaches the breakpoint due to the number increase (number decrease) of the observed views, the displacement of at least one of the objects of the AM3D display, such as ordered and directional optical structure, 2D displays or the 3D image, leads to a decrease (increase) in the number of the observed view to a view number that lies between two adjacent breakpoints of the 3D image, which allows the moving observer to see the increasing (decreasing) numbers of views again. The method according to one of the preceding claims is characterized in that by shifting at least one of the objects of the AM3D display, such as the 2D display, the ordered and directional optical structure, and a viewing part/entire 3D image, by a certain distance depending on the position of the observer, the additional time is gained to satisfy the criterion L _{_gen}> v _{_gen}τ_{_gen}, where τ _{_gen} is the time interval that the AM3D system (AM3D display with detectors) needs to generate a new stereoscopic view instead of the pseudoscopic view, wherein L _{_gen} is the distance from the current position of the viewer to the 3D image break area in which the m + 1 pseudoscopic view would be observed if, in its place, during the shift from the previous view N + m, the new view N + m + 1 had not been generated, where m is an integer, and v_{_gen} is the average speed of movement of the viewer on the interval L _{_gen} . The method according to one of the preceding claims is characterized in that, when the 3D image is shifted, the value of the coefficient matrix $M_{i,j}^n$ The contribution of each of the observing views n to the value of each of the subpixels with coordinates i and j on the surface of the 2D display at different times t can, in the general case, be described as $M_{i,j}^n=M_{i,j}^n\left(\overrightarrow{r_s}\left(t\right)\right)$ be represented, where the two-dimensional vector $\overrightarrow{r_s}\left(t\right)$ denotes the position of the 3D image on the 2D display plane at time t. The procedure according to one of the Claims 1 until 5 , characterized in that the time dependence of the size of the matrix element P _i,j , which determines which part of the subpixel area with coordinates i and j is seen by an observer moving relative to the AM3D display, can be represented as: $P_{i,j}=P_{i,j}\left({\overrightarrow{r}}_0\left(t\right),{\overrightarrow{r}}_{2D}\left(t\right),{\overrightarrow{n}}_{2D}\left(\alpha \left(t\right),\beta \left(t\right),\gamma \left(t\right)\right),{\overrightarrow{r}}_{\text{str}}\left(t\right)\right),$ where the three-dimensional vector $\overrightarrow{r_0}\left(t\right)$ The position of the viewer relative to the AM3D display surface in three dimensions at the moment of time t is denoted by the three-dimensional vector. ${\overrightarrow{r}}_{2D}\left(t\right)$ denotes the position of the surface center of a 2D display in space at the moment of time t, three-component unit vector ${\overrightarrow{n}}_{2D}\left(\alpha \left(t\right),\beta \left(t\right),\gamma \left(t\right)\right)$ perpendicular to the 2D display surface, beginning of which through the end of the $\overrightarrow{r_{2D}}\left(t\right)$ The direction is determined by the Euler angles α(t), β(t), γ(t), and the position of the ordered and directional optical structure is determined by a two-dimensional vector. ${\overrightarrow{r}}_{\text{str}}\left(t\right)$ determined, wherein the beginning of this vector is defined by the intersection of the plane of an ordered and directional optical structure with a straight line defined by the vector ${\overrightarrow{n}}_{2D}\left(\alpha \left(t\right),\beta \left(t\right),\gamma \left(t\right)\right)$ The path is determined, and the end of this vector is determined by the position of the center of the surface of the ordered and directional optical structure, and in all movements it is necessary that the distance between the 2D display planes and the plane of the ordered and directional optical structure of the AM3D display is maintained at every time t. The method according to one of the preceding claims is characterized in that by shifting at least one of such objects of the AM3D display, such as a 2D display, the ordered and directional optical structure, and the viewing stereo pairs of the 3D images by a certain distance determined by the position of the observer, the spatial dimensions of the viewing area of the stereo pairs can be controlled and adapted to a predetermined shape, provided that, at the time of the observer's approach to the predetermined viewing area, the AM3D system succeeds in displaying the subpixels with predetermined values at certain viewing points of the AM3D display and at certain times. The method according to one of the preceding claims is characterized in that various parameters of the 3D image, such as the size of the details of the displayed image and/or the effects of inscreen and outscreen, can be changed in a controlled manner by changing the distance between the viewer and the AM3D display. The method according to one of the preceding claims is characterized in that the AM3D display presented here can be used by one or more observers simultaneously and each observer has its own detector that detects its position relative to the surface of the AM3D display, and it is necessary that when 3D image stereo pairs are shifted, the numbers of the observing stereo pairs also get closer to each other or match when different observers approach each other.