DE102015119753A1 - Augmented reality system - Google Patents

Abstract

The invention relates to an augmented reality (AR) system (10) comprising at least one mobile terminal (12) having the following features: - an image acquisition device (26), - a display (14) for reproducing a video sequence, - an AR control ( 40), which AR system (10) further comprises at least one information carrier (16) on which at least one AR marker (18) is mapped, which AR controller (40) comprises an analysis unit (42) provided with a comparison memory (44) are stored in the comparison pattern, wherein the analysis unit (42) is designed, a in the detection range of the image capture device (26) raten on the information carrier (16) reproduced AR marker (18) due to the comparison with the comparison patterns and to issue a trigger signal to a playback unit (46) which causes the playback of at least one AR sequence (30) correlated to the detected AR marker (18), the AR flag r (18) has a plurality of marker elements (20, 22, 24) and the analysis unit (42) is designed, based on the orientation of the marker elements (20, 22, 24) in the detection range of the image capture device (26), the orientation of the mobile terminal (12) relative to the information carrier (16) to determine and generate corresponding orientation data, wherein the display unit (46) performs the representation of the AR sequence (30) in dependence on the orientation data. The display unit (46) changes the representation of at least part of the AR sequence (30) on the display (14) in dependence on the orientation data in such a way that it deviates from a representation of the AR sequence that is orientation-correlated with respect to the orientation data.

Description

The present invention relates to an Augmented Reality System (AR system), comprising at least one mobile terminal with an image capture device, with a display for reproducing a video sequence and an AR control. Furthermore, the AR system contains at least one information carrier on which at least one AR marker for triggering the reproduction of an AR sequence is arranged in the mobile terminal.

The mobile terminal has a controller with an analysis unit, which analysis unit is connected to a reference memory in which comparison patterns are stored. The image capture device is designed to recognize the AR marker displayed on the information carrier on the basis of the comparison with the comparison pattern, whereupon the analysis unit issues a trigger signal to a reproduction unit, which causes the reproduction unit to reproduce at least one AR sequence correlated to the AR marker. The AR marker has a plurality of marker elements and the analysis unit of the mobile terminal is designed to determine the orientation of the mobile terminal relative to the information carrier based on the orientation of the marker elements in the detection area of the image capture device and to generate orientation data therefrom. The reproducing unit then performs the reproduction of the AR sequence in accordance with the orientation data.

Such an AR system is in the US 2015/0248785 A1 shown. There, an AR marker in the form of an abstracted icon can be viewed from different sides, whereby a house is displayed three-dimensionally on the mobile terminal triggered by the icon, corresponding to the viewing angle of the mobile terminal relative to the icon located on the information carrier. This means that the orientation of the AR sequence on the display always corresponds to the orientation of the terminal relative to the information carrier. Such an AR system may work comparatively well for three-dimensional objects to be viewed from all sides.

However, especially in the storybook area, sometimes a pseudo 3D environment is created by the AR sequence, e.g. by 2D pop-ups, i. the environment is represented as a horizontal base plane and from this base plane different elements are presented on different levels arranged one after the other extending from the base plane upwards. This technique is known in particular for pop-up greeting cards, where with the unfolding of the map different pixels on parallel to each other and successively arranged levels are shown.

For such a 2D pop-up 3D system, the system shown in the U.S. patent application is not usable because the mere side view of such a system would only show the gossamer edges of the 2D pop-up planes.

It is therefore an object of the invention to provide an AR system which can also be realized in an AR environment containing a pseudo 3D environment, e.g. 2D PopUp elements.

The object is achieved by an AR system with the features of claim 1 and by a method according to claim 15. Advantageous developments of the invention are the subject of the associated dependent claims.

According to the invention, the reproduction unit additionally modifies the representation of at least part of the AR sequence on the display in dependence on the orientation data such that it deviates from a representation of the AR sequence that is orientation-correlated with respect to the orientation data.

In other words, the representation, in particular the translational position, the scaling and / or orientation, of the AR sequence on the display of the terminal differs from the orientation of the terminal to the AR marker (corresponds to the orientation data). This has the advantage that for any orientation of the terminal relative to the AR marker different display variants of the AR sequence and / or additional animations can be displayed, which on the one hand as vivid as possible representation of the AR sequence in 2D PopUp elements and, second, allows additional orientation-dependent animations that are not reflective of a pure orientation correlation of the display of the AR sequence in the display relative to the orientation data. If, for example, an AR marker is viewed directly from the side of the terminal, ie its image capture device, then the AR sequence can be displayed rotated by 45 degrees on the display, eg touchscreen, so that the pop-up levels are visible from diagonally forward are what would no longer be the case with a precise side view. Additionally, portions of the AR sequence, eg, animated elements, may additionally be rotated throughout the AR sequence relative to its AR environment. Preferably, at certain orientations, additional animations may be enabled in the AR sequence that is specifically suited for the current location of the terminal relative to the information carrier, such as the movement of elements in the AR sequence. This technique thus allows an in Depending on the orientation data different orientation of elements of the AR sequence relative to each other. This is particularly advantageous in the case of 2D pop-up elements, because they can always be advantageously presented for a three-dimensional impression independently of the actual orientation of the terminal relative to the AR marker. In addition, animations can be released in certain positions of the terminal relative to the information carrier in this way, for example, animations that are activated by moving the touch screen of the terminal.

Preferably, for an AR marker, several display variants are provided for at least part of the AR sequence, e.g. stored. The representation variant of the AR sequence is then selected as a function of the orientation data. In this way, for difficult viewing angles, e.g. directly from above, directly from the side and directly from behind predefined display variants of the AR sequence are reproduced, which allow a realistic or pseudo 3D representation.

The analysis unit is designed to store the orientation data over a temporal range, e.g. 05, to 2 s, and to calculate therefrom dynamic data, which reflect, as it were, the movement of the terminal relative to the information carrier. Depending on the dynamic data, at least parts of the AR sequence can then be animated. This makes it possible to incorporate the dynamics of the orientation of the terminal in the representation of the AR sequence. Thus, e.g. in picture books animals or persons are moved or otherwise animated depending on the movement of the terminal.

Thus, animated elements of the AR sequence may e.g. in a view from the front of the horizontal plane to move approximately vertically upwards. When viewed from above, the direction of movement and orientation of the animated element will then be e.g. tilted back, e.g. 45 degrees to the horizontal, so that the movement of the animated element is visible even when viewed directly from above in a - of course strongly inclined - front view. A 2D PopUp element as well as its direction of movement can thus by the invention at all viewing angles, i. Orientations of the terminal relative to the information carrier or AR marker can be displayed optimally.

Preferably, the presentation variants are stored in a memory connected to the playback unit, which allows a quick representation of the variants without any computational effort. Alternatively, it is possible to calculate each representation variant from base data of the AR sequence. Although the image build-up takes longer, which should not really be noticeable in the powerful processors of modern devices. In any case, this solution saves storage space.

In an advantageous development of the invention, a permissible region of the orientation data is defined for each representation variant of an AR sequence, which leads to the reproduction of the corresponding representation variant. Thus, e.g. be provided for the front view, a first display variant, and further display variants for the view from above, for the two side views and the view from behind. The representation of the AR sequence can thus be optimized for each viewing angle range, which is especially true for pseudo 3D representations, e.g. 2D PopUp elements is beneficial.

Preferably, the allowable range for reproduction of a display variant has a horizontal angle range (about a vertical axis) of 60 to 180 degrees and / or a vertical angle range (about a horizontal axis) of 90 degrees. The different permissible ranges for different representation variants of the AR sequence are best defined by angle ranges of + -30 to 90 degrees about the relevant preferred directions of the information carrier, which preferential directions preferably on the one hand in the horizontal plane the front view, the rear view and the two side views and the other is the view from vertically above.

The AR sequence preferably comprises at least one element animated in an AR environment, which is always displayed independently of the orientation data in a maximum angular deviation of +/- 60 °, in particular of at most +/- 45 ° from the frontal view. The angular deviation is then determined from the orientation data. This has the advantage in 2D pop-up elements that regardless of the arrangement or orientation of the mobile terminal relative to the information carrier or AR marker, the animated element is always to see largely frontal (with a maximum angular deviation of 60 °). The problem with 2D PopUp elements is that they are perceived as pseudo 3D only over a defined angular range in the overall environment. If such a 2D PopUp element is viewed too much from the side, it only appears as the side view of a plane, ie as a stroke. Therefore, the representation of an element in the AR system according to the invention is controlled so that it has an independent of the orientation data proportion, namely always in the allowed frontal area including the allowable angular deviation remains. Even if thus the mobile terminal relative to the information carrier to the side by 75 ° or 80 ° or even 90 ° is pivoted (pivoting about the vertical axis), the maximum allowed deviation in the representation of the animated element is reproduced, and thus always a (tilted or rotated) frontal representation. Thus, the animated element appears to be slightly inclined or rotated relative to the information carrier according to the current position of the terminal, but which tilt / rotation is reduced, for example, so that regardless of the orientation of the mobile terminal relative to the information carrier, eg picture book, the animated element essentially always according to the orientation data is seen more or less head-on.

In a very simple embodiment, e.g. the current orientation of the mobile terminal relative to the information carrier are detected and the representation are chosen such that the corresponding orientation of the animated element is inclined only by half the angle. If e.g. the mobile terminal is rotated by 80 ° laterally relative to the front of the information carrier, so the animated element can be displayed with a lateral rotation of 40 °. In any case, this is so much a reduction in orientation that the animated element is essentially perceived head-on even in this rather extreme side view.

In this way, the AR system according to the invention can be optimally used for 2D pop-up 3D environments.

In an optional embodiment of the invention, the reproduction of the AR sequence in the display is rotated relative to the orientation of the terminal to the information carrier so that the viewing angle in the display deviates from the orientation of the terminal relative to the information carrier or AR marker. If e.g. the image of a picture book is viewed exactly from the side or even from behind, the scene can always be viewed from the front within the allowable angle deviation independently. The AR sequence is thus started and displayed correctly from the front, even if the terminal is pointed at the image from the rear, i. as if the book were read over head.

Based on marker elements of the AR marker in the information carrier, e.g. In the image of a picture book, a first computing unit of the analysis unit calculates the exact orientation of the mobile terminal relative to the image of the picture book (or AR marker) both translationally and rotationally preferably in all spatial directions and takes this data into account in the representation of the AR sequence or the at least one animated element of the AR sequence, preferably the entire AR environment representation.

Preferably, the orientation angle of the representation of the animated element in the environmental representation of the AR sequence correlates to a corresponding orientation angle in the orientation data. This leads to a realistic representation of the element in the AR sequence. If e.g. the mobile terminal sees the image from the right side, the animated element is also seen from the right side in the surrounding representation of the AR sequence, although the orientation angle may also be reduced.

In an advantageous development of the invention, the information carrier is a picture book which has pictures in which AR markers for triggering the sequence are arranged, preferably integrated. The use of a 2D PopUp 3D space in conjunction with picture books is easy for children to understand and looks very appealing.

The analysis unit preferably contains a first computing unit which calculates the orientation data based on the arrangement of the marker elements of the AR marker in the detection area of the image capture device. Essential for the reproduction of the AR sequence is the tilting of the terminal about a horizontal axis (eg between viewing from the front and viewing from above) and / or around a vertical axis (between viewing from the right and viewing from the left), because these are the viewing angles are with whom a picture is usually viewed. Of course, these common viewing angles are also commonly used in the orientation of the terminal relative to the image.

The first computing unit preferably calculates the representation of an animated element in the environment representation of the AR sequence on the basis of the orientation data calculated by the analysis unit. Optionally, it can also calculate the environmental representation of the AR sequence, so that the environment is always displayed from the front, but at a permissible viewing angle from the side and from above. The relative orientation of the animated element in its AR environment representation may differ from the orientation of the AR environment representation and be calculated in dependence on the orientation data.

The calculation of the position of the terminal relative to the image (AR marker) preferably proceeds as follows: The arithmetic unit evaluates the arrangement of the marker elements in the plane of the detection area. In this case, for example, the rotational arrangement of the marker elements in the detection area is recorded, ie how much the marker elements are rotated relative to a reference axis, as a rule the front view of an image, and also the mutual Distance of the marker elements for determining a tilt angle of the mobile terminal relative to the information carrier about a horizontal axis. By tilting about a horizontal axis, the mutual distance of the marker elements changes. Due to this change in the mutual distance, the tilt angle and the tilt direction can be closed. The more the mobile terminal is tilted laterally or forward, the smaller the mutual distance of the marker elements in the corresponding tilting direction. The mutual distance of the marker elements is greatest when the information carrier, such as the image of a picture book is detected directly from above.

The invention also relates to a method for displaying an AR sequence on a display of a mobile terminal, in which method the orientation of the mobile terminal relative to an information carrier in the form of orientation data is detected, and in which the orientation data in the reproduction of the AR sequence be considered on the display. The reproduction of at least a part of the AR sequence on the display then takes place as a function of the orientation data such that at least a part of the AR sequence is displayed on the display in dependence on the orientation data changed from one with respect to the Orientation data orientation-correlated representation of the AR sequence deviates. Example: The AR sequence is a pop-up car, which appears in the middle of the AR marker at a viewing angle β1 of 45 °. Now, if the device is tilted to a viewing angle β2 of 80 °, then e.g. not only tilting the car (thus keeping it well visible) but also changing its position on the AR marker, e.g. it drove to the right due to the tipping process. If the car is tilted back to 45 °, the car returns to the center position. In this way, while the animated element is playing, the viewing position of the mobile device in the animated element's representation in the environment representation of the mobile device is displayed. In addition, the position and rotation and size of the AR sequences in relation to the AR target can be animated. Further properties and advantages of the method are described in connection with the description of the system according to the invention. In particular, the animated element is a 2D PopUp element in a 2D PopUp 3D environment, typically characterized by a horizontal base plane and 2D plane protruding 2D elements arranged one behind the other.

Preferably, the orientation data is spread over a temporal range, e.g. 05, to 2 s, and from this dynamic data are calculated, which more or less reflect the movement of the terminal relative to the information carrier. Depending on the dynamic data, at least parts of the AR sequence can then be animated. This makes it possible to incorporate the dynamics of the orientation of the terminal in the representation of the AR sequence. This considerably expands the scope for rendering AR sequences.

It is obvious to a person skilled in the art that the o.g. Embodiments of the invention can be combined with each other as desired. It is further understood by those skilled in the art that multiple AR sequences can be triggered by an AR marker, or that multiple AR sequences can be provided simultaneously, e.g. the AR sequence, on the one hand, show the image in a 2D PopUp 3D spatiality and, in addition, elements in that sequence can be animated by touching the touch screen, e.g. Make sounds and / or move.

Technical components of the AR system, e.g. AR control, analysis unit, playback unit, memory and computing units can be realized as separate components or integrated with each other. Preferably, these components may be implemented in the form of an app on the processor of the terminal.

The orientation is preferably determined with reference to a reference direction G, which preferably represents the horizontal straight line with respect to the information carrier (image). The following expressions are used synonymously: image information carrier AR markers; Control-AR control

The invention will be described below by way of example with reference to the schematic drawing. In this show:

1 the representation of a mobile terminal over an information carrier with an AR marker containing three marker points;

2 the characteristics of the marker points in determining the translational and rotational position of the mobile terminal relative to the information carrier;

3 a representation of an animated 2D PopUp element in a 2D PopUp 3D environment;

4 the schematic representation of relevant for the AR application technical components;

5a -E the schematic representation of pre-set AR environment representations for four different rotational viewing areas in the form of sectors about a vertical axis;

6a -C is a schematic representation of preset AR environment representations for three different tilt viewing areas about a horizontal axis;

7 - 16 the presentation of 2D PopUp 3D AR sequences based on the images of a picture book.

1 shows an AR system 10 comprising a mobile terminal in the form of a tablet 12 which has a touch screen 14 has, as well as lying in the horizontal plane information carrier 16 , eg the picture of a picture book. The information carrier 16 has at least one AR marker 18 consisting for example of three AR marker elements, in particular marker points 20 . 22 . 24 , However, it is also possible to use other conventional marker elements which are preferably integrated in the image of the picture book, for example lines or two-dimensional elements.

At the bottom of the tablet 12 is an image capture device in the form of a camera 26 arranged, their coverage area 28 in the form of a detection cone 28 is shown.

2 shows the detection area 28 the camera 26 out 1 namely, the circular area of the image lying in the detection cone 16 in which the three marker points 20 . 22 . 24 of the AR marker 18 are shown. The XY coordinates of the three marker points 20 . 22 . 24 in the detection area 28 provide information about the translatory position of the tablet 12 in one to the information carrier 16 parallel plane. The mutual distance of the three marker points 20 . 22 . 24 provides information about the distance of the tablet 12 from the picture 16 , The rotational position according to the rotation arrow R indicates to what extent or to what extent the tablet 12 is rotated relative to the image about a vertical axis z. A determination of the tilt ( 1 : gray arrow) around a horizontal axis x, is possible by the mutual distance of the three marker points 20 . 22 . 24 is evaluated to each other, wherein in a tilting the mutual distance decreases in one spatial direction, while it remains constant in another spatial direction. For example, when tilting the tablet 12 around the horizontal X-axis the mutual distance of the three marker points 20 . 22 . 24 unchanged in the X direction while changing in the Y direction. In this way, an arithmetic unit of the analysis unit of the AR control is capable of the actual position of the tablet 12 relative to the picture 18 to determine the picture book.

3 shows the structure of an AR sequence 30 in the form of a 2D PopUp Pseudo 3D representation. Such a 2D PopUp AR sequence 30 usually includes a horizontal base plane 32 , from which there are several 2D representation levels 34 . 36 extend obliquely upward, preferably at an angle of 60 ° -90 ° to the base plane 32 , The 2D PopUp AR sequence 30 also contains an animated element 38 , which is also a 2D representation, at an angle α preferably 50 ° -90 ° relative to the base plane 32 is arranged. The animated element 38 moves in animation in the direction of the arrow M in its presentation plane, as in the example below 7 - 13 is shown (A hippo hops out of the water). At least the animated element 38 becomes independent of the viewing angle of the terminal relative to the information carrier according to the invention 18 always shown in a frontal view, wherein a slight tilt angle β from the front view G about the vertical z-axis and the horizontal x-axis depending on the orientation data, ie the actual orientation of the terminal 12 relative to the information carrier 18 , is allowed. The tilt angle in the illustration depends on the actual position and orientation of the tablet 12 above the picture 16 of the picture book, so that the orientation of the 2D PopUp 3D spatiality of the AR sequence 30 the orientation of the tablet 12 relative to the picture 16 reflects. So, the 2D PopUp 3D environment allows the AR sequence 30 always be reproduced realistically.

4 shows the structure of an AR control 40 that in the tablet 12 is implemented. The AR control 40 has an analysis unit 42 , in turn, with the camera 26 as well as with a comparison memory 44 connected is. In the analysis unit 42 will be the one from the camera 26 recorded coverage area 28 on the appearance of AR markers 18 examined, which by comparing with in the comparison memory 44 stored comparison patterns are determined.

Furthermore, the AR control has 40 a playback unit 46 that with the touch screen 14 ie the display of the tablet 12 connected is. If an AR marker is detected in the above comparison, the analysis unit returns 42 a trigger signal to the playback unit 46 with which this for reproducing at least one associated AR sequence 30 is initiated. In the analysis unit 42 is connected to a first arithmetic unit, which detects the position and mutual arrangement of the marker points when detecting an AR marker 20 . 22 . 24 of the AR marker and from this the current orientation of the tablet 12 relative to the picture 16 calculated in the form of orientation data. The playback unit 46 is linked to a second arithmetic unit, which consists of the orientation data a customized view angle of the animated element 38 in the AR sequence 30 and possibly the total environmental representation of the AR sequence. Thus, in a simple embodiment, the rotation angle of the tablet about the vertical axis z relative to the frontal straight G could simply be halved, leaving the animated element essentially always in (direct or slightly laterally inclined) frontal view with only a slight angular deviation corresponding to the current twist of the tablet 12 relative to the picture 16 ,

The 5b to 5e show an embodiment of the AR sequence 30 out 3 at which depending on the rotational position of the tablet 12 relative to the picture 16 (to the frontal line G) four different environment representations for the playback of the AR sequence 30 in the display 14 be reproduced. 5a shows what orientation angle of the tablet 12 to the picture 18 to the different environment representations of the 5b to 5e to lead.

In 5a is the front of the picture 18 through a front point 50 marked, while the back by two adjacent rear points 52 is marked. These points are used to indicate the orientation of the environment in the display 14 of the terminal 12 in the four different environment representations of the 5b to 5e to mark. In 5a is also the orientation of the animated element 38 which is shown in the AR sequence 30 is animated. 5a defines four different angular ranges α1 to α4 (about a vertical axis) in which the terminal 12 (that is, its camera 26 ) relative to the picture 16 is oriented. For each of the angular ranges α1 to α4 is an environmental representation according to the 5b to 5e defines which one for the 2D PopUp sequence 30 provides a customized representation that best expresses the spatiality of the 2D PopUp representation, taking into account the respective angular range α1 to α4. The angular range α1 (from the front) is the environmental representation 54 of the 5b assigned. The angle range α2 (from the right) is the environmental representation 56 of the 5c assigned. The angle range α3 (from left) is the environmental representation 58 of the 5d and the angle range α4 (from the back) is the environment representation 60 of the 5e assigned.

In 5b is the representation of the AR sequence reproduced by the playback unit in the display 14 of the tablet 12 is displayed when the tablet 12 in the angular range α1 relative to the image 16 is oriented. While the viewing angle α1 is about 120 degrees, depending on the accurate viewing angle (of the orientation data), the vicinity of the AR sequence is displayed only in an angle range Δu which is about 25 degrees. On the other hand, the angular range of the rotation Δe of the animated element is stronger depending on the orientation data. This angular range Δe of the animated element 38 is about 90 degrees centered around the frontal line G. As a result, the environment is slightly rotated depending on the viewing angle according to the angle range Δu, but remains largely frontal. On the other hand, the animated element 38 rotated according to the angular range .DELTA.e stronger, so that when playing in the display 14 more turned towards the viewer seems.

The 5c and 5d show the two predefined environment representations 56 . 58 according to the angles α2 and α3 5a , As it is based on the points 50 . 52 is apparent in these environmental representations 56 . 58 the basic orientation of the environment 32 . 34 rotated according to the angle, but by far not as strong as the actual orientation of the terminal 12 relative to the picture 16 according to the angles α2 and α3. The angular ranges Δu and Δe again show how strong the environment or the animated element 38 is rotated in response to the actual viewing angle within the corresponding viewing angle α2 and α3, that is, in response to the orientation data.

5e shows the presentation 60 in the event that the picture from the camera 26 of the tablet 12 from behind (viewing angle α4), ie viewed overhead. In this case, the entire AR environment is rotated about 180 degrees, leaving the AR sequence 30 as in the environment representation 54 from the front ( 5b ) is reproduced. It thus becomes independent of the rotational position of the tablet 12 relative to the picture 18 always an appealing 2D PopUp environment 30 reproduced, which reflects a realistic pseudo 3D reality.

The 6a to 6c still show two environment representations 60 . 62 according to the 6b and 6c , which are defined for two different tilt angle ranges β1 and β2 of the tablet in which the tablet 12 around a horizontal axis x, y opposite the picture 18 is tilted. Here we differentiated between the front angle range of about 0 to 50 degrees to the horizontal and the supervision range of 50 degrees to 90 degrees. In the range of 0 to 50 degrees, the AR sequence becomes 30 largely in front view, like in the environment view 60 of the 6b is shown. The inclination angles γ1 and γ2 with which the 2D PopUp levels of the animated element 38 and the background 34 relative to the base plane 32 extend preferably depending on the viewing angle β between 75 degrees and 90 degrees, therefore protrude from the base level 32 in about vertical upward. In the bird's-eye view according to the viewing angle β2, according to FIG 6c the inclination angles γ1 and γ2 are greatly reduced, for example to values between 30 degrees and 75 degrees, depending on the actual viewing angle (orientation data). In this bird's-eye view you do not look at the upper edges of the 2D PopUp elements 34 . 38 but by changing the inclination angle in this viewing area β2, these elements also appear in a tilted front view. Of course, preferably the default environment representations 54 . 56 . 58 of the 5b to 5e for the rotation angle with the environment representations 60 . 62 of the 6b and 6c is combined or integrated for the tilt angle about a horizontal axis, whereby the environment representation is optimized for both the rotational orientation and the tilt position.

The from the current viewing angle (orientation of the tablet 12 to the picture 16 ) different representation of the AR scene 30 in the display 14 thus allows an appealing pseudo 3D representation of an AR scene, even if it contains 2D PopUp elements.

The 7 to 10 show an AR sequence 30a that starts when the tablet 12 in a spatial relationship to the picture book 16a is held. The picture book 16a shows a picture with a scene where a hippopotamus jumps out of the water. According to 7 becomes the tablet 12 held right frontally in front of the picture book, reflecting the appearance of the on the touch screen 14 of the tablet 12 represented AR sequence 30a arranged from the front. Good to see in this scene is the animated hippopotamus 38a in the form of a 2D PopUp representation that jumps from a body of water that forms the base level of the 2D PopUp environment. In the background is an inclined to the base level further 2D PopUp level 34a represented with the background forming plant pictures. In this orientation of the tablet 12 relative to the picture book 16a can the animated element (hippopotamus) 38a be activated in such a way that it touches the touch screen 14 jumps up, or again in the water 32a disappears, so that it only looks out with his head. The viewing angle β is very low here at about 10 °, but the angle γ between the hippopotamus and the ground plane is very large at about 80 °, the hippopotamus protrudes almost vertically out of the ground plane.

In 8th This is very easy to recognize because the tablet 12 relative to the picture book is pivoted by about 80 degrees to the side. The hippopotamus 38a is almost perpendicular to the base plane.

In 9 and 10 Now you can see the same scene almost from a bird's-eye view, the viewing angle β is very large here at about 80 °. In the well-known AR representation, the hippopotamus would now be almost only recognizable as a narrow line, since the 2D image is viewed almost from above. By means of the invention described here, however, the angle of inclination γ of the hippopotamus can also be adjusted, for example, when the viewing angle β is changed, as in the example in this case to approximately 45 °. And thus, the hippopotamus is still clearly visible even from a bird's-eye view (in this example, the inclination angle γ of the hippopotamus was changed by 35 °, ie by half the change of the viewing angle β by 70 °).

The 11 and 12 show another AR sequence from another image 16b , When the tablet is tilted relative to the image, the animated polar bear catches 38a to swing, according to the tilt of the tablet 12 (red rotary arrow). The orientation or dynamics of the orientation (movement or rotation of the tablet) can thus be used to start animations of the AR sequence.

13 and 14 show another AR sequence 30c Polar bear climbing on a tree. In the illustrated orientation of the tablet 12 relative to the picture book 16c For example, a chameleon sitting on a hippopotamus can be animated such that it touches the touch screen 14 appears on a branch of the tree shown. This animation also depends on the orientation data.

Finally show 15 and 16 a fourth AR sequence of the little polar bear floating on a barrel through the waves. Moving the tablet to the right relative to the image will cause the polar bear on the barrel to move to the right in the water and vice versa. It can thus also the dynamics of the orientation of the tablet relative to the picture book for the animation of elements of the AR scene 30d be used.

It should be summarized that the present invention makes the reproduction of AR sequences appear very realistic in particular in pseudo 3D representations, preferably in 2D pop-up 3D representations, and allows orientation-dependent additional animations.

The invention is not limited to the illustrated embodiment, but may be varied as desired within the scope of the following claims.

LIST OF REFERENCE NUMBERS

10

AR (augmented reality) system

12

Terminal (tablet)

14

Display (touch screen)

16

Information carrier (picture)

18

AR marker

20

first AR marker point

22

second AR marker point

24

third AR marker point

26

Image capture device (camera)

28

Image capture area

30

AR sequence

32

base level

34

rear 2D pop-up display layer

36

middle 2D PopUp presentation plane

38

animated 2D-Popup element (hippopotamus)

40

AR control

42

analysis unit

44

Compare memory

46

Playback unit

50

front point of the AR environment

52

rear points of the AR environment

54

first environment representation for front and back

56

second environment representation for right

58

third environment representation for links

60

fourth environment representation for flat or frontal

62

fifth environment representation for steep or supervision

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• US 2015/0248785 A1 [0003]

Claims (20)

AR system ( 10 ) comprising at least one mobile terminal ( 12 ) with the following features - an image capture device ( 26 ), - a display ( 14 ) for playback of a video sequence, - an AR control ( 40 ), which AR system ( 10 ) at least one information carrier ( 16 ), on which at least one AR marker ( 18 ), which AR control ( 40 ) an analysis unit ( 42 ) with a reference memory ( 44 ) are stored in the comparison pattern, wherein the analysis unit ( 42 ) is placed in the detection area of the image capture device ( 26 ) on the information carrier ( 16 ) AR markers ( 18 ) due to the comparison with the comparison patterns and a trigger signal to a playback unit ( 46 ) rendering at least one of the detected AR markers ( 18 ) correlated AR sequence ( 30 ), the AR marker ( 18 ) several marker elements ( 20 . 22 . 24 ) and the analysis unit ( 42 ), based on the orientation of the marker elements ( 20 . 22 . 24 ) in the detection area of the image capture device ( 26 ) the orientation of the mobile terminal ( 12 ) relative to the information carrier ( 16 ) and to generate corresponding orientation data, wherein the playback unit ( 46 ) the representation of the AR sequence ( 30 ) depending on the orientation data, characterized in that - the playback unit ( 46 ) additionally the representation of at least part of the AR sequence ( 30 ) on the display ( 14 ) is changed depending on the orientation data so as to deviate from an orientation-correlated representation of the AR sequence with respect to the orientation data. AR system ( 10 ) according to claim 1, characterized in that for an AR marker ( 18 ) several display variants ( 54 - 62 ) at least part of the AR sequence ( 30 ) exist, and that the representation variant ( 54 - 62 ) of the AR sequence ( 30 ) depending on the orientation data. AR system ( 10 ) according to claim 2, characterized in that the representation variants ( 54 - 62 ) in one with the playback unit ( 46 ) associated storage or any representation variant ( 54 - 62 ) from base data of the AR sequence ( 30 ) is calculated. AR system ( 10 ) according to claim 2 or 3, characterized in that for each representation variant ( 54 - 62 ) of an AR sequence ( 30 ) a permissible region of the orientation data (α1-4, β1-2) is defined, which is used to reproduce the corresponding representation variant ( 54 - 62 ) leads. AR system ( 10 ) according to claim 4, characterized in that the permissible range for the reproduction of a representation variant ( 54 - 62 ) comprises a horizontal viewing area (α1-4) about a vertical axis of 60 to 180 degrees and / or a vertical viewing area (β1-2) about a horizontal axis of 90 degrees. AR system ( 10 ) according to one of the preceding claims, characterized in that the AR sequence ( 30 ) at least one element animated in an AR environment ( 38 ) and that the animated element ( 38 ) is always represented in a maximum angular deviation (Δu, Δe) of + -60 degrees, more preferably + -45 degrees from a frontal view regardless of the orientation data, and the angular deviation (Δu, Δe) is calculated from the orientation data. AR system ( 10 ) according to one of the preceding claims, characterized in that the AR control is designed to enable animations of the AR sequence as a function of the orientation data. AR system ( 10 ) according to claim 7, characterized in that the AR sequence ( 30 ) a representation of the animated 2D PopUp element ( 38 ) in a 2D PopUp 3D environment representation ( 32 . 34 ) includes. AR system ( 10 ) according to one of the preceding claims, characterized in that the reproduction unit ( 46 ) the orientation of different elements ( 32 . 34 . 38 ) of the AR sequence ( 30 ) relative to each other depending on the orientation data. AR system ( 10 ) according to one of the preceding claims, characterized in that the analysis unit ( 42 ) has a first computing unit based on the arrangement of the marker elements ( 20 . 22 . 24 ) of the AR marker ( 18 ) in the detection area of the image capture device ( 26 ) calculates the orientation data. AR system ( 10 ) according to one of the preceding claims, characterized in that the reproduction unit ( 46 ) has a second arithmetic unit which based on the orientation data, the representation of the AR sequence ( 30 ) or an animated element ( 38 ) in the AR sequence ( 30 ). AR system ( 10 ) According to claim 11, characterized, in that the first computing unit, the translational arrangement of the marker elements ( 20 . 22 . 24 ) in the image capture area ( 28 ) for detecting the position of the mobile terminal ( 12 ) relative to the information carrier ( 16 ) in a plane parallel to the information carrier plane ( 16 ) used. AR system ( 10 ) according to claim 11 or 12, characterized in that the first arithmetic unit, the rotary arrangement of the marker elements ( 20 . 22 . 24 ) in the image capture area ( 28 ) for detecting the rotation of the mobile terminal ( 12 ) relative to a reference side of the information carrier ( 16 ) used. AR system ( 10 ) according to one of claims 11 to 13, characterized in that the first arithmetic unit the mutual distance of the marker elements ( 20 . 22 . 24 ) in the image capture area ( 28 ) for detecting the tilting position of the mobile terminal ( 12 ) relative to the information carrier plane ( 16 ) used. Method of representing an AR sequence ( 30 ) on a display ( 14 ) of a mobile terminal ( 12 ), in which method the orientation of the mobile terminal ( 12 ) relative to an information carrier ( 16 ) in the form of orientation data, and in which the orientation data in the representation of the AR sequence ( 30 ) on the display ( 14 ), characterized in that at least part of the AR sequence ( 30 ) on the display ( 14 ) is represented as being different in dependence on the orientation data such that it deviates from a representation of the AR sequence that is orientation-correlated with respect to the orientation data. A method according to claim 15, carried out using a system according to any one of claims 1 to 14. The method of claim 15 or 16, wherein the AR sequence ( 30 ) at least one animated element ( 38 ) in an AR environment ( 32 . 34 ), which animated element ( 38 ) is displayed as a function of the orientation data in an angular range (Δe) of at most +/- 60 degrees, in particular +/- 45 degrees, deviating from the frontal view in the A-environment. Method according to one of claims 15 to 17, wherein as an animated element ( 38 ) a 2D PopUp element in a 2D PopUp 3D environment ( 32 . 34 ) is pictured. Method according to one of Claims 15 to 18, in which animations of the AR sequence are released as a function of the orientation data. Method according to one of Claims 15 to 19, in which the orientation data are recorded over a time range and the dynamic data are calculated from the orientation data, and in that at least parts of the AR sequence are animated in dependence on the dynamic data.

Images