DE102021002346A1 - Exact overlaying of printed matter with moving images using augmented reality - Google Patents

Abstract

The method presented here can be used in the field of augmented reality in order to overlay printed products (2-dimensional) with moving images using mathematical transformation. The moving images are precisely and permanently adapted to the position of the printed product. In the AR application, these then appear as videos instead of the 2-dimensional objects.

Description

State of the art

Augmented Reality (AR) is used in more and more areas and generally describes the overlaying of real content with artificially generated non-real content. A device is always required to use AR, e.g. a smartphone, tablet or AR glasses. It can be expected that the type and characteristics of AR-enabled devices will increase significantly in the future. The non-real content is typically overlaid on top of the real world objects. This creates an enriched perspective, which offers many advantages and possible uses, e.g. for applications in construction, in service, or in computer games. The central tasks for this are the recognition of the real object, the calculation of the non-real content and the enrichment of the real object with the non-real content. Manufacturers of AR devices already offer APIs for this, with which the corresponding software can be developed. Using these APIs, the procedure described here can generally be implemented for all AR devices.

problem definition and solution

In AR applications, the real environment is used as a basis. In order to enable interaction with the user, an anchor, i.e. a stored pattern, is typically required to which the AR application then reacts. Such anchors can also be 2-dimensional printed products (e.g. photos, posters, business cards, flyers, etc.). Additional information about such anchors should be displayed in AR applications.

The method presented here uses these anchors to play a moving image (= video) in their place at exactly the same position and spatial perspective. This turns the 2-dimensional print product into a video in the AR application. The calculation of perspective, rotation, inclination and distance, so that the video can be displayed at the exact position of the anchor, is carried out using object recognition, i.e. the anchor is constantly re-detected in the AR application and the size and orientation of the video is adjusted accordingly.

There is a high degree of interaction between the user and an application. Any 2-dimensional print products can be overlaid with moving images in AR applications.

1. 1. Identifizieren des Ankers innerhalb der AR-Anwendung
2. 2. Finden des zugehörigen Videos
3. 3. Berechnen der räumlichen Position und Perspektive des Ankers
4. 4. Skalieren und Transformieren (Drehen, Neigen, Verzerren) des Videos
5. 5. Abspielen des Videos an der exakten Position und der exakten Größe des Ankers

The procedure is as follows:

1. 1. Identifying the anchor within the AR application
2. 2. Finding the related video
3. 3. Calculate the spatial position and perspective of the anchor
4. 4. Scale and transform (rotate, skew, distort) the video
5. 5. Playing the video at the exact position and size of the anchor

For steps 1 and 2, a content management system is typically used in practical applications, in which the corresponding anchors and their assignment to videos are stored. In principle, it is also possible here to store several videos for an anchor and then select which of the videos should be displayed during playback.

Further refinement

For AR applications, the appropriate anchors must be defined. This is typically done by storing a reference image in the application. These reference images are dynamically integrated into the application and e.g. loaded from a server with a content management system. The same applies to the videos to be played. The resulting application examples are listed below.

Since the field of "Augmented Reality" will continue to develop and change in the future, further development in various directions is conceivable. The method described here can be used whenever a video is superimposed on a 2-dimensional printed product.

• - Plakatwerbung: werden Plakate als Anker verwendet, kann die AR-Anwendung dieses Plakat für Hintergrundinformationen in Video-Form verwenden
• - Fotobücher, Foto-Geschenke (z.B. Tassen, Kissen, Mousepads, Leinwanddrucke): die Sammlung von privaten Fotos kann über die AR-Anwendung mit Videos überlagert werden
• - Visitenkarten: wird eine Visitenkarte als Anker hinterlegt, so kann ein kurzes Video den Herausgeber der Visitenkarte näher beschreiben
• - Bildung: werden z.B. Museums-Exponate als Anker hinterlegt, so können zusätzliche Informationen über den Künstler, den geschichtlichen Hintergrund, etc. als Video abgespielt werden
• - Zeitungen: klassische Zeitungen können die AR-Anwendung nutzen, um Bilder aus Artikeln oder Anzeigen mit Bewegtbild anzureichern
• - Etiketten: 2-dimensionale Druckerzeugnisse können auch auf Objekte geklebt oder als Verpackung von Dingen verwendet werden. Dies können mithilfe des hier vorgestellten Verfahrens ebenfalls mit Videos überlagert werden

A number of possible application examples are described below.

• - Billboard advertising: if billboards are used as anchors, the AR application can use this billboard for background information in video form
• - Photo books, photo gifts (e.g. mugs, pillows, mouse pads, canvas prints): the collection of private photos can be overlaid with videos via the AR application
• - Business cards: if a business card is stored as an anchor, a short video can describe the publisher of the business card in more detail
• - Education: if, for example, museum exhibits are stored as anchors, additional information about the artist, the historical chen background, etc. can be played as a video
• - Newspapers: classic newspapers can use the AR application to enrich images from articles or advertisements with moving images
• - Labels: 2-dimensional print products can also be stuck on objects or used as packaging for things. These can also be overlaid with videos using the method presented here

Claims (1)

It is a process for connecting moving images (= video) and print products using Augmented Reality (AR), in which videos are played on AR-capable devices instead of the print products, with the video being played at the exact position of the print product, too if its position, distance, perspective angle or condition is changed. Central to this process is the permanent, exact positioning of the moving image over the real position of the printed product, even if this changes.

Images