DE102023203159A1 - Method for determining at least one degree of freedom of a camera, computer program, machine-readable storage medium and electronic control unit or automation arrangement - Google Patents

Abstract

The invention relates to a method for determining at least one degree of freedom of a camera 1 relative to a code arrangement 2 from a camera image of the camera 1, wherein the code arrangement 2 has a dot matrix with a plurality of base symbols 20, wherein a rough position of the base symbols 20 is encoded in the code arrangement 2, wherein a start field 28 is determined with base symbols 20, wherein the start field 28 has at least three base symbols 20, wherein two independent main directions along the dot matrix in the camera image are estimated via the base symbols 20 of the start field 28, wherein the base symbols 20 of the start field form valid base symbols, wherein in a search step, starting from at least one valid base symbol 20, further base symbols 20 are searched for along at least one of the main directions and wherein, if the search is successful, the further base symbols 20 are marked as valid base symbols 20, wherein the search step is carried out several times, wherein on the basis of the valid Basic symbols 20 or a subset thereof, the coarse position of at least one of the basic symbols 20 in the code arrangement and/or the coarse position of the coordinate system of the point grid is determined as two degrees of freedom of the camera relative to the code arrangement 2.

Description

State of the art

The invention relates to a method for determining at least one degree of freedom of a camera relative to a code arrangement in the camera image of the camera according to claim 1. The invention also relates to a computer program, a machine-readable storage medium with the computer program and an electronic control unit or an automation arrangement with the electronic control unit.

In modern manufacturing applications and other applications, it is necessary to detect the position of a moving body relative to its surroundings. There are a variety of sensors for this purpose, such as displacement sensors, coded positions that are read out, etc. All existing sensors have specific advantages for the respective application, which can be based on accuracy, simplicity, cost-effective integration, robustness or other properties.

In the printed publication DE 10 2016 216 221 A1 The applicant's application, which represents the closest prior art, proposes a method for positioning and/or determining the position of objects in a space and/or on a surface, which uses a two-dimensional code arrangement. The two-dimensional code arrangement has basic symbols, the basic symbols being arranged on a surface and the basic symbols on the surface forming a two-dimensional, periodic grid. The position of a camera relative to the code arrangement can be determined on the basis of the code arrangement.

The publication DE 10 2016 216 196 A1 The applicant discloses a sensor system, wherein the sensor system uses the method for positioning and/or position determination of the preceding document.

disclosure of the invention

A method for determining at least one degree of freedom of a camera relative to a code arrangement from a camera image of the camera with the features of claim 1, an electronic control unit or an automation arrangement with the electronic control unit, a computer program and a machine-readable storage medium with the features of the independent claims are proposed. Preferred or advantageous embodiments of the invention emerge from the subclaims, the following description and the attached figures.

The method according to the invention serves to determine at least one degree of freedom of a camera relative to a code arrangement from a camera image of the camera.

The procedure is based on the procedure of the printed document DE 10 2016 216 221 A1 and/or the application of the publication DE 10 2016 216 196 A1 , the disclosure of which is incorporated by reference into the present disclosure.

Definition Camera:

The camera comprises in particular an optical sensor unit (also referred to as an imaging sensor element, camera chip or image sensor) for recording the camera image and an imaging optic. In particular, the camera is designed as a color camera or as a black and white camera. The optical sensor unit is designed to provide a single camera image of a section of the code arrangement and/or a sequence of camera images, for example in the form of a video. The camera has an image sensor as the optical sensor unit. The camera can comprise a lens, wherein the lens is preferably designed as a wide-angle lens.

In particular, the camera implements a vanishing perspective and/or a central perspective. In particular, parallel edges are not shown parallel to the image when viewed at an angle, but optically merge into an imaginary point, the so-called vanishing point. In particular, the camera, especially including the lens, implements a perspective distortion that is desired for the application.

Definition of optical axis 5:

The imaging optics are ideally a rotationally symmetrical optical system, where the axis of symmetry is the optical axis. It is characterized by the fact that a light beam does not experience any deflection along the optical axis when passing through the optics.

Definition of image center 7:

The intersection point of the optical axis 5 with the optical sensor unit 12 is referred to as the image center 7. This can, but does not necessarily have to, coincide with the geometric center of the optical sensor unit 12.

Definition of camera image:

The camera image of the camera is in particular designed as a matrix which has pixels at the matrix points, such as 8-bit grayscale points or color points.

Definition code arrangement:

The code arrangement can be imaged by the camera in particular, whereby the absolute position of a machine or a machine part can be measured in one to six degrees of freedom on the basis of the camera image. In particular, the code arrangement is designed to be detected by the camera by non-contact reading, whereby a multi-dimensional actual position determination can be carried out by the non-contact detection by the camera. The two-dimensional code arrangement can preferably be used in a production and/or testing system in which workpieces and/or testing and/or work equipment must be positioned.

Definition of basic symbol:

The two-dimensional code arrangement comprises basic symbols, the basic symbols being arranged on a surface and the basic symbols on the surface forming a two-dimensional periodic grid, a dot grid. The basic symbols are preferably geometric figures, such as circles, squares, triangles or lines. The basic symbols are particularly preferably designed as circles. The basic symbols are also referred to as dots. They preferably represent digits of a number system. In particular, the two-dimensional code arrangement comprises at least two different basic symbols. In one possible embodiment of the invention, the two-dimensional periodic grid comprises empty spaces at grid locations and/or grid locations that are not occupied by basic symbols, as parcel symbols.

The basic symbols are arranged on the surface, whereby the surface can be a curved or a non-curved surface. For example, the surface is the floor of a production and/or testing facility. The basic symbols are arranged in the surface to form a two-dimensional grid, whereby the area centers of gravity of the basic symbols preferably form grid points in the point grid.

Definition of dot matrix of code arrangement:

The raster points are also referred to below as raster positions and/or grid positions. In particular, the two-dimensional periodic point grid forms a two-dimensional grid.

Definition of parcel:

The area is preferably divided into similar, regularly arranged plots, the plots having, for example, a square, rectangular, triangular or hexagonal basic shape. Similar plots are understood to mean plots of the same size and/or shape in particular. Preferably, each plot comprises n basic symbols. The plots comprise in particular an integer number of basic symbols and in particular an even number of basic symbols. Preferably, the plots comprise more than ten basic symbols, in particular more than twenty basic symbols and in particular more than forty basic symbols. Furthermore, the number of basic symbols in a plot is preferably less than one hundred. The plots formed by the basic symbols can be optically be indicated in the code arrangement, such as by a border, or not be indicated visually and form only a mental and/or logical unit.

Definition of parcel area:

The plots have at least a first and a second plot area. An X plot area comprises the first plot area and a Y plot area comprises the second plot area. In particular, each plot area occupies a contiguous area or several distributed, non-contiguous sub-areas within the plot.

Each parcel area comprises several basic symbols. In particular, the X parcel area and the Y parcel area comprise the same number of basic symbols. In particular, the at least two parcel areas are arranged in the parcel such that they have a p-fold rotational symmetry with respect to the center of the parcel as the pivot point, wherein the p-fold rotational symmetry is, for example, a two-fold, three-fold or four-fold rotational symmetry.

Definition of parcel symbol:

The plots each have at least one plot symbol, which represents a fixed reference point within each plot. The plot symbol enables the reading and/or decoding of the basic symbols in a fixed order. The plot symbols are arranged in particular regularly and/or periodically in the two-dimensional periodic grid of the basic symbols. The plot symbols can be located on or next to the grid points. The plot symbols are each arranged in particular at the same position within a plot, such as in the center of a plot. The plot symbols are, for example, other graphic elements than the basic symbols, such as triangles, hexagons or lines. Alternatively and/or additionally, the plot symbols are represented by omitting one or more basic symbols in a plot. In one possible embodiment, the plot symbol forms the point of symmetry of the p-fold rotational symmetry of the plot. In particular, the reading direction and/or decoding sequence, i.e. the sequence in which the basic symbols must be read within the X-parcel area and/or within the Y-parcel area, is specified. Preferably, the reading direction and/or decoding sequence corresponds to the specification of which basic symbols are to be read and/or decoded one after the other.

Definition of coding:

In the X-parcel area, an X-coordinate value is encoded by the base symbols and a Y-coordinate value is encoded in the Y-parcel area by the base symbols. In particular, the X-coordinate value and the Y-coordinate value are the coordinates of a base symbol in the parcel, wherein the coordinate is specified in a Cartesian coordinate system of the area spanned by the base symbols. Alternatively and/or additionally, the X-coordinate value and the Y-coordinate value can also specify a position within the area as coordinates in another coordinate system, such as an oblique coordinate system, in cylindrical coordinates or spherical coordinates.

Definition of preferred design of the dot matrix:

In a particularly preferred embodiment of the invention, the two-dimensional periodic grid is a rectangular grid, with the plots also being rectangular. In particular, the rectangular grids and the rectangular plots are square grids and/or square plots. Preferably, the distance between the base symbols along a length and width axis of the rectangular grid is the same. For example, for a square grid with square plots, the number of base symbols in the X and Y directions of the two-dimensional periodic grid is the same. In particular, the X plot area and the Y plot area each consist of two spatially separated, rectangular partial areas within a plot, with the partial areas having a longitudinal extension. The longitudinal extension of the partial areas of the X plot area is preferably perpendicular to the longitudinal extension of the partial areas of the Y plot area. Preferably, the area occupied by the X plot area can be converted into the area occupied by the Y plot area by a 90° rotation.

In a particularly preferred embodiment of the invention, the periodic grid has a grid length and a grid width. The grid length extends in the X direction of a Cartesian coordinate coordinate system, the grid width in the Y direction. The coordinate system thus formed assigns a uniquely determined position vector (X, Y) to each point of the coding area.

A number of g consecutive basic symbols forms in particular an overall sequence. The overall sequence is entered into the X-plot areas, in particular in several plots adjacent in the X direction. The basic symbols are entered into the X-plot areas in particular according to their order in the overall sequence, preferably in ascending X direction of the plots, within each plot in a previously determined reading and/or decoding order. The number g is sufficiently large so that the X-plot areas of all plots adjacent in the X direction can be completely filled. In particular, g is greater than fifty, in particular greater than a thousand and especially greater than a million. The contents of the X-plot areas of plots adjacent in the Y direction are identical. It is particularly preferred that a section of t consecutive basic symbols in the overall sequence forms a partial sequence. In particular, each section of t consecutive basic symbols of the overall sequence forms a partial sequence, with the consecutive basic symbols following one another in decoding and/or reading order. In particular, the overall sequence is designed such that each partial sequence of t consecutive basic symbols is contained only once in the overall sequence when read forwards and each partial sequence read backwards is not contained in the overall sequence when read forwards. Preferably, a partial sequence comprises at least five consecutive basic symbols, in particular at least twenty consecutive basic symbols and in particular at least thirty basic symbols. Furthermore, the partial sequence preferably comprises fewer than fifty basic symbols and in particular fewer than thirty basic symbols. In particular, t < g applies.

In a particularly preferred embodiment of the invention, the base symbols are designed to encode digits to a number base b. The number base b is preferably the base of a place value system. Preferably, the number base b = 2, the base of a binary system, where the digits of the binary system include 0 and 1. It is also possible for the number base b = 10 and to form the base of a decimal system, where the decimal system includes the digits 0, 1, 2, 3, 4, 5, 6, 7, 8 and 9. Alternatively, the number base b = 16, the base of a hexadecimal system, where the hexadecimal system includes the digits 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, A, B, C, D, E and F. In particular, the number base b can be selected as desired, where the digits of the number base include the elements 0, 1, ... b - 1. Alternatively and/or additionally, the number base b is the base of an additive number system, such as the Roman numeral system.

In a particularly preferred embodiment of the invention, the number base b = 2 is selected. In particular, the selected number base b = 2 forms a binary system. The binary system and/or the number system with the number base b = 2 comprises two digits, in particular the digit 0 and the digit 1. Preferably, two different base symbols encode the digits 0 and 1 of the binary system. The two different base symbols are formed by two circles, a first circle with a radius R1 and a second circle with a radius R2. In particular, the radius R1 is selected to be smaller than the radius R2. In particular, the ratio of radius R2 to radius R1 is greater than the square root of 2. For example, the radius R2 is selected such that it is smaller than half the grid dimension of the code grid, so that two adjacent circles with R2 in the periodic grid do not touch each other. This design is based on the consideration that, on the one hand, a particularly high information density is achieved, and on the other hand, reliable readability is achieved with standard image processing methods such as segmentation using blob analysis.

In one possible embodiment of the invention, the overall sequence is designed such that a checksum of a partial sequence is less than half of the maximum possible checksum of a partial sequence. In particular, the checksum for any partial sequence selected from the overall sequence is less than half of the maximum possible checksum of a partial sequence. For example, the checksum of any partial sequence with length t for a number system with base b is less than t·(b - 1)/2.

In a particularly preferred embodiment of the invention, the two-dimensional code arrangement comprises an inverted overall sequence. The inverted overall sequence preferably represents an inversion of the overall sequence. The inverted overall sequence is preferably entered into the Y-plot areas of several plots adjacent in the Y direction. The basic symbols are entered into the Y-plot areas in particular according to their order in the inverted overall sequence, preferably in ascending Y direction of the plots, within each plot in the previously determined reading and/or decoding order. The number g is sufficiently large so that the Y-plot areas of all plots adjacent in the Y direction can be completely filled. In particular, the contents of the Y-plot areas of plots adjacent in the X direction are identical.

In a possible embodiment of the invention, the base symbol at the k-th position of the inverted overall sequence is the coded digit m(k), where the digit m(k) with the digit n(k) coded by the base symbol at the k-th position in the overall sequence satisfies the relation: m(k) = b - n(k) - 1. For example, the inverted overall sequence and the overall sequence for the choice of a dual system with the base b = 2 form the relation: m(k) = 1 - n(k).

Definition of starting point:

The section particularly preferably comprises a reference point. The reference point is, for example, the intersection point of the optical axis of the camera with the plane in which the code arrangement lies. The method is preferably designed to determine the position of the camera in relation to the work area on the basis of the image recorded by the sensor unit, such as the X coordinate (rx) and the Y coordinate (ry) of the reference point in a coordinate system which is spanned by the work area and/or the basic symbols and/or the code arrangement.

Initially, the procedure only determines the rough position in the form of the coded X coordinate and the coded Y coordinate as two degrees of freedom. The rough position alone provides a reliable and usable position for the rough positioning of the camera.

By determining the X coordinate and the Y coordinate in the code arrangement as a rough position, the relative position of the camera in relation to the code arrangement can be deduced. For example, if the rough position is determined in the immediate vicinity of the reference point, the position of the camera can be deduced by knowing the course of the optical axis of the camera. A quasi-exact position of the camera is obtained by making the reference point equal to the coded position and knowing the angle of the optical axis to the code arrangement. The simplest case is that the optical axis is aligned perpendicular to the code arrangement.

Definition of degrees of freedom:

• rx: Ortskoordinate in dem Koordinatensystem der Codeanordnung (als Grobposition und/oder Feinposition).
• ry: Ortskoordinate in dem Koordinatensystem der Codeanordnung (als Grobposition und/oder Feinposition).

In a particularly preferred embodiment of the invention, the method is designed to determine the position and/or the location of the work module with respect to the work area in up to six degrees of freedom. The degrees of freedom are in particular:

• rx: Location coordinate in the coordinate system of the code arrangement (as coarse position and/or fine position).
• ry: Location coordinate in the coordinate system of the code arrangement (as coarse position and/or fine position).

• rz: Abstand von dem Referenzpunkt zu der Kamera entlang der optischen Achse.
• phiz: Z-Drehwinkel der Kamera um die optische Achse.
• phix: erster Nickwinkel der optischen Achse zu der Codeanordnung insbesondere in Bezug auf die erste Hauptrichtung.
• phiy: zweiter Nickwinkel der optischen Achse zu der Codeanordnung insbesondere in Bezug auf die zweite Hauptrichtung.

In particular, rx and ry refer to the reference point, in particular the point where the optical axis penetrates the code arrangement.

• rz: Distance from the reference point to the camera along the optical axis.
• phiz: Z-rotation angle of the camera around the optical axis.
• phix: first pitch angle of the optical axis to the code arrangement, particularly with respect to the first main direction.
• phiy: second pitch angle of the optical axis to the code arrangement, particularly with respect to the second main direction.

Definition X, Y coordinate (rx, ry):

For example, the six degrees of freedom of the position and/or attitude of the camera relative to the workspace include the coordinates X and Y with respect to the Cartesian coordinate system spanned by the base symbols, in particular in a coordinate system (X _c , Y _c ) of the code arrangement in the plane of the code arrangement.

Definition distance between reference point and optical center of the camera (rz):

Furthermore, the 6 degrees of freedom include the distance Z or r _z of the camera to the reference point along the optical axis, where the distance e.g. denotes the distance between the reference point and e.g. the optical center of the camera lens.

Definition of angle of rotation:

Furthermore, the three independent rotation angles of the camera are determined in a camera coordinate system. Two of the rotation angles are specifically defined as the intermediate angle between the optical axis and the code arrangement (phix, phiy). The third rotation angle (phiz) describes the rotation of the camera around the optical axis.

Definition of the coordinate system of the code arrangement:

The coordinate system (X _c , Y _c ) of the code arrangement lies in the plane of the code arrangement.

• Das Koordinatensystem (X_I, Y_I) des Bildaufnehmers liegt in der Ebene des Bildaufnehmers oder in dem Kamerabild.

Definition of the coordinate system of the camera's image sensor and/or in the camera image:

• The coordinate system (X _I, Y _I ) of the image sensor lies in the plane of the image sensor or in the camera image.

Definition of camera coordinate system:

The coordinate system (X, Y, Z) of the camera has its origin at the optical center of the camera lens, with the negative Z direction coinciding with the optical axis of the camera.

Generalization:

While the coordinate systems in this case are specified as examples with regard to the origin and orientation, equivalent coordinate systems can also be used.

Definition of procedure:

In some variants, the method is characterized in that the X coordinate value and the Y coordinate value are decoded and/or determined on the basis of the camera image recorded by the camera. In a possible further development of the method, the orientation of the section with respect to three coordinate axes is determined on the basis of the section of the code arrangement recorded in the image. In particular, the orientation of the section recorded in the image with respect to the X axis and the Y axis of the Cartesian coordinate system, which is spanned by the basic symbols, is determined. Alternatively and/or additionally, the coordinates of the position of the reference point in the code arrangement are determined by decoding the section of the code arrangement contained in the image. In a particularly preferred embodiment of the invention, the position of the work module in relation to the work area is determined in up to six degrees of freedom in the method. In particular, the position of the camera in the work area is determined in three Cartesian coordinates X, Y, Z, where X and Y are the coordinates of the coordinate system spanned by the base symbols and Z is the distance of the work module from the work area. The method also determines, for example, the three Euler angles of the camera.

Alternatively and/or in addition, the two-dimensional code arrangement is used in areas outside of automation technology, for example for monitoring movement sequences in nature and the environment, biology and medicine, architecture, consumer electronics and/or sensor technology.

• Die zentrale Aufgabe der Erfindung ist es, ein leistungsfähiges, zuverlässiges und preiswertes Verfahren zur Lokalisierung und Identifikation von einem oder mehreren Objekten in einem Arbeitsraum zur Verfügung zu stellen in bis zu sechs Freiheitsgraden (6D Lokalisierung = simultane Positionserfassung in 6 Bewegungsfreiheitsgraden).

The general advantages of the invention are depending on the variant:

• The central task of the invention is to provide a powerful, reliable and inexpensive method for localizing and identifying one or more objects in a workspace in up to six degrees of freedom (6D localization = simultaneous position detection in 6 degrees of freedom).

The method processes image data provided by an imaging measurement system, for example a camera. A smartphone can also be used, for example, to capture images and determine the position using the method according to the invention, with the method being implemented as an algorithm in an application that is executed by the smartphone's embedded computer or in the cloud.

To determine the position of an object in up to six degrees of freedom, the camera captures a two-dimensional code arrangement that is associated with the object to be located.

• - Bis zu voller 6D Positionsinformation: Translation (X,Y,Z) und Rotation (φ_X,φ_Y,φ_Z)
• - Absolute Positionsinformation (nicht inkrementell), d.h. eine Referenzierungsfahrt wie bei inkrementellen Sensoren entfällt
• - Sehr großer Messbereich:
• - in X, Y „nahezu unendlich“, nur durch die Codeanordnung begrenzt. Beispiele: eindeutig codierbare Fläche bei Verwendung des in [B] beschrieben Codes und Punktraster 2,5mm:
  • bei Parzellengröße 5x5 Dots: 2m x 2m
  • bei Parzellengröße 7x7 Dots: 8km x 8km
  • bei Parzellengröße 9x9 Dots: 550.000km x 550.000km
  • in Z: einzige Dimension mit Begrenzung. Anpassbar an die Applikation über Kameraauflösung, Objektiv und Rastermaß der Codeanordnung.
  • in φ_X, φ_Y: aktuell +/- 60°
  • unendlich, wenn das Objekt auf allen Seiten einen Punktecode trägt (wie in 5a, b)
  • in φ_Z unendlich (0°-360°)
• - Hohe Messrate, beispielsweise 100Hz - 10.000Hz
• - Kurze Latenzzeit (Zeit von der Bildaufnahme bis zur Ausgabe des 6D Positionsmesswertes), z.B. 10ms - 100µs.
• - Hohe Messgenauigkeit in allen sechs Freiheitsgraden:
  • aktuell 1 µm und 0,01 ° Wiederholbarkeit (bei 3 sigma), bei einem Coderastermaß von 2,5mm
• - Maximale Lesesicherheit, auch bei lokalen Störungen im Bild oder Schwankungen der Bildhelligkeit
• - Preiswert realisierbar, sowohl das Sensorsystem als auch die Codeanordnung
• - Zusatzdaten lesbar, zeitgleich mit der Lokalisierung. Z.B. kann ein ID-Code zur Identifikation des Objektes gelesen werden, der in die Codeanordnung integriert ist.
• - Skalierbar hinsichtlich Genauigkeit, Messbereich in Z, Redundanz und Messrate durch Änderung von Objektiv, Auflösung Bildaufnehmer, Rastermaß Codeanordnung, Größe des Lesefeldes, Rechenleistung etc.
• - Einfache Integration in vorhandene Systeme, beispielsweise als Smartphone-App

The method according to the invention offers the following properties depending on the variant:

• - Up to full 6D position information: translation (X,Y,Z) and rotation (φ_X,φ_Y,φ_Z)
• - Absolute position information (not incremental), i.e. a referencing run as with incremental sensors is not required
• - Very large measuring range:
• - in X, Y “almost infinite”, limited only by the code arrangement. Examples: clearly encodeable area when using the code described in [B] and dot grid 2.5mm:
  • for plot size 5x5 dots: 2m x 2m
  • for plot size 7x7 dots: 8km x 8km
  • for plot size 9x9 dots: 550,000km x 550,000km
  • in Z: only dimension with limitation. Adaptable to the application via camera resolution, lens and grid size of the code arrangement.
  • in φ_X, φ_Y: currently +/- 60°
  • infinite if the object has a point code on all sides (as in 5a , b)
  • in φ_Z infinite (0°-360°)
• - High measurement rate, e.g. 100Hz - 10,000Hz
• - Short latency (time from image capture to output of the 6D position measurement value), e.g. 10ms - 100µs.
• - High measurement accuracy in all six degrees of freedom:
  • currently 1 µm and 0.01 ° repeatability (at 3 sigma), with a code pitch of 2.5 mm
• - Maximum reading reliability, even with local disturbances in the image or fluctuations in image brightness
• - Cost-effective, both the sensor system and the code arrangement
• - Additional data can be read at the same time as the localization. For example, an ID code for identifying the object that is integrated into the code arrangement can be read.
• - Scalable in terms of accuracy, measuring range in Z, redundancy and measuring rate by changing the lens, image sensor resolution, grid size of code arrangement, size of the reading field, computing power, etc.
• - Easy integration into existing systems, for example as a smartphone app

• - Präzise Lokalisierung von Achsen oder Läufern in einem Achssystem oder Robotersystem, um eine Echtzeit-Lageregelung in bis zu sechs Dimensionen zu realisieren.
• - Aufbau von absoluten Positioniersystemen und Robotern, die ohne Referenzfahrt auskommen
• - Vereinfachung von Positioniersystemen, da mehrere eindimensional messende Sensoren durch einen mehrdimensional messenden Sensor ersetzt werden können.
• - Visual Servoing, z.B. zur Nachführung eines Robotergreifers auf ein bewegtes Ziel.
• - Monitoring und Tracking von mehreren Objekten im Raum durch zyklische Positionsmessung
• - Aufzeichnung und Analyse von Bewegungsabläufen
• - Schwingungsanalyse von Maschinen in 6 Dimensionen.
• - Bestimmung der relativen Lage von zwei Objekten. Eine Kamera, beispielsweise in einem Hand-held device, erfasst in einem Bild zwei Objekte, die jeweils mit einem Punktecode ausgestattet sind. Das vorgeschlagene Verfahren erlaubt die Lokalisierung beider Objekte in Bezug auf das gemeinsame Kamera-Koordinatensystem. Durch eine vektorielle Differenzbildung kann die relative Lage der beiden Objekte zueinander ermittelt werden. Die Position der Kamera wird dabei rechnerisch eliminiert, d.h. das Verfahren ist in weitgehend unabhängig von der Kameraposition.
• - Bestimmung der relativen Lage von mehr als zwei Objekten, die jeweils mit Kameras und/oder Codeanordnungen ausgerüstet sind.
• - Aufbau eines Lokalisierungsnetzwerkes, z.B. in einer Fertigungshalle, um mehrere stationäre oder bewegte Objekte relativ zueinander oder relativ zu einem Hallen-Koordinatensystem zu lokalisieren. Beispielsweise lokalisieren sich mehrere autonome Fahrzeuge, die jeweils mit einer Kamera ausgerüstet sind, indem sie eine Codeanordnung erfassen, die an der Hallendecke angebracht ist.

This makes it easy to implement a wide range of automation applications that were previously not or not economically feasible, for example

• - Precise localization of axes or sliders in an axis system or robot system to realize real-time position control in up to six dimensions.
• - Construction of absolute positioning systems and robots that do not require reference travel
• - Simplification of positioning systems, since several one-dimensional measuring sensors can be replaced by one multi-dimensional measuring sensor.
• - Visual servoing, e.g. for tracking a robot gripper on a moving target.
• - Monitoring and tracking of multiple objects in space through cyclic position measurement
• - Recording and analysis of movement sequences
• - Vibration analysis of machines in 6 dimensions.
• - Determination of the relative position of two objects. A camera, for example in a hand-held device, captures two objects in one image, each of which is equipped with a point code. The proposed method allows the localization of both objects in relation to the common camera coordinate system. The relative position of the two objects to each other can be determined by forming a vectorial difference. The position of the camera is eliminated computationally, ie the method is largely independent of the camera position.
• - Determining the relative position of more than two objects, each equipped with cameras and/or code arrangements.
• - Setting up a localization network, e.g. in a production hall, to localize several stationary or moving objects relative to each other or relative to a hall coordinate system. For example, several autonomous vehicles, each equipped with a camera, localize themselves by detecting a code arrangement attached to the hall ceiling.

The code arrangement has a particularly flat dot grid with a plurality of basic symbols, wherein the centers of the basic symbols are preferably located at the grid points and wherein a rough position of the basic symbols is encoded in the code arrangement.

As part of the method, a starting field is determined. The starting field has at least three basic symbols, preferably adjacent to one another, which are arranged at an angle to one another. The connecting lines between the basic symbols of the starting field define two independent main directions along the dot grid in the camera image. In the event that the starting field has exactly three basic symbols, these are arranged at an angle to one another, for example. They define a coordinate system of the flat dot grid, with one of the at least three basic symbols forming the origin of the coordinate system. In a preferred development of the invention, the starting field has nine basic symbols, which are arranged squarely and/or rectangularly. In particular, the starting field has an edge length with three basic symbols. The basic symbols of the starting field are valid basic symbols. Valid basic symbols are understood to mean in particular the basic symbols that are either defined as valid within the framework of the process or classified as valid basic symbols based on image features and/or set as valid by finding the basic symbols and successfully processing the basic symbols with a usable processing result. Image structures that are not classified as valid basic symbols are interpreted as image interference and excluded from further processing.

In the method, in a search step, starting from at least one valid basic symbol, further particularly valid basic symbols are searched for along the main directions.

If the search is successful, especially if additional base symbols are found, the additional base symbols are marked as valid base symbols.

It is intended that the search step is carried out several times. In the event that new basic symbols are found as valid basic symbols in a search step, the search step is carried out either from the basic symbol of the start field or from the newly found, valid basic symbols. In this way, the dot grid is gradually supplemented with further valid basic symbols, starting from the start field with valid basic symbols. The search step is repeated until a sufficient number of valid basic symbols have been found, in particular so that decoding can take place.

Based on the valid basic symbols or a selected subset thereof, in particular by decoding the valid basic symbols, the coarse position of at least one of the basic symbols in the code arrangement is determined as two degrees of freedom of the camera relative to the code arrangement.

In particular, the valid basic symbols or a subset thereof are decoded, whereby at least one degree of freedom of the camera relative to the code arrangement is determined from the decoded coarse position and optionally additional coarse orientation of the coordinate system of the point grid in the camera image.

It is a consideration of the invention that the basic symbols in the code arrangement and/or in the camera image can be searched for and found, for example, using surface functions of image processing. However, finding basic symbols, for example using blob analysis or pattern matching, requires frequent access to the pixels of the camera image and thus leads to complex image processing.

By searching exclusively in the main directions starting from the search field, which defines the two main directions of the point grid and thus the point grid, in particular the coordinate system of the point grid, in the camera image, the a priori knowledge about the structure of the point grid can be used so that the search is not area-oriented, but line-oriented along the main directions. In practical terms, it is sufficient to search for another base symbol starting from a valid base symbol in the main direction along a line. It is obvious that by reducing an area search to a line search, the number of pixel accesses and/or the effort for image processing is significantly reduced. The method according to the invention thus enables a very efficient implementation of the method.

In a preferred development of the invention, the respective center point position, in particular the center of gravity position of the basic symbol in the camera image, is determined for the basic symbols. This determines a large number of intersection points in the point grid on the basis of the valid basic symbols.

It is intended that the search step is carried out starting from the center position of the respective valid base symbol, in particular a line-oriented search in a predetermined search direction, in particular along one of the main directions. While finding a base symbol initially only allows an approximate determination of the position of the base symbol in the dot grid and/or camera image, the exact position in the dot grid and/or camera image is determined by detecting the center position and thus making the dot grid more specific. It is advantageous that the line-oriented search step in particular is carried out from the center position of the base symbol, as this prevents neighboring base symbols from being accidentally missed.

In a preferred development of the invention, two connection vectors are determined in the two independent main directions along the point grid in the camera image on the basis of the starting field and/or on the basis of further valid base symbols. By determining the connection vectors, the position of the nearest and/or neighboring base symbol sought can be estimated by extrapolation based on the known valid base symbol within the framework of a linear combination of the connection vectors.

Particularly preferably, the determination of the center position of a base symbol is carried out in such a way that a starting point in the base symbols is moved over several intermediate steps to a center position along the main directions until the position in which the center position is arranged centrally in the base symbol in the main directions is found as the center position. Because the starting position is moved to the center position in such a way that it is always located centrally in the main directions, it is only necessary to search for the boundary of the base symbol in the main directions for each shifting step and then to move the starting position to the center between the boundaries.

In the example of a circular base symbol, the boundary of the base symbol is searched for by linear search in a first main direction of the camera image, starting from an initial position within the base symbol. The boundary of the base symbol is then searched for in the negative first main direction. Averaging the boundaries leads to a more precise position estimate. This step is repeated in the same way for the second main direction, and then again for the first main direction.

In this way, the shift from the initial position to the center position is performed computationally efficiently through a sequence of line evaluations along the main directions.

In a subsequent step, the area of the basic symbols in the camera image is determined, whereby the area forms a datum for the coding of the code arrangement. In particular, the area is used to classify the basic symbol.

The date can be a 0 or a 1 in the binary system, for example, where the base symbols are, for example, circular areas of different sizes. In the event that the respective center position and the extent of the base symbol in the main directions is known from the shifting steps, the area of the base symbol can be easily determined using these values. In this way, the center position in the point grid and then the area of the base symbol are determined very efficiently.

The area of the base symbol in the camera image is strongly influenced by perspective distortion. For example, the area decreases with increasing distance of the camera from the base symbol and with increasing tilt angle of the camera (elliptical distortion). These influences can lead to incorrect classification, for example if similar base symbols with very different distances from the camera appear in a camera image. Therefore, the standardized area of the base symbol is preferably used to classify the base symbol.

To calculate the standardized area, a reference area is determined that is subject to approximately the same perspective distortions as the base symbol. A parallelogram is used as the reference area, which is spanned by the two vectors from the grid position of the base symbol to the grid positions adjacent in the direction of the main axis. The reference area is the value of the cross product of both vectors.

In particular, the normalized area of the base symbol is calculated as the quotient of the area of the base symbol and the reference area.

In a preferred embodiment of the invention, a reading field is defined in the camera image and/or in the flat dot matrix and a corresponding two-dimensional basic symbol matrix, in particular a dot matrix, is formed, with each point in the reading field being assigned an entry in the basic symbol matrix and with the position and the area and/or the date of the basic symbols being entered in the basic symbol matrix. The rough position and optionally additional orientation of the coordinate system of the dot matrix can be decoded from the data in the basic symbol matrix and/or the rough position of at least one basic symbol in the code arrangement can be determined as the two degrees of freedom.

In a preferred development of the invention, at least one starting position is determined in a preliminary step for creating the starting field. The starting position is determined arbitrarily. The starting position is preferably located near or adjacent to the reference point. If the starting position happens to be within a base symbol, the starting position is taken as the initial position for a starting base symbol. If the starting position is outside a base symbol, a neighboring base symbol is searched for as a starting base symbol along search rays starting from the starting position. In the preliminary step, no computationally intensive area-oriented image processing takes place; instead, a base symbol is searched for only along the search rays starting from the starting position. The boundary of the base symbol can be detected, for example, by a contrast change along the search direction in the camera image. It is possible for the method to use at least or exactly 8 or 16 search rays, which start in the starting position and are arranged in regular angular steps distributed over 360°. A minimum radius can be defined for the search beams, from which a further base symbol is searched. Optionally, a maximum search radius can be specified. If the maximum search radius is reached without finding a neighboring base symbol, the start position is discarded and an alternative start position is selected.

In a subsequent step, starting from the position of the starting base symbol, a neighboring base symbol is searched for as the first auxiliary starting base symbol along further search beams. The search for the first auxiliary starting base symbol can be carried out with the same distribution of search beams as previously described.

In a subsequent step, a second auxiliary start base symbol is searched for, starting from the starting base symbol and the first auxiliary start base symbol, in a direction that is angled to the connection (and/or along a first main direction) between the starting base symbol and the first auxiliary start base symbol. However, fewer search beams can be used than before, which are distributed in particular over a smaller angular range, so that the search is accelerated. This takes advantage of the fact that the first three base symbols should be arranged at an angle to one another, so that there is a priori knowledge of the direction in which the second auxiliary start base symbol is being searched for. For example, only one, two or three to six search beams are used, which are distributed around a main search direction that is oriented perpendicular to the connection between the starting base symbol and the first auxiliary start base symbol.

To complete the process, starting from the starting base symbol, all base symbols of the starting base symbol that are immediately adjacent in the main axis direction or diagonal direction of the point grid are searched for based on the connection between the starting base symbol and the first auxiliary starting base symbol and the connection between the starting base symbol and the second auxiliary starting base symbol by linear combination of the connections. In particular, the connection vectors and/or main directions can be derived from the first three base symbols found. These form a two-dimensional coordinate system of the point grid. To improve the process, the center positions can be determined in order to define the connection vectors as precisely as possible. The result is a starting field with 3 x 3 valid base symbols.

In the search step, neighboring base symbols are searched for starting from the starting field and/or from the other valid base symbols that were found in an earlier search step. The method preferentially searches for base symbols at grid points that have at least two valid neighbors in the main directions or in the diagonal directions. All neighbors of a grid point provide independent position estimates through extrapolation, which, when averaged, lead to a more accurate position estimate of the grid point.

The requirement that only raster points and/or those with at least two valid base symbols as neighbors are searched for also promotes a planar expansion of the captured area and avoids linear expansion in the form of lances, dendrites or tips. The planar expansion is more robust and more tolerant of disturbances in the image than the linear expansion.

The basic symbols define a first and an independent second main direction along the point grid. Depending on the viewing angles, the two independent main directions are shown perpendicularly or obliquely to each other in the camera image.

A basic symbol matrix, in particular a two-dimensional one, is determined from the camera image, with each grid point of the code arrangement being assigned an entry in the basic symbol matrix and with the center position of the basic symbols in the camera image being entered in the basic symbol matrix. An entry in the basic symbol matrix thus refers to the position of the basic symbol in the assigned grid point, in particular in a coordinate system of the image sensor and/or in a coordinate system of the camera image.

The dot grid in the camera image is particularly perspectively distorted, so that although the rows and columns of the dot grid are each formed in a straight line, the rows and columns are generally arranged at an oblique angle to one another in the perspective distortion.

In addition, optical distortion may occur, which can optionally be compensated by rectification.

Preferably, a first straight line function in a coordinate system of the camera image is determined from the data of the basic symbol matrix for a first straight line with a first function argument. The first straight line is always parallel to the first main direction of the dot grid, regardless of the first function argument, if the first straight line is mentally transferred to the coordinate system of the code arrangement. By changing the first function argument, the first straight line is shifted parallel in the second main direction in the coordinate system of the code arrangement. Thus, by changing the first function argument, the straight line can be placed, for example, on a row of the dot grid, both in the coordinate system of the code arrangement and in the coordinate system of the camera image.

In particular, this can be implemented as follows:

1. a) Die Mittelpunkte der Basissymbole bilden in der Codeebene ein zweidimensionales Punktraster mit regelmäßig angeordneten geradlinigen Reihen und Spalten
2. b) Die Kamera bildet die Codeebene perspektivisch verzerrt auf dem Bildsensor ab. Im Kamerabild erscheinen die Reihen im Allgemeinen als aufgefächerte Linien, die einen gemeinsamen Fluchtpunkt besitzen. Gleiches gilt auch für die Spalten.
3. c) Durch Verzeichnung des Objektives erscheinen die Reihen und Spalten im Kamerabild als gekrümmte Linien.

Initial situation:

1. a) The centers of the basic symbols form a two-dimensional dot grid in the code plane with regularly arranged straight rows and columns
2. b) The camera images the code plane on the image sensor with a distorted perspective. In the camera image, the rows generally appear as fanned-out lines that have a common vanishing point. The same applies to the columns.
3. c) Due to distortion of the lens, the rows and columns in the camera image appear as curved lines.

1. a) Durch Rektifizierung wird die Verzeichnung des Objektives mathematisch eliminiert, die gekrümmten Linien werden in gerade Linien überführt.
2. b) Durch lineare Interpolation werden Geraden an die Reihen angepasst und mit einem ganzzahligen Index als Funktionsargument entsprechend ihrer Abfolge nummeriert. Sie bilden ein erstes Geradenbündel. Sinngemäß wird mit den Spalten verfahren, sie bilden ein zweites Geradenbündel.
3. c) Jede Gerade wird durch einen Geradenwinkel und einen Achsenabschnittswert beschrieben. Die Geradenwinkel eines Bündels bilden eine Zahlenfolge, die durch quadratische Interpolation angenähert wird. Als Funktionsargument dient der ganzzahlige Geradenindex. Das Polynom besitzt drei Interpolationsparameter. Sinngemäß wird mit dem Achsenabschnittswert verfahren, woraus sich drei weitere Interpolationsparameter ergeben. Jedes Geradenbündel wird also von sechs Interpolationsparametern vollständig und kompakt beschrieben.
4. d) Durch Einsetzen rationaler Funktionswerte in die Interpolationsfunktion können auch Geraden berechnet werden, die zwischen zwei Reihen oder Spalten liegen.

Process steps in the camera coordinate system:

1. a) By rectification, the distortion of the lens is mathematically eliminated, the curved lines are converted into straight lines.
2. b) Linear interpolation is used to fit straight lines to the rows and number them according to their sequence using an integer index as a function argument. They form a first bundle of straight lines. The same procedure is followed for the columns, which form a second bundle of straight lines.
3. c) Each straight line is described by a straight line angle and an intercept value. The straight line angles of a bundle form a sequence of numbers that is approximated by quadratic interpolation. The integer straight line index serves as the function argument. The polynomial has three interpolation parameters. The intercept value is used in the same way, resulting in three further interpolation parameters. Each straight line bundle is therefore completely and compactly described by six interpolation parameters.
4. d) By inserting rational function values into the interpolation function, straight lines that lie between two rows or columns can also be calculated.

Alternatively or additionally, a second straight line function is determined on the basis of the basic symbol matrix in a coordinate system of the camera image for a second straight line with a second function argument. Regardless of the second function argument, the second straight line is always parallel to the second main direction of the dot grid when the second straight line is transferred to the coordinate system of the code arrangement. By changing the second function argument, the second straight line is shifted parallel in the first main direction in the coordinate system of the code arrangement. Thus, by changing the second function argument, the straight line can be placed on a row of the dot grid, for example, both in the coordinate system of the code arrangement and in the coordinate system of the camera image.

It should be noted that the terms row and column are for naming purposes only and do not imply a specific orientation of the dot grid.

The at least one further degree of freedom of the camera is determined based on at least one of the straight line functions. Optionally, the at least one further degree of freedom of the camera can be determined based on both straight line functions.

A further consideration is that the amount of data should be reduced on the way from the camera image to the at least one degree of freedom in order to enable a faster and/or more efficient calculation of the at least one degree of freedom. While the camera image with an image size of 200 x 200 pixels, for example, still has 40,000 values, the basic symbol matrix is already reduced to the position and accordingly only has 225 values for the positions in a section of the camera image with an edge length of the point grid of 15 basic symbols, for example. By deriving the at least one linear function, the amount of data is reduced to the parameters of the linear function.

It has been shown that 6 parameters (optional) plus one datum per linear function are sufficient to transfer the essential information content of the basic symbol matrix or the camera image to the at least one degree of freedom, so that the amount of data in the example is reduced from 225 entries to 12 or 14 entries. This significantly reduces the effort required to calculate the at least one degree of freedom.

A further advantage of the implementation is that the rows and columns of the basic symbol matrix in the code arrangement are arranged parallel to one another and at regular intervals, so that the derivation of the straight line function in the camera image also carries out a type of averaging over the basic symbol matrix, with the straight line functions describing an averaged information of the basic symbol matrix. The straight line functions compress the information while simultaneously improving the information content.

The method according to the invention thus allows a computationally efficient implementation of the method for determining at least one degree of freedom of the camera relative to the code arrangement from the camera image. From an application point of view, the determination can be carried out, for example, on a microcontroller, which can determine the at least one degree of freedom at least 100 times per second. In this way, it is possible to carry out real-time applications with the method, for example in production.

In a preferred development, the first straight line function is determined on the basis of at least two rows, preferably more than two rows, in particular on the basis of all rows of the basic symbol matrix. Alternatively or additionally, the second straight line function is determined on the basis of at least two columns, preferably more than two columns and in particular on the basis of all columns of the basic symbol matrix. This development underlines that the straight line function carries averaged and/or condensed information across multiple rows or columns.

In a preferred embodiment of the invention, a first best-fit line is formed for each row along the first main directions. The first straight line function is formed on the basis of a plurality of the first best-fit lines. Because a first best-fit line is formed by a row, this first best-fit line can be adapted to the course of the row, so that the first best-fit line already forms an averaged and/or condensed information of the underlying row. The first straight line function is formed on the basis of a plurality of the first best-fit lines, with a second averaging or condensing taking place here, so that the first straight line function is formed by a double averaging of the original information.

Alternatively or additionally, a second best fit line is formed for each column along the second main directions. The second straight line function is formed on the basis of a plurality of the second best fit lines. Because a second best fit line is formed through a column, this second best fit line can be adapted to the course of the column, so that the second best fit line already forms an averaged and/or condensed information of the underlying column. The second straight line function is formed on the basis of a plurality of the second best fit lines, whereby a second averaging or condensing takes place here, so that the second straight line function is formed by a double averaging of the original information.

In a preferred implementation, the first function argument is designed as an integer first count value of the rows and/or the second function argument is designed as an integer second count value of the columns. For an integer first count value in the first straight line function, the first straight line thus corresponds to a first best-fit line. In the same way, for an integer second count value as the second function argument in the second straight line function, the second straight line corresponds to one of the columns of the basis symbol matrix. However, the first straight line and/or the second straight line is not exactly the first best-fit line or the second best-fit line, since the straight line function has undergone the second averaging/compression, so that it is a corrected first best-fit line or corrected second best-fit line.

In a preferred implementation, the best-fit lines are described by a straight line angle as the intersection angle and at least one axis intersection point with one or the coordinate system of the image sensor and/or of the camera image. For the best-fit line, the straight line angle and the axis intersection point of at least or exactly one coordinate axis of the coordinate system are sufficient to clearly determine the best-fit line in the coordinate system.

Through this implementation, the positions of the best-fit lines in the point grid of the corresponding row or column are reduced to two values.

In a preferred development, the straight line function is formed by a combination of a straight line angle function of the straight line angle depending on the function argument of the straight line function and an axis intersection function of the axis intersection depending on the function argument of the straight line function. The straight line function is thus also determined by the straight line angle and at least one axis intersection.

It is preferred that the straight line angle function is designed as a second degree polynomial and/or the axis intersection function is designed as a second degree polynomial, wherein the polynomials have the function argument of the respective straight line function as the function argument. This ensures that the The angle of the line and/or the axis intersection can be determined depending on the function argument. By choosing a second degree polynomial, the approximation can be carried out particularly easily, so that the computational efficiency is further increased.

In a preferred embodiment, depending on the straight line angle of the straight line function with the coordinate system, the axis intersection point of the coordinate axis of the coordinate system is selected which leads to a smaller intermediate angle to a perpendicular to the respective coordinate axis. Furthermore, a datum is assigned to the straight line function which encodes the selected coordinate axis. The idea here is that only the axis intersection point of a single coordinate axis is necessary to describe the straight line, but not the axis intersection points with both coordinate axes. In order to achieve the greatest possible information, the axis intersection point is selected whose assigned crossing angle is more perpendicular to the crossed coordinate axis.

A reference point is preferably arranged in the camera image. The reference point can be positioned as desired. In preferred embodiments, as described later, the reference point is formed as an intersection point of the optical axis with the image sensor and/or with the camera image. The reference point is thus predetermined by the camera in the coordinate system of the camera and/or in the camera image.

A first axis intersection function is preferably formed in the coordinate system of the camera image from a first straight line. The first straight line is aligned in the coordinate system of the code arrangement parallel to the first main direction of the dot matrix. The axis intersection function has a first function argument, whereby by changing the first function argument the first straight line in the coordinate system of the code arrangement is shifted parallel in the second main direction. Depending on the first function argument, the first axis intersection function defines a first axis intersection along a first axis of the coordinate system of the camera image, whereby the first axis runs through the reference point. By shifting the first straight line by changing the first function argument in the coordinate system of the code arrangement, the first straight line in the coordinate system of the camera and/or the camera image is thus shifted - distorted in perspective - in such a way that the axis intersection moves along the first axis.

Furthermore, a second axis intersection function in the coordinate system of the camera image is preferably formed by a second straight line. The second straight line is aligned in the coordinate system of the code arrangement parallel to the second main direction of the dot matrix. The axis intersection function has a second function argument, whereby by changing the second function argument the second straight line in the coordinate system of the code arrangement is shifted parallel in the second main direction. Depending on the second function argument, the second axis intersection function defines a second axis intersection along a second axis of the coordinate system of the camera image, whereby the second axis runs through the reference point. By shifting the second straight line by changing the second function argument in the coordinate system of the code arrangement, the second straight line in the coordinate system of the camera and/or the camera image is thus shifted - perspectively distorted - in such a way that the axis intersection moves along the second axis.

Based on the axis intersection functions, the first and second function arguments are determined in such a way that the reference point forms the first and second axis intersection points. Figuratively speaking, the first function argument is varied until the first straight line in the coordinate system of the camera and/or the camera image runs through the reference point and/or the first axis intersection point lies on the reference point. In the same way, the second function argument is varied until the second straight line in the coordinate system of the camera and/or the camera image runs through the reference point and/or the second axis intersection point lies on the reference point.

Subsequently, based on the first and second function arguments and the coarse position, the fine position of the reference point in the coordinate system of the code arrangement in the plane of the code arrangement is determined as a further degree of freedom. In theory, the coarse position of the base symbol in the coordinate system of the code arrangement is first determined and then a shift of the base symbol to the reference point is determined based on the function arguments.

The method has the advantage that the rough position of the base symbol can be determined by decoding the code arrangement. Subsequently, the displacement up to the reference point is calculated on the basis of the axis intersection functions, whereby by using the axis intersection functions the calculation can be carried out with few calculation values and thus computationally efficient. This has the advantage that the method can be carried out in real-time applications even with digital data processing equipment, especially microcontrollers, with low computing power.

In a preferred embodiment of the invention, the first function argument is designed as a count value of the rows and/or the second function argument as a count value of the columns in the dot matrix. Figuratively speaking, the position of the base symbol, whose rough position is known, is shifted to the reference point by whole steps or partial steps in the grid dimension of the dot matrix.

As already discussed, the reference point is particularly preferably designed as an intersection point of the optical axis of the camera with the image sensor. The reference point is thus defined as a constructive position in the coordinate system of the camera and/or the camera image. However, it is not absolutely necessary for the reference point and/or the intersection point to be located exactly in the middle of the camera image and/or on the camera's image sensor. Rather, the position of the reference point can be defined via calibration.

In one possible embodiment of the invention, the axis intersection function is designed as a straight line function for describing the straight line, in particular as described above. In this case, the axis intersection function comprises a complete mathematical description of the straight line depending on the respective function arguments.

Alternatively, a straight line function is used, in particular as described above, which is formed by a combination of a straight line angle function of the straight line angle as the angle of intersection of the straight line with one of the axes of the coordinate system of the camera image depending on the function argument and the axis intersection function. The straight line is thus completely described by an axis intersection and a straight line angle. This division has the advantage that to determine the fine position only the axis intersection function needs to be determined and/or evaluated, without any information about the straight line angle. This design further increases the efficiency of the method.

The fine position can then be determined based on the function arguments and a known grid size of the dot grid. Figuratively speaking, the fine position is determined in such a way that, for example, starting from the position of the base symbol with the known coarse position, a fraction of a grid size must be moved in the first main direction and a fraction of the grid size in the second main direction in order to get to the reference point in the coordinate system of the code arrangement. This representation is particularly computationally efficient.
It is preferably proposed that the first and/or the second function argument of the straight line function is determined in such a way that the first and the second straight line intersect the reference point. For example, the first and the second straight line are shifted by varying the function argument in such a way that they intersect the reference point.

A rotation angle Phiz of the camera around the optical axis is derived as a degree of freedom of the camera relative to the code arrangement based on the first and/or the second straight line that intersect the reference point. The derivation is possible because the straight line functions are defined in the camera image. It is thus possible to determine the rotation angle Phiz as a straight line angle in the coordinate system of the camera and/or the camera image. Theoretical considerations have shown that the straight line angle in the coordinate system of the camera and/or the camera image corresponds to the rotation angle Phiz of the camera around the swivel axis, without having to transform the coordinate systems from the coordinate system of the camera and/or the camera image to the coordinate system of the code arrangement. It is thus possible to determine the rotation angle Phiz in a simple manner based on at least one straight line function. The further development thus shows a way in which the rotation angle Phiz can be determined with the greatest precision and without great computational effort. It should be emphasized that the determination is particularly easy if the intersection point of the optical axis of the camera through the camera image or through the image sensor is chosen as the reference point. With this special constellation, the angle of rotation Phiz can be derived particularly easily. The method thus allows a simple and at the same time highly accurate determination of the angle of rotation Phiz from the camera image, which is then available as a degree of freedom of the camera relative to the code arrangement.

In a preferred embodiment of the invention, the basic symbol matrix is determined from the camera image, wherein the position of the basic symbols in the camera image is entered in the basic symbol matrix. Thus, a location in the base symbol matrix refers to the position of the base symbol in the point grid, in particular in a coordinate system of the image sensor and/or in a coordinate system of the camera image. The first and/or the second straight line function is preferably determined on the basis of the base symbol matrix.

Particularly preferably, a straight line function is used which is formed by a combination of a straight line angle function of the straight line angle as the angle of intersection of the straight line with one of the axes of the coordinate system of the camera image depending on the function argument and an axis intersection function. The axis intersection function defines an axis intersection along an axis of the coordinate system of the camera image depending on the first function argument, the axis running through the reference point. By shifting the straight line by changing the first function argument in the coordinate system of the code arrangement, the straight line is shifted - distorted in perspective - in the coordinate system of the camera and/or the camera image in such a way that the axis intersection moves along the axis. One straight line function or combination of straight line angle function and axis intersection function is assigned to the first main direction and the other straight line function or combination of straight line angle function is assigned to the second main direction. The advantage of the division is that the function argument can be determined via the axis intersection function and subsequently the angle of rotation Phiz can be read out from the straight line angle function based on the determined function argument.

In a preferred development of the invention, a first value for the angle of rotation Phiz is determined from the first straight line function and a second value for the angle of rotation Phiz is determined from the second straight line function, and the angle of rotation Phiz is then determined as the average of the two values. Because the angle of rotation Phiz can be determined independently from both straight line functions, two independent values are obtained, which can then be averaged to determine the angle of rotation Phiz in order to improve the measurement accuracy. Alternatively, a plausibility check can be carried out and one of the two values can be rejected if it is not plausible or not valid.

A reference point or the reference point is arranged in the camera image. The reference point is formed as an intersection point of the optical axis with the image sensor and/or with the camera image. The reference point is thus specified by the camera in the coordinate system of the camera and/or in the camera image.

• Es wird ein oder der Drehwinkel Phiz - auch Z-Drehwinkel genannt - der Kamera um die optische Achse der Kamera bestimmt. Der Drehwinkel phiz kann besonders bevorzugt über die Geradenfunktion und/oder Schnittpunkfunktion und/oder die Achsenschnittpunktfunktion bestimmt werden.

From the camera image, the following values are determined as characteristic values in the reference point in a preferred further development:

• A rotation angle Phiz - also called Z rotation angle - of the camera around the optical axis of the camera is determined. The rotation angle phiz can particularly preferably be determined using the straight line function and/or the intersection point function and/or the axis intersection point function.

Furthermore, at least one or the local grid dimension of the dot grid in the camera image is determined. The local grid dimension indicates the distance between two adjacent straight lines of the dot grid in the camera image. The local grid dimension can particularly preferably be determined via the straight line function and/or the intersection point function and/or the axis intersection point function. The dot grid in the code arrangement is regular and/or designed with a regular grid dimension. By recording the code arrangement with the camera, which leads to the camera image, the grid dimension is distorted so that the grid dimension changes across the camera image. The local grid dimension at the reference point is understood to be the value of the grid dimension at the reference point.

Furthermore, at least a first and a second local angular divergence of the dot grid in the camera image is determined. In principle, the straight lines in the dot grid in the code arrangement are arranged parallel to one another. However, the mapping of the code arrangement onto the camera image distorts the dot grid so that the straight lines each assume a difference angle that is not equal to 0 between two adjacent straight lines. The local angular divergence of the dot grid in the camera image is understood to be the difference angle between two adjacent straight lines of the dot grid at the reference point. This is a first local angular divergence for the first main direction and a second local angular divergence for the second main direction. The first and second local angular divergences can be determined in particular via the straight line function and/or the straight line angle function.

The above-mentioned parameters describe - from a physical point of view - three degrees of freedom of the camera relative to the code arrangement, namely a distance rz between the code arrangement and the Camera at the reference point and/or along the optical axis as well as two pitch angles phix and phy, which describe an intermediate angle of the optical axis and the plane of the code arrangement in the first and second main directions. The above-mentioned characteristic values are therefore sufficient to determine the above-mentioned three degrees of freedom.

A further consideration is that the parameters mentioned can be determined in a simple manner from the camera image. In addition to a "manual" determination of the parameters in the camera image, it is also possible to derive them using digital image processing methods. Knowing the parameters allows conclusions to be drawn about the three degrees of freedom mentioned.

By selecting the above-mentioned parameters, a new method for determining the above-mentioned degrees of freedom is proposed, which is characterized by the use of only a few parameters to determine the degrees of freedom. This makes it possible to design the method in a computationally efficient and highly accurate manner.

In a preferred embodiment of the invention, the above-mentioned characteristic values are used in an imaging model which describes the optical imaging of the code arrangement onto the image sensor, taking into account the position and orientation of the camera relative to the code arrangement.
The model describes the physical and thus analytical relationship between the mentioned parameters as input values and the three mentioned degrees of freedom as output values. This relationship thus leads to the determination of the three degrees of freedom.

In one possible embodiment of the invention, three equations are determined for the three degrees of freedom, which form a system of equations. It should be emphasized that the three equations represent possible representations of the analytical relationships between the characteristic values and the three degrees of freedom; other mathematical or analytical representations are possible. However, the physical relationship can be represented particularly simply and compactly using the three equations and/or the system of equations.

In particular, the equation for determining the first pitch angle phix is a function of the first local angular divergence dalphac0, the distance rz between the code arrangement and the camera and the second pitch angle phiy.

mit c_grid: Rasterabstand des Punktrasters in [m] Die Gleichung zur Bestimmung des zweiten Nickwinkels phiy ist insbesondere eine Funktion von der zweiten lokalen Winkeldivergenz dalphac1, dem Abstand rz zwischen der Codeanordnung und der Kamera sowie dem ersten Nickwinkel phix.In particular, the first pitch angle is determined using the following equation: $\text{tan}{\phi }_X=\frac{-d{\mathrm{<figure-callout\; id="0"\; label="\alpha \; C"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">\alpha </figure-callout>}}_C\cdot r_Z}{c_{Grid}\cdot {\text{<figure-callout id="2" label="cos" filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png" state="{{state}}">cos</figure-callout>}}^2{\phi }_Y}$

with c _grid : grid spacing of the point grid in [m] The equation for determining the second pitch angle phiy is in particular a function of the second local angular divergence dalphac1, the distance rz between the code arrangement and the camera and the first pitch angle phix.

In particular, the second pitch angle is determined using the following equation: $$\text{tan}{\phi }_Y=\frac{-d{\mathrm{<figure-callout\; id="1"\; label="\alpha \; C"\; filenames="DE102023203159A1\_0157.png,DE102023203159A1\_0158.png"\; state="\{\{state\}\}">\alpha </figure-callout>}}_C\cdot r_Z}{c_{Grid}\cdot \text{cos}{\phi }_X}$$

$r_{Z10}=\frac{b\cdot c_{grid}\cdot \text{cos }{\phi }_X}{g_{C1}\cdot \text{cos }{\phi }_Z}$

Alt. 2 rot_status =1 $r_{Z01}=\frac{b\cdot c_{grid}\cdot \text{cos }{\phi }_Y}{g_{C0}\cdot \left(-\text{sin }{\phi }_Z-\text{tan }{\phi }_X\text{sin}{\phi }_Y\text{cos }{\phi }_Z\right)}$

$r_{Z11}=\frac{b\cdot c_{grid}\cdot \text{cos }{\phi }_X}{g_{C1}\cdot \text{sin }{\phi }_Z}$

Mit

• b: Bildweite der Kamera im [m]
• g_C0, g_C1: lokales Rastermaß des Punktrasters im Kamerabild, in [m]

In particular, the equation for the distance rz between the code array and the camera is a function as follows: Alt. 1: rot_status =0 ${\mathrm{<figure-callout\; id="00"\; label="r\; Z"\; filenames="DE102023203159A1\_0158.png"\; state="\{\{state\}\}">r</figure-callout>}}_Z=\frac{b\cdot c_{Grid}\cdot \text{cos}{\phi }_{\mathrm{<figure-callout\; id="0"\; label="Y\; G\; C"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">Y</figure-callout>}}}{G_C}$

${\mathrm{<figure-callout\; id="10"\; label="r\; Z"\; filenames="DE102023203159A1\_0155.png,DE102023203159A1\_0156.png"\; state="\{\{state\}\}">r</figure-callout>}}_Z=\frac{b\cdot c_{Grid}\cdot \text{cos}{\phi }_{\mathrm{<figure-callout\; id="1"\; label="X\; G\; C"\; filenames="DE102023203159A1\_0157.png,DE102023203159A1\_0158.png"\; state="\{\{state\}\}">X</figure-callout>}}}{G_C}$

Alt. 2 rot_status =1 $r_{Z01}=\frac{b\cdot c_{Grid}\cdot \text{cos}{\phi }_{\mathrm{<figure-callout\; id="0"\; label="Y\; G\; C"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">Y</figure-callout>}}}{G_C}$

${\mathrm{<figure-callout\; id="11"\; label="r\; Z"\; filenames="DE102023203159A1\_0155.png,DE102023203159A1\_0156.png"\; state="\{\{state\}\}">r</figure-callout>}}_Z=\frac{b\cdot c_{Grid}\cdot \text{cos}{\phi }_{\mathrm{<figure-callout\; id="1"\; label="X\; G\; C"\; filenames="DE102023203159A1\_0157.png,DE102023203159A1\_0158.png"\; state="\{\{state\}\}">X</figure-callout>}}}{G_C}$

With

• b: Image distance of the camera in [m]
• g _C0 , g _C1 : local grid size of the point grid in the camera image, in [m]

The columns refer to two independent variants and the rows refer to two alternatives depending on the definition and/or convention of the angle of rotation Phiz.

Alternatively or additionally, the distance between the code arrangement and the camera is determined as the average of the variants and/or functions using the following equation: $$r_Z=\frac{r_{Z0k}+r_{Z1\mathrm{<figure-callout\; id="2"\; label="k"\; filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png"\; state="\{\{state\}\}">k</figure-callout>}}}{2}$$

In principle, the system of equations can be solved analytically. In a preferred development of the invention, the system of equations, comprising the three equations, is solved iteratively. Initial values are initially specified and then the degrees of freedom are determined in an optimization routine in iterative steps.

In particular, the method can provide that a camera image is first recorded by the camera, then at least one degree of freedom of the camera is determined relative to the code arrangement, and then, for example, an actuator of the automation arrangement is controlled. For example, the at least one degree of freedom can be output on an optical output device, such as a display. The at least one degree of freedom can be used to control and/or regulate the position of the actuator of the automation arrangement by using it as an ACTUAL value. A robot with the sensor unit of the automation arrangement can, for example, determine its absolute position relative to the code arrangement and output this as ACTUAL information or move to a predefinable further position, with the robot continuing to orient itself on the code arrangement with regard to its ACTUAL position.

A further subject matter of the invention relates to a control unit and/or an automation arrangement with the control unit, wherein the latter is designed to carry out the method as described above. Optionally, the control unit comprises the camera and/or is connected to it for data purposes.

A further subject matter of the invention relates to a computer program which is designed to carry out the method described above when the computer program is executed on a digital data processing device and/or on the control unit.

Another object of the invention relates to a machine-readable storage medium with the computer program.

• 1 ein Flussdiagramm des Gesamtverfahrens mit einem Ausführungsbeispiel des erfindungsgemäßen Verfahrens;
• 2 eine schematische Darstellung des optischen Modells in 3D;
• 3 eine schematische Darstellung des optischen Modells in 2D;
• 4 eine schematische Darstellung des optischen Modells in 2D;
• 5, 6, 7 eine schematische Veranschaulichung der Koordinatensysteme;
• 8 eine schematische Veranschaulichung des Koordinatensystems für das Abbildungsmodell
• 9 das Abbildungsmodell in der 8 mit verkippter Codeebene;
• 10 eine Veranschaulichung der Codeanordnung;
• 11 eine weitere Veranschaulichung der Codeanordnung;
• 12 ein Punktraster einer Codeanordnung mit zwei beispielhaft eingezeichneten Lesefeldern;
• 13 verschiedene Winkellagen des Lesefelds;
• 14 ein Beispiel für ein Lesefeld;
• 15 einen beispielhaften Aufbau einer Parzelle in der Codeanordnung;
• 16 weitere beispielhafte Aufbauten einer Parzelle in der Codeanordnung;
• 17 Flussdiagramm einer Validitätsprüfung;
• 18 Details der Validitätsprüfung;
• 19 beispielhaft ein Kamerabild mit einer identifizierten Startbasis und dem Lesefeld;
• 20 ein Beispiel für eine Basissymbol-Matrix mit eingetragenem Datum der Fläche;
• 21 ein Flussdiagramm zur Bestimmung der Startbasis;
• 22 mehrere vordefinierte Startpositionen im Bildfeld der Kamera;
• 23 Veranschaulichung des Verfahrens zur Bestimmung der Startbasis;
• 24 Veranschaulichung des Verfahrens zur Bestimmung der Startbasis;
• 25 Flussdiagramm der Funktion Center_Pos;
• 26 Veranschaulichung der Funktion Center_Pos;
• 27a,b Veranschaulichung der Funktion Nearest_dot;
• 28 Flussdiagramm vom Ablauf zur Erfassung aller Dots/Basissymbole im Lesefeld;
• 29 a-d Veranschaulichung des Ablaufs zur Erfassung aller Dots/Basissymbole im Lesefeld;
• 30 Veranschaulichung des Erfassens von Basissymbolen in einem verzerrten Raster;
• 31 Veranschaulichung der Rektifizierung;
• 32 Flussdiagramm zur Klassifizierung der Basissymbole;
• 33 Flussdiagramm zur Bestimmung der Grobposition;
• 34a,b Basissymbol-Matrix mit eingetragenem Datum und mit Lesespuren;
• 35 Veranschaulichung der Dekodierung der Grobposition;
• 36 Veranschaulichung der Ausgleichsgeraden für die Geradenfunktion;
• 37 Veranschaulichung des Geradenwinkels;
• 38a-c Veranschaulichung der Ausgleichsgeraden sowie der Geradenfunktion;
• 39 Veranschaulichung der Bestimmung der Feinposition und des Drehwinkels phiz (Z-Drehwinkel);
• 40 Flussdiagramm zur Bestimmung des Kamerabstands und der Nickwinkel.

Further features, advantages and effects of the invention emerge from the following description of preferred embodiments and the attached figures. These show:

• 1 a flow chart of the overall method with an embodiment of the method according to the invention;
• 2 a schematic representation of the optical model in 3D;
• 3 a schematic representation of the optical model in 2D;
• 4 a schematic representation of the optical model in 2D;
• 5 , 6 , 7 a schematic illustration of the coordinate systems;
• 8 a schematic illustration of the coordinate system for the imaging model
• 9 the mapping model in the 8 with tilted code level;
• 10 an illustration of the code arrangement;
• 11 a further illustration of the code arrangement;
• 12 a dot matrix of a code arrangement with two exemplary reading fields drawn in;
• 13 different angular positions of the reading field;
• 14 an example of a reading field;
• 15 an example structure of a plot in the code arrangement;
• 16 further exemplary structures of a plot in the code arrangement;
• 17 Flowchart of a validity test;
• 18 Details of the validity check;
• 19 example, a camera image with an identified starting base and the reading field;
• 20 an example of a base symbol matrix with the area date entered;
• 21 a flow chart for determining the starting base;
• 22 several predefined starting positions in the camera's field of view;
• 23 Illustration of the procedure for determining the starting base;
• 24 Illustration of the procedure for determining the starting base;
• 25 Flowchart of the function Center_Pos;
• 26 Illustration of the function Center_Pos;
• 27a ,b Illustration of the Nearest_dot function;
• 28 Flowchart of the process for capturing all dots/basic symbols in the reading field;
• 29 a -d Illustration of the process for detecting all dots/basic symbols in the reading field;
• 30 Illustration of the detection of basic symbols in a distorted grid;
• 31 Illustration of rectification;
• 32 Flowchart for classifying basic symbols;
• 33 Flowchart for determining the coarse position;
• 34a ,b Basic symbol matrix with entered date and with reading traces;
• 35 Illustration of coarse position decoding;
• 36 Illustration of the best-fit line for the straight line function;
• 37 Illustration of the straight line angle;
• 38a -c Illustration of the best-fit line and the straight line function;
• 39 Illustration of the determination of the fine position and the rotation angle phiz (Z rotation angle);
• 40 Flowchart for determining camera distance and pitch angles.

A method is disclosed as an implementation of an algorithm for precisely determining the absolute 6D position of a camera 1 with respect to a planar code arrangement 2 recorded by the camera 1, wherein the algorithm receives a camera image 3 as input information.

Code arrangement 2 serves simultaneously as an analog and digital scale in two dimensions, X and Y. It consists of symbols arranged in a regular grid. Preferably, it is a binary code with two symbols arranged in a square grid: a small round dot for a digital 0 and a large dot for a 1. The dot type contains the digital information, while the dot center contains the analog information.

The camera 1 captures a section of the code arrangement 2 and transmits the camera image 3 to a computer or any data processing device. The algorithm selects a reading field in the camera image 3 with the minimum size of a code cell (e.g. 7 x 7 dots) and uses this to calculate the position of the camera 1 in relation to the code arrangement 2 in six dimensions. To do this, the method uses both the digitally coded position information and the precisely measured center positions of all dots in the measuring field or reading field. These form a grid that is distorted by the camera perspective and the distortion of the lens.

• Schritt 100: Einlesen des Kamerabildes 3 in den Arbeitsspeicher des Rechners.
• Schritt 200: optional: Erste Prüfung der Bildqualität anhand von Kenndaten wie Helligkeit und Kontrast.
• Schritt 300: XY-Grobpositionsauswertung und optional z-Grobwinkelauswertung
• Schritt 310: Suche eines Startfeldes (launch pad) von 3x3 Dots im Lesefeld
• Schritt 320: Erfassung der Dots in einem Lesefeld im Kamerabild
• Schritt 330: Präzise Messung der Mittelpunktsposition und Fläche der Dots
• Schritt 340: Optional: Mathematische Korrektur der Objektivverzeichnung durch Rektifizierung der Dot-Positionen. Dadurch werden die im Kamerabild 3 gekrümmten Linien des Dot-Rasters in gerade Linien überführt.
• Schritt 350: Flächenbasierte Klassifizierung der Dots, Zuordnung zu den binären Ziffern 0 und 1
• Schritt 360: Lesen des digitalen Codes in den Achsrichtungen X und Y, Bestimmung der absoluten Grobposition in X-, Y- und optional φ_Z-Richtung als z-Grobwinkelauswertung.
• Schritt 400: Verzerrungsauswertung
• Schritt 410: Anpassung einer Ausgleichsgeraden (beam) an jede Reihe und jede Spalte des Dot-Rasters des Lesefelds
• Schritt 420: Anpassung einer Ausgleichsfunktion an ein erstes Geradenbündel (bunch) durch Interpolation aller Geraden in den Reihen und einer zweiten Ausgleichsfunktion durch Interpolation aller Geraden in den Spalten des Code-Rasters. Jedes Geradenbündel wird durch 6 Interpolationsparameter (bunch-Daten) vollständig beschrieben. Mit Hilfe der Interpolationsfunktion können daraus auch interpolierte Geraden berechnet werden, die zwischen zwei gemessenen Geraden liegen.

The position detection procedure includes the following steps (see 1 ):

• Step 100: Reading camera image 3 into the computer’s memory.
• Step 200: optional: First check of image quality using parameters such as brightness and contrast.
• Step 300: XY coarse position evaluation and optional z coarse angle evaluation
• Step 310: Find a launch pad of 3x3 dots in the reading field
• Step 320: Detection of dots in a reading field in the camera image
• Step 330: Precise measurement of the center position and area of the dots
• Step 340: Optional: Mathematical correction of the lens distortion by rectifying the dot positions. This converts the curved lines of the dot grid in camera image 3 into straight lines.
• Step 350: Area-based classification of the dots, assignment to the binary digits 0 and 1
• Step 360: Reading the digital code in the X and Y axis directions, determining the absolute coarse position in the X, Y and optionally φ _Z directions as a z-coarse angle evaluation.
• Step 400: Distortion evaluation
• Step 410: Fitting a best fit line (beam) to each row and column of the dot grid of the reading field
• Step 420: Adaptation of a compensation function to a first bunch of straight lines by interpolating all straight lines in the rows and a second compensation function by interpolating all straight lines in the columns of the code grid. Each bunch of straight lines is completely described by 6 interpolation parameters (bunch data). Using the interpolation function, interpolated straight lines can also be calculated that lie between two measured straight lines.

• Schritt 500: XY-Feinpositionsauswertung:
  • Die Position in X- und Y-Richtung wird aus der XY-Grobposition und der Lage des Rasters in Bezug auf die Lage der optischen Achse (Bildmittelpunkt) berechnet.
• Schritt 600: Z-Winkelauswertung aus dem Winkel der (interpolierten) Geraden im Bildmittelpunkt
• Schritt 700: Z-Positions- und XY-Winkelauswertung
• Schritt 710: Herleitung von interpolierten Kennwerten aus den bunch-Daten, die das perspektivisch verzerrte Raster im Bildmittelpunkt beschreiben:
  • - Rastermaß der Geraden beider Geradenbündel
  • - Winkeldivergenz der Geraden beider Geradenbündel

Calculation of the 6D camera position from parameters obtained from the measured dot positions that describe the distorted grid in the camera image. The equations for calculating the position result from an inverse optical imaging model of camera 1 and the laws of geometric optics:

• Step 500: XY fine position evaluation:
  • The position in the X and Y directions is calculated from the XY coarse position and the position of the grid in relation to the position of the optical axis (image center).
• Step 600: Z-angle evaluation from the angle of the (interpolated) straight line in the image center
• Step 700: Z position and XY angle evaluation
• Step 710: Derivation of interpolated parameters from the bunch data that describe the perspective-distorted grid in the image center:
  • - Grid dimension of the straight lines of both straight line bundles
  • - Angular divergence of the lines of both bundles of lines

The camera position in Z-direction is mainly determined by the grid size of the straight line in the image center

The camera angles φ _X and φ _Y are mainly determined from the angular divergence of the two straight line bundles in the image center.

An iterative algorithm numerically solves the system of equations relating the quantities Z, φ _X and φ _Y.

Step 800: Output of the 6D position information and the optional validity information resulting from the results of numerous diagnostic functions in the process.

With minimal computing effort, the algorithm concentrates the camera image data on a few pieces of data relevant to position determination, increases accuracy through interpolation and behaves robustly in the event of camera image disturbances. The following sequence positions are of particular importance (numerical values as an example): Expiration position I: Input data camera image (e.g. 200x200 pixels) 40,000 values Process position II (step 300): converts the camera image data into dot data 2,250 values Process position III (step 400/500): converts the dot data into beam data 90 values Process position IV (step 400/500): converts the beam data into bunch data 12 values Process position V (step 500/600/700): converts the bunch data into a 6D position 6 values

1. a) Schnelles Auffinden der Dots im Kamerabild 3 durch mathematische Modellierung des perspektivisch verzerrten Coderasters in der Codeanordnung 2. Das Verfahren stützt sich auf die bereits gemessenen Dot-Positionen. → Vorteile: schnelles Auffinden der Dots, die zeitaufwändige Suche im Pixelraster entfällt. Robust gegenüber Störungen im Kamerabild: Dots werden als „nicht valide“ klassifiziert, falls sie an der erwarteten Position nicht gefunden werden. Nicht valide Dots werden nicht weiterverarbeitet, behindert jedoch im Allgemeinen nicht den weiteren Ablauf
2. b) Stack-basierter Algorithmus zur schnellen und robusten Suche der Dots im Coderaster der Codeanordnung 2. → Vorteil: fehlertolerant, da das Suchverfahren bei fehlerhaften Dots nicht abbricht, sondern diese allseitig umschließt. Präzise (hohe Ausbeute an gefundenen Dots pro Lesefeld) und robust (durch priorisierte Erfassung von Dots mit vielen validen Nachbarn)
3. c) Funktion Center_Pos: schnelles Verfahren zur Subpixel-genauen Positions- und Flächenbestimmung von Dots im Kamerabild → Vorteil: schnell (wenig Pixelzugriffe), genau (durch Subpixeling), robust (Verwendung dynamischer Kontrastschwellen statt Grauwertschwellen)
4. d) Rektifizierung der Dot-Positionen anstatt des gesamten Kamerabildes. → Vorteil: schnell (Rektifizierung von nur 225 Positionen statt aller 40.000 Pixel im Kamerabild)
5. e) Lokale Normierung der Dot-Fläche auf die Fläche der Dot-Rasterzelle. Zur Klassifizierung der Dots wird die normierte Dot-Fläche verwendet. → Vorteil: robustes Lesen des digitalen Codes (Klassifikation der Dots ist tolerant gegen perspektivische Verzerrung und gegen eine Änderung des Abstandes zwischen Kamera und Dot)
6. f) Redundantes Lesen der digitalen Codes, Fehlererkennung und z.T. Fehlerkorrektur → Vorteil: Geringe Fehlerrate bei Störungen im Kamerabild, korrektes Lesen trotz nicht valider Dots.

The drain positions II - V have the following special features. Drain position II:

1. a) Fast location of the dots in the camera image 3 by mathematical modeling of the perspective-distorted code grid in the code arrangement 2. The method is based on the dot positions that have already been measured. → Advantages: Fast location of the dots, no time-consuming search in the pixel grid. Robust against disturbances in the camera image: Dots are classified as "not valid" if they are not found at the expected position. Invalid dots are not processed further, but generally do not hinder the further process
2. b) Stack-based algorithm for fast and robust search of dots in the code grid of code arrangement 2. → Advantage: fault-tolerant, as the search process does not stop at faulty dots, but encloses them on all sides. Precise (high yield of found dots per reading field) and robust (through prioritized detection of dots with many valid neighbors)
3. c) Function Center_Pos: fast method for subpixel-accurate position and area determination of dots in the camera image → Advantage: fast (few pixel accesses), accurate (through subpixeling), robust (use of dynamic contrast thresholds instead of gray value thresholds)
4. d) Rectification of the dot positions instead of the entire camera image. → Advantage: fast (rectification of only 225 positions instead of all 40,000 pixels in the camera image)
5. e) Local normalization of the dot area to the area of the dot raster cell. The normalized dot area is used to classify the dots. → Advantage: robust reading of the digital code (classification of the dots is tolerant of perspective distortion and a change in the distance between the camera and the dot)
6. f) Redundant reading of the digital codes, error detection and partial error correction → Advantage: Low error rate in case of disturbances in the camera image, correct reading despite invalid dots.

• g) Komprimierte und präzise Darstellung der aus dem Kamerabild 3 extrahierten Dot-Positionen als bunch-Parameter (nur 12 Werte) → Vorteil: schnell (da geringe Datenmenge), präzise (durch Mittelung und best-fit Interpolation in zwei Stufen: Dots zu Ausgleichsgeraden und Ausgleichsgeraden zu Geradenbündeln), fehlertolerant und robust (durch Ausschluss nicht valider Dots von der Weiterverarbeitung).

Expiry position III and IV:

• g) Compressed and precise representation of the dot positions extracted from camera image 3 as bunch parameters (only 12 values) → Advantage: fast (due to small amount of data), precise (through averaging and best-fit interpolation in two steps: dots to best-fit lines and best-fit lines to bundles of lines), fault-tolerant and robust (through exclusion of invalid dots from further processing).

• h) Mathematisches Verfahren zur Transformation der 12 bunch-Parameter in eine 6D Position.

Drain position V:

• h) Mathematical procedure for transforming the 12 bunch parameters into a 6D position.

• • Volle 6D Positionsmessung, absolute Positionsinformation
• • Gleichzeitige Erfassung aller Dimensionen mit einem Messvorgang (in einem Kamerabild).
• • Hohe Positions-Messrate und geringe Latenzzeit. Das Verfahren erfordert nur eine geringe Anzahl von Rechenschritten eines Computers. Mit einem embedded Computer werden typische Messraten von etwa 100Hz - 10000Hz erreicht, so dass der Sensor beispielsweise auch in einem geschlossenen Lageregelungskreis eingesetzt werden kann.

The benefits include some or all of the following improvements:

• • Full 6D position measurement, absolute position information
• • Simultaneous recording of all dimensions with one measurement process (in one camera image).
• • High position measurement rate and low latency. The process requires only a small number of calculation steps on a computer. With an embedded computer, typical measurement rates of around 100Hz - 10000Hz are achieved, so that the sensor can also be used in a closed position control loop, for example.

• • Hohe Lesesicherheit durch redundantes Lesen des Punktecodes. Die Positionserfassung ist auch bei Störungen im Bildfeld der Kamera möglich, wie zum Beispiel bei lokalen Verdeckungen, nicht validen Dots oder unter ungünstigen Beleuchtungsverhältnissen. Durch eine Validitätsprüfung der Verarbeitungsschritte werden fehlerhafte Zustände erkannt und die Ausgabe unzuverlässiger Positionswerte verhindert.
• • Hohe Genauigkeit der Positionsmessung in allen Freiheitsgraden. Dies wird durch verschiedene Maßnahmen erreicht wie
  • ◯ Mittelung über zahlreiche Dots pro Kamerabild 3, beispielsweise mehr als 100 oder 1000
  • ◯ Klassifizierung von Dots nach Validität, Ausschluss von nicht validen Dots von der weiteren Verarbeitung
  • ◯ Subpixel-genaue Positionsbestimmung der Kanten der Dots
  • ◯ Verwendung von Kontrastschwellen statt Grauwertschwellen, dadurch hohe Robustheit gegenüber lokalen Änderungen der Bildhelligkeit
  • ◯ Runde Dots, daher kaum Mittelpunktsfehler bei Kippung oder Rotation der Codeebene
• • Unbegrenzter Messbereich in φ_Z
• • Nahezu unbegrenzter Messbereich in X und Y. Beispielsweise wird bei Verwendung eines Codes mit einer Codezelle von 9 x 9 Dots und einem Dot-Raster von 2,5mm ein Messbereich von 550.000km x 550.000km erreicht.
• • Großer Winkel-Messbereich φ_X, φ_Y von etwa +/- 50°, bei hoher Genauigkeit. Erweiterung auf 360° durch räumlich verteilte Codeanordnungen auf dem Objekt.
• • Ein inverses Abbildungsmodell auf Basis der geometrischen Optik wird genutzt, um die Position der Kamera aus den Kamerabilddaten zu errechnen.
• • Das Verfahren kommt ohne eine aktive Beleuchtung aus, so dass Kamerabilder verarbeitet werden können, die unter Umgebungslicht aufgenommen wurden, beispielsweise mit einem Smartphone.
• • Minimale Fehlerrate. Das Verfahren bestimmt die Validität eines Positionsmesswertes, indem es jeden Verarbeitungsschritt mit verschiedenen Diagnosemethoden auf Fehler und Plausibilität überprüft. Die Validität wird zusammen mit dem Positionsmesswert ausgegeben. Dabei wird das Ziel verfolgt, nur valide Messwerte zur weiteren Verarbeitung auszugeben.

The method also achieves a high reading speed by minimizing the number of accesses to camera image points (pixels). No time-consuming area-based camera image operations are carried out. In addition, the amount of data is significantly reduced with each processing step.

• • High reading reliability through redundant reading of the dot code. Position detection is possible even when there are disturbances in the camera's image field, such as local obscurations, invalid dots or under unfavorable lighting conditions. A validity check of the processing steps detects faulty states and prevents the output of unreliable position values.
• • High accuracy of position measurement in all degrees of freedom. This is achieved by various measures such as
  • ◯ Averaging over numerous dots per camera image 3, for example more than 100 or 1000
  • ◯ Classification of dots according to validity, exclusion of invalid dots from further processing
  • ◯ Subpixel-accurate positioning of the edges of the dots
  • ◯ Use of contrast thresholds instead of gray value thresholds, thus high robustness against local changes in image brightness
  • ◯ Round dots, therefore hardly any center errors when tilting or rotating the code plane
• • Unlimited measuring range in φ _Z
• • Almost unlimited measuring range in X and Y. For example, when using a code with a code cell of 9 x 9 dots and a dot grid of 2.5mm, a measuring range of 550,000km x 550,000km is achieved.
• • Large angle measuring range φ _X , φ _Y of approximately +/- 50°, with high accuracy. Extension to 360° through spatially distributed code arrangements on the object.
• • An inverse imaging model based on geometric optics is used to calculate the position of the camera from the camera image data.
• • The process does not require active lighting, so that camera images taken under ambient light, for example with a smartphone, can be processed.
• • Minimum error rate. The procedure determines the validity of a position measurement value by checking each processing step for errors and plausibility using various diagnostic methods. The validity is output together with the position measurement value. The aim is to only output valid measurement values for further processing.

• • Ein einzelner Messaufnehmer, insbesondere ausgebildet als Kamera 1, erfasst die Position eines oder mehrerer Objekte in jeweils sechs Dimensionen. Gegenüber einem System mit vielen verteilten Sensoren zur Erfassung einzelner Freiheitsgrade reduziert sich der Installationsaufwand und die Komplexität des Gesamtsystems, was zu Kostenvorteilen führt.
• • Einsparung von Systemkomponenten durch Erfassung aller 6 Freiheitsgrade. Beispielsweise benötigt ein Drehwinkel-Encoder meist ein Drehlager für die Messachse, damit seitliche Positionsabweichungen der Codescheibe nicht zu Messfehlern führen. Mit dem vorgeschlagenen Verfahren kann die mechanische Führung entfallen, da die volle 6D Lageinformation der Codescheibe als Codeanordnung simultan erfasst wird. Eine seitliche Positionsabweichung des Sensors führt nicht zu einem Fehler in der Winkelmessung. Dadurch werden die Kosten und die mechanische Komplexität des Positionsmesssystems reduziert.

• • Es kann ein Positioniersystem realisiert werden, bei dem die Aufteilung der Positionssensorik auf mehrere Achsen baulich nicht möglich ist. Beispielsweise kann ein levitierender planarer Roboter realisiert werden, der in sechs Dimensionen positionierbar ist, ohne dass eine bauliche Verbindung zwischen dem Roboter und dem darunter liegenden Stator besteht.
• • Durch die simultane Erfassung mehrerer Dimensionen werden Messfehler reduziert, die bei verteilten Sensorsystemen durch unterschiedliche Messzeitpunkte entstehen können.
• • Die Kosten des Sensors sind weitgehend unabhängig von der Anzahl der erfassten Freiheitsgrade. Daher kann er auch vorteilhaft in Anwendungen eingesetzt werden, die weniger als 6 Freiheitsgrade benötigen. Die zusätzlich bereitgestellte Information ermöglicht Zusatzfunktionen, beispielsweise die Eigendiagnose von Systemen oder die permanente Beobachtung des Betriebszustandes („Condition Monitoring“).
• • Das Verfahren ist skalierbar, z.B. durch Variation des Rastermaßes der Codeanordnung und Anpassung der abbildenden Optik an die Codeanordnung. Die Auflösung kann über viele Größenordnungen variieren, beispielsweise vom Nanometer-Bereich (Anwendungsbeispiel: Nanometer-Positioniersystem) bis in den m-Bereich (Anwendungsbeispiel: Automatisiertes Landen eines Flugzeugs oder einer Drohne auf einem Flugplatz, wobei das Flugfeld mit einer Codeanordnung markiert ist).
• • Das Verhältnis von Auflösung zu Messbereich kann viele Größenordnungen umfassen. Wird beispielsweise ein Wegmesssystem mit 10nm Auflösung mit einer 10m langen Codeanordnung kombiniert, beträgt das Verhältnis von Auflösung zu Messbereich 1:10⁹.
• • Ein Sensor mit dem Verfahren besitzt eine hohe Einsatzflexibilität, da er einfach installiert werden kann und durch Parametrierung für spezifische Anwendungen konfiguriert werden kann. Dies ist insbesondere vorteilhaft bei häufig wechselnden Einsatzbedingungen.
• • Das Verfahren kann Zusatzinformationen lesen, die im Positionscode enthalten sind. Beispielsweise können zusätzlich zur Position Objekt-Identifikationsdaten gelesen werden.
• • Bei zyklischer Bildaufnahme liefert das Verfahren für jedes Einzelbild eine unabhängige Positionsschätzung, sie ist nicht auf Vorinformation aus vorherigen Bildern angewiesen. Daher entspricht die Messrate der Bildwiederholrate.

Further advantages are:

• • A single sensor, in particular designed as camera 1, records the position of one or more objects in six dimensions. Compared to a system with many distributed sensors for recording individual degrees of freedom, the installation effort and the complexity of the overall system are reduced, which leads to cost advantages.
• • Saving system components by recording all 6 degrees of freedom. For example, a rotary angle encoder usually requires a rotary bearing for the measuring axis so that lateral position deviations of the code disk do not lead to measurement errors. With the proposed method, the mechanical guide can be omitted because the full 6D position information of the code disk is recorded simultaneously as a code arrangement. A lateral position deviation of the sensor does not lead to an error in the angle measurement. This reduces the costs and mechanical complexity of the position measuring system.

• • A positioning system can be implemented in which the division of the position sensors into several axes is structurally impossible. For example, a levitating planar robot can be implemented that can be positioned in six dimensions without a structural connection between the robot and the stator underneath.
• • The simultaneous recording of several dimensions reduces measurement errors that can arise in distributed sensor systems due to different measurement times.
• • The costs of the sensor are largely independent of the number of degrees of freedom recorded. It can therefore also be used advantageously in applications that require fewer than 6 degrees of freedom. The additional information provided enables additional functions, such as self-diagnosis of systems or permanent observation of the operating status (“condition monitoring”).
• • The method is scalable, e.g. by varying the grid size of the code arrangement and adapting the imaging optics to the code arrangement. The resolution can vary over many orders of magnitude, for example from the nanometer range (application example: nanometer positioning system) to the m range (application example: automated landing of an aircraft or a drone at an airfield, where the airfield is marked with a code arrangement).
• • The ratio of resolution to measuring range can cover many orders of magnitude. For example, if a position measuring system with 10 nm resolution is combined with a 10 m long code arrangement, the ratio of resolution to measuring range is 1:10 ⁹ .
• • A sensor using this method has a high degree of flexibility in use, as it can be easily installed and configured for specific applications through parameterization. This is particularly advantageous when operating conditions change frequently.
• • The method can read additional information contained in the position code. For example, object identification data can be read in addition to the position.
• • With cyclic image acquisition, the method provides an independent position estimate for each individual image; it does not rely on prior information from previous images. Therefore, the measurement rate corresponds to the frame rate.

The method enables the localization of at least one camera 1 in relation to at least one object 8 in six degrees of freedom, with at least one code arrangement being applied to the surface of each object 8. A camera 1 captures the code arrangements 2 on the objects 8 and displays them completely or in sections in the camera image 3. Using the proposed method and other common mathematical/technical methods, the 6D position of the objects 8 is calculated from the camera image 3.
The code arrangement 2 forms a flat, digitally coded scale. It contains several different symbols, preferably circular symbols (dots), which are arranged in a regular, preferably square grid and enable position determination in 6 degrees of freedom.

• • ein bildgebendes Sensorelement, üblicherweise einen Kamera-Chip, insbesondere den Bildaufnehmer 12.
• • Ein Abbildungssystem, welches den Maßstab scharf und kontrastreich auf das Sensorelement abbildet, insbesondere das Objektiv 9. Das Abbildungssystem umfasst insbesondere ein Weitwinkelobjektiv als Objektiv 9, da zur Bestimmung aller sechs Freiheitsgrade eine zentralperspektivische Abbildung erforderlich ist.
• • Eine Schnittstelle zur Ausgabe der Kamerabilddaten und/oder des Kamerabilds 3. Optional umfasst das Kamerasystem der Kamera 1
• • ein Beleuchtungssystem, insbesondere zur Beleuchtung der Codeanordnungen. Dadurch wird die Kamera 1 unabhängiger von den Beleuchtungsbedingungen in der Umgebung. Im Blitzbetrieb werden kurze Messzeiten ermöglicht, so dass auch schnell bewegte Objekte erfasst werden können. Als Leuchtmittel werden beispielsweise Leuchtdioden eingesetzt.
• • Mittel zur Abschirmung von Fremdlicht, z.B. eine Blende oder optische Filter. Es kann ein spektrales Filter im Strahlengang des Abbildungssystems vorhanden sein, damit nur das Licht aus einem begrenzten Wellenlängenbereich das Sensorelement erreicht. Idealerweise werden einfarbige Leuchtmittel mit gleicher Wellenlängencharakteristik wie das Farbfilter eingesetzt.

Camera 1 comprises at least

• • an imaging sensor element, usually a camera chip, in particular the image sensor 12.
• • An imaging system which images the scale sharply and with high contrast onto the sensor element, in particular the lens 9. The imaging system comprises in particular a wide-angle lens as lens 9, since a central perspective image is required to determine all six degrees of freedom.
• • An interface for outputting the camera image data and/or the camera image 3. Optionally, the camera system of camera 1 includes
• • a lighting system, in particular for illuminating the code arrangements. This makes the camera 1 less dependent on the lighting conditions in the environment. In flash mode, short measurement times are possible so that even fast-moving objects can be detected. Light-emitting diodes, for example, are used as lighting sources.
• • Means for shielding from extraneous light, e.g. a diaphragm or optical filter. A spectral filter can be present in the beam path of the imaging system so that only light from a limited wavelength range reaches the sensor element. Ideally, single-color lamps with the same wavelength characteristics as the color filter are used.

In addition, a computer system is required as a digital data processing device for information processing, e.g. designed as a control unit. It receives the digitized camera image data of camera image 3 as input information, and at the output it provides the determined 6D positions of the identified objects and optionally an assessment of the validity of the position measurement values.

The method is implemented as an algorithm in the computer system. The algorithm is executed either on request from outside or cyclically, for example in a fixed time frame to record the movement paths of objects (tracking).

The computer system can be designed as an embedded system, parallel computer, GPU, FPGA, ASIC or cloud system, for example. A smartphone can also be used as the overall system for position detection, whereby the integrated camera 1, possibly with active lighting, is used to capture images. The method can be implemented in a smartphone application and run embedded in the smartphone or in the cloud.

2 shows a typical arrangement for determining the absolute position in 6 degrees of freedom. The camera 1 records the camera image 3 of a planar code arrangement 2. From the perspective-distorted camera image 3, the method according to the invention calculates the 6D position of the camera in the coordinate system of the code arrangement 2 and outputs it as a position vector (r _X , r _Y , r _Z ) and angle vector (φ _X , φ _Y , φ _Z ).

The coordinates r _X and r _Y are indicated in the rectangular coordinate system (X _C , Y _C ) of the code arrangement 2. They indicate the intersection point 4 of the optical axis 5 of the camera 1 with the plane in which the code arrangement lies. Since the optical axis 5 is perpendicular to the plane of the image sensor 12, it can be designated by a point 7 in the camera image 3.

The method determines the intersection point (X, Y) even if the code arrangement 2 is only shown at the edge of the image field and not at the location of the optical axis 5.

The distance r _Z lies on the optical axis 5 and extends from the intersection point 4 to the optical center of the lens of the camera 1. Since the optical axis 5 is not always perpendicular to the code arrangement 2, r _Z generally forms a non-orthogonal coordinate system with (X _C , Y _C ). Using the angle vector, the position vector (r _X , r _Y , r _Z ) can be transformed into a rectangular coordinate system.

The method can be used to locate several simultaneously recorded objects 8 in the camera's field of view, even if they partially overlap. In order to be able to read the position, at least one area the size of a code parcel (e.g. 7x7 dots) must be recognizable in the camera image 3.

3 and 4 show schematically the beam path of camera 1. In the 3 the camera 1 with a lens 9 and an image sensor 12 can be seen, the viewing direction is directed downwards to the code level of the code arrangement 2. The optical axis 5 is shown as a dash-dotted vertical line, the focal points of the lens 9 as points 10,11.

A vector arrow G on the code plane of the code arrangement 2 is imaged as a vector arrow B on the image sensor 12 according to the laws of ray optics. A line of sight 14 intersects the optical axis 5 at a point which is referred to here as the optical center 13 of the objective 9.

mit:

B

Bildgröße

b

Bildweite

G

Gegenstandsgröße

g

Gegenstandsweite

f

Brennweite

In 4 the complete beam path is shown, the objective 9 is simplified to a lens. Ray optics with the ray theorem and the lens equation forms the basis for the mathematical method for determining position. The following applies: $$\begin{array}{cc}\text{B}/\text{G}=\text{b}/\text{G}& \text{and}1/\text{f}=1/\text{b}+1/\text{G}\end{array}$$

with:

B

image size

b

focal length

G

object size

G

object range

f

focal length

5 , 6 , 7 illustrate the coordinate systems involved:

The two-dimensional coordinate system 15 of the image sensor 12 is spanned by the axes (X _I, Y _I ) of the camera chip. A position on the image sensor 12 is specified in image points [pixels], with the pixel rows and columns of the camera chip being numbered consecutively. In this example, an image sensor 12 with 200 x 200 pixels is assumed. The numbering results in an integer X and Y position value for each pixel. Subpixeling, a method for interpolating gray values between neighboring pixels, can also result in real numbers appearing as the camera position during the evaluation. To convert the unit of position from [pixels] to [m], the position is multiplied by the distance between neighboring pixels p _grid on the camera chip in the unit [m/pixel]. The pixel spacing p _grid is a property of the image sensor 12. Each pixel provides an integer gray value.

The coordinate system 16 of the code arrangement 2 (X _C , Y _C ) is spanned by the main axes of the code arrangement 2, which correspond to the square grid of the dots. It is extended by a third axis Z _C to form a three-dimensional coordinate system, which is perpendicular to the code arrangement 2, with the code arrangement 2 arranged at Z _C = 0. The unit is either [dots], i.e. the consecutive numbering of the dot rows and columns, or [m] - after multiplication by the dot grid spacing c _grid in [m/dot]. The code arrangement 2 uniquely encodes the local position in the coordinate system (X _c , Y _C ).

The three-dimensional camera coordinate system (X, Y, Z) 17 is firmly connected to the camera 1. Its origin is on the optical axis 5 between the image sensor 12 and the code arrangement 2, at a distance of twice the image width b from the image sensor 12. The optical axis 5 forms the Z axis, the camera 1 looks in the direction -Z.

The coordinate system for the mapping model in 8 corresponds to the camera coordinate system 17 (X, Y, Z). Here, b is the image distance of the camera 1. The image plane is at the height (0, 0, 2b) and is parallel to the X/Y plane of the camera coordinate system 17. As a model, a further "virtual" image plane 18 can be constructed in the (X, Y) plane of the coordinate system 17, whereby the virtual image is mirrored point-symmetrically with respect to the real image on the image sensor 12 in relation to the image center.

• • in Translation: (r_X, r_Y, r_Z) und
• • in Rotation: (φ_X, φ_Y, φ_Z).

The code plane of the code arrangement 2 is located at the height (0, 0, z ₀ ) with (z ₀ < 0). With the camera orientation (0°, 0°, 0°), the code plane is parallel to the X/Y plane of the camera coordinate system 17 and the axes (X _C , Y _C ) of the coordinate system 16 of the code arrangement 2 point in the same direction as the axes (X _I, Y _I ) of the coordinate system 15 of the image sensor 12.
The procedure determines the 6D position of the camera

• • in translation: (r _X , r _Y , r _Z ) and
• • in rotation: (φ _X , φ _Y , φ _Z ).

The center of rotation to which the tilt and rotation angles phix and phy of camera 1 refer is at the point (0, 0, z ₀ ). The roll angle phiz is measured around the optical axis 5.

9 shows the imaging model with tilted code plane of code arrangement 2. The value r _Z is the distance between the center of rotation (0, 0, z ₀ ) and the optical center (0, 0, b) of the lens, so that: r _Z = b - z ₀ .

• • Optisches Zentrum 13 des Objektivs 9: $$\overrightarrow{F}=\left(\begin{array}{c}0\\ 0\\ b\end{array}\right)$$
  
• • Zweidimensionale Koordinaten von Dots oder Punkten in der Ebene der Codeanordnung in Koordinatensystem 16 der Codeanordnung 2 $${\overrightarrow{P}}_{i,j}^{\prime }={\overrightarrow{P}}_0+\left(\begin{array}{c}i\\ j\end{array}\right)\cdot c_{grid}=\left(\begin{array}{c}{x}_0+i\cdot c_{grid}\\ {\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+j\cdot c_{grid}\end{array}\right)$$
  
  mit ${\overrightarrow{P}}_0=\left(x_0,y_0\right)$
  
  dem Flächenschwerpunkt eines ausgewählten Basissymbols und i, j ganzen Zahlen
• • Räumliche Koordinaten von Dots oder Punkten in dem Kamerakoordinatensystem 17: $${\overrightarrow{P}}_{i,j}=\left[\underset{\_}{R}\left(\begin{array}{c}{x}_0+i\cdot c_{grid}\\ {\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+j\cdot c_{grid}\\ 0\end{array}\right)\right]+\left(\begin{array}{c}0\\ 0\\ z_0\end{array}\right)$$
  
  mit der Rotationsmatrix für Vektoren im dreidimensionalen Raum: $$\begin{array}{l}\underset{\_}{R}=\left(\begin{array}{ccc}{R}_{11}& R_{12}& R_{13}\\ R_{21}& R_{22}& R_{23}\\ R_{31}& R_{32}& R_{33}\end{array}\right)\\ =\left(\begin{array}{ccc}\text{cos }{\phi }_Y\text{cos }{\phi }_Z& \text{sin }{\phi }_X\text{sin }{\phi }_Y\text{cos }{\phi }_Z+\text{cos }{\phi }_X\text{sin }{\phi }_Z& \text{sin }{\phi }_X\text{sin }{\phi }_Z-\text{cos }{\phi }_X\text{sin }{\phi }_Y\text{cos }{\phi }_Z\\ -\text{cos }{\phi }_Y\text{sin }{\phi }_Z& \text{cos }{\phi }_X\text{cos }{\phi }_Z-\text{sin }{\phi }_X\text{sin }{\phi }_Y\text{sin }{\phi }_Z& \text{cos }{\phi }_X\text{sin }{\phi }_Y\text{sin }{\phi }_Z+\text{sin }{\phi }_X\text{cos }{\phi }_Z\\ \text{sin }{\phi }_Y& -\text{sin }{\phi }_X\text{cos }{\phi }_Y& \text{cos }{\phi }_X\text{cos }{\phi }_Y\end{array}\right)\end{array}$$
  
  Bildpunkt in der virtuellen Bildebene im Kamera-Koordinatensystem 17: $$\overrightarrow{B}=\left(\begin{array}{c}{B}_X\\ B_Y\\ 0\end{array}\right)$$
  
• • Sichtstrahl von $\overrightarrow{F}\text{ zu }{\overrightarrow{P}}_{i,j}:$
  
  $$\overrightarrow{G}=\overrightarrow{F}+\lambda \left({\overrightarrow{P}}_{i,j}-\overrightarrow{F}\right);\lambda =\mathrm{0..1}$$
  
• • Schnittpunkt des Sichtstrahls mit der virtuellen Bildebene 18: $$\left(\begin{array}{c}{B}_X\\ B_Y\\ 0\end{array}\right)=\left(\begin{array}{c}0\\ 0\\ b\end{array}\right)+\lambda \left[\underset{\_}{R}\left(\begin{array}{c}{x}_0+i\cdot c_{grid}\\ {\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+i\cdot c_{grid}\\ 0\end{array}\right)+\left(\begin{array}{c}0\\ 0\\ z_0\end{array}\right)-\left(\begin{array}{c}0\\ 0\\ b\end{array}\right)\right]$$
  
  $$\begin{array}{l}0=b+\lambda \left(R_{31}\left(x_0+i\cdot c_{grid}\right)+R_{32}\left({\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+j\cdot c_{grid}\right)+{\mathrm{<figure-callout\; id="0"\; label="z"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">z</figure-callout>}}_0-\text{b}\right)\\ \Rightarrow \lambda =-\text{b}/(R_{31}\left(x_0+i\cdot c_{grid}\right)+R_{32}\left({\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+j\cdot c_{grid}\right)+{\mathrm{<figure-callout\; id="0"\; label="z"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">z</figure-callout>}}_0-\\ \text{b})\end{array}$$
  
• • Virtuelles Bild des Punkts oder Dots: $$\begin{array}{l}\left(\begin{array}{c}{B}_X\\ B_Y\end{array}\right)=\frac{-\text{b}}{\left(R_{31}\left(x_0+i\cdot c_{grid}\right)+R_{32}\left({\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+j\cdot c_{grid}\right)+{\mathrm{<figure-callout\; id="0"\; label="z"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">z</figure-callout>}}_0-\text{b}\right)}\cdot \\ \left(\begin{array}{c}{R}_{11}\left(x_0+i\cdot c_{grid}\right)+R_{12}\left({\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+j\cdot c_{grid}\right)\\ R_{21}\left(x_0+i\cdot c_{grid}\right)+R_{22}\left({\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+j\cdot c_{grid}\right)\end{array}\right)\end{array}$$
  
• • Für das reale Abbild muss das negative Vorzeichen weggelassen werden: Für den Schnittpunkt 4 mit dem Index (i=0; j=0) erhält man die Bildkoordinaten im realen Bild: $$\left(\begin{array}{c}{B}_X\\ B_Y\end{array}\right)=\frac{\text{b}}{\left(R_{31}{x}_0+R_{32}{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+{\mathrm{<figure-callout\; id="0"\; label="z"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">z</figure-callout>}}_0-\text{b}\right)}\cdot \left(\begin{array}{c}{R}_{11}{x}_0+R_{12}{y}_0\\ R_{21}{x}_0+R_{22}{y}_0\end{array}\right)$$
  

Based on the ray theorem and geometric optics, the mapping of a point (x ₀ , y ₀ ) in the code plane to a point (B _X , B _Y ) in the image plane can be mathematically described as a mapping equation in the camera coordinate system 17:

• • Optical center 13 of the lens 9: $$\overrightarrow{F}=\left(\begin{array}{c}0\\ 0\\ b\end{array}\right)$$
  
• • Two-dimensional coordinates of dots or points in the plane of the code arrangement in coordinate system 16 of code arrangement 2 $${\overrightarrow{P}}_{i,j}^{\prime }={\overrightarrow{P}}_0+\left(\begin{array}{c}i\\ j\end{array}\right)\cdot c_{Grid}=\left(\begin{array}{c}{x}_0+i\cdot c_{Grid}\\ {\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+j\cdot c_{Grid}\end{array}\right)$$
  
  with ${\overrightarrow{P}}_0=\left(x_0,y_0\right)$
  
  the centroid of a selected base symbol and i, j integers
• • Spatial coordinates of dots or points in the camera coordinate system 17: $${\overrightarrow{P}}_{i,j}=\left[\underset{\_}{R}\left(\begin{array}{c}{x}_0+i\cdot c_{Grid}\\ {\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+j\cdot c_{Grid}\\ 0\end{array}\right)\right]+\left(\begin{array}{c}0\\ 0\\ z_0\end{array}\right)$$
  
  with the rotation matrix for vectors in three-dimensional space: $$\begin{array}{l}\underset{\_}{R}=\left(\begin{array}{ccc}{R}_{11}& R_{12}& R_{13}\\ R_{21}& R_{22}& R_{23}\\ R_{31}& R_{32}& R_{33}\end{array}\right)\\ =\left(\begin{array}{ccc}\text{cos}{\phi }_Y\text{cos}{\phi }_Z& \text{sin}{\phi }_X\text{sin}{\phi }_Y\text{cos}{\phi }_Z+\text{cos}{\phi }_X\text{sin}{\phi }_Z& \text{sin}{\phi }_X\text{sin}{\phi }_Z-\text{cos}{\phi }_X\text{sin}{\phi }_Y\text{cos}{\phi }_Z\\ -\text{cos}{\phi }_Y\text{sin}{\phi }_Z& \text{cos}{\phi }_X\text{cos}{\phi }_Z-\text{sin}{\phi }_X\text{sin}{\phi }_Y\text{sin}{\phi }_Z& \text{cos}{\phi }_X\text{sin}{\phi }_Y\text{sin}{\phi }_Z+\text{sin}{\phi }_X\text{cos}{\phi }_Z\\ \text{sin}{\phi }_Y& -\text{sin}{\phi }_X\text{cos}{\phi }_Y& \text{cos}{\phi }_X\text{cos}{\phi }_Y\end{array}\right)\end{array}$$
  
  Image point in the virtual image plane in the camera coordinate system 17: $$\overrightarrow{B}=\left(\begin{array}{c}{B}_X\\ B_Y\\ 0\end{array}\right)$$
  
• • Visibility of $\overrightarrow{F}\text{to}{\overrightarrow{P}}_{i,j}:$
  
  $$\overrightarrow{G}=\overrightarrow{F}+\lambda \left({\overrightarrow{P}}_{i,j}-\overrightarrow{F}\right);\lambda =\mathrm{0..1}$$
  
• • Intersection of the visual ray with the virtual image plane 18: $$\left(\begin{array}{c}{B}_X\\ B_Y\\ 0\end{array}\right)=\left(\begin{array}{c}0\\ 0\\ b\end{array}\right)+\lambda \left[\underset{\_}{R}\left(\begin{array}{c}{x}_0+i\cdot c_{Grid}\\ {\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+i\cdot c_{Grid}\\ 0\end{array}\right)+\left(\begin{array}{c}0\\ 0\\ z_0\end{array}\right)-\left(\begin{array}{c}0\\ 0\\ b\end{array}\right)\right]$$
  
  $$\begin{array}{l}0=b+\lambda \left(R_{31}\left(x_0+i\cdot c_{Grid}\right)+R_{32}\left({\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+j\cdot c_{Grid}\right)+{\mathrm{<figure-callout\; id="0"\; label="z"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">z</figure-callout>}}_0-\text{b}\right)\\ \Rightarrow \lambda =-\text{b}/(R_{31}\left(x_0+i\cdot c_{Grid}\right)+R_{32}\left({\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+j\cdot c_{Grid}\right)+{\mathrm{<figure-callout\; id="0"\; label="z"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">z</figure-callout>}}_0-\\ \text{b})\end{array}$$
  
• • Virtual image of the point or dot: $$\begin{array}{l}\left(\begin{array}{c}{B}_X\\ B_Y\end{array}\right)=\frac{-\text{b}}{\left(R_{31}\left(x_0+i\cdot c_{Grid}\right)+R_{32}\left({\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+j\cdot c_{Grid}\right)+{\mathrm{<figure-callout\; id="0"\; label="z"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">z</figure-callout>}}_0-\text{b}\right)}\cdot \\ \left(\begin{array}{c}{R}_{11}\left(x_0+i\cdot c_{Grid}\right)+R_{12}\left({\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+j\cdot c_{Grid}\right)\\ R_{21}\left(x_0+i\cdot c_{Grid}\right)+R_{22}\left({\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+j\cdot c_{Grid}\right)\end{array}\right)\end{array}$$
  
• • For the real image, the negative sign must be omitted: For the intersection point 4 with the index (i=0; j=0) you get the image coordinates in the real image: $$\left(\begin{array}{c}{B}_X\\ B_Y\end{array}\right)=\frac{\text{b}}{\left(R_{31}{x}_0+R_{32}{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+{\mathrm{<figure-callout\; id="0"\; label="z"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">z</figure-callout>}}_0-\text{b}\right)}\cdot \left(\begin{array}{c}{R}_{11}{x}_0+R_{12}{y}_0\\ R_{21}{x}_0+R_{22}{y}_0\end{array}\right)$$
  

2.4.3 Code arrangement

The method requires a surface-coded scale in the code arrangement 2, a partial area of which is read with an imaging sensor, for example the camera 1, so that the position of the sensor and/or the camera 1 in relation to the scale can be determined in up to six spatial directions from the image information. Further details on the code arrangement 2 can be found in the publication DE 10 2016 216 221 A1 of the applicant, the content of which is incorporated into the present disclosure via referencing, in particular with regard to the formation of the code arrangement as well as the decoding and the variants.

The coded scale is designed as a sensor-readable marking as a code arrangement 2 on a surface that extends essentially in two dimensions, but can also have curvatures. For a better description, it is assumed in the following that the coded scale is printed as an optically readable pattern on a flat surface, without restricting the claim to other marking and sensor principles and curved surfaces.

The coded scale of code arrangement 2 is formed by arranging different basic symbols in a regular grid. The basic symbols carry two pieces of information: their shape encodes digital information and their center of gravity marks a specific position on the surface.

In a simple case, the digital information is encoded in a binary number system with the base b=2. Then only two base symbols 20 are used, e.g. a small and a large circle, which symbolize the values "0" and "1". Their centroid (center of the circle) marks a grid point on the surface.

The centroids of the basic symbols 20 form a periodic two-dimensional pattern on the scale plane of the code arrangement 2, for example in a square grid as in 10 with the same base spacing c _grid of adjacent symbols in X and Y directions.

By dividing the plane into equally sized, area-filling plots 19, the basic symbols within a plot 19 are combined into a logical unit. Square plots 19 are preferably used. 10 shows a dot matrix with square plots 19, each of which can hold 7 x 7 binary symbols, which corresponds to a maximum information content of 49 bits. The 10 The lines and squares shown are for illustrative purposes only and are not shown in the actual code arrangement 2. 11 shows an exemplary code arrangement 2 without lines with the basic symbols 20 in the plots 19.

The regular arrangement of the base symbols 20 is overlaid by a regular arrangement of parcel symbols 21 by marking each parcel 19 in the same way with a parcel symbol 21. The parcel symbol 21 can be represented, for example, by omitting a base symbol 20.

In 10 , 11 the parcel symbol 21 consists of a blank space in the dot grid, which is located in the middle of each parcel 19. The parcel symbol 21 makes it possible to recognize the position of the parcel 19 in order to read the basic symbols 20 along a reading track in the correct order. Since the parcel symbol 21 occupies a grid space, the information content of a parcel 19 is reduced to 48 bits in this example.

A reading field 22 is a field on the coded scale of the code arrangement 2 which is at least the size of a plot 19. It is bound to the point grid, but not to the grid of the plots 19.

12 shows a dot grid of a code arrangement 2 with two exemplary reading fields 22. The position of the reading field 2 in the coordinate system of the dot grid is defined by its center position 23. It can vary in integer steps of the base distance c _grid .

In addition, the reading field 22 has an angular position with respect to the coordinate system of the dot grid, which can vary in steps of 90° in a square grid. In 12 The angle position is visualized by marking a corner of the reading field. The possible angle positions are shown in 13 shown. The 12 The reading fields 22 shown as examples are clearly described by the following information: • Reading field 22, left: Position = (4,11); angular position = 0° • Reading field 22, right: Position = (12,6); angular position = 90°

The code of the code arrangement 2 is constructed in such a way that the basic symbols 20 in a reading field 22 contain sufficient information to digitally encode the location (X, Y) and the direction of the reading field 22 in the coordinate system 16 of the code arrangement 2. X and Y are specified in integer multiples of the basic distance c _grid (rough position). The fine position in fractions of the basic distance c _grid and the exact angle in fractions of 90° are not digitally encoded; they are determined by an exact location determination of the basic symbols 20 in the camera coordinate system 17.

14 shows an example of a reading field 22 with 15x15 spaces for basic symbols 20. It provides more information than the minimum required reading field 22 of the size of a plot 19, here 7x7 symbols. The redundant information is used for error detection and/or error correction.

15 shows an example of the structure of a plot 19 with 7x7 grid points. In the middle of the plot 19 is the plot symbol 21. The two fields 24 mark the X plot areas in which the continuous code for the position in the X direction is shown. The X plot areas comprise 24 grid points, accordingly they represent a code with a length of 24 bits.

To read the code, the basic symbols 20 are read column by column from left to right and within each column from top to bottom. The reading order in the X-parcel areas 24 (reading track) is in raster point coordinates (X _C , Y _C ): (0.6); (0.5); (0.4); (1.6); (1.5); (1.4); (2.6); (2.5); (2.4); (3.6); (3.5); (3.4); (3.2); (3.1); (3.0); (4.2); (4.1); (4.0); (5.2); (5.1); (5.0); (6.2); (6.1); (6.0).

The two fields 25 mark the Y-plot areas in which the continuous code for the position in the Y direction is shown. The reading order corresponds to that of the X-plot areas 24, but is rotated 90° anti-clockwise. The reading track therefore runs line by line from bottom to top and within each line from left to right.

The code from the X-parcel areas 24 (X-code) is a partial sequence with a length of t=24 digits from a total sequence of g digits, where g is much larger than t. The number g is sufficiently large so that the X-parcel areas 24 of all parcels 19 adjacent in the X direction can be completely filled. In particular, g is greater than fifty, in particular greater than a thousand and especially greater than a million. The contents of the X-parcel areas 24 of parcels adjacent in the Y direction are identical. The overall sequence is designed such that each partial sequence of t consecutive basic symbols 20 is contained exactly once in the overall sequence when read forwards and each partial sequence read backwards is not contained in the overall sequence when read forwards.
The code from the Y parcel areas 25 (Y code) is shown inverted, ie each digit z is replaced by the digit (b-1-z). For the binary system with b=2, this corresponds to a bit-wise inversion. The inverted Y code is a partial sequence with t=24 bits in length from an overall sequence, whereby the same overall sequence can be used as for the X code. The inverted Y code also only appears once in the overall sequence and does not appear in the overall sequence when read backwards.

The contents of the Y-plot areas 25 of neighboring plots 19 in the X-direction are identical.

The overall sequence is designed in such a way that a checksum of a subsequence is less than half of the maximum possible checksum of a subsequence. In particular, the checksum for any subsequence selected from the overall sequence is less than half of the maximum possible checksum of a subsequence. For example, the checksum of any subsequence with length t for a number system with base b is less than q = t·(b - 1)/2.

The subsequences of the overall sequence encode a coordinate value, for example the X-coordinates of the start position of the reading process. Likewise, a subsequence of the inverted overall sequence encodes a Y coordinate, for example the position from which the partial sequence is read. The X code and the Y code read in a plot 19 are assigned to the X and Y coordinates of a reference point in the plot 19, for example the center of the reading field.

Each reading field 22 contains exactly one parcel symbol 21, t basic symbols from X parcel areas 24 and t basic symbols from Y parcel areas 25. The position of the parcel grid and thus the position of the X and Y parcel areas 24, 25 in the reading field 22 as well as the associated reading sequence can be derived from the position of the parcel symbol 21. The basic symbols 20 read according to the reading sequence result in a digit sequence with t positions, which - if necessary after inversion - is a partial sequence of the overall sequence.

If the orientation of a reading field 22 is unknown, it is initially not known which of the two axes is the X-axis and which is the Y-axis. This is determined by calculating the cross sum of the read code and comparing it with q: the cross sum of an X-code is smaller than q, the cross sum of a Y-code is larger than q. The direction of the coordinate axes of the reading field 22 in relation to the coordinate axes of the code arrangement 2 is clearly derived from the direction of the read codes in relation to the overall sequence (forward or backward).

In this way, the position and orientation of a reading field 22 in the coordinate system 16 of the code arrangement 2 can be determined, the position in integer steps of the grid width and the orientation in integer steps of 90°.

In 16 , on the left, plots 19 with 9 rows and 9 columns are shown. In addition to the plot areas X and Y 24, 25, which represent the position code for the respective axis of the coordinate system 16, a plot area for additional data 26 is provided. It comprises (5x5) grid points, whereby the middle grid point is left out for the plot symbol 21, so that 24 grid points remain, each of which can accommodate a base symbol 20. In this way, 24 bits of additional information can be shown in each plot 19, which are not required for position determination and are read in a fixed order. 16 , on the right shows another code arrangement 2 with additional data. Here, four parcel areas 26 with 10 grid points each are provided for additional data, so that a total of 40 bits of additional information are available per parcel 19.

The algorithm for determining the position is in 1 as a sequence of data processing steps, which are explained in detail below using examples. Standard methods of mathematics and image processing as well as for diagnosis and error detection are not explained in detail.

The method is optimized in particular for high accuracy in position determination and for rapid execution. The sequence of steps quickly reduces the data volume, which supports rapid execution.

In process position I in step 100, the digital image data is transferred from camera 1 to the computer as a grayscale matrix. Step 200 is used to roughly check the validity of the image data using key values. Further image processing steps may follow, e.g. to prepare the image data or to segment it into individual code areas. These are not shown in detail here. The data volume for a 200x200 camera image 3 is 40,000 pixels and thus 40,000 bytes in grayscale.

In the process position, the symbols (dots) contained in the image are localized and entered into a dot matrix according to their arrangement in the dot code grid, which corresponds to the size of a reading field (here: 15x15 dots). Each symbol is measured in terms of its position and area. The positions are rectified to correct lens distortions. The areas of the symbols are standardized to eliminate the influence of perspective distortion. The symbol type is classified based on the standardized area. This reduces the data volume to 225 dots with meta information and thus 9000 bytes. In order to classify unoccupied grid points or incorrectly displayed symbols as such, a local model of the dot code grid in the camera image is created from the positions of the successfully identified symbols. Symbols are searched for at all grid points of the model within the reading field. Due to local occlusions, image errors or the image field limitation, not all symbols can always be identified. In this case, the corresponding raster positions are classified as “not valid”.

In the sequence position III, best fit lines (beams) are adjusted to the rows and columns of valid dots in the reading field 22. Each best fit line is described by a straight line angle in the image field of the camera 1 and an intersection point with the X or Y axis of the image field coordinate system. Each reading field 22 provides two bundles of best fit lines, corresponding to the two main axis directions of the code arrangement 2. In this example, each bunch of straight lines comprises up to 15 straight lines. In this way, the position data of 15 x 15 = 225 dots is reduced to the data of 15 + 15 = 30 best fit lines.

In the sequence position IV, the straight lines of each bundle are interpolated by two second degree polynomials: one polynomial describes the straight line angles, a second the axis intersections of the straight lines of a straight line bundle. The polynomial parameters of the two bundles are represented by 12 real values, which corresponds to a reduction by a factor of 5 compared to the straight line representation.

In the sequence position V, an inverse mathematical mapping model is also used to calculate the 6D position of camera 1 from the polynomial parameters of the two straight line bundles. In addition, the validity of the position value is estimated by evaluating a large number of diagnostic results from the individual steps of the program sequence. The 6D camera position and validity are output to the higher-level system in step 800.

Step 200 - Review/validity test of image data

Statistical data are used to check whether an evaluable camera image 3 is available (see flow chart in 17 ). In this example, the image brightness B and contrast C are estimated and checked for compliance with specified limits. If the specified limits are exceeded, further evaluation is aborted and the result is classified as invalid.
To save computing time, only a small number of pixels are included in the determination of the key figures, for example the gray values at the intersection points of the straight lines and circles in the intersection point pattern 27 ( 18 ).

In the sub-step 210, a set of pixels is selected which are determined according to an arbitrary intersection pattern 27. The pixels at the intersection points in the intersection pattern 27 are used. Thus, G _i is a grayscale value of the pixels at the positions i, i = 1..n.

$$\text{C}=\frac{max\left(G_i\right)-min\left(G_i\right)}{max\left(G_i\right)+min\left(G_i\right)+1}$$

In sub-step 220, the image brightness B and contrast C are estimated: $$B=mean\left(G_i\right)$$

$$\text{C}=\frac{max\left(G_i\right)-min\left(G_i\right)}{max\left(G_i\right)+min\left(G_i\right)+1}$$

In sub-step 230, the image brightness B and the contrast C are evaluated with the following conditions: $$\begin{array}{c}\left(B\le B_{min}\right)\text{or}\left(B\ge B_{max}\right)\\ \left(C<C_{min}\right)\end{array}$$

If one of the conditions is met, further evaluation is aborted and the result is classified as invalid.

• Ziel ist es, in einem Lesefeld 22 vorgegebener Größe (hier: 15x15 Rasterpunkte) die Dots als Basissymbole 20 zu identifizieren, zu vermessen und zu klassifizieren. Als Ergebnis wird eine zweidimensionale Dot-Matrix 29 oder allgemein Basissymbol-Matrix mit den Daten der Dots ausgegeben, wobei die Indizes der Matrix 29 den Reihen und Spalten der Codeanordnung 2 zugeordnet sind. Zu jedem Dot werden Kennwerte ermittelt.

Dot data capture/steps 310 to 350:

• The aim is to identify, measure and classify the dots as basic symbols 20 in a reading field 22 of a predetermined size (here: 15x15 grid points). The result is a two-dimensional dot matrix 29 or generally a basic symbol matrix with the data of the dots, whereby the indices of the matrix 29 are assigned to the rows and columns of the code arrangement 2. Characteristic values are determined for each dot.

• • Dot.Typ = 2: großer Dot; logische „1“
• • Dot. Typ = 1: kleiner Dot; logische „0“
• • Dot. Typ = 0: fehlender Dot; Parzellensymbol 21

The Dot.Typ size contains the classification result. If Dot.Typ is positive, the dot was evaluated as valid:

• • Dot.Type = 2: large dot; logical “1”
• • Dot. Type = 1: small dot; logical “0”
• • Dot. Type = 0: missing dot; parcel symbol 21

If Dot.Typ is negative, the dot could not be clearly identified.

In sub-step 310, a starting base/starting field (launch pad) 28 is first searched for in the image area and/or in the reading field 22 of the camera image 3 with the dot code of the code arrangement 2. This is understood here to be a field of, for example, 3x3 adjacent dots of type 1 or 2.

19 shows an example of a camera image 3 with an identified starting base 28 and the reading field 22 in which - starting from the starting base 28 - the dots are searched for as base symbols 20.

In sub-step 320, the starting base 28 is used to build a local model of the point grid in the area surrounding the starting base 28. Additional dots are searched for at the neighboring grid locations predicted by the model, precisely measured if successful, and entered into the dot matrix 29 as valid dots. This process is repeated cyclically so that more and more valid dots are registered around the starting base 28 until the reading field 22 of 15x15 dots is completely covered. The search process is robust against local reading errors: if individual dots cannot be clearly identified or exceed the edge of the image field, they are marked as invalid in the dot matrix 29 and excluded from further processing. 20 shows the result of a successfully classified dot matrix 29 based on the camera image 3 in the 19 .

In substep 330, the center positions and areas of the dots are measured.

Sub-step 340 converts the measured positions of all valid dot positions in the reading field 22 into rectified coordinates. This step serves to compensate for the distortion errors of the wide-angle lens (barrel distortion). After rectification, points that lie on a straight line in the code coordinate system 16 also lie on a straight line in the rectified image in the coordinate system of the image sensor 15. Instead of computer-assisted rectification, optical rectification can also be carried out with a corresponding lens 9, so that sub-step 340 is optional.

Based on the measured data, the dots are classified in sub-step 350. Then, in sub-step 360, the digital code is read in both spatial directions and, using the code table, converted into an integer position indication and a rough direction indication in 90° steps. The determination of the rough position in the upper step 300 is thus completed.

To substep 310 - Determination of the starting base

The process for determining the starting base 28 is shown in the flow chart 21 shown.

Substep 310.1: First, a starting position 30 is determined to search for the first dot of the starting base 28. 22 shows several predefined starting positions 30 in the image field of camera 1. The search begins at one of these starting positions. If the process fails (e.g. due to a disturbance in the image field), a second search is started at the second starting position, and so on, until sub-step 310.1 can be successfully completed. If all starting positions have been used and no search was successful, sub-step 310.1 is aborted with a negative result.

wird der nächstgelegene Dot gesucht. In 23 wird ein Dot an Position ${\overrightarrow{P}}_0$

als Startbasissymbol gefunden.Substep 310.2: Starting from the starting position ${\overrightarrow{P}}_{Start}/30$

the nearest dot is searched for. In 23 a dot is placed at position ${\overrightarrow{P}}_0$

as starting base symbol.

der nächstgelegene Dot gesucht und mit ${\overrightarrow{P}}_1$

als erster Hilfsstartbasisymbol bezeichnet.Sub-step 310.3: Then, starting from ${\overrightarrow{P}}_0$

the nearest dot is searched and with ${\overrightarrow{P}}_1$

referred to as the first auxiliary start base symbol.

gebildet.Substep 310.4: Then a connection vector ${\overrightarrow{E}}_1=\left({\overrightarrow{P}}_1-{\overrightarrow{P}}_0\right)$

formed.

wird der nächstgelegene Dot in orthogonaler Richtung zum Verbindungsvektor ${\overrightarrow{E}}_1=\left({\overrightarrow{P}}_1-{\overrightarrow{P}}_0\right)$

gesucht und mit ${\overrightarrow{P}}_2$

als zweites Hilfsstartbasissymbol bezeichnet.Substep 310.5: Starting from ${\overrightarrow{P}}_0$

the nearest dot in the orthogonal direction to the connection vector ${\overrightarrow{E}}_1=\left({\overrightarrow{P}}_1-{\overrightarrow{P}}_0\right)$

searched and with ${\overrightarrow{P}}_2$

referred to as the second auxiliary starting base symbol.

spannen ein schiefwinkliges Koordinatensystem mit den Achsen ${\overrightarrow{E}}_1=\left({\overrightarrow{P}}_1-{\overrightarrow{P}}_0\right)$

und ${\overrightarrow{E}}_2=\left({\overrightarrow{P}}_2-{\overrightarrow{P}}_0\right)$

auf, wobei ${\overrightarrow{P}}_0$

den Ursprung des Koordinatensystems bildet.Substep 310.6: The three dots ${\overrightarrow{P}}_0,{\overrightarrow{P}}_1\text{and}{\overrightarrow{P}}_2$

span an oblique coordinate system with the axes ${\overrightarrow{E}}_1=\left({\overrightarrow{P}}_1-{\overrightarrow{P}}_0\right)$

and ${\overrightarrow{E}}_2=\left({\overrightarrow{P}}_2-{\overrightarrow{P}}_0\right)$

on, whereby ${\overrightarrow{P}}_0$

forms the origin of the coordinate system.

wird es durch Tausch der Achsen ${\overrightarrow{E}}_1$

und ${\overrightarrow{E}}_2$

in ein rechtshändisches Koordinatensystem überführt, siehe 24, links.Substep 310.7: If it is a left-handed coordinate system $\left(\text{Condition:}\left({\overrightarrow{E}}_1\times {\overrightarrow{E}}_2\right)<0\right),$

it is done by exchanging the axles ${\overrightarrow{E}}_1$

and ${\overrightarrow{E}}_2$

converted into a right-handed coordinate system, see 24 , left.

und ${\overrightarrow{E}}_2$

werden alle acht zu ${\overrightarrow{P}}_0$

benachbarten Dot-Positionen abgeschätzt (24, zweites von links) und als Startfeld 28 zur Suche und Vermessung von Dots genutzt. Als Ergebnis steht eine Startbasis 28 von 3x3 Dots zur Verfügung (24, zweites von rechts, rechts).Substep 310.8: By linear combination of vectors ${\overrightarrow{E}}_1$

and ${\overrightarrow{E}}_2$

all eight will be ${\overrightarrow{P}}_0$

neighboring dot positions are estimated ( 24 , second from the left) and used as starting field 28 for searching and measuring dots. As a result, a starting base 28 of 3x3 dots is available ( 24 , second from the right, right).

Substep 310.9: If a dot is not found during this procedure, it is assumed that it is a parcel symbol 21 (missing dot).

um eine Dot-Position in die zum fehlenden Dot gegenüberliegende Richtung verschoben und der Ablauf wird mit der erneuten Suche von ${\overrightarrow{P}}_1$

und ${\overrightarrow{P}}_2$

fortgesetzt. Bei dem zweiten Durchlauf ist davon auszugehen, dass alle 3x3 Dots identifiziert werden, da das nächste Parzellensymbol 21 eine Parzelle 19 weit (hier: 7 Dots) entfernt liegt und die Startbasis nur 3x3 Dots groß ist.In sub-step 310.10, ${\overrightarrow{P}}_0$

shifted by one dot position in the direction opposite to the missing dot and the process is continued with the new search of ${\overrightarrow{P}}_1$

and ${\overrightarrow{P}}_2$

continued. In the second run, it can be assumed that all 3x3 dots are identified, since the next parcel symbol 21 is one parcel 19 away (here: 7 dots) and the starting base is only 3x3 dots in size.

Subsequently, a sub-step 310.11 can be carried out for error detection, whereby if an error is detected in a sub-step 310.12, a new starting position 30 is used and the procedure is repeated starting from sub-step 310.2.

As part of the previously described process, the challenge is to identify a dot in the image field of camera 1 starting from an estimated starting position and to measure its position and area with subpixel accuracy. In order to achieve a short measurement time, this process must be carried out very quickly, since a total of 225 dots must be identified in a reading field 22.

For this purpose, the Center_Pos function described below is used, which is optimized for a minimal number of pixel accesses. 25 shows the flow chart.

übergeben. Falls ${\overrightarrow{P}}_0$

außerhalb des Suchbereiches liegt, wird die Suche abgebrochen und ein negatives Ergebnis zurückgemeldet. Andernfalls werden die 8 Schritte a) - h) ausgeführt, siehe auch 26 a - h:

• a) Mit einem grauwertbasierten zweidimensionalen Gradientenverfahren wird aus ${\overrightarrow{P}}_0$
  
  ein optimierter Startpunkt ${\overrightarrow{P}}_1$
  
  berechnet, der näher am Zentrum des Dots liegt. Für die Berechnung des Gradienten sind lediglich 4 Pixel-Zugriffe im Umfeld des Startpunktes ${\overrightarrow{P}}_0$
  
  erforderlich.
• b) Ausgehend vom optimierten Punkt ${\overrightarrow{P}}_1$
  
  wird in +X- und -X-Richtung mit Methoden zur Kantenerkennung der Rand des Dots gesucht. Der Abstand von ${\overrightarrow{P}}_{}$${\stackrel{}{}}_1$
  
  zum rechten Rand ist r₀, zum linken Rand r₁.
• c) ${\overrightarrow{\mathrm{<figure-callout\; id="1"\; label="P\; \to "\; filenames="DE102023203159A1\_0157.png,DE102023203159A1\_0158.png"\; state="\{\{state\}\}">P</figure-callout>}}}_1$
  
  wird auf der X-Achse in Bezug auf die linke und rechte Kante zentriert, daraus ergibt sich der optimierte Punkt ${\overrightarrow{P}}_2$
  
• d) Ausgehend vom Punkt ${\overrightarrow{P}}_2$
  
  wird in +Y- und -Y-Richtung mit Methoden zur Kantenerkennung der Rand des Dots gesucht. Der Abstand von ${\overrightarrow{P}}_{}$${\stackrel{}{}}_2$
  
  zum oberen Rand ist r₂, zum unteren Rand r₃.
• e) ${\overrightarrow{\mathrm{<figure-callout\; id="2"\; label="P\; \to "\; filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png"\; state="\{\{state\}\}">P</figure-callout>}}}_2$
  
  wird auf der Y-Achse in Bezug auf die obere und untere Kante zentriert, daraus ergibt sich der optimierte Punkt ${\overrightarrow{P}}_3.$
  
• f)Ausgehend vom Punkt ${\overrightarrow{P}}_3$
  
  wird erneut in +X- und -X-Richtung mit Methoden zur Kantenerkennung der Rand des Dots gesucht. Der Abstand von ${\overrightarrow{P}}_{}$${\stackrel{}{}}_3$
  
  zum rechten Rand ist r₄, zum linken Rand r₅.
• g) ${\overrightarrow{\mathrm{<figure-callout\; id="3"\; label="P\; \to "\; filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0158.png"\; state="\{\{state\}\}">P</figure-callout>}}}_3$
  
  wird auf der X-Achse in Bezug auf die linke und rechte Kante zentriert. Daraus ergibt sich der optimierte Punkt ${\overrightarrow{P}}_4,$
  
  der als Mittelpunktsposition des Dots mit Subpixel-Genauigkeit ausgegeben wird.
• h) Die Fläche des Dots wird als Fläche des umschreibenden Rechtecks berechnet, multipliziert mit dem Faktor $\frac{\pi }{}$$\frac{}{4}$
  
  zur Umrechnung der quadratischen Fläche in eine Kreisfläche bzw. elliptische Fläche: $$Dot.Area=\left({\mathrm{<figure-callout\; id="2"\; label="r"\; filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png"\; state="\{\{state\}\}">r</figure-callout>}}_2+r_3\right)\cdot \left({\mathrm{<figure-callout\; id="4"\; label="r"\; filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png"\; state="\{\{state\}\}">r</figure-callout>}}_4+r_5\right)\cdot \frac{\mathrm{<figure-callout\; id="4"\; label="\pi "\; filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png"\; state="\{\{state\}\}">\pi </figure-callout>}}{4}$$
  

When the function Center_Pos is called, the estimated center position of the dot ${\overrightarrow{P}}_0$

handed over. If ${\overrightarrow{P}}_0$

outside the search area, the search is aborted and a negative result is returned. Otherwise, the 8 steps a) - h) are carried out, see also 26 a - h:

• a) Using a gray value based two-dimensional gradient method, ${\overrightarrow{P}}_0$
  
  an optimized starting point ${\overrightarrow{P}}_1$
  
  which is closer to the center of the dot. To calculate the gradient, only 4 pixel accesses are required in the vicinity of the starting point ${\overrightarrow{P}}_0$
  
  necessary.
• b) Starting from the optimized point ${\overrightarrow{P}}_1$
  
  The edge of the dot is searched for in the +X and -X directions using edge detection methods. The distance from ${\overrightarrow{P}}_1$
  
  to the right edge is r ₀ , to the left edge r ₁ .
• c) ${\overrightarrow{P}}_1$
  
  is centered on the X-axis with respect to the left and right edges, resulting in the optimized point ${\overrightarrow{P}}_2$
  
• d) Starting from the point ${\overrightarrow{P}}_2$
  
  The edge of the dot is searched for in the +Y and -Y directions using edge detection methods. The distance from ${\overrightarrow{P}}_2$
  
  to the upper edge is r ₂ , to the lower edge is r ₃ .
• e) ${\overrightarrow{P}}_2$
  
  is centered on the Y-axis with respect to the upper and lower edges, resulting in the optimized point ${\overrightarrow{P}}_3.$
  
• f)Starting from the point ${\overrightarrow{P}}_3$
  
  The edge of the dot is searched again in the +X and -X directions using edge detection methods. The distance from ${\overrightarrow{P}}_3$
  
  to the right edge is r ₄ , to the left edge r ₅ .
• G) ${\overrightarrow{P}}_3$
  
  is centered on the X-axis with respect to the left and right edges. This results in the optimized point ${\overrightarrow{P}}_4,$
  
  which is output as the center position of the dot with subpixel accuracy.
• h) The area of the dot is calculated as the area of the surrounding rectangle, multiplied by the factor $\frac{\pi }{4}$
  
  to convert the square area into a circular area or elliptical area: $$DOt.Area=\left(r_2+r_3\right)\cdot \left(r_4+r_5\right)\cdot \frac{\pi }{4}$$
  

nächstgelegenen Dot zu finden. Diese Aufgabe kann mit der nachfolgend beschriebenen Funktion Nearest_dot gelöst werden. Die Funktion erhält die Eingangsdaten

Startposition

rmin

minimaler Suchradius

rmax

maximaler Suchradius

As part of the previously described process, there is also the challenge of finding the starting position in camera image 3 ${\overrightarrow{P}}_0$

to find the nearest dot. This task can be solved using the Nearest_dot function described below. The function receives the input data

starting position

rmin

minimum search radius

rmax

maximum search radius

• Falls die Startposition innerhalb eines Dots liegt und ND = false ist, wird der Dot am Ort der Startposition ausgegeben. Falls ND = true ist, wird der Dot am Ort der Startposition ignoriert und der nächstgelegene Nachbar-Dot ausgegeben.

ND Boolean size (ND = "neighbor dot"):

• If the start position is within a dot and ND = false, the dot at the location of the start position is output. If ND = true, the dot at the location of the start position is ignored and the nearest neighboring dot is output.

tastet die Funktion das Bildfeld entlang von 16 Suchstrahlen 31 im Winkelabstand von 22,5° ab (27a, b). Der Radius aller Suchstrahlen 31 wird schrittweise erhöht, beginnend mit dem Radius r_min, bis zum maximalen Radius r_max.
Falls ND = false (27a), werden sowohl Kanten gesucht, die in einen Dot hineinführen als auch Kanten, die aus einem Dot herausführen. Falls ND = true ( 27b), werden nur Kanten gesucht, die in einen Dot hineinführen. Sobald eine Kante gefunden wurde, wird die Suche abgebrochen. Die Kante befindet sich an der Position ${\overrightarrow{P}}_1$

sie gehört zu dem gesuchten nächstgelegenen Dot. Ausgehend von ${\overrightarrow{P}}_1$

wird mit der Funktion Center_Dot die Mittelpunktsposition und die Fläche des gefundenen Dots bestimmt und ausgegeben.Starting from the starting point ${\overrightarrow{P}}_0$

The function scans the image field along 16 search beams 31 at an angular distance of 22.5° ( 27a , b). The radius of all search beams 31 is increased step by step, starting with the radius r _min , up to the maximum radius r _max .
If ND = false ( 27a) , both edges leading into a dot and edges leading out of a dot are searched for. If ND = true ( 27b) , only edges that lead into a dot are searched for. As soon as an edge is found, the search is aborted. The edge is located at the position ${\overrightarrow{P}}_1$

it belongs to the nearest dot we are looking for. Starting from ${\overrightarrow{P}}_1$

The Center_Dot function is used to determine and output the center position and the area of the found dot.

Substep 320 - Detecting the dots in the reading field

The flow chart in 28 shows the process for detecting all dots in the reading field 22. The previously determined starting base 28 of e.g. 3x3 dots serves as input information.

In addition, step 330: Precise measurement of the center position and area of the dots is integrated into sub-step 320.

In the first sub-step 320.1, the center position 32 of the reading field 22 is determined. This is selected so that the reading field 22 has as much coverage as possible with the image area in which the code is displayed, so that as many valid dots as possible can be recorded. This is usually the case when the center of the reading field 22 is in the center of the code area. If the code area takes up the entire image area, as in the present example, the center position 32 of the reading field is placed as close to the center of the image as possible (see 29a) .

At the beginning of the procedure, only the 3x3 dots of the starting base 28 are known. All other dots in the reading field 22 are still unknown, i.e. their exact position and area have not yet been measured. The aim of the procedure is to gradually measure and classify the unknown dots.

und Dot. ${\overrightarrow{C}}_2$

bestimmt (30), die zum jeweils nächsten Nachbardot in den Hauptachsenrichtungen 1 und 2 zeigen. Sie spannen aufgrund der perspektivischen Verzerrung ein schiefwinkliges Koordinatensystem auf. Durch Linearkombination der Vektoren Dot. ${\overrightarrow{C}}_1$

und Dot. ${\overrightarrow{C}}_2$

können die Rasterpositionen der unbekannten benachbarten Dots abgeschätzt werden. Die Positionsschätzung für einen unbekannten Dot kann verbessert werden, indem die Schätzwerte von mehreren validen Nachbarn extrapoliert und gemittelt werden.
Je mehr valide Nachbarn ein unbekannter Dot besitzt, umso besser kann dessen Position geschätzt werden. Um die Zuverlässigkeit des Lesevorgangs zu erhöhen, werden nur unbekannte Dots untersucht, die mindestens zwei valide Nachbarn haben. Für jeden unbekannten Dot wird die Anzahl der validen Nachbarn gezählt. Diese kann zwischen 0 und 8 liegen, siehe 29c): die mittlere Dot-Position hat 8 Nachbarpositionen (dunkel).Substep 320.2: For each known dot, the vectors Dot. ${\overrightarrow{C}}_1$

and Dot. ${\overrightarrow{C}}_2$

certainly ( 30 ), which point to the nearest neighboring dot in the main axis directions 1 and 2. Due to the perspective distortion, they form an oblique coordinate system. By linear combination tion of the vectors Dot. ${\overrightarrow{C}}_1$

and Dot. ${\overrightarrow{C}}_2$

The grid positions of the unknown neighboring dots can be estimated. The position estimation for an unknown dot can be improved by extrapolating and averaging the estimates from several valid neighbors.
The more valid neighbors an unknown dot has, the better its position can be estimated. To increase the reliability of the reading process, only unknown dots that have at least two valid neighbors are examined. The number of valid neighbors is counted for each unknown dot. This can be between 0 and 8, see 29c ): the middle dot position has 8 neighboring positions (dark).

At the beginning of the search, only the 3x3 dots of the starting base are valid. The only unknown dots with 2 or more valid neighbors are in 29b) shown in a more prominent way. 29 d) shows schematically the unknown dots with 2 valid neighbors 20 a and with 3 valid neighbors as 20 b. All other unknown dots (20c) have no valid neighbors yet.

The algorithm is based on a stack, where in substep 320.3 all unknown dots with at least 2 valid neighbors are entered into the stack.

• • Unterschritt 320.3: Der oberste Eintrag wird als unbekannter Dot vom Stack geholt und in den folgenden Schritten bearbeitet.
• • Unterschritt 320.4: Die Position des unbekannten Dots wird geschätzt, indem die extrapolierten Positionen aller validen Nachbarn gemittelt werden. Falls der unbekannte Dot außerhalb des erwarteten Lesefeldes 22 liegt, wird er nicht weiter bearbeitet. Liegt der unbekannte Dot innerhalb des erwarteten Lesefeldes 22, wird er an der geschätzten Position mit der Funktion Center_Pos identifiziert und vermessen.
• • Unterschritt 320.5: Falls der unbekannte Dot nicht identifiziert werden kann, wird er als defekt klassifiziert (Unterschritt 320.6) und der nächste unbekannte Dot wird vom Stack geholt.
• • Schritt 330/Unterschritt 320.7: Falls der Dot identifiziert wurde, wird seine Position und Fläche mit der Funktion Center_Pos gemessen und er wird als valider neuer Dot in die Dot-Matrix 29 eingetragen. Auch die Vektoren Dot. ${\overrightarrow{C}}_1$
  
  und Dot. ${\overrightarrow{C}}_2$
  
  werden für diesen Dot berechnet und eingetragen.
• • Unterschritt 320.8: Für alle 8 Nachbarn des neuen validen Dots wird die Zahl der validen Nachbarn um 1 erhöht. Falls dadurch ein unbekannter Dot zwei oder mehr valide Nachbarn hat, wird dieser zur Vermessung auf den Stack gelegt.
• • Unterschritt 320.9: Dieser Ablauf wird wiederholt, bis der Stack leer ist. Dann wurden alle Dots des Lesefeldes bearbeitet.
• • Unterschritt 320.10: Falls nicht genügend Dots zur Positionsbestimmung gefunden wurden, wird der Lesevorgang als nicht valide bewertet.

The stack is processed in a loop:

• • Substep 320.3: The top entry is popped from the stack as an unknown dot and processed in the following steps.
• • Substep 320.4: The position of the unknown dot is estimated by averaging the extrapolated positions of all valid neighbors. If the unknown dot is outside the expected reading field 22, it is not processed further. If the unknown dot is within the expected reading field 22, it is identified and measured at the estimated position using the Center_Pos function.
• • Substep 320.5: If the unknown dot cannot be identified, it is classified as defective (substep 320.6) and the next unknown dot is popped from the stack.
• • Step 330/sub-step 320.7: If the dot has been identified, its position and area are measured using the Center_Pos function and it is entered into the dot matrix 29 as a valid new dot. The vectors Dot. ${\overrightarrow{C}}_1$
  
  and Dot. ${\overrightarrow{C}}_2$
  
  are calculated and entered for this dot.
• • Substep 320.8: For all 8 neighbors of the new valid dot, the number of valid neighbors is increased by 1. If this results in an unknown dot having two or more valid neighbors, it is placed on the stack for measurement.
• • Sub-step 320.9: This process is repeated until the stack is empty. Then all dots of the reading field have been processed.
• • Substep 320.10: If not enough dots were found to determine the position, the reading process is evaluated as invalid.

To sub-step 340 - rectification

Imaging with the wide-angle lens can result in a barrel-shaped distortion of the image field. Lines of dots that lie on a straight line in the code plane of the code arrangement 2 may lie on a curved line in the camera image 3.

This curvature can be eliminated by a mathematical procedure, called rectification. For computing time reasons, rectification is only applied to the center positions 32 of the dots, not to all pixels of the input image. 31 shows an example of a non-rectified grid (33) and a rectified grid (34).

liegt im Schnittpunkt der optischen Achse 5 mit dem Kamerachip des Bildaufnehmers 12, bei perfekter Montage der Kamera 1 ist das üblicherweise die Mitte des Kamerachips. Andernfalls wird das Zentrum durch einen einmaligen Kalibriervorgang der Kamera 1 bestimmt. Mit einem Polynom 2. Ordnung wird eine Position $\overrightarrow{P}$

im Bildfeld der Kamera 3 in die rektifizierte Position $\overrightarrow{R}$

umgerechnet: $$\overrightarrow{R}=\left(p_0+p_{1}D+p_2{D}^2\right)\cdot \overrightarrow{D}+\overrightarrow{IC}$$

mit

Position in Bezug auf das Zentrum der Verzeichnung

Abstand zum Zentrum der Verzeichnung

The center of distortion $\overrightarrow{IC}$

is located at the intersection point of the optical axis 5 with the camera chip of the image sensor 12. If the camera 1 is perfectly mounted, this is usually the center of the camera chip. Otherwise, the center is determined by a one-time calibration process of the camera 1. A position is determined using a 2nd order polynomial. $\overrightarrow{P}$

in the image field of camera 3 into the rectified position $\overrightarrow{R}$

converted: $$\overrightarrow{R}=\left({\mathrm{<figure-callout\; id="0"\; label="p"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">p</figure-callout>}}_0+p_{1}D+p_2{D}^2\right)\cdot \overrightarrow{D}+\overrightarrow{IC}$$

with

position relative to the center of distortion

distance to the center of distortion

nicht dessen Richtung. Die Polynomparameter p₀, p₁, p₂ sind spezifisch für das Objektiv 9 und werden in einem Kalibriervorgang an dieses angepasst. Alternativ hierzu kann ein verzerrungsfreies Objektiv 9 verwendet werden.The distortion only changes the length of the vector $\overrightarrow{D},$

not its direction. The polynomial parameters p ₀ , p ₁ , p ₂ are specific to the lens 9 and are adapted to it in a calibration process. Alternatively, a distortion-free lens 9 can be used.

To substep 350 - Classification of dots

The small and large dots symbolize the values "0" and "1" of the binary number system. The classification of the dots is based on a threshold value for the standardized area of the dots. The process is shown in the flow chart 32 shown.

Perspective distortion has a strong influence on the area of the dots in the image field. For example, dots further away are shown smaller, and circular dots on an inclined plane are shown as an ellipse.

und Dot. ${\overrightarrow{C}}_2$

am Ort des Dots, also die Abstandsvektoren vom Dot zu den in den Hauptachsenrichtungen 1 und 2 benachbarten Dots, aufspannen.Substep 350.1: To compensate for these effects, the area of the dots is normalized with respect to the area of their dot cell. The dot cell is the parallelogram that contains the vectors Dot. ${\overrightarrow{C}}_1$

and Dot. ${\overrightarrow{C}}_2$

at the location of the dot, i.e. the distance vectors from the dot to the neighboring dots in the main axis directions 1 and 2.

und Dot. ${\overrightarrow{C}}_2$

aufgespannt wird. Die Fläche der Zelle ist $\left|Dot.{\overrightarrow{C}}_1\times Dot.{\overrightarrow{C}}_2\right|.$

Die normierte Fläche des Dots wird folgendermaßen berechnet: $$Dot.Are{a}_{scaled}=Dot.Area/\left|Dot.{\overrightarrow{C}}_1\times Dot.{\overrightarrow{C}}_2\right|$$

30 shows a perspectively distorted dot in a distorted grid, which is locally defined by the vectors dot. ${\overrightarrow{C}}_1$

and Dot. ${\overrightarrow{C}}_2$

The area of the cell is $\left|DOt.{\overrightarrow{C}}_1\times DOt.{\overrightarrow{C}}_2\right|.$

The normalized area of the dot is calculated as follows: $$DOt.Are{a}_{scaled}=DOt.Area/\left|DOt.{\overrightarrow{C}}_1\times DOt.{\overrightarrow{C}}_2\right|$$

Substep 350.2: The threshold value is determined using statistical methods from the normalized areas Dot. Area _scaled of all dots.

• • Fehlender Dot: Typ 0
• • Normierte Fläche kleiner als Schwellwert: Typ 1; sonst: Typ 2.

Substep 350.3: All dots are then classified:

• • Missing Dot: Type 0
• • Normalized area smaller than threshold: Type 1; otherwise: Type 2.

Substep 350.3: If errors occur, the status is set to invalid; if no errors occur, the dot is valid.

To substep 360 - Reading the code

After the dots are classified, the dot matrix 29 contains all the information to determine the 6D position.

Genaue Dot-Mittelpunkte in den originalen Bildkoordinaten Teilschritt 310 und 320, Funktion: Center_Pos: Subpixel genaues Erfassen des Dot-Mttelpunkts $Do{t}_{i,j}.\overrightarrow{R}$

Dot-Mittelpunkte in den rektifizierten Bildkoordinaten Teilschritt 340, Funktion: Rectify: Rektifizierung der $Do{t}_{i,j}.\overrightarrow{P}$

$Do{t}_{i,j}.\overrightarrow{C1}Do{t}_{i,j}.\overrightarrow{C2}$

Lokaler Distanzvektor zwischen benachbarten Dots in der ersten und der zweiten Rasterrichtung. Teilschritt 310 und 320, berechnet aus den Dot-Mittelpunkten in den originalen Bildkoordinaten $Do{t}_{i,j}.\overrightarrow{P}$

Dot _i,j . Area Dotfläche in den originalen Bildkoordinaten [Pixel] Teilschritt 310 und 320, Center_Pos: Dotflächenerfassung Dot _i,j . Typ Klassifizierter DotTyp Teilschritt 340, Dot-Klassifizierung auf Basis der Dot-Fläche Dot _i,j . VN Anzahl der validen Nachbardots Teilschritt 310 und 320, Zählung der validen Nachbarn The dot matrix 29 contains the following information: Dot data Description Data is provided through the following steps: $DO{t}_{i,j}.\overrightarrow{P}$

Exact dot centers in the original image coordinates Substep 310 and 320, Function: Center_Pos: Subpixel accurate detection of the dot center $DO{t}_{i,j}.\overrightarrow{R}$

Dot centers in the rectified image coordinates Substep 340, Function: Rectify: Rectification of the $DO{t}_{i,j}.\overrightarrow{P}$

$DO{t}_{i,j}.\overrightarrow{\mathrm{<figure-callout\; id="1"\; label="C"\; filenames="DE102023203159A1\_0157.png,DE102023203159A1\_0158.png"\; state="\{\{state\}\}">C</figure-callout>}1}DO{t}_{i,j}.\overrightarrow{\mathrm{<figure-callout\; id="2"\; label="C"\; filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png"\; state="\{\{state\}\}">C</figure-callout>}2}$

Local distance vector between neighboring dots in the first and second raster direction. Substeps 310 and 320, calculated from the dot centers in the original image coordinates $DO{t}_{i,j}.\overrightarrow{P}$

Dot _i,j . Area Dot area in the original image coordinates [pixels] Substep 310 and 320, Center_Pos: Dot area detection Dot _i,j . Type Classified DotType Substep 340, Dot classification based on the dot area Dot _i,j . VN number of valid neighboring dots Substeps 310 and 320, counting the valid neighbors

The flow chart is in 33 The digital coarse position in X and Y directions is determined by reading the X and Y bit chains in the dot matrix.

Sub-step 360.1: The parcel symbol 21 “empty dot” (type 0) serves as a reference point to identify the position of the bit chains in the dot matrix 29. If several entries with type 0 are included, these can be checked against each other because the “empty dot” repeats regularly in the parcel grid (here: 7x7 dots).

Sub-step 360.2: Starting from the parcel symbol 21, the position of the reading tracks for the codes in the two axes of the code plane is determined. It is initially unknown which of the sequences are assigned to the X-axis and which to the Y-axis and in which direction (forwards or backwards) the code is read. The codes are read along the reading track and saved as Code_0 and Code_1. 34a) shows examples of the dot types that are entered in the dot matrix. The parcel symbol (type 0) is highlighted in grey. in 34b) The reading track for the two axes is also marked. The reading tracks of the two axes are rotated by 90° to each other, with the parcel symbol 21 forming the center of rotation.

Since the reading field (15x15 dots) is significantly larger than a parcel (7x7), the code can be read redundantly: A copy of Code_0 is located in the parcels adjacent in the j-direction (35a), a copy of Code_1 is located in the parcels adjacent in the i-direction (36a).

Redundancy also exists in the length of the readable code. A code length of 24 bits in each direction is sufficient for determining the location, i.e. the content of a 7x7 parcel. However, the larger reading field with 15x15 dots delivers 51 or 54 bits in each direction. The redundant information is used to detect and correct individual incorrectly read bits in Code_0 and Code_1.

• • Code_0: 111.111.011.111.011.111.111.111.111.111.111.111.011.011.111.111.111.111
• • Code_1: 000.000.000.101.000.001.000.000.000.000.100.110.000.000.000.000.000.000

In this example, the following codes are read:

• • Code_0: 111.111.011.111.011.111.111.111.111.111.111.111.011.011.111.111.111.111
• • Code_1: 000,000,000,101,000,001,000,000,000,000,100,110,000,000,000,000,000,000

Substep 360.3: An arbitrary section with t=24 consecutive bits is selected from Code_0 and checked for completeness.

Substep 360.4: The checksum of Code_0 is then calculated. If this is less than t/2 = 12, Code_0 is the X code and Code_1 is the Y code. If it is greater than t/2, Code_0 is the Y code and Code_1 is the X code. The Y code is inverted. The X code remains unchanged.

• - Es wird ein wird ein beliebiger Abschnitt mit t=24 aufeinanderfolgenden Bits ausgewählt und auf Vollständigkeit geprüft: 000100000000001010100000

Sub-step 360.5: Then Code_0 and Code_1 are searched for in the code table using an error-tolerant string search. The bit position with the best match is output as the search result; if the deviation is too great, the search result is assessed as invalid. In order to transform Code_0 and Code_1 into integer location coordinates (X, Y), the bit position of the codes is converted into a spatial position according to the code structure, as shown in the example in 35 shown schematically:

• - An arbitrary section with t=24 consecutive bits is selected and checked for completeness: 000100000000001010100000

• 010101010000000010000000000100010000000010000000000101010000000010000 00001000101000000001 ...

• - Jeder X-Dot bekommt eine Bitposition innerhalb der Codetabelle zugeordnet.
• - Die Bitposition entspricht dem Anfangspunkt bei der Lesereihenfolge innerhalb einer Parzelle.
• - Jeder Y-Dot bekommt eine Bitposition innerhalb der Codetabelle zugeordnet.

This is searched in the code table in forward direction and found at bit position 37 as underlined:

• 010101010000000010000000000100010000000010000000000101010000000010000 00001000101000000001 ...

• - Each X-Dot is assigned a bit position within the code table.
• - The bit position corresponds to the starting point of the reading order within a parcel.
• - Each Y-dot is assigned a bit position within the code table.

360.6 Validity Check

According to the 35 this corresponds to the dot position Xc=10=Pos ₀ . The Yc value for Pos ₁ is determined in an analogous manner. The reading direction (Dir_0 and Dir_1) is also determined for both codes by searching for the codes in both directions in the code table: 0 = found in the forward direction, 1 = found in the backward direction. From the reading directions of Code_0 and Code_1, the rough orientation of the reading window 22 is determined in steps of 90° according to the following table.

As a result of sub-step 360, the integer coarse position (r _X , _int , r _Y,int ) and the orientation of the reading window 22 are available.

Step 400 - Determining the beam data

To determine the fine position, the rectified position data of the valid dots in the dot matrix 29 are evaluated. 36 shows an example of the dots of a reading field in the camera image after rectification.

Step 410: Adjustment of best-fit lines

For this purpose, best fit lines (beams) are adjusted to the rows and columns of valid dots in the reading field 22. The best fit line is calculated using standard mathematical methods to minimize errors from the position data of the valid dots in the respective row or column. Invalid dots are excluded from the process.

• ◯ SX_ki Schnittpunkt mit der X_I-Achse
• ◯ SY_ki Schnittpunkt mit der Y_I-Achse
• ◯ α_ki Winkel der Gerade i im Bildfeld

Each regression line i of the bunch k = 0..1 is described by the following parameters ( 37 ):

• ◯ SX _ki intersection point with the X _I axis
• ◯ SY _ki intersection with the Y _I axis
• ◯ α _ki angle of the line i in the image field

The best-fit lines fitted to the dot rows form a first bundle of lines (bunch ₀ ) 36, where the lines are parallel to the Y _c axis. The lines of the dot columns form a second bundle of lines (bunch ₁ ) 37, where the lines are parallel to the X _c axis. In this way, the position data of 15 x 15 = 225 dots are reduced to the data of 15 + 15 = 30 best-fit lines.

The interpolation also has the effect of averaging the dot positions on each line, so that the best fit lines are robust against individual fluctuations in the individual dot positions. This increases the stability and accuracy of the 6D position value.

In 36 The best-fit lines for a reading field with 15x15 dots are shown, the lines from bunch ₀ as 36 and the lines from bunch ₁ as 37.

Step 420 - Determining the bunch data

In this step, the three line parameters SX _ki , SY _ki , α _ki of each bundle of regression lines 36, 37 are interpolated with a second degree polynomial. The interpolation process only takes valid lines into account. Invalid lines, for which there are too few valid dots available for interpolation, are excluded from the bunch interpolation. For this purpose, a weighted interpolation is carried out using standard mathematical methods, which is based on the minimization of error squares. Valid lines are given a weight of 1.0, while invalid lines are given a weight of 0.0 and are thus hidden.

• • SX_k(i) = psx_k0 + i · psx_k1 + i² · psx_k2
• • SY_k(i) = psy_k0 + i · psy_k1 + i² · psy_k2
• • α_k(i) = pα_k0 + i · pα_k1 + i² · pα_k2

Approach interpolation function:

• • SX _k (i) = psx _k0 + i · psx _k1 + i ² · psx _k2
• • SY _k (i) = psy _k0 + i · psy _k1 + i ² · psy _k2
• • α _k (i) = pα _k0 + i · pα _k1 + i ² · pα _k2

Here, i is the consecutive number of the straight line (line index) in the respective straight line bundle.

General approach to interpolation of line parameters: $$P_k\left(i\right)={\mathrm{<figure-callout\; id="0"\; label="p\; k"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">p</figure-callout>}}_k+i\cdot {\mathrm{<figure-callout\; id="1"\; label="p\; k"\; filenames="DE102023203159A1\_0157.png,DE102023203159A1\_0158.png"\; state="\{\{state\}\}">p</figure-callout>}}_k+{\mathrm{<figure-callout\; id="2"\; label="i"\; filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png"\; state="\{\{state\}\}">i</figure-callout>}}^2\cdot {\mathrm{<figure-callout\; id="2"\; label="p\; k"\; filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png"\; state="\{\{state\}\}">p</figure-callout>}}_k;\text{with}{P}_k\left(i\right)=S{X}_k\left(i\right),S{Y}_k\left(i\right)\text{or}{\alpha }_k\left(i\right)$$

• • In Bündel 1 wird eine Referenz-Gerade ausgewählt, die sich nahe am Mittelpunkt 32 des Lesefeldes 22 befindet.
• • Falls der Winkel der Referenz-Geraden im Bereich $\left[\frac{-1}{4}\pi ,\frac{1}{4}\pi \right]$
  
  oder $\left[\frac{3}{4}\pi ,\frac{5}{4}\pi \right]$
  
  liegt (Sektoren auf der X_I-Achse in 38a)), handelt es sich um eine flache Gerade. Dann wird für alle Geraden in Bündel 0 (36) der Schnittpunkt SX_0i mit der X-Achse spezifiziert und für alle Geraden in Bündel 1 (37) der Schnittpunkt SY_1i mit der Y-Achse (38b)). Als Merker wird rot_status auf 0 gesetzt.
• • Falls der Winkel der Referenz-Geraden außerhalb des genannten Bereiches liegt (Sektoren auf der Y_I-Achse in 38 a)), handelt es sich um eine steile Gerade. Dann wird für alle Geraden in Bündel 0 (36) der Schnittpunkt SY_0i mit der Y-Achse spezifiziert und für alle Geraden in Bündel 1 (37) der Schnittpunkt SX_1i mit der X-Achse ( 38c)). Als Merker wird rot_status auf 1 gesetzt.

For a mathematically unambiguous description of a straight line, it is sufficient to specify only one of the two axis intersection points SX _ki or SY _ki together with the line angle α _ki . Ideally, the intersection point is specified with the axis that is as perpendicular to the straight line as possible: for flat straight lines, the intersection point with the Y axis is preferably specified, and for steep straight lines, the intersection point with the X axis. Therefore, the following procedure is used:

• • In bundle 1, a reference line is selected which is located close to the center 32 of the reading field 22.
• • If the angle of the reference line is in the range $\left[\frac{-1}{4}\pi ,\frac{1}{4}\pi \right]$
  
  or $\left[\frac{3}{4}\pi ,\frac{5}{4}\pi \right]$
  
  (sectors on the X _I axis in 38a) ), it is a flat line. Then the intersection point SX _0i with the X-axis is specified for all lines in bundle 0 (36) and the intersection point SY _1i with the Y-axis ( 38b) ). As a flag, rot_status is set to 0.
• • If the angle of the reference line is outside the specified range (sectors on the Y _I axis in 38 a) ), it is a steep line. Then the intersection point SY _0i with the Y-axis is specified for all lines in bundle 0 (36) and the intersection point SX _1i with the X-axis ( 38c )). As a flag, rot_status is set to 1.

This step further reduces the amount of data: a bundle of straight lines is described by only 6 parameters, three parameters for α _ki and three parameters for one of the axis intersection points, SX _ki or SY _ki . For two bundles, there are 12 parameters plus the marker rot_status.

Two bundles 36, 37 are always described by 12 parameters and the Boolean variable rot_status, regardless of the number of straight lines or the size of the measuring field. In the next step, the 6D position is calculated from these parameters.

The interpolation also has the effect of averaging the straight lines of each bundle and thus all dots, so that the interpolated values are robust against individual fluctuations of individual straight lines or dots. This increases the stability and accuracy of the 6D position value.

Step 6 - Calculating the 6D camera position

• • Teilschritt 500: Position r_X und r_Y
• • Teilschritt 710: Berechnung von Kennwerten des verzerrten Rasters
• • Teilschritt 600: Kamera-Winkel φ_Z
• • Teilschritt 700: Position r_z
• • Teilschritt 700: Kamera-Winkel φ_X und φ_Y
• • Teilschritt 700: Iterativer Algorithmus zur Lösung des Gleichungssystems für φ_X, φ_Y und r_Z

From the 12 interpolation parameters of the straight line bundles (Table 9.1), the 6 coordinates of the camera position are calculated in the following order:

• • Substep 500: Position r _X and r _Y
• • Substep 710: Calculation of characteristics of the distorted grid
• • Substep 600: Camera angle φ _Z
• • Substep 700: Position r _z
• • Substep 700: Camera angle φ _X and φ _Y
• • Substep 700: Iterative algorithm for solving the system of equations for φ _X , φ _Y and r _Z

The calculation includes the derivation of parameters that describe the measured grid at the location of the optical axis 5. These parameters are linked to the 6D camera position via a system of equations that results from the imaging model and the laws of ray optics.

The camera position is determined by solving the system of equations. In addition, the validity of the position values is estimated by evaluating a large number of diagnostic results from the individual steps of the program sequence.

To substep 500 - Calculating the camera position r _X and r _Y

$$r_Y=\left(r_{Y,int}+r_{Y,fract}\right)\cdot c_{grid};\text{Einheit: }\left[\text{m}\right]$$

$$\begin{array}{l}\begin{array}{cc}\text{mit }\left(r_{X,int},r_{Y,int}\right)& \text{: Ganzzahlige Position des Lesefeldes in }\left[\text{Dot}\right],\text{ wie in}\end{array}\\ \text{Schritt 300 ermittelt}\end{array}$$

$$\begin{array}{cc}\left(r_{X,fract},r_{Y,fract}\right)& :\text{ reeller Bruchteil der Lesefeld-Position in }\left[\text{Dot}\right]\end{array}$$

$$c_{grid}:\text{Dot}-\text{Rasterabstand in }\left[\text{m}/\text{Dot}\right].$$

The camera position in X and Y is calculated as the sum of the integer coarse position (r _X,int , r _Y,int ) and a real fraction [0..1] (fine position), multiplied by the dot grid spacing: $$r_X=\left(r_{X,int}+r_{X,fract}\right)\cdot c_{Grid};\text{Unit:}\left[\text{m}\right]$$

$$r_Y=\left(r_{Y,int}+r_{Y,fract}\right)\cdot c_{Grid};\text{Unit:}\left[\text{m}\right]$$

$$\begin{array}{l}\begin{array}{cc}\text{with}\left(r_{X,int},r_{Y,int}\right)& \text{: Integer position of the reading field in}\left[\text{Dot}\right],\text{as in}\end{array}\\ \text{<figure-callout id="300" label="Step" filenames="DE102023203159A1\_0153.png" state="{{state}}">Step</figure-callout> 300 determined}\end{array}$$

$$\begin{array}{cc}\left(r_{X,fract},r_{Y,fract}\right)& :\text{real fraction of the reading field position in}\left[\text{Dot}\right]\end{array}$$

$$c_{Grid}:\text{Dot}-\text{Grid spacing in}\left[\text{m}/\text{Dot}\right].$$

The reference point for measuring the camera position is the intersection point 4 of the optical axis 5 with the code arrangement 2, in the camera image 3 the intersection point of the optical axis 5 with the image sensor 12 ( 2 , point (4) or image of the intersection point 4 Image center 4 in 39 ). This image center IC (4) is determined by the position of the lens 9 in relation to the camera chip; it does not have to be identical to the center of the camera chip. It is determined in a calibration process and stored in the program as a constant two-dimensional vector (IC _X , IC _Y ). We calculate the fractions r _X,fract and r _Y,fract as the intersection point of the interpolated line with the image center 4. For the two beams of rays k = 0 and k = 1 (36, 37), the interpolation equation for the axis intersection point applies: $$\begin{array}{l}{S}_k\left(i\right)=p{s}_{\mathrm{<figure-callout\; id="0"\; label="k"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">k</figure-callout>}0}+i\cdot p{s}_{\mathrm{<figure-callout\; id="1"\; label="k"\; filenames="DE102023203159A1\_0157.png,DE102023203159A1\_0158.png"\; state="\{\{state\}\}">k</figure-callout>}1}+{\mathrm{<figure-callout\; id="2"\; label="i"\; filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png"\; state="\{\{state\}\}">i</figure-callout>}}^2\cdot p{s}_{\mathrm{<figure-callout\; id="2"\; label="k"\; filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png"\; state="\{\{state\}\}">k</figure-callout>}2};\text{with}{S}_k\left(i\right)=S{X}_k\left(i\right)\text{or}S{Y}_k\left(i\right),\text{depending on}\\ \text{red\_status}.\end{array}$$

$$·\text{ für rot}\_\text{status}=1:\left(\begin{array}{l}{S}_0\left(r_{X,fract}\right)\\ S_1\left(r_{Y,fract}\right)\end{array}\right)=\left(\begin{array}{l}I{C}_Y\\ I{C}_X\end{array}\right)$$

Figure 39 shows the center lines for both bundles of lines k = 0 and k = 1.
It is $$·\text{for red}\_\text{status}=0:\left(\begin{array}{l}{S}_0\left(r_{X,fract}\right)\\ S_1\left(r_{Y,fract}\right)\end{array}\right)=\left(\begin{array}{l}I{C}_X\\ I{C}_Y\end{array}\right)$$

$$·\text{for red}\_\text{status}=1:\left(\begin{array}{l}{S}_0\left(r_{X,fract}\right)\\ S_1\left(r_{Y,fract}\right)\end{array}\right)=\left(\begin{array}{l}I{C}_Y\\ I{C}_X\end{array}\right)$$

verschoben. Dies wird erfüllt durch die Gleichungen $$·\text{ für Geradenbündel }0:r_{X,fract}{}^2+r_{X,fract}{}^2\cdot \frac{p{s}_{01}}{p{s}_{01}}+\frac{\left(p{s}_{00}-T_X\right)}{p{s}_{02}}=0$$

$$·\text{ für Geradenbündel 1}:r_{Y,fract}{}^2+r_{Y,fract}{}^2\cdot \frac{p{s}_{11}}{p{s}_{12}}+\frac{\left(p{s}_{10}-T_Y\right)}{p{s}_{12}}=0$$

$$·\text{ mit }\left(T_X,T_Y\right)=\{\begin{array}{cc}\left(I{C}_X,I{C}_Y\right)& fürrot\_status=0\\ \left(I{C}_Y,I{C}_X\right)& fürrot\_status=1\end{array}$$

By suitable choice of the interpolation parameters (i, j) the axis intersection point for both center lines is placed in the image center $\overrightarrow{IC}$

This is fulfilled by the equations $$·\text{for straight line bundles}0:r_{X,fra\mathrm{<figure-callout\; id="2"\; label="c\; t"\; filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png"\; state="\{\{state\}\}">c</figure-callout>}t}{}^2+r_{X,fra\mathrm{<figure-callout\; id="2"\; label="c\; t"\; filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png"\; state="\{\{state\}\}">c</figure-callout>}t}{}^2\cdot \frac{p{s}_{01}}{p{s}_{01}}+\frac{\left(p{s}_{00}-T_X\right)}{p{s}_{02}}=0$$

$$·\text{for straight line bundle 1}:r_{Y,fra\mathrm{<figure-callout\; id="2"\; label="c\; t"\; filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png"\; state="\{\{state\}\}">c</figure-callout>}t}{}^2+r_{Y,fra\mathrm{<figure-callout\; id="2"\; label="c\; t"\; filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png"\; state="\{\{state\}\}">c</figure-callout>}t}{}^2\cdot \frac{p{s}_{11}}{p{s}_{12}}+\frac{\left(p{s}_{10}-T_Y\right)}{p{s}_{12}}=0$$

$$·\text{with}\left(T_X,T_Y\right)=\{\begin{array}{cc}\left(I{C}_X,I{C}_Y\right)& fürrOt\_status=0\\ \left(I{C}_Y,I{C}_X\right)& fürrOt\_status=1\end{array}$$

$$·r_{Y,fract}=\frac{-1}{2\cdot p{s}_{12}}\left(sign\left(p{s}_{11}\right)\cdot \sqrt{p{s}_{11}^2-4\cdot p{s}_{12}\cdot \left(p{s}_{10}-T_Y\right)}-p{s}_{11}\right)$$

The solution of the equations leads to the desired fractions of the line indices r _X,fract and r _Y,fract : $$·r_{X,fract}=\frac{-1}{2\cdot p{s}_{02}}\left(siGn\left(p{s}_{01}\right)\cdot \sqrt{p{s}_{01}^2-4\cdot p{s}_{02}\cdot \left(p{s}_{00}-T_X\right)}-p{s}_{01}\right)$$

$$·r_{Y,fract}=\frac{-1}{2\cdot p{s}_{12}}\left(siGn\left(p{s}_{11}\right)\cdot \sqrt{p{s}_{11}^2-4\cdot p{s}_{12}\cdot \left(p{s}_{10}-T_Y\right)}-p{s}_{11}\right)$$

To substep 710 - Calculation of characteristics of the distorted grid

Further parameters that characterize the distorted raster are obtained from the interpolation equations. The fine positions r _X,fract and r _Y,fract are inserted into the interpolation equations to obtain the parameters in the image center. The parameters are required to determine the position in the dimensions r _Z , φ _x , φ _Y and φ _Z .

a) Grid size of the axes intersection points of the straight lines for both bundles of straight lines: g _c0 , g _c1 .

$$\text{mit }{r}_{fract}=\{\begin{array}{l}{r}_{X,fract}fork=0\\ r_{Y,fract}fork=1\end{array}$$

The grid dimension is determined by deriving the interpolation function for the axis intersection point S _k (i) with respect to the dimensionless straight line index i at the location of the image center: $$\text{for}bunc{h}_k:G_{Ck}=p_{Grid}\cdot {\frac{\partial s_k\left(i\right)}{{\partial }_i}|}_{i=r_{fract}}=p_{Grid}\cdot \left(p{s}_{\mathrm{<figure-callout\; id="1"\; label="k"\; filenames="DE102023203159A1\_0157.png,DE102023203159A1\_0158.png"\; state="\{\{state\}\}">k</figure-callout>}1}+2\cdot r_{fract}\cdot p{s}_{k2}\right)$$

$$\text{with}{r}_{fract}=\{\begin{array}{l}{r}_{X,fract}fOrk=0\\ r_{Y,fract}fOrk=1\end{array}$$

b) Straight line angle in the image center: α _c0 , α _c1 (see 37 ), results from the interpolation function for the straight line angle: $$\text{for}bunc{h}_k:G_{Ck}={\alpha }_k\left(r_{fract}\right)={\mathrm{<figure-callout\; id="0"\; label="\alpha \; k"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">\alpha </figure-callout>}}_k+r_{fract}\cdot {\alpha }_{\mathrm{<figure-callout\; id="1"\; label="k"\; filenames="DE102023203159A1\_0157.png,DE102023203159A1\_0158.png"\; state="\{\{state\}\}">k</figure-callout>}1}+r_{fra\mathrm{<figure-callout\; id="2"\; label="c\; t"\; filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png"\; state="\{\{state\}\}">c</figure-callout>}t}^2\cdot {\mathrm{<figure-callout\; id="2"\; label="\alpha \; k"\; filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png"\; state="\{\{state\}\}">\alpha </figure-callout>}}_k^2$$

c) Angular divergence of the bundles of straight lines in the image center: dα _C0 , dα _C1 . This is the angle difference between adjacent straight lines close to the image center. It results from the derivative Determination of the interpolation function for the line angle according to the dimensionless line index i at the location of the image center: $$\text{for}bunc{h}_k:d{\alpha }_{Ck}={\frac{\partial {\alpha }_k\left(i\right)}{{\partial }_i}|}_{i=r_{fract}}={\mathrm{<figure-callout\; id="1"\; label="\alpha \; k"\; filenames="DE102023203159A1\_0157.png,DE102023203159A1\_0158.png"\; state="\{\{state\}\}">\alpha </figure-callout>}}_k+2\cdot r_{fract}\cdot {\alpha }_{\mathrm{<figure-callout\; id="2"\; label="k"\; filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png"\; state="\{\{state\}\}">k</figure-callout>}2}$$

To substep 600 - Calculation of the camera angle φ _Z

We determine the straight line angle φ _Z from the mapping equation derived in 2.4.2: $$\begin{array}{l}\left(\begin{array}{l}{B}_X\\ B_Y\end{array}\right)=\frac{\text{b}}{\left(R_{31}{x}_0+R_{32}{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+r_Z\right)}\cdot \left(\begin{array}{l}{R}_{11}{x}_0+R_{12}{y}_0\\ R_{21}{x}_0+R_{22}{y}_0\end{array}\right);\text{with}{r}_Z={\mathrm{<figure-callout\; id="0"\; label="z"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">z</figure-callout>}}_0-b:\text{distance code level}/\\ \text{opt}.\text{center}\end{array}$$

By deriving the image coordinates B _X and B _Y with respect to the code raster coordinate x ₀ we obtain $$\begin{array}{cc}\frac{\partial B_X}{\partial x_0}=b\frac{\left(R_{11}{\mathrm{<figure-callout\; id="32"\; label="R"\; filenames="DE102023203159A1\_0158.png,DE102023203159A1\_0165.png"\; state="\{\{state\}\}">R</figure-callout>}}_{32}-R_{12}{R}_{31}\right){\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+R_{11}{r}_Z}{{\left(R_{31}x+R_{32}{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+r_Z\right)}^2};& \frac{\partial B_Y}{\partial x_0}=b\frac{\left(R_{21}{\mathrm{<figure-callout\; id="32"\; label="R"\; filenames="DE102023203159A1\_0158.png,DE102023203159A1\_0165.png"\; state="\{\{state\}\}">R</figure-callout>}}_{32}-R_{22}{R}_{31}\right){\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+R_{21}{r}_Z}{{\left(R_{31}x+R_{32}{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+r_Z\right)}^2}\end{array}$$

The slope of a straight line of the straight line bundle bunch ₁ in the camera image is $${\mathrm{<figure-callout\; id="1"\; label="m"\; filenames="DE102023203159A1\_0157.png,DE102023203159A1\_0158.png"\; state="\{\{state\}\}">m</figure-callout>}}_1=\frac{\partial B_Y/\partial x_0}{\partial B_X/\partial x_0}=\frac{\left(R_{21}{\mathrm{<figure-callout\; id="32"\; label="R"\; filenames="DE102023203159A1\_0158.png,DE102023203159A1\_0165.png"\; state="\{\{state\}\}">R</figure-callout>}}_{32}-R_{22}{R}_{31}\right){\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+R_{21}{r}_Z}{\left(R_{11}{\mathrm{<figure-callout\; id="32"\; label="R"\; filenames="DE102023203159A1\_0158.png,DE102023203159A1\_0165.png"\; state="\{\{state\}\}">R</figure-callout>}}_{32}-R_{12}{R}_{31}\right){\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+R_{11}{r}_Z}=\frac{\left(\text{sin}{\phi }_X\text{sin}{\phi }_Z-\text{cos}{\phi }_X\text{sin}{\phi }_Y\text{cos}{\phi }_Z\right)\cdot {\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0-\text{cos}{\phi }_Y\text{sin}{\phi }_Z\cdot r_Z}{\left(-\text{sin}{\phi }_X\text{cos}{\phi }_Z-\text{cos}{\phi }_X\text{sin}{\phi }_Y\text{sin}{\phi }_Z\right)\cdot {\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+\text{cos}{\phi }_{Y}cOs{\phi }_Z\cdot r_Z}$$

We consider the slope in the center at y ₀ = 0 and obtain m ₁ | _{y 0 =0} = - tan(φ _Z ). m ₁ | _{y 0 =0} is the slope of the line from bunch ₁ in the image center. It corresponds to the characteristic value α _C1 from 6.2 b), ie m ₁ | _{y 0 =0} = tan(α _C1 ).

This leads to the equation for the camera angle φ _Z : φ _Z = α _C1

To substep 700 - Calculation of the camera position r _Z

The camera position r _Z is determined from the grid spacing of the axis intersection points g _Ci for both straight line bundles bunch _i . Since there are two straight line bundles in each image, two r _Z positions can be determined in principle. With the cases rot_status = k, four values r _Zik with (i, k = 0..1) result. The mapping equation for 9 : $$\begin{array}{l}\begin{array}{cc}\left(\begin{array}{c}{B}_X\\ B_Y\end{array}\right)=\frac{\text{b}}{\left(R_{31}{x}_0+R_{32}{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+r_Z\right)}\cdot \left(\begin{array}{c}{R}_{11}{x}_0+R_{12}{y}_0\\ R_{21}{x}_0+R_{22}{y}_0\end{array}\right)& \text{; with}{r}_Z=b-z_0:\text{distance code level}/\end{array}\\ \text{opt}\text{. center}\end{array}$$

$$

For example, the value r _Z00 is determined from the measured grid spacing g _C0 of the axis intersections on the B _X axis for rot_status = 0. On the B _X axis, the following applies: B _Y = 0, from the mapping equation follows $R_{21}{x}_0+R_{22}{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0=0<figure-callout\; id="0"\; label="\Rightarrow \; y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">\Rightarrow </figure-callout>y_{}{}_0=-\frac{R_{21}}{R_{22}}{x}_0$

$$

$$

By inserting this term into the equation for B _X we get $B_X=\frac{b\cdot \left(R_{11}{\mathrm{<figure-callout\; id="22"\; label="R"\; filenames="DE102023203159A1\_0159.png,DE102023203159A1\_0160.png"\; state="\{\{state\}\}">R</figure-callout>}}_{22}-R_{12}{R}_{21}\right)x_0}{\left(R_{22}{\mathrm{<figure-callout\; id="31"\; label="R"\; filenames="DE102023203159A1\_0172.png"\; state="\{\{state\}\}">R</figure-callout>}}_{31}-R_{21}{R}_{32}\right)x_0+R_{22}{\mathrm{<figure-callout\; id="00"\; label="r\; Z"\; filenames="DE102023203159A1\_0158.png"\; state="\{\{state\}\}">r</figure-callout>}}_Z}$

$$

bei $x_0^{\prime }=0;$

mit $x_0^{\prime }=\frac{x_0}{c_{grid}}$

Geradenindex in [dots]
Daraus folgt $\frac{\partial B_X}{\partial x_0^{\prime }}=\frac{\partial B_X}{\partial x_0}\cdot c_{grid}=\frac{b\cdot c_{grid}\cdot \left(R_{11}{R}_{22}-R_{12}{R}_{21}\right)R_{22}\cdot r_{Z00}}{{\left(\left(R_{22}{R}_{31}-R_{21}{R}_{32}\right)x_0+R_{22}{r}_{Z00}\right)}^2}$

Bei x₀ = 0 erhalten wir $g_{C0}={\frac{\partial B_X}{\partial x_0^{\prime }}|}_{x_0=0}=\frac{b\cdot c_{grid}\cdot \left(R_{11}{R}_{22}-R_{12}{R}_{21}\right)}{R_{22}{r}_{Z00}},$

und aufgelöst nach $$\begin{array}{l}{r}_{Z00}:\\ r_{Z00}=\frac{b\cdot c_{grid}\cdot \left(R_{11}{R}_{22}-R_{12}{R}_{21}\right)}{g_{C0}\cdot R_{22}}\end{array}$$

The grid spacing g _C0 corresponds to the derivative $\frac{\partial B_X}{\partial x_0^{\prime }}$

at $x_0^{\prime }=0;$

with ${\mathrm{<figure-callout\; id="0"\; label="x"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">x</figure-callout>}}_0^{\prime }=\frac{x_0}{c_{Grid}}$

line index in [dots]
It follows $\frac{\partial B_X}{\partial x_0^{\prime }}=\frac{\partial B_X}{\partial x_0}\cdot c_{Grid}=\frac{b\cdot c_{Grid}\cdot \left(R_{11}{\mathrm{<figure-callout\; id="22"\; label="R"\; filenames="DE102023203159A1\_0159.png,DE102023203159A1\_0160.png"\; state="\{\{state\}\}">R</figure-callout>}}_{22}-R_{12}{R}_{21}\right){\mathrm{<figure-callout\; id="22"\; label="R"\; filenames="DE102023203159A1\_0159.png,DE102023203159A1\_0160.png"\; state="\{\{state\}\}">R</figure-callout>}}_{22}\cdot r_{Z00}}{{\left(\left(R_{22}{\mathrm{<figure-callout\; id="31"\; label="R"\; filenames="DE102023203159A1\_0172.png"\; state="\{\{state\}\}">R</figure-callout>}}_{31}-R_{21}{R}_{32}\right)x_0+R_{22}{r}_{Z00}\right)}^2}$

For x ₀ = 0 we get $G_{C0}={\frac{\partial B_X}{\partial x_0^{\prime }}|}_{x_0=0}=\frac{b\cdot c_{Grid}\cdot \left(R_{11}{R}_{22}-R_{12}{R}_{21}\right)}{R_{22}{r}_{Z00}},$

and dissolved after $$\begin{array}{l}{r}_{Z00}:\\ {\mathrm{<figure-callout\; id="00"\; label="r\; Z"\; filenames="DE102023203159A1\_0158.png"\; state="\{\{state\}\}">r</figure-callout>}}_Z=\frac{b\cdot c_{Grid}\cdot \left(R_{11}{\mathrm{<figure-callout\; id="22"\; label="R"\; filenames="DE102023203159A1\_0159.png,DE102023203159A1\_0160.png"\; state="\{\{state\}\}">R</figure-callout>}}_{22}-R_{12}{R}_{21}\right)}{{\mathrm{<figure-callout\; id="0"\; label="G\; C"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">G</figure-callout>}}_C\cdot {\mathrm{<figure-callout\; id="22"\; label="R"\; filenames="DE102023203159A1\_0159.png,DE102023203159A1\_0160.png"\; state="\{\{state\}\}">R</figure-callout>}}_{22}}\end{array}$$

By inserting the matrix elements R _ij we obtain the equation for r _Z00 : $${\mathrm{<figure-callout\; id="00"\; label="r\; Z"\; filenames="DE102023203159A1\_0158.png"\; state="\{\{state\}\}">r</figure-callout>}}_Z=\frac{b\cdot c_{Grid}\cdot \text{cos}{\phi }_{\mathrm{<figure-callout\; id="0"\; label="Y\; G\; C"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">Y</figure-callout>}}}{G_C}$$

In the same way, the position r _Z01 is calculated for rot_status=1, approach: ${\mathrm{<figure-callout\; id="0"\; label="G\; C"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">G</figure-callout>}}_C={\frac{\partial B_Y}{\partial x_0^{\prime }}|}_{x_0=0}$

$$

für $r_{Z11}:g_{C1}={\frac{\partial B_X}{\partial y_0^{\prime }}|}_{y_0=0}$

The positions r _Z1k are derived from g _C1 , approach for $r_{Z10}:G_{C1}={\frac{\partial B_Y}{\partial y_0^{\prime }}|}_{y_0=0};$

for $r_{Z11}:{\mathrm{<figure-callout\; id="1"\; label="G\; C"\; filenames="DE102023203159A1\_0157.png,DE102023203159A1\_0158.png"\; state="\{\{state\}\}">G</figure-callout>}}_C={\frac{\partial B_X}{\partial {\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0^{\prime }}|}_{{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0=0}$

$r_{Z10}=\frac{b\cdot c_{grid}\cdot \text{cos }{\phi }_X}{g_{C1}\cdot \text{cos }{\phi }_Z}$

1 $\begin{array}{l}{r}_{Z01}\\ =\frac{b\cdot c_{grid}\cdot \text{cos }{\phi }_Y}{g_{C0}\cdot \left(-\text{sin }{\phi }_Z-\text{tan }{\phi }_X\text{sin }{\phi }_Y\text{cos }{\phi }_Z\right)}\end{array}$

$r_{Z11}=\frac{b\cdot c_{grid}\cdot \text{cos }{\phi }_X}{g_{C1}\cdot \text{sin }{\phi }_Z}$

The results are summarized in Table 10.1. Table: Calculation of the camera position values r _Zik red_status r _Z0k (derived from bunch ₀ ) r _Z1k (derived from bunch ₁ ) 0 $\begin{array}{l}{\mathrm{<figure-callout\; id="00"\; label="r\; Z"\; filenames="DE102023203159A1\_0158.png"\; state="\{\{state\}\}">r</figure-callout>}}_Z\\ =\frac{b\cdot c_{Grid}\cdot \text{cos}{\phi }_{\mathrm{<figure-callout\; id="0"\; label="Y\; G\; C"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">Y</figure-callout>}}}{G_C}\end{array}$

${\mathrm{<figure-callout\; id="10"\; label="r\; Z"\; filenames="DE102023203159A1\_0155.png,DE102023203159A1\_0156.png"\; state="\{\{state\}\}">r</figure-callout>}}_Z=\frac{b\cdot c_{Grid}\cdot \text{cos}{\phi }_{\mathrm{<figure-callout\; id="1"\; label="X\; G\; C"\; filenames="DE102023203159A1\_0157.png,DE102023203159A1\_0158.png"\; state="\{\{state\}\}">X</figure-callout>}}}{G_C}$

1 $\begin{array}{l}{r}_{Z01}\\ =\frac{b\cdot c_{Grid}\cdot \text{cos}{\phi }_{\mathrm{<figure-callout\; id="0"\; label="Y\; G\; C"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">Y</figure-callout>}}}{G_C}\end{array}$

${\mathrm{<figure-callout\; id="11"\; label="r\; Z"\; filenames="DE102023203159A1\_0155.png,DE102023203159A1\_0156.png"\; state="\{\{state\}\}">r</figure-callout>}}_Z=\frac{b\cdot c_{Grid}\cdot \text{cos}{\phi }_{\mathrm{<figure-callout\; id="1"\; label="X\; G\; C"\; filenames="DE102023203159A1\_0157.png,DE102023203159A1\_0158.png"\; state="\{\{state\}\}">X</figure-callout>}}}{G_C}$

können die Gleichungen zusammengefasst werden: $$\begin{array}{cc}{r}_{Z0k}=\frac{b\cdot c_{grid}\cdot \text{cos }{\phi }_Y}{g_{C0}\cdot \left(\text{cos}\left({\phi }_Z+\beta \right)-\text{tan }{\phi }_X\text{sin }{\phi }_Y\text{sin}\left({\phi }_Z+\beta \right)\right)}& r_{Z1k}=\frac{b\cdot c_{grid}\cdot \text{cos }{\phi }_X}{g_{C1}\cdot \text{cos}\left({\phi }_Z-\beta \right)}\end{array}$$

By introducing the angle $\beta =rOt\_status\cdot \frac{\mathrm{<figure-callout\; id="2"\; label="\pi "\; filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png"\; state="\{\{state\}\}">\pi </figure-callout>}}{2}$

the equations can be summarized: $$\begin{array}{cc}{r}_{Z0k}=\frac{b\cdot c_{Grid}\cdot \text{cos}{\phi }_{\mathrm{<figure-callout\; id="0"\; label="Y\; G\; C"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">Y</figure-callout>}}}{G_C}& r_{Z1k}=\frac{b\cdot c_{Grid}\cdot \text{cos}{\phi }_{\mathrm{<figure-callout\; id="1"\; label="X\; G\; C"\; filenames="DE102023203159A1\_0157.png,DE102023203159A1\_0158.png"\; state="\{\{state\}\}">X</figure-callout>}}}{G_C}\end{array}$$

r_Z To output only one value r _Z per image, the mean of both z-values is calculated: $r_Z=\frac{r_{Z0k}+r_{Z1\mathrm{<figure-callout\; id="2"\; label="k"\; filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png"\; state="\{\{state\}\}">k</figure-callout>}}}{2}$

r _Z

To substep 700 - Calculation of the camera angles φ _X , φ _Y

der beiden Geradenbündel bunch_k im Bildzentrum abgeleitet. Im Schritt 600 haben wir die Geradensteigung m₁ für das Geradenbündel bunch₁ als Funktion des Geradenindex y₀ hergeleitet: $$m_1\left(y_0\right)=\frac{\left(\text{sin }{\phi }_X\text{sin }{\phi }_Z-\text{cos }{\phi }_X\text{sin }{\phi }_Y\text{cos }{\phi }_Z\right)\cdot y_0-\text{cos }{\phi }_Y\text{sin }{\phi }_Z\cdot r_Z}{\left(-\text{sin }{\phi }_X\text{cos }{\phi }_Z-\text{cos }{\phi }_X\text{sin }{\phi }_Y\text{sin }{\phi }_Z\right)\cdot y_0+\text{cos }{\phi }_{Y}cos{\phi }_Z\cdot r_Z}$$

The angles φ _X , φ _Y are calculated from the angle divergence $r_Z=\frac{r_{Z0k}+r_{Z1\mathrm{<figure-callout\; id="2"\; label="k"\; filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png"\; state="\{\{state\}\}">k</figure-callout>}}}{2}$

of the two straight line bundles bunch _k in the image center. In step 600 we derived the straight line slope m ₁ for the straight line bundle bunch ₁ as a function of the straight line index y ₀ : $$m_1\left(y_0\right)=\frac{\left(\text{sin}{\phi }_X\text{sin}{\phi }_Z-\text{cos}{\phi }_X\text{sin}{\phi }_Y\text{cos}{\phi }_Z\right)\cdot {\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0-\text{cos}{\phi }_Y\text{sin}{\phi }_Z\cdot r_Z}{\left(-\text{sin}{\phi }_X\text{cos}{\phi }_Z-\text{cos}{\phi }_X\text{sin}{\phi }_Y\text{sin}{\phi }_Z\right)\cdot {\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0+\text{cos}{\phi }_{Y}cOs{\phi }_Z\cdot r_Z}$$

[dots] ab, um die Winkeldivergenz zu erhalten: $$\frac{\partial {\alpha }_1\left(y_0\right)}{\partial y_0^{\prime }}=\frac{\partial {\alpha }_1\left(y_0\right)}{\partial y_0}\cdot c_{grid}=\frac{-c_{grid}\cdot \text{cos }{\phi }_X\text{sin }{\phi }_Y\text{cos }{\phi }_Y\cdot r_Z}{{\left(y_0\cdot \text{sin }{\phi }_X-r_Z\cdot \text{cos }{\phi }_Y\right)}^2+y_0^2\cdot {\text{cos}}^2{\phi }_X{\text{sin}}^2{\phi }_Y}$$

• ▪ Im Bildzentrum y₀ = 0 ist die Winkeldivergenz ${\frac{\partial {\alpha }_1\left(y_0\right)}{\partial {\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0^{\prime }}|}_{{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0=0}=\frac{-c_{grid}\cdot \text{cos }{\phi }_X\text{tan }{\phi }_Y}{r_Z}$
  
• ▪ Gleichsetzen mit der gemessenen Winkeldivergenz dα_C1 ergibt die Gleichung für $$\begin{array}{cc}{\phi }_Y:& \text{tan }{\phi }_Y=\frac{-d{\mathrm{<figure-callout\; id="1"\; label="\alpha \; C"\; filenames="DE102023203159A1\_0157.png,DE102023203159A1\_0158.png"\; state="\{\{state\}\}">\alpha </figure-callout>}}_C\cdot r_Z}{c_{grid}\cdot \text{cos }{\phi }_X}\end{array}$$
  

From this we calculate the straight line angle α ₁ (y ₀ ) = tan(m ₁ (y ₀ )) and derive it according to ${\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0^{\prime }=\frac{y_0}{c_{Grid}}$

[dots] to obtain the angular divergence: $$\frac{\partial {\alpha }_1\left(y_0\right)}{\partial {\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0^{\prime }}=\frac{\partial {\alpha }_1\left(y_0\right)}{\partial {\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0}\cdot c_{Grid}=\frac{-c_{Grid}\cdot \text{cos}{\phi }_X\text{sin}{\phi }_Y\text{cos}{\phi }_Y\cdot r_Z}{{\left({\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0\cdot \text{sin}{\phi }_X-r_Z\cdot \text{cos}{\phi }_Y\right)}^2+{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0^2\cdot {\text{<figure-callout id="2" label="cos" filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png" state="{{state}}">cos</figure-callout>}}^2{\mathrm{<figure-callout\; id="2"\; label="\phi \; X\; sin"\; filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png"\; state="\{\{state\}\}">\phi </figure-callout>}}_X{\text{sin}}^{}{}^2{\phi }_Y}$$

• ▪ At the image center y ₀ = 0 the angular divergence is ${\frac{\partial {\alpha }_1\left(y_0\right)}{\partial {\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0^{\prime }}|}_{{\mathrm{<figure-callout\; id="0"\; label="y"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">y</figure-callout>}}_0=0}=\frac{-c_{Grid}\cdot \text{cos}{\phi }_X\text{tan}{\phi }_Y}{r_Z}$
  
• ▪ Equating with the measured angular divergence dα _C1 gives the equation for $$\begin{array}{cc}{\phi }_Y:& \text{tan}{\phi }_Y=\frac{-d{\mathrm{<figure-callout\; id="1"\; label="\alpha \; C"\; filenames="DE102023203159A1\_0157.png,DE102023203159A1\_0158.png"\; state="\{\{state\}\}">\alpha </figure-callout>}}_C\cdot r_Z}{c_{Grid}\cdot \text{cos}{\phi }_X}\end{array}$$
  

• ▪ Wir leiten den Winkel α₀(x₀) = tan(m₀(x₀)) nach $x_0^{\prime }=\frac{x_0}{c_{grid}}$
  
  ab und erhalten die Winkeldivergenz $$\frac{\partial {\alpha }_0\left(x_0\right)}{\partial x_0^{\prime }}=\frac{\partial {\alpha }_0\left(x_0\right)}{\partial x_0}\cdot c_{grid}=\frac{-c_{grid}\cdot \text{sin }{\phi }_X\text{ cos }{\phi }_{\mathrm{<figure-callout\; id="2"\; label="X\; cos"\; filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png"\; state="\{\{state\}\}">X</figure-callout>}}{\text{ cos}}^{}{}^2{\phi }_Y\cdot r_Z}{{\left(x_0+r_Z\cdot \text{cos }{\phi }_X\right)}^2-x_0^2\cdot {\text{<figure-callout id="2" label="cos" filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png" state="{{state}}">cos</figure-callout>}}^2{\phi }_X}$$
  
• ▪ Im Bildzentrum x₀ = 0 ist die Winkeldivergenz ${\frac{\partial {\alpha }_0\left(x_0\right)}{\partial x_0^{\prime }}|}_{x_0=0}=\frac{-c_{grid}\cdot {\text{<figure-callout id="2" label="cos" filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png" state="{{state}}">cos</figure-callout>}}^2{\phi }_Y\text{ tan }{\phi }_X}{r_Z}$
  
• ▪ Gleichsetzen mit der gemessenen Winkeldivergenz dα_C0 ergibt die Gleichung für $$\begin{array}{cc}{\phi }_X:& \text{tan }{\phi }_X=\frac{-d{\mathrm{<figure-callout\; id="0"\; label="\alpha \; C"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">\alpha </figure-callout>}}_C\cdot r_Z}{c_{grid}\cdot {\text{<figure-callout id="2" label="cos" filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png" state="{{state}}">cos</figure-callout>}}^2{\phi }_Y}\end{array}$$
  

In the same way, the angle φ _X results from the slope m ₀ of a straight line in bunch ₀ . Analogous to the calculation in 600 we obtain $$m_0\left(x_0\right)=\frac{\left(-\text{sin}{\phi }_X\text{sin}{\phi }_Z+\text{cos}{\phi }_X\text{sin}{\phi }_Y\text{cos}{\phi }_Z\right)\cdot x_0+\left(\text{cos}{\phi }_X\text{cos}{\phi }_Z-\text{sin}{\phi }_X\text{sin}{\phi }_Y\text{sin}{\phi }_Z\right)\cdot r_Z}{\left(\text{sin}{\phi }_X\text{cos}{\phi }_Z+\text{cos}{\phi }_X\text{sin}{\phi }_Y\text{sin}{\phi }_Z\right)\cdot x_0+\left(\text{cos}{\phi }_X\text{sin}{\phi }_Z+\text{sin}{\phi }_X\text{sin}{\phi }_{Y}cOs{\phi }_Z\right)\cdot r_Z}$$

• ▪ We derive the angle α ₀ (x ₀ ) = tan(m ₀ (x ₀ )) according to $x_0^{\prime }=\frac{x_0}{c_{Grid}}$
  
  and obtain the angle divergence $$\frac{\partial {\alpha }_0\left(x_0\right)}{\partial x_0^{\prime }}=\frac{\partial {\alpha }_0\left(x_0\right)}{\partial x_0}\cdot c_{Grid}=\frac{-c_{Grid}\cdot \text{sin}{\phi }_X\text{cos}{\phi }_{\mathrm{<figure-callout\; id="2"\; label="X\; cos"\; filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png"\; state="\{\{state\}\}">X</figure-callout>}}{\text{cos}}^{}{}^2{\phi }_Y\cdot r_Z}{{\left(x_0+r_Z\cdot \text{cos}{\phi }_X\right)}^2-x_0^2\cdot {\text{<figure-callout id="2" label="cos" filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png" state="{{state}}">cos</figure-callout>}}^2{\phi }_X}$$
  
• ▪ At the image center x ₀ = 0 the angular divergence is ${\frac{\partial {\alpha }_0\left(x_0\right)}{\partial x_0^{\prime }}|}_{x_0=0}=\frac{-c_{Grid}\cdot {\text{<figure-callout id="2" label="cos" filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png" state="{{state}}">cos</figure-callout>}}^2{\phi }_Y\text{tan}{\phi }_X}{r_Z}$
  
• ▪ Equating with the measured angular divergence dα _C0 gives the equation for $$\begin{array}{cc}{\phi }_X:& \text{tan}{\phi }_X=\frac{-d{\mathrm{<figure-callout\; id="0"\; label="\alpha \; C"\; filenames="DE102023203159A1\_0156.png,DE102023203159A1\_0157.png"\; state="\{\{state\}\}">\alpha </figure-callout>}}_C\cdot r_Z}{c_{Grid}\cdot {\text{<figure-callout id="2" label="cos" filenames="DE102023203159A1\_0154.png,DE102023203159A1\_0155.png" state="{{state}}">cos</figure-callout>}}^2{\phi }_Y}\end{array}$$
  

To substep 700 - Iterative calculation of the camera position Z and the camera angles φ _X , φ _Y

There are mutual dependencies between the functions φ _X , φ _Y and Z: $$\begin{array}{ccc}{\phi }_X=f\left({\phi }_y,r_Z\right);& {\phi }_Y=f\left({\phi }_X,r_Z\right)& {\phi }_Z=f\left({\phi }_X,r_y\right)\end{array}$$

• Unterschritt 700.1: Berechnung der reellen Bruchteile der Lesefeld-Position in [Dot] (r_X,fract, r_Y,fract)
• Unterschritt 700.2: Berechnung der Position r_x und r_y
• Unterschritt 700.3: Berechnung der Kamerarotation/Kamerawinkels φ_Z
• Unterschritt 700.4: Initialisiere die Kamerawinkel phix = 0; phiy = 0
• Unterschritt 700.5: Initialisiere einen Zähler für Iterationen mit z.B. 4
• Unterschritt 700.6: Berechnung von r_ZOk und r_Z1k und schließlich r_Z=(r_Z0k+r_Z1k)/2
• Unterschritt 700.7: Berechnung von Kamerawinkel phix, phiy
• Unterschritt 700.8: Counter = 0? Neuberechnung, ansonsten Abbruch der Iteration
• Unterschritt 700.9: Fehlerabfrage

Therefore, the equations are solved in an iterative process. With each iteration step, the accuracy of the camera position Z and the camera angles φ _X and φ _Y increases. The algorithm is shown in the flow chart 40 shown.

• Substep 700.1: Calculation of the real fractions of the reading field position in [Dot] (r _X,fract , r _Y,fract )
• Substep 700.2: Calculating the position r _x and r _y
• Substep 700.3: Calculation of the camera rotation/camera angle φ _Z
• Substep 700.4: Initialize the camera angles phix = 0; phiy = 0
• Substep 700.5: Initialize a counter for iterations with e.g. 4
• Substep 700.6: Calculate r _ZOk and r _Z1k and finally r _Z =(r _Z0k +r _Z1k )/2
• Substep 700.7: Calculation of camera angle phix, phiy
• Substep 700.8: Counter = 0? Recalculation, otherwise abort the iteration
• Substep 700.9: Error query

• • Kameraposition (r_x, r_y, r_z)
• • Kamerawinkel (φ_X, φ_Y, φ_Z)
• • Validitätsinformationen

The algorithm converges quickly, after about 4 iterations the result is sufficiently stable. Finally, the following results are output:

• • Camera position (r _x , r _y , r _z )
• • Camera angle (φ _X , φ _Y , φ _Z )
• • Validity information

List of reference symbols:

1

camera

2

code arrangement

3

Picture

4

Intersection point of the optical axis 5 with the code arrangement 2; reference point;

5

optical axis

6

empty

7

intersection of the optical axis with the image sensor

8

Objects with at least one code arrangement 2

9

lens

10

focal point of the lens 9

11

focal point of the lens 9

12

image sensor

13

optical center of the lens 9

14

line of sight

15

Coordinate system (X _I, Y _I ) of the image sensor 12 in the plane of the image sensor 12

16

Coordinate system (X _C , Y _C ) of code arrangement 2 in the plane of code arrangement 2

17

coordinate system (X, Y, Z) of camera 1

18

virtual image plane

19

plot

20

base symbol

21

parcel symbol

22

reading field

23

center position

24

X-plot areas

25

Y-plot areas

26

parcel area for additional data

27

intersection pattern

28

launch pad

29

dot matrix

30

starting positions

31

search beams

32

center position of reading field 22

33

non-rectified dot matrix

34

rectified dot matrix

35

reading track in a first code direction

36

reading track in a second code direction

37

first straight line bundle/bunch ₀

38

second straight bundle/bunch ₁

QUOTES INCLUDED IN THE DESCRIPTION

This list of documents listed by the applicant was generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited patent literature

• DE 102016216221 A1 [0003, 0007, 0174]
• DE 102016216196 A1 [0004, 0007]

Claims (17)

Method for determining at least one degree of freedom of a camera (1) relative to a code arrangement (2) from a camera image of the camera (1), wherein the code arrangement (2) has a dot matrix with a plurality of base symbols (20), wherein a rough position of the base symbols (20) is encoded in the code arrangement (2), wherein a start field (28) is determined with base symbols (20), wherein the start field (28) has at least three base symbols (20), wherein two independent main directions along the dot matrix in the camera image are estimated via the base symbols (20) of the start field (28), wherein the base symbols (20) of the start field form valid base symbols, wherein in a search step, starting from at least one valid base symbol (20), further base symbols (20) are searched for along at least one of the main directions, and wherein, if the search is successful, the further base symbols (20) are considered valid base symbols (20), wherein the search step is carried out several times, wherein on the basis of the valid basic symbols (20) or a subset thereof the rough position of at least one of the basic symbols (20) in the code arrangement and/or the rough position of the coordinate system of the point grid is determined as two degrees of freedom of the camera relative to the code arrangement (2). procedure according to claim 1 , characterized in that the base symbols (20) of the start field (28) define two connection vectors in the two independent main directions along the point grid in the camera image. procedure according to claim 1 or 2 , characterized in that the respective center position of the base symbol (20) in the camera image is determined for the base symbols (20), wherein the search step is carried out starting from the center position of the respective base symbol (20). Method according to one of the preceding claims, characterized in that the area of the base symbols (20) in the camera image is determined, wherein the area forms a datum in the coding of the code arrangement (2). Method according to one of the preceding claims, characterized in that in a preliminary step for creating the starting field (28) at least one starting position (30) is determined and an adjacent base symbol (20) is searched for as a starting base symbol along search rays (31) starting from the starting position (30). procedure according to claim 5 , characterized in that starting from the position of the starting base symbol along search rays (31) an adjacent base symbol (20) is searched for as a first auxiliary starting base symbol. procedure according to claim 6 , characterized in that starting from the starting base symbol and the first auxiliary starting base symbol, a second auxiliary starting base symbol is searched for in a direction perpendicular to a connection between the starting base symbol and the first auxiliary starting base symbol. procedure according to claim 7 , characterized in that starting from the starting base symbol, all immediately adjacent base symbols (20) of the starting base symbol are searched for on the basis of the connection between the starting base symbol and the first auxiliary starting base symbol and the connection between the starting base symbol and the second auxiliary starting base symbol by linear combination of the connections. Method according to one of the preceding claims, characterized in that starting from the starting field (28) and/or the valid basic symbols (20), adjacent basic symbols (20) are searched for which have at least two valid neighbors in the main directions or in the diagonal directions. Method according to one of the preceding claims, characterized in that a reading field (22) is defined in the camera image and/or planar dot matrix and a basic symbol matrix corresponding thereto is formed, wherein each point in the reading field (22) is assigned an entry in the basic symbol matrix, wherein the position and the in particular standardized area or the datum of the basic symbols (20) are entered in the basic symbol matrix, wherein the rough position of the coordinate system of the dot matrix and/or of at least one basic symbol in the code arrangement is determined as the two degrees of freedom on the basis of the basic symbol matrix. Method according to one of the preceding claims, characterized in that on the basis of the basic symbol matrix a first straight line function with a first function argument is derived in a coordinate system of the camera image for a first straight line, wherein the first straight line in the coordinate system of the code arrangement (2) is aligned parallel to the first main direction, wherein by changing the first function argument the first straight line is shifted parallel in the coordinate system of the code arrangement (2) in the second main direction, and/or a second straight line function with a second function argument is derived in a coordinate system of the camera image for a second straight line, wherein the second straight line in the coordinate system of the code arrangement (2) is aligned parallel to the second main direction, wherein by changing the second function argument the second straight line is shifted parallel in the coordinate system of the code arrangement (2) in the first main direction, wherein at least one further degree of freedom of the camera is determined on the basis of at least one of the straight line functions. Method according to one of the preceding claims, characterized in that a reference point is arranged in the camera image, with a first axis intersection function in the coordinate system of the camera image of a first straight line, wherein the first straight line in the coordinate system (X _C , Y _C ) of the code arrangement is aligned parallel to the first main direction (2) of the dot matrix, with a first function argument, wherein by changing the first function argument the first straight line in the coordinate system (X _C , Y _C ) of the code arrangement is shifted parallel in the second main direction, wherein the first axis intersection function determines a first axis intersection along a first axis of the coordinate system (X _I, Y _I ) of the camera image depending on the first function argument, wherein the first axis runs through the reference point, with a second axis intersection function in the coordinate system of the camera image of a second straight line, wherein the second straight line in the coordinate system (X _C , Y _C ) of the code arrangement is aligned parallel to the second main direction, with a second function argument, wherein by changing the second function argument, the second straight line in the coordinate system (X _C , Y _C ) of the code arrangement is shifted parallel in the first main direction, wherein the second axis intersection function determines a second axis intersection along a second axis of the coordinate system (X _I, Y _I ) of the camera image as a function of the second function argument, wherein the second axis runs through the reference point, wherein on the basis of the axis intersection functions the first and the second function argument are determined such that the reference point forms the first and the second axis intersection, wherein on the basis of the first and the second function argument and the coarse position the fine position of the reference point in the coordinate system (X _C , Y _C ) of the code arrangement (2) in the plane of the code arrangement (2) is determined as at least one further degree of freedom. Method according to one of the Claims 11 or 12 , characterized in that the first and/or the second function argument is determined such that the first and the second straight lines intersect the reference point, wherein an angle of rotation (phiz) of the camera about the optical axis is derived as a degree of freedom of the camera (1) relative to the code arrangement (2) on the basis of the first and/or the second straight line. Method according to one of the preceding claims, characterized in that a or the reference point is arranged in the camera image, the reference point being formed as an intersection point of the optical axis of the camera with the image sensor and/or the camera image, the following values being determined as characteristic values from the camera image in the reference point: - a or the angle of rotation (phiz) of the camera about the optical axis of the camera; - a first local grid dimension (gc0) of the point grid in the camera image in the first main direction and/or a second local grid dimension (gc1) of the point grid in the camera image in the second main direction; - a first local angular divergence (dalphac0) of the dot matrix in the camera image in the first main direction, wherein the first local angular divergence (dalphac0) describes the difference angle between two adjacent straight lines in the first main direction in the dot matrix - a second local angular divergence (dalphac1) of the dot matrix in the camera image in the second main direction, wherein the second local angular divergence (dalphac1) describes the difference angle between two adjacent straight lines in the second main direction in the dot matrix; wherein the following further three degrees of freedom of the camera relative to the code arrangement are determined on the basis of the characteristic values: - a distance (rz) between the code arrangement and the camera; - two independent pitch angles (phix; phy) of the optical axis to the code arrangement. Electronic control unit or automation arrangement with the electronic control unit, wherein the control unit is designed in terms of programming and/or circuitry to carry out the method according to one of the preceding claims. Computer program, wherein the computer program is designed to carry out the method according to one of the Claims 1 until 14 to be carried out if the computer program is installed on a computer or on a control unit or the automation device according to claim 15 is executed. Machine-readable storage medium, on which the computer program is stored according to claim 16 is stored.

Images