DE102017000962A1 - Method for determining a head pose of a head - Google Patents

Abstract

The invention relates to a method for determining a head pose of a head (H). According to the invention, a head target (TH) is arranged on the head (H) and a head calibration unit (C) is arranged on the head (H) such that a calibration unit coordinate system (KKS) of the head calibration unit (C) is provided with a head coordinate system (HKS) of the head (H ), wherein a pose of the head target (TH) and the calibration unit coordinate system (KKS) is determined at the same time by means of a reference measuring system to its base and from this the head pose is calculated.

Description

The invention relates to a method for determining a head pose of a head.

From the prior art, it is generally known to attach a target on the head and to determine a pose of the target to its base by means of a reference measuring system. Thereafter, a coordinate transformation from the target to the head coordinate system is determined.

The invention is based on the object to provide a comparison with the prior art improved method for determining a head pose.

The object is achieved by a method for determining a Kopfpose with the features of claim 1.

Advantageous embodiments of the invention are the subject of the dependent claims.

In a method according to the invention for determining a head pose of a head, a head target is arranged on the head and a head calibration unit is arranged on the head such that a calibration unit coordinate system of the head calibration unit coincides with a head coordinate system of the head, whereby a pose of the head target and the calibration unit coordinate system are simultaneously determined by means of a reference measurement system is determined to its base and from this the head pose is calculated.

The method of the present invention allows more accurate transformation into the head coordinate system, requires less initial calibration effort, and allows easier setup of a head calibration system that includes the head target placed on the head, the head calibration unit, and the reference measurement system.

Furthermore, a free head position during the calibration process (<1 s) is made possible by the method according to the invention. The prerequisite for this is that both targets, ie. H. the head target and a calibration unit target of the head calibration unit, found simultaneously, d. H. can be captured.

The calibrated transformation into the head coordinate system is as accurate as the reference measurement system, since all transformations are obtained in this system (simultaneous recording).

There is no static structure with a known geometry required. The method according to the invention and the components for carrying it out can also be used, for example, in a conventional office or at another location.

When used in a video-based driver observation of a driver of a vehicle, many images can be recorded cost-effectively with an associated precise head pose. These can be used to train and evaluate machine learning methods for head pose estimation and detection. In particular, current methods require a large amount of training data (> 100,000). Traditional approaches use manual annotation, which takes a lot of time.

Embodiments of the invention are explained in more detail below with reference to drawings.

Showing:

1 a schematic representation of a head coordinate system,

2 a schematic representation of a head target arranged head target,

3 a schematic representation of another arranged on a head head target,

4 a schematic representation of another arranged on a head head target,

5 a schematic representation of a head with a head target arranged thereon and a Kopfkalibriereinheit,

6 a schematic representation of the Kopfkalibriereinheit from 5 .

7 a schematic representation of a recording of a driver observation camera in a vehicle with coordinate systems of a head of a driver of the vehicle and a head target, and

8th a schematic representation of a three-dimensional visualization of coordinate systems of a head, a head target and a base of a reference measurement system.

Corresponding parts are provided in all figures with the same reference numerals.

Based on 1 to 8th In the following, various procedures for determining a head pose of a person's head H are described. Such a head position determination is used for example for calibrating a video-based driver observation system in a vehicle 1 , Such a video-based driver observation system is exemplary in 8th shown.

For clarity, some terms will be defined first, which will be used in the following.

A head coordinate system HKS is a defined coordinate system fixedly defined to the (bony) head H of the human, as in 1 exemplified. For example, a zero point of the head coordinate system HKS lies in the nasion N, an x-direction X of the head coordinate system HKS runs in the viewing direction, a y-direction Y of the head coordinate system HKS runs in the direction of the left eye and a z-direction Z of the head coordinate system HKS runs in the direction of Forehead. Thus, a yz plane passes through nasion N and chin K, and an xz plane is such that the face is mirror symmetric to the xz plane.

A coordinate transformation is a description of translational and rotational relationships between two coordinate systems.

Calibration, as used herein, is finding a suitable coordinate transformation between two coordinate systems.

A head pose is a coordinate transformation between the head coordinate system HKS and a reference coordinate system, for example, a camera coordinate system or a vehicle coordinate system.

A calibration device, in particular a Kopfkalibriereinheit C in the sense used here is, such as in 5 shown, a device for applying the head H to find the calibration. This calibration device, in particular head calibration unit C, specifies the position of the head coordinate system HKS by clearly placing it on.

When developing and evaluating systems for estimating the head pose, an accurate reference measurement system is needed to determine the head pose. The aim is to enable an accurate measurement of both the head position and the head rotation in a reference coordinate system at all times. In driver observation, these measurements are used in training and the evaluation of camera-based head position estimators.

In such a driver observation, the head pose may be used in downstream methods to obtain impressions, in particular information about the condition of the driver. For example, the head pose can be used for a driver behavior analysis and, as a result, for example, for predicting driver behavior. Furthermore, the head pose can be used, for example, for the operation of a three-dimensional display, as this requires an exact knowledge of the Nasion N in the display coordinate system. In addition, the head pose can be used, for example, for detection of tiredness, for example for detecting eyelid closure and / or eyelid striking and / or for determining the PERCLOS measurement.

For automatic longitudinal and lateral control of the vehicle 1 For example, the head pose may be used to determine the driver's attention and / or line of sight to no longer perform this automatic longitudinal and lateral control in response to "hands-on-wheel", ie, depending on whether the driver's hands are on one Steering wheel of the vehicle 1 but in Dependence on "head-on-road", ie depending on whether the head H, in particular the viewing direction, of the driver is directed to the roadway.

For example, in the case of camera-based head pose estimation systems, it is important not to obscure the driver's face with metrology. This head pose estimation is usually performed by a non-invasive measurement.

The following will, in particular, based 5 , a reference measuring system that allows a precise measurement of the head pose through a new calibration routine without obscuring the driver's face.

Measuring systems for determining the coordinate transformation of one or more well-defined objects, referred to below as targets, to the reference coordinate system are already known from the prior art. Underlying technologies include, for example, electromagnetic sensors, inertial sensors, laser-based optical sensors, image-based methods (for example, the perspective n-point algorithm described in US Pat Lepetit, V .; Moreno-Noguer, M .; Fua, P. (2009). "EPnP: An Accurate O (n) Solution to the PnP Problem". International Journal of Computer Vision 81 (2): 155-166 ) and the so-called optical marker tracking.

These reference measurement systems can determine the pose of the target to their base. If the target is attached as a head target T _H at the head H, as in the 2 to 8th the coordinate transformation from the head target T _H to the head coordinate system HKS must still be determined, ie the calibration must be carried out. This coordinate transformation can either be predefined or measured by the attachment. By surveying any attachment of the head target T _H head H is possible. 2 shows such a head H arranged on the head target T _H , wherein by means of a dashed arrow P, the coordinate transformation from the head target T _H to the head coordinate system HKS is shown.

At this point, the new method described below sets in, since by means of this new method, and in particular by means of the head calibration unit C used for this purpose, such a coordinate transformation from the head target T _H to the head coordinate system HKS is found, as will be described in detail later. This coordinate transformation from the head target T _H to the head coordinate system HKS can be used for determining the head pose, as long as the head target T _H maintains its position on the head H, that in particular does not slip. If the position of the head target T _H relative to the head H is identical to the position during the calibration, the transformation of the head target T _H into the head coordinate system HKS can be used, also with retroactive effect. This new method and the head calibration unit C used for this purpose can be used in particular in video-based driver observation.

Methods are already known from the prior art, which use the above-mentioned reference measuring systems for Kopfposenschätzung.

For example, a rotation estimation without translation estimation is known. Such methods only estimate the rotation of the transformation, but neglect the translation.

Furthermore, a relative pose estimation is known. For this purpose, methods use an initial "straight-ahead view" of the wearer of the head target T _H as a calibration. Pose changes of the head target T _H are determined relative to the initial pose of the head target T _H. Here, high variances occur through the perception of the wearer and also between wearers, ie a repeatability is impaired.

Furthermore, a system is known which uses a laser-based optical sensor as a reference measurement system for driver observation. As a calibration device, a frame is used, which is on the steering wheel of the vehicle 1 is hung up. To calibrate, the driver must place his head H on the rack. For calibration, a chain of coordinate transformations is set up to find the transformation from the head H to the head target T _{H of} the laser-based optical sensor. In this case, a chain of coordinate transformations must be measured, namely from the head H to the calibration device, from here to the zero position of the steering wheel, from here to the vehicle CAD, from here to the laser-based optical sensor and from here to the head target T _{H of} the laser-based optical sensor. This can lead to the accumulation of measurement errors and poor reproducibility. This chain must be carried out for each setup of the reference measuring system, for example for each test carrier etc.

In another method known from the prior art, a tiara carried centrally on the head H, as in FIG 3 shown, or a chessboard worn centrally on the head H, as in 4 shown, used as a head target T _H , for each of which algorithmically the pose is found in the camera coordinate system. However, a calibration to the head coordinate system HKS does not take place.

Another method known from the prior art is the so-called iterative point-cloud model fitting. The estimation of the head pose can be made by fitting a flexible model to a point cloud of the head H. This is usually done by an iterative algorithm. This can not always find the global optimum due to noisy point cloud coordinates. Furthermore, no statement about the accuracy of the fit is possible, especially if the flexible model can not represent the variation of the point cloud.

The following in particular based 5 The method described avoids these disadvantages of the methods described above. In particular, it does not need a long chain of coordinate transformations and finds an absolute calibration.

transformiert einen Punkt ^BP = [x y z 1]^T (2) aus einem Basiskoordinatensystem B eines Referenzmesssystems in einen Punkt ^AP in einem Koordinatensystem A:

In the following, all transformations take place in homogeneous coordinates. A homogeneous matrix:

transform a point ^B P = [xyz 1] ^T (2) from a base coordinate system B of a reference measuring system into a point ^A P in a coordinate system A:

The assumption is a reference measuring system which can simultaneously determine the pose of at least two targets T _H , T _C to the base B, ie to the base coordinate system B of the reference measuring system. The reference measuring system advantageously comprises two cameras and one infrared illumination.

The head target T _H is attached to the head H as in 5 shown. The calibration unit target T _C is attached to a head calibration unit C. The two targets T _H , T _C , for example, each as a so-called marker trees, comprising a plurality, in particular infrared radiation reflecting, markers M, formed, which can be illuminated with the infrared illumination of the reference measuring system and detected by the cameras. Two cameras and infrared lighting make the reference measuring system particularly robust. Alternatively, however, it is also possible, for example, to use a reference measuring system with only one camera and without infrared illumination. In particular, a camera-based detection of the marker trees with the infrared radiation-reflecting markers, in particular their pose, by the reference measuring system is required.

des Kalibriereinheittargets T_C zum Kalibriereinheitkoordinatensystem KKS an der Kopfkalibriereinheit C ist bekannt (via Konstruktionsplan). Das Kalibriereinheitkoordinatensystem KKS hat seinen Ursprung in einem Nasionstempel NS der Kopfkalibriereinheit C. Die x-Richtung X verläuft analog zum Kopfkoordinatensystem HKS zur Kopfkalibriereinheit C hin. Die negative z-Richtung Z verläuft vom Nasion N zu einem Kinnstempel KS der Kopfkalibriereinheit C. Die y-Richtung Y verläuft entlang zweier Wangenstempel eines Doppelstempels DS der Kopfkalibriereinheit C.A calibration unit coordinate system KKS of the head calibration unit C is selected so that it coincides when resting on the head H with the head coordinate system HKS of the head H. The pose

of the calibration unit target T _C to the calibration unit coordinate system KKS at the head calibration unit C is known (via construction plan). The calibration unit coordinate system KKS has its origin in a nasion stamp NS of the head calibration unit C. The x-direction X is similar to the head coordinate system HKS to Kopfkalibriereinheit C. The negative z-direction Z runs from the nasion N to a chin stamp KS of the head calibration unit C. The y-direction Y runs along two cheek stamps of a double punch DS of the head calibration unit C.

The Kopfkalibriereinheit C is designed so that it can be reproducibly placed on each head H. The Nasionstempel NS is located centrally on the Nasion N, the Kinnstempel KS can be moved along the Z-axis of the head coordinate system HKS to lie on the chin K. A double die DS, comprising the two cheek punches, is then slid along the negative X axis toward the head H and pressed against the cheekbones W, as in 5 which shows the head calibration with the Kopfkalibriereinheit C.

The Nasionstempel NS is here centrally on the Nasion N. The chin stamp KS is shifted to chin height. The two cheek stamps in the form of the double stamp DS lie with equal force on the cheeks, ie on the cheekbones W. In 6 is the Kopfkalibriereinheit C again shown separately.

vom Kopftarget T_H zum Kopf H kann nun errechnet werden:

The sought transformation

from the head target T _H to the head H can now be calculated:

The head coordinate system HKS and the calibration unit coordinate system KKS are identical during the defined placement of the head calibration unit C on the head H, represented by the identity mapping id:

stammen vom Referenzmesssystem. Somit kann die gesuchte Transformation analytisch berechnet werden (3 Matrixmultiplikationen von homogenen Matrizen).Thus, equation (5) holds by resting the head calibration unit C on the head H:

come from the reference measuring system. Thus, the sought transformation can be calculated analytically (3 matrix multiplications of homogeneous matrices).

The innovation of this Kopfkalibriersystems thus consists in the idea of an advantageously constructed Kopfkalibrators, ie the Kopfkalibriereinheit C, and a calibration routine, which finds the pose of the head H carried on the head target T _H in the head coordinate system HKS.

In the video-based driver observation, the transformation from the reference measurement system to the camera coordinate system can similarly be found via a calibration target holding a calibration checkerboard and a target in a known geometric arrangement.

The 7 and 8th show a calibrated system in a prototype use in video-based driver observation. It shows 7 a photograph of the driver observation camera with the head coordinate system HKS of the head H and a head target coordinate system KTS of the head target T _H and 8th shows a three-dimensional visualization of coordinate systems of head H, head target T _H and base of the reference measuring system, ie the head coordinate system HKS, the head target coordinate system KTS and the base coordinate system B.

The solution described allows a more accurate transformation, requires less initial calibration effort, and allows for easier setup of the calibration system. In addition, the solution described allows a free head position during the calibration process (<1 s). The prerequisite is that both targets T _H , T _C can be found simultaneously. The calibrated transformation is as accurate as the reference measurement system, since all transformations are obtained in this system (simultaneous acquisition).

There is no need for a static structure with a known geometry. The system can also be used in the office, for example.

When used in the video-based driver observation, many images can be recorded cost-effectively with an associated precise head pose. These can be used to train and evaluate machine learning methods for head pose estimation and detection. In particular, current methods require a large amount of training data (> 100,000). Traditional approaches use manual annotation, which takes a lot of time. This is avoided by the solution described.

LIST OF REFERENCE NUMBERS

1

vehicle

B

Basiskoordinatensytem

C

Kopfkalibiereinheit

DS

double temple

H

head

HKS

Head coordinate system

K

chin

KKS

Kalibriereinheitkoordinatensystem

KS

chin temple

KTS

Head target coordinate system

M

marker

N

nasion

NS

Nasionstempel

P

arrow

T _C

Kalibriereinheittarget

T _H

Kopftraget

W

cheekbone

X

x-direction

Y

y-direction

Z

z-direction

QUOTES INCLUDE IN THE DESCRIPTION

This list of the documents listed by the applicant has been generated automatically and is included solely for the better information of the reader. The list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions.

Cited non-patent literature

• Lepetit, V .; Moreno-Noguer, M .; Fua, P. (2009). "EPnP: An Accurate O (n) Solution to the PnP Problem". International Journal of Computer Vision 81 (2): 155-166 [0035]

Claims (4)

Method for determining a head pose of a head (H), characterized in that a head target (T _H ) is arranged on the head (H) and a head calibration unit (C) is arranged on the head (H) such that a calibration unit coordinate system (PPS) the Kopfkalibriereinheit (C) coincides with a head coordinate system (HKS) of the head (H), by means of a reference measuring system at the same time a pose of the head target (T _H ) and the Kalibrierereinheitkoordinatensystems (KKS) is determined to its base and from the Kopfpose is calculated. A method according to claim 1, characterized in that the head calibration unit (C) is arranged at the head (H) such that the calibration unit coordinate system (KKS) of the head calibration unit (C) coincides with the head coordinate system (HKS) of the head (H) by a nasion stamp (NS) of the Kopfkalibriereinheit (C) is placed centrally on the Nasion (N) of the head (H), a chin stamp (KS) of the Kopfkalibriereinheit (C) in z-direction (Z) of the Kopfkoordinatensystems (HKS) is moved until it on the chin (K) of the head (H) rests and a double punch (DS) in the negative x-direction (X) of the head coordinate system (HKS) in the direction of the head (H) is moved until it to the cheekbones (W) of the Head (H) evenly. Method according to one of the preceding claims, characterized in that the head target (T _H ) and the calibration unit target (T _C ) comprise infrared radiation-reflecting markers (M), wherein the reference measuring system, in particular camera-based, detects the infrared radiation-reflecting markers (M). A method according to claim 3, characterized in that the reference measuring system comprises two cameras and an infrared illumination, wherein the markers (M) are detected by means of the two cameras.

Images