ITSA20080017A1 - ANALYTICAL METHOD FOR 360 DEGREE VISION AND FOR THE CREATION OF FLAT IMAGES WITHOUT BENDING AND DISTORTION STARTING FROM IMAGES CAPTURED WITH FISHEYE AND PAL OPTICS (PANORAMIC ANULAR LENS). - Google Patents

Description

DESCRIPTION OF THE INVENTION

TITLE ANALYTICAL METHOD FOR 360-DEGREE VISION AND FOR THE CREATION OF FLAT IMAGES WITHOUT CURVATURE AND DISTORTION FROM IMAGES CAPTURED WITH FISHEYE AND PAL OPTICS (PANORAMIC ANULAR LENS)

SUMMARY

A procedure is defined for the Local Perspective Correction of Curvature and Distortion in images captured with digital devices (mainly but not exclusively video cameras and cameras) with Fisheye and PAL (Panoramic Anular Lens) curved optics. The final result ? to produce one or more? images - flat, without curvature and distortion - from a single image captured with a single Vision system with optics capable of allowing a 360 vision? along the axis perpendicular to the line of sight and with an opening angle along the line of sight of 180? and beyond (depending on the optics used). In other words, a procedure (algorithm) is presented that is capable of allowing a real-time correction of curvature and distortion effects on panoramic and wide-angle images. In order to make the final image more usable to the user, different presentations can be created such as: 1) a single navigable image (ie with an immersive effect in the resumed anribienie); 2) more? images as if the environment was taken from more? cameras placed at different angles and therefore capable of covering 360? of interest in the area. It is therefore a? Combination invention that realizes one or more? flat image without curvature and without distortion through the use of commercial digital cameras / cameras with Fish Eye or PAL optics and with a computer procedure that implements the proposed method and allows to achieve the inventive result (see Figure 1).

DESCRIPTION

The result referred to in the summary is achieved through a function called CONVERT. This function has the following inputs / outputs:

Input: Vector of RGBA type elements (Red, Green, Blue and Alpha) representing the fisheye or PAL image

Output: Vector of RGBA type elements (Red, Green, Blue and Alpha) representing the image without curvature and distortion.

The CONVERT function, that is its algorithmic implementation can? be schematized as follows:

Step 1: Supporting Variables Precaution;

Step 2: Calculation of the parameters of the virtual camera (virtual point of view);

Step 3: Calculation of the vectors of the frustum (pyramid with a rectangular base;

Step 4: Calculation of the size of the frustum;

Step 5: Unwarped procedure (core)

Before passing high detailed description of the transformation, let's define (also graphically) some important concepts:

? Transformation chain: phases of transformation of? Fisheye or PAL input image into an output image without warp and distortion (Figure 2). The camera, even in the presence of Fisheye or PAL optics? able to output rectangular 2D images. For? in this case, the output image of the camera is not? all shooting image, but since the Fisheye or PAL optics have a 360 view, this image will present? of the sectors in which you will not have? no image; in figure 2, you can? verify this effect: from the camera we obtain a rectangular image where the area of the red circle represents the part taken by the camera, while in the external part of the circle we have no image (since the optic is not able to take a rectangular area but only circular-shaped area, in the non-shooting sectors we have a black image). On the output 2D image of the camera, the unwarped procedure is applied to sectors, thus obtaining? an image without curvature and distortion.

* Frustum: frustum is defined as the pyramid with a rectangular base derived from the shooting area by a camera. To better understand, think of the camera; in taking a photo, what is framed? a rectangular area. The frustum are imaginary lines that start from the camera's optics and connect with the vertices of the shooting area, or rather with the rectangle. In figure 3? illustrated the fustrum in the case of Fisheye or PAL optics.

? Antialiasing Definition:? a technique to reduce the aliasing effect when a low resolution sign is displayed at high resolution. Antiatiasing softens the lines by smoothing the edges and homogenizing the image. So the image is sampled as if instead of a single pixel there were more? and the final pixel is assigned the medium color (supersampling). Each pixel is divided into pi? pixels that have an independent color. To calculate the final color of the pixels, an "average" of all the pixels is made, generally giving greater importance to the central pixels. The subpixel mask? the set of "fragments" of pixels. A great mask? of 4x4 pixels (16 subpx) even if it pays in performance compared to the pi? fast 2x2 pixels or 3x3 pixels.

Let's analyze in detail the proposed transformation model.

Step 1. Precaution of supporting variables

A three-component vector is used to define within the three-dimensional (3D) space the position and direction of the virtual camera (Virtual Viewpoint - PW).

XYZ vp = {0,0, 0}, vd, vu, staves;

The increments to be used during the unwarped procedure are calculated. These increases represent the movements that must be made at each step of the procedure (rescaling),

y p p g;

with

? Fishradius, radius of the input image evaluated in pixels (in this specific case it is usually initialized to fishwidth / 2);

? Fishaperture, visual delta width in degrees of the input camera (typically it assumes a value of 180?);

? Antialiasing, degree of reduction of the aliasing effect (scaling). It is usually set at 2.

? Perspwidth, size (width) that should? have the output image;

? Perspheight, size (length) that should? have the output image

? DTOR, additional parameter to transform an angle expressed in degrees into an angle expressed in radians

Step 2. Calculation of virtual camera parameters (virtual point of view)

In this phase the position and direction of the virtual camera (PW) is calculated; this camera will be? used to shoot a portion of the image subjected to the unwarped procedure (it is not the camera used to create the input image, but a virtual camera that will be positioned within the 3D space to shoot only the part of the fisheye image or PAL which will have to be subjected to the unwarped procedure).

In addition to the vd direction vector we must also specify the camera orientation, that is an up vector (camera orientation). The persptheta (0? To 360?) And perspphi (0 to? / 2) angles will be used to move the camera along the circular ring of the fisheye or PAL image.

The vector vd represents the direction vector of the camera;

The vector vu represents the orientation vector of the delta camera (vector up);

The vector vu has the same x and y components as the vector vd, except for the z component, where we have that vu.z has the value -1. This notation means that the verse of vu? opposite to the vector vd (head-toe inversion).

i hi C P d t (d)

The CrossProduct function performs the vector product (or external product, or cross produce) of two vectors of the three-dimensional space, namely:

The Normalize function normalizes a vector to three dimensions (the length of each single component will be between 0 and 1), that is:

}

Step 3. Calculation of the vectors of the frustum (pyramid with rectangular base)

In this phase, the 4 vectors are calculated that from the position of the virtual camera (by default the virtual camera is arranged in coordinates (0,0,0)) point towards the rectangle that encloses the portion of the fisheye or PAL image to be subjected to the unwarped procedure (in figure 1 these points are represented by p1, p2, p3 and p4). The four vectors cos? constructed represent the 4 vertices in the 3D space of the frustum (as in the figure above);

with

? perspaperture, amplitude, in degrees, of the optical aperture of the virtual camera;

? perspwidth, size (width) of the imaged rectangle;

? perspheight, size (length) of the imaged rectangle;

? VectorSum, a function that accepts 4 vectors in 3D space and 4 increments and calculates their sum; the result obtained? a vector in 3D space. The calculation is performed ne! following way:

XYZ VectorSum (float d1, XYZ p1, fioat d2, XYZ p2, float d3, XYZ p3, float d4, XYZ p4)

The 4 results obtained, pi, p2, p3 and p4, represent the 4 vectors that connect the 4 vertices of the rectangle taken by the camera from the center of the virtual camera (imagine that the 4 vectors all start from the same point and have as their final point the 4 vertices of the rectangle; a pyramid with a rectangular base is formed).

Step 4. Calculation of the size of the frustum

Once the 4 vectors are obtained, we move on to the calculation of the dimensions of the frustum of the perspective projection (the dimensions of the shooting rectangle of the virtual camera)

For the calculation of the two sides of the rectangle the vectors p1, p2 and p4 are used (as in the upper diagram).

Step 5. Unwarped procedure (core)

After configuring the variables, let's move on to the description of the unwarped procedure:

// For each pixel of the perspective image (of the frustum box)

1

// Calculate the position of the pixel inside the array

// mono dimensional which contains the pixel information

;

// Save the information of the pixel present in the array (inside the 2 innermost helicles, the // pixels are searched through the unwarped procedure to eliminate aliasing. For each pixel, the // information of the neighboring pixels is added)

The (we leave the 2 innermost loops)

// After obtaining the information of the fisheye / PAL pixel and transformed into a pixel perspective, write the // result in the output vector of the Convert function

Claims (1)

CLAIMS TITLE; ANALYTICAL METHOD FOR 360-DEGREE VISION AND FOR THE CREATION OF FLAT IMAGES WITHOUT CURVATURE AND DISTORTION STARTING FROM IMAGES CAPTURED WITH FISHEYE AND PAL OPTICS (PANORAMIC ANULAR LENS) The novelty? of the patent is briefly reported below. Yup ? created a procedure, or an algorithm (directly implementable in programming language) to correct the curvature / distortion of images captured with 1. Fisheye optics; 2. PAL (Panoramic Anular Lens) through a mechanism of creation d? virtual camera (aka Virtual Point of View - PVV). Compared to this PVV, an image of assigned size is created without any distortion / curvature effect, i.e. a perfectly flat image, as if it had been acquired by a device with traditional optics (i.e. not curved, not fisheye, not PAL). Unlike other similar solutions that simply unroll the curved image leaving distortion effects, the proposed solution also locally corrects the distortion providing an output image equivalent to that obtained with classic optics capable of covering all 360 degrees. . Each algorithm / method / solution (computer / numerical / computational) both software and firmware / hardware for the creation of one or more? flat images starting from curved images (i.e. with distortion / curvature as obtained with Fisheye / PAL optics) written in any programming language that is a translation of the metacode presented and for which an industrial patent is requested or that it is a? substantial change? of the same potr? be the subject of claims. Specifically for? Not substantial modification? any form of code that differs from the patented one must be understood only for aspects that are not functionally indispensable to the correct procedure for eliminating the curvature / distortion.