ES2249162A1 - Method for creating three-dimensional object model during e.g. architectural engineering design work, involves providing digital video camera on swivel base, and linking intersection points of projective rays by software - Google Patents

Abstract

Three-dimensional modeling procedure using boundary conditions. A procedure is described that allows processing of digital images obtained from different angles and heights of an object, in order to determine or digitally reproduce the object. For this, the procedure uses the treatment of the data obtained from the different intersections of projective rays with the edge or delimitation of the object in each captured image. Relating through software the points obtained from different perspectives of the object, located on a rotating base and obtained with the help of a digital video camera, the reproduction of the external figure that defines it is achieved, determining through the user characteristics that will be relevant. in terms of the quality of the modeling obtained, which will depend on the application to be achieved. This procedure is especially indicated for design engineering, architecture, restoration, fine arts and geometric control, and can be extrapolated to other fields or applications that fit the purpose or result inherent to this type of project.

Description

Three-dimensional modeling procedure using edge conditions.

Object of the invention

The present invention relates to a three-dimensional modeling procedure using conditions of edge, which provides essential novelty and remarkable features advantages over known and used means for same purposes in the current state of the art.

More particularly, the invention proposes the development of a procedure whereby it is possible achieve the modeling of the external surface of certain objects or pieces, with the help of a digital video camera and a Rotary stand used as. object support medium during image capture from a multiplicity of vision points

For the application of the procedure, the use of a specific management software, through a conventional computer equipment, designed exclusively for your application to the development of the proposed procedure

The scope of the invention is It is mainly comprised within the sector related to design engineering, architecture, restoration, fine arts and geometric control, being able to be extrapolated to other fields or applications that fit the purpose or result it poses This type of project.

Background and summary of the invention

The determination of the shape of a convex object  It has traditionally been resolved for use manually by applying complex robotized methods with five movements; Another more automated solution is the use of scanners based on the emission and reception of a laser beam, having all these methods a very high acquisition cost due to the necessary instrumentation for them.

It is therefore proposed with this invention, a new methodology based on image analysis, using a video camera of the highest possible quality (camera digital), providing this method as a result of the final form of the object with a resolution more than enough for any type of requirement. This resolution allows you to observe an object from all points of view and analyze the edge of the image (object edge).

The proposed procedure allows starting from from the edge of the object (determined in the image) build the full three-dimensional model fully automatically with Millimeter accuracy for an object with dimensions considerable, obtaining if the object is of smaller size a Greater precision.

The method employs a rotating base on which the object and a digital video camera will be placed to get all points of view of the object, generating a model three-dimensional of it with the original textures on the model and a characterization of the accuracies obtained in the model determination (all applying a series of algorithms developed for the treatment of images).

This modeling method has certain advantages over the known, as it can be used by Any user proving very economical, has no limited space or interaction with the environment, it has no special computer requirements being able to work in any current computer, and above all, is not a method Aggressive with the environment.

The investigation and application of the algorithm developed for the treatment of captured images has generated a deep analysis, which concludes that by the use of this procedure can be obtained in a very cheap the digital model of any object, being able to become a very useful tool mainly in design engineering, architecture, restoration, fine arts and geometric control.

Brief description of the drawings

These and other features and advantages of the invention, will become more clearly apparent from the detailed description that follows in a preferred way of embodiment, given by way of illustration only and not limiting, with reference to the accompanying drawings, in the that:

Figure 1 represents by means of a drawing schematic the view of the plant where you can see the contour of the base of the rotating base on which it is located a figurative object surrounded by the arrows that represent the area of covered object at each moment of the video capture.

Figure 2 shows, like the figure previous and by means of a schematic drawing the view of the plant in which you can see the outline of the base of the base rotating on which is a figurative object surrounded by a pair of arrows that represent the limit or right edge of the object defined by the camera image.

Figure 3 represents by means of a drawing schematic, like the previous figure, the view of the plant in which can be seen the outline of the base of the base rotating on which is a figurative object surrounded by border lines defined by the image.

Figure 4 schematically shows a possible sector in which the double arrows are represented by  cylindrical coordinates that delimit it.

Figure 5 represents, an initial cylinder a model by the edge lines that model it, which is defined  by removing external material by applying successively the cuts that suggest the edge vectors that Define the images.

Description of a preferred embodiment

As indicated above, the detailed description of the invention will be carried out taking into account the representations of the drawings annexes, through which the same references are used numerical to designate the same or similar parts. In this sense, and following the procedure of the invention itself set out in the following paragraphs an example in which they clarify determining aspects for development and understanding of the invention itself.

The main objective of the procedure is to  get the three-dimensional modeling of the object automatically, for this, the following data are available according to the Algorithms developed during the development of the invention: External orientation parameters of all images corresponding to the first round, internal geometry of the camera, main distance and radial distortion correction, mapping or equivalence between the images of the first and the second round, in order to use the orientation parameters external achieved in the first round, and the data tables with the pixel coordinates of the edges of the object in each of the Second round images.

With all this data it is intended to obtain automatic way, the three-dimensional modeling of the object and its precision. In addition, the results are complemented by the output three-dimensional graph (using VRML) of the model obtained in both format of wires, faces or textures.

As described above, to check the correct operation of the proposed algorithms help software has been developed minimizing performance of the user in tests made with different objects.

The objective is to automatically obtain the three-dimensional modeling of the object. The concept of "automatic" implies not using at all times the stereoscopic vision for the identification of points on the object (so this modeling procedure would not make sense since it is a classic photogrammetry work with a peculiar form of procedure orientation ), nor the correlation for the automatic identification of homologous points (since the object may lack appropriate textures for this purpose, like most sculptures).
flush).

The methodology presented consists in using only the points of the edges of the object to define it of completely automatic, without identifying at any time Homologous points with some technique such as correlation.

Therefore, no homologous points are available, but of a series of lines in the space that involve the points of the edges of the images defining the outer shape of the object. In an approach to the problem, by having a single image with its object edge points, you get a inclusion zone and other object exclusion zone, as You can see in Figure 1, where you can see the base rotating, referenced with the number 1, the object to be modeled, with the  number 2, and the different representations by arrows of the possible projective rays, referenced with the number 3 in the figure.

The border lines of the image behave as if the object was the envelope of the defined straight beam by the edge. Having the double condition of tangency and inclusion to the object, so both properties can define the object in the job. However, the first condition (tangency) has a geometric definition approachable in the direct problem, where known the object you want to know the tangents. Your solution is you will get through the Frenet trihedron in space, applied on the surface in question, studying the tangent plane (the perpendicular to normal at the point of the studied surface). On the contrary, the problem raised in this procedure is the reverse, that is, the envelopes are known and it is desired to obtain the object.

The problem of obtaining straight beams and his enveloping figure is always posed as an equation differential that has to be integrated. Differential equation represents the straight lines of the beam and its envelope. If you know the differential equation that contains all the lines and their object envelope you could get the equation of the lines as well as of the object. But the available lines do not adapt to a simple differential equation, just as the object does not it has a simple differential equation that represents it, so so much is useless to continue in the direction of integrating differential equations.

The resolution of the problem is approached from the point of view of inclusion: The object is contained within the space defined by the edges. If a cylindrical coordinate system is defined in the object, so that a point on its surface has coordinates P (f, H, r) being: f polar angle, H height and r distance
polar.

The position of point P, as indicated in Figure 2 with the number 4, will comply in all the images that the distance from a point on the Z axis (f, H, r = 0) to any line of the edge that is at that height and in that direction is greater than r. There is only one subtraction that equals and that corresponds to the line that passes through that point of the edge.

A value of r is obtained for each value of the height H (second coordinate) and for each direction f (first coordinate). The value of r (polar distance) is obtained by applying A minimum function.

In each frame i for each pixel j (of the border of the object in the image), we study the projective line that produces and its relation to a height H and a direction f anyone.

The first coordinate r of the model will result in minimum value of distance D (i, j) (referenced in the Figure 3 with a double arrow numbered with the number 5) of the axis of rotation to the line (at height H and in the direction f), as it can be seen in Figure 3. The distance will be obtained by calculating the minimum value in each direction of the axis distance: r = min [D (i, j)].

The object will be defined by the intersection or common area among all inclusion zones produced by all The images of the video stream.

Entering more in the applied methodology, as He explained previously the coordinate system used to define The object model will be a cylindrical coordinate system. Is that is, the model will consist of sectors that will have a certain resolution (number of sectors in each horizontal plane and number of horizontal planes).

Looking now at Figure 4, you can see the coordinate sector (f, H, r), being referenced in the drawing each parameter f, H and r, through a double arrow, with the numbers 6, 7 and 8 respectively. Each sector will intersect at object at a point on its surface, which will be defined by a angle on the axis of the X (f), a height above the platform lower (H), and a radius (r) to the Z axis.

For the application of the algorithm, the number of sectors used for modeling, initially specified in the software that has been developed for subsequent calculations. It is also convenient to limit modeling to a height margin or horizontal sector

As described below, the modeling of the object must be performed in two consecutive steps: a first step which consists in the actual modeling of the object without entering in no assessment, and a second step in which a improvement of the modeling obtained in the previous step and a series of indicators to evaluate the characteristics of the model got.

In the first step, in the initial modeling of the object is part of a cylinder determined by an interval of height, with so many sectors per plane, and so many horizontal planes as indicated in the parameters entered by the user in the software. Therefore, in a first example if you have 90,000 sectors (300 X 300, 300 sectors and 300 planes) so that when the value of r (polar distance) is known to each will correspond after modeling to as many points on the object surface.

From each sector according to its spatial position, it it has by definition the coordinates f and H. The coordinate r will be the distance to calculate to place the point on the surface of the object. At first this coordinate is initialized with a upper bound which may be for example the radius of the platform.

The algorithm to be applied is twofold and needs define the projective rays that cross each sector and the minimum distance to those rays.

The software as graphic outputs of this first model can display several images, in which each sector is represents with a point in the space located in its position central. Later, graphical outputs will be created using wire, face, and textured modeling meshing and triangulating The surface obtained. In these figures obtained you can see that there are only points on the surface of the object and not within the same.

In a second step, of improvement and valuation of modeling, it is observed that in the previous section each sector came defined by its coordinates f, H and the radius r that was calculated as the minimum distance to the nearest straight line of all lines that crossed this sector. When selecting the minimum distance, is exposed to the line that defines it, is created from a badly defined edge point (1 pixel of error is easy to commit in detecting an edge point in any image). In This section proposes the solution to this possible error, as well as the determination of the precision in the modeling obtained.

In each sector obtained (f, H, r), it is determined which lines have been the closest (according to a value previously fixed) and with these statistical operations are carried out with their minimum radios, being necessary to carry out this process in two phases: a first phase where there is a determination of nearest lines within an established range (accumulation of straight lines), and a second phase where a statistical study is carried out of the geometric distribution of these.

As the first phase, in the accumulation of straight lines  decide on the minimum distance for the accumulation of straight, which will normally correspond to 1 pixel measured on the object, the minimum distance for accumulation being specified of straight lines in each sector through software.

In this case, for example, if a line passes to less than 3 millimeters of a sector its identification is stored in a list belonging to that sector. In this way the software after performing the calculation with all sectors will create an informative table.

The table will form a table with all the sectors represented, in this case they will be 40,000, organized by rows according to coordinate H and columns according to coordinate f. In the table you can see, for each sector, the number of lines that have been found at a distance less than specified of 3 mm.

The realized software presents tools of visual check of the proper functioning of the algorithm. To the select a cell from the table, for example the cell corresponding to a certain sector f and height H, the information on the total number of lines that pass less than 3 millimeters of said sector, for example 409 straight, in addition to a graph in which the distribution of these lines appears according to the distances less than 3 millimeters.

The horizontal and vertical axes of the graph to the referred to above, shown by the software, will appear multiplied by a certain factor, for example a factor of 10, so that it is observed that, for example, 4 lines appear at the minimum distance of the object, 31 lines pass between 0 and 0.3 mm, 57 straight pass between 0.3 and 0.6 mm, etc.

In addition, it is displayed in space, through a graphic representation, the definition of the sector according to these lines indicated above, where it is shown from different points of view, like these 409 straight lines whose coordinates define the sector studied (f, H, r).

At this point it can be determined that the sector studied is a well defined sector, since it commission 409 lines corresponding to a number of edge points of a large number of different images.

However, when the study does not focus on a  edge point or discontinuity of the object but it is a point that presents a certain spatial generality in its environment, such as for example it is a point on a wall of the box where the number of concurrent lines on that edge descends remarkably, on that case the definition of the sector will not be as optimal as in the case previous.

For example, a sector in the previous situation it would be a sector that would have coordinates H = 0.225175 and f = 0.125664, which only has 24 lines for its definition available. Therefore the definition of this sector will have less quality than the sector seen previously that had 409 straight.

It can be concluded, saying that the number of lines that involve the sector is a binding factor in the good definition of the same, and that the methodology presented here detects right way and more precisely the edge areas and discontinuities of objects, unlike other techniques which are precisely not applicable in these cases (correspondence of images by correlation).

Finally, comment that the statisticians calculated in the case of the straight lines that cross a sector are: the mean distance to the object, the standard deviation and the standard deviation of the mean.

The procedure outlined so far, accumulated a line in the sector every time a projective ray it crossed that sector. The proposed improvement is to accumulate the straight, but using weights to give more importance to some Straight than others.

According to Figure 5, line R, referenced with number 9, crosses 5 different sectors whose distance is It is less than 3 millimeters from the object considered.

Without using weights as until now to each of  these sectors would increase by one unit the number of lines that define it (as long as the line passes less than 3 millimeters of the object).

Using weights the amount to add to the counter of the sector that indicates the number of lines that define it would be of 1 divided by 5 (line R helps define 5 sectors).

In the table cells provided by the software developed, the number of straight lines accumulated in each sector without using weights, and below the number of lines with weights.

In the studied sector of 409 accumulated lines, of all of them, for example, will show that only 26.8 lines Accumulated are real. The most precise sector defined will be given by: submit a high number of cumulative lines and submit a quality factor, with values greater than 0.2.

The quality factor will be defined by the quotient between the number of accumulated lines and the number of lines  accumulated using weights, obtaining a result of factor of quality that will range between 0 and 1. As a last step in improving the modeling, straight lines whose residue is too much are removed high.

At this point, the software provides so explanatory, some graphs distributed according to the number of sectors that present certain values of number of lines, factor of quality and corrections to apply. In some graphs it will be shown the number of lines that involve each sector, both with straight lines accumulated as with straight lines using weights. And other graphs show the number of sectors that present a certain quality factor, this factor can oscillate between 0 and 1. The sectors above 0.2 are very good defined.

In another section of the visualization software modeling, the results are also offered above mentioned but three-dimensionally on the object by Colored legends on their faces.

Attending now to the precision obtained in the model, the quality of the model can be encrypted from two points of view that as described below are opposed: number of points used in modeling and the quality of same.

From the point of view of the number of points that define the model you can say that precision comes directly defined by the number of heights and the number of sectors, although the number of points that define the model is the product of both, also the operation time will depend on the Number of this product. A poorly defined model with a number of 200 x 200 sectors, requires 40,000 points, a very model Defined 2,000 x 2,000 points, requires 4,000,000 points. Since the cumulative straight process is repetitive we will say that the operating time grows proportionally to the number of modeling sections.

After checking the time the evaluation costs of a simple model (200 x 200), you can easily foresee the maximum reasonable dimension according to the time you want to invest in the generation of the model.

From the point of view of the quality of points there are two influencing factors, the pixel size and the number of straight lines accumulated.

The images obtained that have 200,000 pixels  approximately, using a higher resolution image (4,000,000 pixels) more detail would be obtained in the definition of the edge. If the support has a size of 500 x 500 mm and is desired a resolution at the edge of 0.25 mm will require an image of 2,000 x 2,000 pixels easy to obtain with a digital camera, it is say, you could calculate the required image size based on of the expected edge resolution.

On the other hand, the definition of the model looks enhanced if the edge obtained is defined by a large number of projective rays (pixels). The number of projective rays that go to influence on average in any zone will depend on the size of pixel and the size of the sectors. If the size of the sectors it is 3 mm and the pixel size of 1 mm, a average of 9 pixels per sector; with more pixels accumulated on well defined edges.

If you want well defined points you can choose (without changing the camera) by defining the model with few points, whereby many pixels will accumulate at each point of the model and You will be sure that this point is correct. Yes for him On the contrary, a model with many points is desired. comply with the definition of the point from a few pixels. A minimum figure for the average number of accumulated pixels that guarantee the result, it would be at least 4 pixels per point,  achieving optimal results with values greater than 16 pixels by point

The above procedure presents a limitation by its own definition: the inability to detect the surface of the object in the concave areas of it.

Indeed, if the object is available It has a concave cavity or area, as the procedure is based on that projective rays turn out to be the envelopes of the object, The concave areas have no outside tangent. Even though I know consider all object edge points of all available images, said object will appear modeled without that concave cavity or area.

Still, the number of points calculated they can automatically constitute a high percentage of the total surface of the object, and also the procedure is capable of recognize those areas where modeling may have been created incorrect, through the study of the sectors that present a small quality factor

Therefore, another complementary method is proposed. to correct these irregularities, through vision stereoscopic, performing multiple resections in space taking advantage of the large number of images available.

To this end, in a following section the software allows to calculate from the sectors that define the modeling obtained, quadrangular or triangular meshes, in format of wires, faces or textures of the object itself, with the option for the representation to have colored ones of the model faces based on some parameters that help the study of the object, such as the number of lines that define the sector, the quality factor or the standard deviation of these lines.

In the wire model, the meshing of these models are created by joining the average coordinates of four sectors contiguous Since four points cannot be coplanar, they can be also create triangular models, especially for the purpose of create faces and apply textures to them later.

In the face model, a color differentiation established based on the factor of quality [0-1]. The application of a scale logarithmic may be adequate to increase resolution in low values As indicated above, the areas of the Object edge will present values closer to unity.

As for the texturing of the object, it consists of apply the corresponding image portion to each triangle of the model. Due to the large number of triangles that form the models (tests with a resolution of 1,440,000 have been performed triangles), it is currently not feasible to create an image for each texture of each triangle, since the file created in VRML would be huge and the computer could not handle it easily.

The provisional solution adopted is to give a solid color to each triangle, which will correspond to the color of the pixel of the coordinate point of the center of each triangle or that of Your average The drawback of this method is that if it is done modeling at a small resolution (less than 300 x 300 sectors, 360,000 triangles), the size of the faces is too large to apply a color only and textures appear granulated

The texture color is taken from the image more appropriate. For each coordinate sector (f, H) the image whose projection center is perpendicular to it, and the collinearity condition achieves the value of pixel

Finally, the software application developed allows the comparison of two modeling performed on the same object under different conditions, with the purpose of  graphically show the differences that appear between them modeling mode depending on different conditions in making video, and where parameters such as which are cited below:

- Shots made at different distances from the object,

- Different value of the focal length of the camera when using different optical zoom factor in both you take,

- Different lighting conditions,

- Different inclinations in the angle of the camera, and

- Coordinate system between the two shots at Do not place the object in the same position on the platform.

The difficulty that arises at the time of compare different models of the same object, is that there are than reduce them to the same terrain coordinate system. TO then, they must also be reduced to the same system of cylindrical coordinates, so that the sectors of both models match in coordinates (F, H) and can be evaluated differences between the r coordinates of both models. For the purpose of comparison of the two models, both should be calculated with the same resolution and present the same intervals of heights and angles

To carry out the comparison of the two models and have them in the same coordinate system, applies a transformation of similarity without deformation. The transformation to apply is a transformation of Helmert in space (Pérez, 2001). The parameters of the transformation are 6, namely: rotation (w, j, k) and translation (T_ {X}, T_ {Y}, T_ {Z}), already that the homotech factor H is worth the unit in this case.

If the coordinates of a control point in the two models are, respectively, (X_ {0}, Y_ {0}, Z_ {0}) and (X_ {1}, Y_ {1}, Z_ {1}), the transformation is given by:

X_ {1} - [(r_ {11} \ times X_ {0} + r_ {12} \ times Y_ {0} + r_ {13} \ times Z_ {0}) - T_ {X}] = 0

Y_ {1} - [(r_ {21} \ times X_ {0} + r_ {22} \ times Y_ {0} + r_ {23} \ times Z_ {0}) - T_ {Y}] = 0

Z_ {1} - [(r_ {31} \ times X_ {0} + r_ {32} \ times Y_ {0} + r_ {33} \ times Z_ {0}) - T_ {Z}] = 0

materializing the adjustment after linearize the system through:

A_ {u, n} \ times X_ {n, 1} - K_ {u, 1} = R_ {n, 1}

where:

u = equations

n = unknowns

In an application example 6 points are used of control, so that the system comprises 18 equations and 6 unknowns

In order to carry out the comparison of coordinates of the sectors of the two models, it is necessary make an adjustment of the transformed points (the Helmert transformation to convert the coordinates of the points of the second models on the first), on the sectors of the first model. This process introduces an error in the comparison of up to half a sector, since each coordinate (X, Y, Z) transformed, the sector of the first model is sought more next.

The numerical comparison of the values of the unknowns obtained from the shots taken, is performed subtracting the r coordinates of the two models. The average of Differences is preferably more than 1 millimeter.

Finally, a visual presentation of  These differences in space. The differences represent the subtraction of the r coordinates of the second model with respect to those of first model. The visual presentation can show the differences positive colored in a different color to the differences negative

As will be understood, although these differences are  small, can be reduced considerably with the use of modeled with higher resolution than the example considered (in the example described, 300 x 300 sectors have been considered).

Claims (8)

1. Three-dimensional modeling procedure using edge conditions, applicable to any object that in its natural state has a mostly convex character with respect to its outer contour with a view to achieving a modeling thereof, which is characterized in that it comprises performing a treatment of the object (2) by analyzing its observed edges from a multiplicity of different perspectives based on those of digital images of the object obtained with the help of a digital video camera and with said object positioned on a rotating platform or stand (1) located at a predetermined distance from said digital capture camera, and because the modeling process is done in two  consecutive phases of which a first phase consists of the proper modeling of the object without going into valuation some, and of which a second phase provides an improvement of modeling obtained in the previous phase, performing the calculation of a set of indicators for the evaluation of characteristics of the model achieved, being all controlled by means of a specific application software, and where the digital treatment of images obtained with respect to the object (2) includes the analysis of the intersection of projective rays (3) with the edges that define said object (2) in each of the image sequences captured, as well as the division into sectors of each of the parts of the object (2) defined in a digital image. 2. Method according to claim 1, characterized in that in the first initial modeling phase, a cylinder determined by a height interval is started, in which are comprised as many sectors per plane and as many horizontal planes as indicated in the parameters entered by the user in the software, the spatial position of each sector being determined by its polar coordinates (F, H, r), corresponding to the polar angle (6), the height (7) and the polar distance (8), the reference origin being located at the central point of the base of the rotating base (1), and determining the minimum distance (r) the minimum distance to the nearest straight line between all those that cross a certain sector. 3. Method according to claims 1 and 2, characterized in that the second phase of modeling improvement and assessment includes the determination of the closest lines of each sector (F, H, r) and the performance of statistical operations on the basis at the minimum radii, according to a two-stage process of which a first stage includes the determination of the nearest lines within a pre-established range (accumulation of lines), and the second stage includes the performance of a statistical study of the distribution geometric of those. 4. The method according to claim 3, characterized in that the first stage of the modeling improvement and evaluation process determined in the first phase of the procedure includes the preparation of an informative table with the lines identified within the pre-established distance range. 5. Method according to claim 3, characterized in that the statistical study provided in the second stage of the process of improvement and assessment of the modeling determined in the first phase of the procedure, in relation to the straight lines that cross a sector, is determined by parameters such as the mean distance to the object, the standard deviation and the average standard deviation. 6. Method according to the preceding claims, characterized in that the management software visually displays graphs distributed according to the number of sectors that have certain values of number of lines, a certain quality factor, and the corrections to be applied. 7. Method according to claims 1 to 5, characterized in that the application software is also capable of displaying, in another modeling visualization, the same results in three-dimensional form, with the inclusion of color legends on the faces of the object. 8. The method according to claims 1 to 7, characterized by the determination of a model quality factor, defined in relation to two points of view, namely the number of points used in the modeling, which preferably has to be greater than a predetermined minimum value depending on the number of sectors of the model and the points defined by each sector, and their quality, determined by the size of the pixels and by the number of lines accumulated.