KR102386982B1 - Method and apparatus for camera calibration using image analysis - Google Patents

Abstract

A camera calibration method using image analysis includes the steps of: receiving, by a calibration apparatus, an image; extracting, by the calibration apparatus, one or more objects from the image; selecting, by the calibration device, searching for an image matching an image of an object to be used as the reference object, and performing, by the calibration device, camera calibration using size information corresponding to the searched image may include

Description

Camera calibration method and apparatus using image analysis {Method and apparatus for camera calibration using image analysis}

The present invention relates to a method and apparatus for automatically performing camera calibration using image analysis. More particularly, it relates to a method of extracting an object from the image by analyzing an image captured or being photographed by a camera, and automatically performing camera calibration using information on the extracted object, and an apparatus for performing the method.

In image analysis, camera calibration refers to a process of calculating a transformation matrix to obtain information on the actual size of an object detected in an image. Although an actual object exists in a three-dimensional space, an object in an image exists in a two-dimensional screen, so distortion naturally occurs. In other words, on the screen, near objects appear larger, and distant objects appear smaller on the screen.

Even if the distortion is corrected and displayed small on the screen, if information on the actual size of the small displayed object, that is, the width, length, and height in three-dimensional space, can be obtained, a lot of information can be obtained through image analysis. For this, a transformation matrix capable of converting 2D coordinates of an image into actual 3D coordinates is required. That is, a transformation matrix that can transform the (x, y) coordinates of the image into the actual (X, Y, Z) is required.

Conventionally, in order to obtain such a transformation matrix, a reference object whose width and length are already known was photographed with a camera, and the coordinates of the reference object were analyzed in the image to perform camera calibration. Although the calibration program helped in the process, manual work such as positioning the reference object at the desired location and setting the coordinates of the reference object was not completely eliminated.

For example, looking at the invention of KR 1545633 B1 "Vehicle Stereo Camera Calibration Method and System" of the Korea Electronics Research Institute, it can be seen that a calibration marker detachable from the bonnet of the vehicle is used to perform the calibration of the vehicle stereo camera. As such, in the prior art, there was a step that consumes time and money, such as having to manually install a calibration marker every time calibration is performed.

If the camera you want to use is a camera with PTZ function (Pan, Tilt, Zoom), you need to calibrate the camera every time after PTZ operation to analyze the image to obtain object information. In addition, even if the camera does not have a PTZ function, if the position of the camera is shifted due to external factors, it is inconvenient to manually calibrate the camera each time.

Accordingly, there is a need for a method capable of automatically performing camera calibration without human manual intervention.

The technical problem to be solved by the present invention is to provide a method and an apparatus for automatically performing camera calibration using image analysis.

The technical problems of the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by those skilled in the art from the following description.

In a camera calibration method using image analysis according to an aspect of the present invention for solving the above technical problem, the calibration apparatus receives an image, the calibration apparatus extracts one or more objects from the image, selecting, by a calibration device, an object to be used as a reference object from among the one or more objects; searching, by the calibration device, for an image matching an image of an object to be used as the reference object; and, by the calibration device, to the searched image The method may include performing camera calibration using the corresponding size information.

In an embodiment, the receiving of the image may include receiving one of a real-time streaming image or a recorded and stored image.

In another embodiment, the extracting of the one or more objects from the image may include detecting the outline of the object by performing an algorithm for detecting the outline on the image.

In another embodiment, the selecting of the object to be used as the reference object includes calculating an area of the object for each extracted object and selecting an object to be used as a reference object by scoring the area can do.

In another embodiment, the selecting of the object to be used as the reference object includes calculating a ratio of the area of the object to the area of the extraction area for each extracted object, and scoring the ratio to determine the reference object It may include the step of selecting an object to be used as

In another embodiment, the step of searching for the image may include storing the image of a specific object and size information of the specific object by using web crawling to correspond to each other in a database.

In another embodiment, after performing camera calibration, tracking an object used as a reference object and when camera calibration is required in another frame of the image, re-calibrate the camera using the size information of the tracked reference object It may include performing steps.

A camera calibration method using image analysis according to an aspect of the present invention for solving the above technical problem includes the steps of: receiving, by a calibration apparatus, an image of a person; and extracting, by the calibration apparatus, a person from the image and confirming, by the calibration apparatus, the extracted face by recognizing the person's face, and performing, by the calibration apparatus, camera calibration using key information of the identified person.

According to some embodiments of the present invention, by analyzing an image, camera calibration may be performed not only on an image currently being photographed but also on an image captured in the past. That is, conventionally, image analysis is possible only by photographing an image after performing camera calibration using a reference object. According to the present invention, even an image captured without performing camera calibration can be calibrated later.

In addition, when performing camera calibration, it is possible to improve problems that require manual work, such as a person repeatedly installing a reference object and inputting coordinates. This can improve the accuracy of camera calibration and streamline the process. In addition, unnecessary waste of human resources and money can be reduced.

Effects of the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art from the following description.

1 is an exemplary diagram for explaining camera calibration.
2 is an exemplary view for explaining a conventional camera calibration method.
3 is a flowchart of a conventional camera calibration method.
4 is a hardware configuration diagram of a camera calibration apparatus using image analysis according to an embodiment of the present invention.
5A to 5F are exemplary views for explaining each configuration of a camera calibration apparatus using image analysis according to an embodiment of the present invention.
6A to 6B are exemplary diagrams for explaining a criterion for detecting an object through image analysis in an embodiment of the present invention.
7 is a flowchart of an automated camera calibration method using face recognition according to an embodiment of the present invention.
8 is a flowchart of an automated camera calibration method using product recognition according to an embodiment of the present invention.
9 is an exemplary diagram illustrating a case in which camera calibration is applied to an image of a vehicle black box according to an embodiment of the present invention.
9B is an exemplary diagram illustrating a case of performing camera calibration by tracking an object according to an embodiment of the present invention.
10 is a flowchart of a camera calibration method using image analysis according to an embodiment of the present invention.

Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Advantages and features of the present invention, and a method for achieving them will become apparent with reference to the embodiments described below in detail in conjunction with the accompanying drawings. However, the present invention is not limited to the embodiments published below, but may be implemented in various different forms, and only these embodiments allow the publication of the present invention to be complete, and common knowledge in the technical field to which the present invention pertains. It is provided to fully inform the possessor of the scope of the invention, and the present invention is only defined by the scope of the claims. Like reference numerals refer to like elements throughout.

Unless otherwise defined, all terms (including technical and scientific terms) used herein may be used with the meaning commonly understood by those of ordinary skill in the art to which the present invention belongs. In addition, terms defined in a commonly used dictionary are not to be interpreted ideally or excessively unless clearly defined in particular. The terminology used herein is for the purpose of describing the embodiments and is not intended to limit the present invention. In this specification, the singular also includes the plural unless specifically stated otherwise in the phrase.

As used herein, "comprises" and/or "comprising" refers to the presence of one or more other components, steps, operations and/or elements mentioned. or addition is not excluded.

Hereinafter, the present invention will be described in more detail with reference to the accompanying drawings.

1 is an exemplary diagram for explaining camera calibration.

Referring to FIG. 1 , when a three-dimensional object 131 in a real space is photographed with the camera 210, a two-dimensional object 135 projected in two dimensions in a 2D image plane is used. expression can be seen. In spite of being expressed in two dimensions, when a person recognizes a two-dimensional image, a sense of distance may be felt due to perspective.

The problem is that distortion occurs while the 3D object 131 is projected onto the 2D object 135 . As a result, there is a difference between the coordinates in the three-dimensional space and the coordinates in the two-dimensional space. Camera calibration is to compensate for such a difference. If camera calibration is used, it is possible to know how many meters in height the 3D object 131 is in the real space of the 2D object 135 measuring 320 pixels in height in the image. there is.

As such, camera calibration refers to a process of obtaining internal parameters in order to restore an object 135 on a two-dimensional image to an actual object 131 on a three-dimensional space. Based on the projected two-dimensional data, external factors such as lenses, sensors, and shooting angles are removed and the actual size is measured. To this end, the correlation between the two-dimensional space and the three-dimensional space can be identified through the process of comparing the pixel information of the object extracted from the image and the size of the real object.

In order to perform camera calibration, conventionally, an object of known size is artificially placed in a three-dimensional space and photographed to perform camera calibration. A conventional camera calibration method will be described with reference to FIGS. 2 to 3 .

2 is an exemplary view for explaining a conventional camera calibration method.

Referring to FIG. 2 , a conventional camera calibration process can be seen. Conventionally, the marker 110 was photographed with the camera 210 to obtain input data of the calibration program 120 . And after loading the image including the marker 110 in the calibration program 120, the coordinates of the marker 110 were identified. Conventionally, in order to easily identify the coordinates of the marker 110, as shown in FIG. 2, the marker 110 in which black and white intersect in a checkered pattern is mainly used. Alternatively, in addition to the marker 110 shown in FIG. 2, a square or rectangular panel made of a specific color was used as a reference object.

The calibration program 120 analyzes the loaded image and extracts the coordinates of each vertex of the marker 110 or the reference object. A transformation matrix can be obtained by comparing the size information of the actual marker 110 or the reference object with the coordinate information in the image. However, even if the marker 110 in which black and white intersect in a checkered pattern or a reference object having a specific color is used, the coordinates of each vertex are often not correctly extracted. In this case, as in the example of FIG. 2 , a process of manually setting the coordinates of each vertex by a person is additionally required.

As shown in FIG. 2 , in the existing camera calibration process, 1) a process of installing a marker 110 or a reference object in a space where the camera 210 is shooting, and 2) a calibration program 120 for analyzing the captured image. A person had to directly perform the process of setting the coordinates of the marker 110 or the reference object. Therefore, when the position of the camera 210 is changed, human intervention is required to perform camera calibration again.

Of course, in the case of a stationary camera that takes a picture of a specific space at a specific angle, it is sufficient to perform camera calibration only once in the process of installing the camera 210 , so it is less inconvenient. However, like a PTZ camera, if the space in which the camera 210 shoots can be changed from time to time, it is almost impossible to perform camera calibration again each time.

3 is a flowchart illustrating the conventional camera calibration process described in FIG. 2 by dividing a step requiring user intervention and a step automatically performed through the calibration program 120 as shown in FIG. 3 .

3 is a flowchart of a conventional camera calibration method.

In FIG. 3 , in the conventional camera calibration process, a manually performed step is shown in the left User Action area, and an automatically performed step is shown in the right Camera Calibration Program area.

First, in order to perform camera calibration, the marker 110 is positioned where the camera 210 is looking so that the marker 110 can be projected on an image (S1100). Next, when the installation of the marker 110 is finished, the image analysis mode is switched to the camera calibration mode to perform camera calibration (S1200). As shown in FIG. 3 , steps S1100 and S1200 are manually performed steps requiring human intervention.

When preparation for performing camera calibration is completed, the camera 210 captures a space including the marker 110 (S1300). Then, the calibration program 120 loads the captured image, analyzes the loaded image, and checks whether the marker 110 is automatically detected (S1400). If the marker 110 is automatically detected, the coordinates of the marker 110 are extracted from the image (S1600). Steps S1300, S1400, and S1600 are steps that can be automatically performed within the calibration program 120 . However, when the marker 110 is not automatically detected even after analyzing the captured image, human intervention is required again (S1500).

Therefore, in the prior art, in order to reduce manual labor, that is, to easily detect the marker 110 in a photographed image, as shown in FIG. 2 , the marker 110 in which black and white are crossed in a checkered pattern is mainly used. However, there may be cases in which it is difficult to detect the marker 110 depending on the photographing environment. For example, when the camera 210 takes pictures of an outdoor space and the space is dark, or the main color of the space where the camera 210 is photographed and the color of the marker 110 are similar, the marker 110 is identified by the protective color effect. This may be difficult.

In this case, it is necessary for a person to manually specify the coordinates of the marker 110 in the camera calibration program 120 (S1500). After automatically extracting the coordinates of the marker 110 in the camera calibration program 120 (S1600) or extracting them manually (S1500), a transformation matrix is calculated by comparing the coordinates extracted from the image with the actual coordinates. (S1700).

That is, in reality, an object having a size of x1 (meter) is digitized as a transformation matrix by a correlation expressed by a size (pixel) of x2 in an image captured by the camera. After the transformation matrix is obtained, the transformation matrix is transmitted to the image analysis algorithm and the camera calibration process is terminated (S1800).

In this process, the process of photographing the marker 110 may be repeatedly performed in order to increase the accuracy of the transformation matrix. That is, the steps indicated by dotted lines in FIG. 3 may be repeatedly performed. The camera 210 installs the marker 110 at another location in the space being photographed, extracts the coordinates by taking an image, and installs the marker 110 at another location and extracts the coordinates by taking the image. can be done The more input data is collected in this way, the more accurate the transformation matrix can be. Of course, human manual labor is also repeatedly required in this process.

The present invention proposes a method and apparatus capable of automatically performing camera calibration in order to solve a problem requiring human manual operation. To this end, it is necessary to be able to solve two kinds of manual tasks: 1) installing the marker 110 and 2) extracting the coordinates of the marker 110 (if necessary). To this end, in the present invention, the camera calibration is performed by selecting a reference object from an image already taken or from an image being photographed, rather than artificially installing a reference object and photographing.

4 is a hardware configuration diagram of a camera calibration apparatus using image analysis according to an embodiment of the present invention.

Referring to FIG. 4 , the camera calibration apparatus 10 using image analysis according to an embodiment of the present invention receives an image 220 through an external input. At this time, the received image 220 may be an image that has already been photographed or an image that is currently being photographed. That is, both recorded video and real-time streaming can be used as inputs. As such, the camera calibration apparatus 10 using the image analysis receives the image 220 photographed through the external camera 210 through the receiver 310 .

The detector 320 analyzes the received image 220 to detect objects to be used as reference objects. In this case, an object can be selected according to two criteria. One is the size of the object and the other is the proportion of the object in the detection area. If the size of an object is an absolute standard, the ratio of an object is a relative standard. This will be described in more detail later with reference to FIGS. 6A to 6B .

The registration unit 330 builds a database of objects detected by the detection unit 320 and objects to be compared. That is, the registration unit 330 collects images of objects and object information by manual or automated methods in advance, and builds them into a database. The database of objects constructed in this way is later utilized by the matching unit 340 .

The matching unit 340 compares and matches the object detected in the image 220 with the object stored in the database. Through this, by using the object information stored together with the object image in the database, it is possible to know the information of the object detected in the image. That is, information on the object detected in the image 220 is grasped by using a database built in advance.

The processing unit 350 performs camera calibration by comparing the information on the 2D image of the object detected from the image 220 identified through the matching unit 340 with the information on the 3D real space. The process of calculating the transformation matrix using the coordinates of the two-dimensional image and the coordinates of the three-dimensional real space may use a conventional camera calibration algorithm.

3 and 4, in the camera calibration method according to an embodiment of the present invention, the process of artificially installing a reference object in a space in which the camera 210 is photographing is omitted. Instead, camera calibration is performed by detecting an object that can be a reference object in the image.

Conventionally, it is possible to analyze an image only by performing camera calibration before capturing an image. That is, in the case of shooting without performing camera calibration, size information could not be grasped through image analysis. In contrast, in the present invention, since a reference object is selected through image analysis and camera calibration is performed separately from shooting, camera calibration is possible even after shooting.

In the present invention, by omitting an artificial pre-processing task of installing a reference object required to perform camera calibration, image capturing and camera calibration can be performed independently. Even for an image captured without performing camera calibration in advance, camera calibration is performed and image analysis is performed to determine actual size information of objects in the image. Through this, it is possible to further increase the efficiency in analyzing not only the image being photographed but also images that have already been taken.

5A to 5F are exemplary views for explaining each configuration of a camera calibration apparatus using image analysis according to an embodiment of the present invention.

5A is a diagram for explaining an image 220 used as an input in the present invention. The image 220 is essential data that is basically required for analysis. Here, the image 220 refers to various image information input to the system. That is, in addition to the image being photographed, that is, the real-time image used in the conventional camera calibration, it includes an image already captured. In the present invention, the image 220 is a concept that includes not only real-time streaming but also stored video files.

In the present invention, the camera calibration is not performed by installing a reference object before photographing with the camera 210, but by analyzing the image 220 being photographed or photographed to select a reference object to perform camera calibration, so that not only real-time streaming data but also real-time streaming data is performed. Camera calibration can also be performed on videos that have already been recorded and stored. As described above, according to an embodiment of the present invention, it is possible to solve the problem that the conventional camera calibration method has to perform before capturing the image 220 .

5B is a diagram for explaining the receiver 310 of the camera calibration apparatus 10 according to an embodiment of the present invention. Referring to FIG. 5B , the camera calibration apparatus 10 of the present invention receives an image 220 captured by an external camera 210 and uses it as an input. In this process, the receiver 310 may decode the image 220 . The decoded image may be later analyzed in units of frames through the detector 320 to extract an object and determine whether to use it as a reference object.

5C is a view for explaining the detection unit 320 of the camera calibration apparatus 10 according to an embodiment of the present invention. Referring to FIG. 5C , the detection unit 320 of the present invention serves to extract an object 230 that can be a reference for calibration from a received image. That is, in the conventional camera calibration method, a reference object is installed and photographed before an image is taken, whereas in the present invention, a reference object is selected from an image already taken.

By detecting the object 230 through image analysis and comparing the image of the detected object 230 with a database built in advance, it can replace a reference object installed before taking a conventional image. To this end, the detector 320 extracts objects 230 that can be used as reference objects. The extracted objects 230 may determine whether to use them as reference objects according to several criteria. In this process, scoring is performed according to a criterion, and only objects with high scores can be used as reference objects with priority.

In addition, error can be reduced by performing camera calibration using a plurality of objects as reference objects instead of using only one reference object. A detailed standard related to whether to use an object from among the objects extracted from the image as a reference object will be described in more detail with reference to FIGS. 6A to 6B .

In the process in which the detector 320 extracts the object, an edge detection algorithm for analyzing an image and detecting an outline may be used. An algorithm for detecting an outline analyzes the image 220 to determine a point where the brightness discontinuously changes as an outline, and extracts a pixel at the point. Since there are various algorithms such as Sobel, Prewitt, Robert, Laplacian, and Canny for detecting the outline, one or more of them may be used to detect the boundary and extract the object 230 .

5D is a view for explaining the registration unit 330 of the camera calibration apparatus 10 according to an embodiment of the present invention. Referring to FIG. 5D , the registration unit 330 of the present invention builds the database 331 through two paths. Among them, the first is a method of crawling the web page 240 in an automated manner (Auto Data Gathering), matching the image and information of a specific object, and storing it in the database 331 .

For example, if you visit various shopping malls or homepages of products, you can see product-related information listed in text at the bottom and images to explain the product. A bot automatically collects such information, purifies and processes the data, and stores it in the database 331 . Therefore, the objects 230 that can be used as reference objects in the present invention are often industrial products. That is, it can be seen a lot in the vicinity, and images of products having the same width, length, and height of each object can be used as a reference object.

Large and standardized objects such as automobiles, televisions, and refrigerators can be used as reference objects. However, the reference object is not necessarily limited only to these industrial products. As will be seen later in FIG. 7 , an image of a person may also be used as a reference object. That is, objects that can be used as reference objects 1) can be compared with an image of a previously registered database, 2) size information (horizontal, vertical, horizontal, vertical, height), it is sufficient if two conditions are satisfied.

The rest of the method of building the database 331 in the register 330 is a manual method (Manual Data Gathering) that provides a user interface (UI), so that an image of a specific object and size information of the specific object can be input from the user. there is. The user may build the database 331 by predicting and storing objects that can be detected by the camera 210 in advance. However, the passive method will be an auxiliary method provided by the registration unit 330 to build the database 331 .

5E is a view for explaining the matching unit 340 of the camera calibration apparatus 10 according to an embodiment of the present invention. Referring to FIG. 5E , the matching unit 340 of the present invention compares the object 251 extracted by the detection unit 320 with the object 259 stored in the database 331 . When the extracted object 251 and the registered object 259 match, the size information matched with the registered object 259 is searched and transmitted to the processing unit 350 .

The processing unit 350 may perform camera calibration using size information (unit: pixel) on the screen of the extracted object 251 and size information (unit: meter) of the registered object 259 in real space. there is. For example, when a fire extinguisher is detected in the image 220 and the matching unit 340 searches the database 331, when a matching fire extinguisher is found, the fire extinguisher specification stored in the database 331 is processed by the processing unit 350 can be forwarded to

5F is a diagram for explaining the processing unit 350 of the camera calibration apparatus 10 according to an embodiment of the present invention. Referring to FIG. 5F , it is possible to see the process of recovering the size information displayed in pixels of the extracted object 251 from the image 220 as an object in real space to obtain the actual size information displayed in meters of the object. there is.

As such, the processing unit 350 performs camera calibration with size information of the matched object and camera information (focal length, resolution, sensor width, sensor height, camera height, etc.). In this process, a specific calculation process or algorithm related to camera calibration may utilize matlab or opencv used in a conventional camera calibration method.

Up to now, each component of the camera calibration apparatus 10 according to an embodiment of the present invention has been described with reference to FIGS. 5A to 5F . Here, a criterion regarding which object is used as a reference object among a plurality of objects 230 detected by the detection unit 320 through edge detection in the image 220 will be described with reference to FIGS. 6A to 6B .

6A to 6B are exemplary diagrams for explaining a criterion for detecting an object through image analysis in an embodiment of the present invention.

Whether to use an object as a reference object among the objects 230 detected through the contour detection by the detector 320 may be selected based on two criteria. One is the size of the detected object, which may be scored through a process of obtaining an area from the 2D image 220 . The other is the ratio of the detected object to the detection area, which may be scored through a process of comparing areas in the 2D image 220 .

The size of the object 230 is a criterion in absolute terms, and the ratio is a criterion in a relative dimension. Since the objects 230 extracted from the image 220 should be compared with the images of the objects stored in the database 331 through the matching unit 340, the larger the size of the extracted objects 230, that is, the area in the two-dimensional area. The wider the width, the easier it is to compare images in the matching unit 340 .

Also, since the image 220 is basically a two-dimensional plane, it has a rectangular shape. Here, the closer the contour of the detected object 230 is to a rectangle, the easier it is to compare the images in the matching unit 340 . That is, even if the area is the same, it is easier to compare an object closer to a rectangle with the image of the object stored in the database 331 .

Referring to FIG. 6A , it can be seen that the ratio of the object A 231a and the object B 231b is compared. The object A 231a is an L-shaped object in the two-dimensional image 220, and is a figure having a short width of 30 pixels, a long width of 100 pixels, a short length of 40 pixels, and a length of 80 pixels. The ratio that the object A 231a occupies in the rectangular area 100x80 can be obtained by dividing the area of the object A 231a by the area of the rectangular area 80x100. For object A(231a), through the formula (100 * 40 + (80 - 40) * 30) / (100 * 80) * 100 = 65% , It can be seen that the actual calculation result is 65%.

Similarly, the object B 231b is a trapezoidal object in the 2D image 220 and is a figure having a height of 80 pixels, an upper side length of 80 pixels, and a bottom side length of 100 pixels. It can be seen that the ratio of the object B 231b in the rectangular area 100x80 is 90% through the formula ((100 + 80) * 80 / 2) / (100 * 80) * 100 = 90%. It can be seen that the object B 231b is a figure closer to a rectangle than the object A 231a. In terms of the ratio only, object B (231b) is more suitable for the reference object than object A (231a).

Referring to FIG. 6B , a process of obtaining the size of the object 230 can be seen. It is based on the fact that it is easy to compare when the size of the object is larger even if the ratio is the same. In FIG. 6B , an L-shaped object A 231a and an object C 231c can be seen in the same way. Object A (231a) and object C (231c) are similar, and the similarity ratio is 2:1. Since the shape of the figure is the same, if the ratio is calculated, both the object A 231a and the object C 231c are 65% as shown in FIG. 6A .

However, when calculating the actual area, object A(231a) is (100 * 40 + (80 - 40) * 30) = 4000 + 1200 = 5200, and object C(231c) is (50 * 20 + (40 - 20) * 15) = 1000 + 300 = 1300. Since the similarity ratio of object A (231a) and object C (231c) is 2:1, it can be confirmed that the area ratio is 4:1, which is the square. Even if the proportion of objects in the rectangular detection area is the same, the larger the size of the object, the easier it is for the matching unit 340 to perform the comparison. Therefore, the object A 231a is more suitable for the reference object than the object C 231c. Do.

An object to be used as a reference object may be selected from among the objects 230 extracted from the image 220 based on two criteria of size and ratio. In this process, scoring may be performed for each extracted object 230 by performing scoring and scaling. For example, the final score of the extracted object 230 may be calculated by dividing the area of the object 230 by the screen resolution, performing scaling to a value less than 100, and multiplying this value by a ratio.

Based on the calculated final score, the top 5 objects can be used as reference objects to perform matching and perform camera calibration. Through this, it is possible to minimize an error that may occur when one reference object is used. However, the scoring and scaling process described now is only an example, and in addition to this, an object to be used as a reference object may be selected by comparing the numerical value of each object 230 by digitizing the size and ratio according to various methods.

7 is a flowchart of an automated camera calibration method using face recognition according to an embodiment of the present invention.

You can easily come across articles related to the recent technology leak. As security becomes so important, most companies control access by databaseizing employees' face photos or fingerprints. In addition, not only face photos or fingerprints, but also employee body information, such as height and weight, are sometimes used for security.

If it is assumed that a company in which the face photos and keys of employees are previously constructed as a database 331 , camera calibration may be performed for several surveillance cameras installed in the company using the information constructed in the database 331 . That is, instead of separately performing camera calibration for each camera while installing surveillance cameras throughout the company, the camera is simply installed, and camera calibration is performed using the employee's face picture and key information built into the database 331 . You may.

Referring to FIG. 7 , the surveillance camera analyzes an image being shot (S2100), and recognizes a face (S2200). In the process of recognizing a face like contour detection, various conventional face detection algorithms may be used. When the face is not detected, the image is continuously monitored (S2100), and when the face is detected, it can be compared with the face image stored in the database 331 (S2300).

If it does not match the face image stored in the database 331, the image is continuously monitored again (S2100), and when it matches the face image stored in the database 331, information about the face object, that is, the person ) inquires (S2500). Through this, camera calibration can be performed by using that the height of a person represented by x pixels in the image is x' cm in real space (S2600).

Conventionally, camera calibration had to be performed by installing cameras throughout the company and installing a reference object for each camera, but in the present invention, camera calibration is performed just by passing a person whose face image is stored in the database 331 in the installed camera. can By reducing the manual work required for camera calibration, the efficiency of image analysis can be increased.

8 is a flowchart of an automated camera calibration method using product recognition according to an embodiment of the present invention.

Referring to FIG. 8 , a process of performing camera calibration on an image captured in a specific indoor space can be seen. That is, in the case of FIG. 8 , camera calibration is performed on an already stored image. In the present invention, since a reference object is selected in an image, there is an advantage that such a post-mortem camera calibration is possible.

First, the recorded image is analyzed (S3100), and an object is recognized (S3200). In this process, an edge detection algorithm can be used. When the object is not detected, the image is continuously monitored (S3100), and when the object is detected, it can be compared with images of products stored in the database 331 (S3300).

If it does not match the TV image stored in the database 331, the image is continuously monitored again (S3100), and when it matches the TV image stored in the database 331, the corresponding object, that is, information about the TV is stored in the database 331 Inquire at (S3500). Through this, camera calibration can be performed by using that the height of the product represented by x pixels in the image is x' cm in the real space (S3600).

If the camera is calibrated, even when a thief is found in the store, the camera calibration is performed on the recorded image using the display installed in the store, and the thief's key can be inferred through this. As such, the reference object that can be used in the present invention may be a human as shown in FIG. 7 or an industrial product as shown in FIG. 8 .

9A is an exemplary diagram illustrating a case in which camera calibration is applied to an image of a vehicle black box according to an embodiment of the present invention.

Since the present invention performs camera calibration by analyzing an image, a camera calibration method further using image analysis may be applied. That is, after camera calibration is performed by selecting an object from a specific image, the object can be continuously tracked and reused when camera calibration is required again.

Referring to FIG. 9A , it can be seen that the vehicle 261a in the next lane is detected in the vehicle black box and camera calibration is performed based on this. Even when the vehicle 261a in the next lane moves and the size of the vehicle 261a in the next lane is reduced and accurate recognition is impossible (261b), since there is information that has already been calibrated in advance, the position of the camera is changed and camera calibration is required again. Even in a situation, camera calibration may be performed again using the camera calibration previously applied to the vehicle 261a in the next lane.

For example, assuming that the vehicle 261a in the next lane is a vehicle having dimensions of 3495*1495*1500mm in real space as the Matiz, the width is detected as 80 pixels in the first scene of FIG. 9A, and camera calibration is performed based on this. In the future, when the angle of view captured by the black box changes while the vehicle is driving or the screen being photographed is changed due to the location adjustment of the black box, camera calibration may need to be performed again.

In this case, in the prior art, there was an inconvenience in that the reference object had to be manually positioned again and the camera had to be calibrated. However, in the present invention, the camera calibration can be performed again without the need to track the object on which the camera calibration has been performed before and compare it with the image of the object stored in the database 331 .

That is, as in the second screen, even if the size of the car Matiz 261b is reduced on the screen, camera calibration can be performed for the vehicle 261b detected this time using the information of the vehicle 261a previously detected on the screen. there is. In the second screen, even if the size of the vehicle 261b is reduced to 40 pixels, the matching unit 340 does not search the database 331 for the vehicle 261b that is reduced in size, but instead takes the vehicle 261a detected in the first screen and uses the camera. Calibration can be performed. Through this, even when the camera calibration is performed using the small image object 261b as the reference object, the accuracy of the camera calibration may be increased.

This is because the present invention performs camera calibration using image analysis, so even if there is no suitable object to be used as a reference object in the frame of the image to be currently calibrated, if there is an object that can be used as a reference object before and after the image frame, the object is tracked This has the advantage of being able to calibrate the camera of the current frame.

9B is an exemplary diagram illustrating a case of performing camera calibration by tracking an object according to an embodiment of the present invention.

Referring to FIG. 9B , it can be seen that object A, object B, and object C are exposed for a predetermined time on the screen based on the time axis. At this time, assuming that the object most suitable as a reference object among the three objects is object C, size information can be obtained in the order of object C > object B > object A. That is, if camera calibration is required to analyze the image at the time point A was photographed, camera calibration is performed based on the image at the time point object C was photographed to obtain information on the size of object B in real space, and using this Again, the size of the object A may be obtained from the image at the point in time when the object A is photographed.

10 is a flowchart of a camera calibration method using image analysis according to an embodiment of the present invention.

Referring to FIG. 10 , in the method for calibrating a camera using image analysis according to an embodiment of the present invention, first, it is checked whether there is camera calibration information in the current image ( S4100 ). When the current camera calibration information is valid, since additional camera calibration is not required, the process ends (S4200).

However, if the current camera calibration information needs to be updated by performing camera calibration again due to a camera position change or setting change, a step of detecting an object in the image is performed (S4300). For the detected objects, an object to be used as a reference object is selected based on the size and ratio (S4400).

When an object to be used as a reference object is selected, it is transmitted to the matching unit 340 (S4500), and the matching unit 340 compares it with the image of the object stored in the database 331 (S4600). In the case of matching with the image stored in the database 331, the size information in the real space is checked by inquiring the specifications of the image (S4700).

Through this process, the size information in the image and the size information in real space are matched, and camera calibration is performed based on this to convert the two-dimensional (x,y) coordinates into three-dimensional (X, Y, Z) coordinates. Calculate the transformation matrix that can be done. By using the calculated transformation matrix, size information of an object other than the object used as a reference object in the image in real space can be restored.

Although embodiments of the present invention have been described with reference to the accompanying drawings, those of ordinary skill in the art to which the present invention pertains can realize that the present invention can be implemented in other specific forms without changing the technical spirit or essential features. you will be able to understand Therefore, it should be understood that the embodiments described above are illustrative in all respects and not restrictive.

Claims (8)

Receiving, by the calibration device, an image;
extracting, by the calibration device, one or more objects from the image;
selecting, by the calibration device, an object to be used as a reference object from among the one or more objects;
searching, by the calibration device, an image matching an image of an object to be used as the reference object; and
Comprising the step of performing, by the calibration apparatus, camera calibration converting the size of an area occupied by pixels representing an object to be used as the reference object in the image using size information in real space of the searched image,
A method of camera calibration using image analysis. The method of claim 1,
Receiving the image comprises:
Receiving one of a real-time streaming video or recorded and stored video,
A method of camera calibration using image analysis. The method of claim 1,
Extracting one or more objects from the image comprises:
Comprising the step of detecting the outline of the object by performing an algorithm for detecting the outline on the image,
A method of camera calibration using image analysis. The method of claim 1,
The step of selecting an object to be used as the reference object comprises:
Comprising the step of calculating the area of the object for each extracted object and selecting an object to be used as a reference object by scoring the area,
A method of camera calibration using image analysis. The method of claim 1,
The step of selecting an object to be used as the reference object comprises:
Comprising the step of calculating the ratio of the area of the object to the area of the extraction area for each extracted object, and selecting an object to be used as a reference object by scoring the ratio,
A method of camera calibration using image analysis. The method of claim 1,
The step of retrieving the image is
Comprising the step of storing the image of a specific object and the size information of the specific object in correspondence using web crawling in a database,
A method of camera calibration using image analysis. The method of claim 1,
tracking an object used as a reference object after performing camera calibration; and
Comprising the step of performing camera calibration again using the size information of the tracked reference object,
A method of camera calibration using image analysis. Receiving, by the calibration device, an image of a person;
extracting, by the calibration device, a human object from the image;
confirming, by the calibration device, an identity by recognizing the face of the extracted human object; and
performing camera calibration in which the calibration device converts the size of an area occupied by pixels representing the extracted human object in the image using key information in the real space of the person whose identity has been confirmed doing,
A method of camera calibration using image analysis.

Images