TWI382267B - Auto depth field capturing system and method thereof - Google Patents

Description

Automatic depth of field capture system and automatic depth of field capture method

The present invention relates to an image capture system, and more particularly to an automatic depth of field capture system for a camera and an automatic depth of field capture method.

Since the beginning of the twentieth century, television has evolved from black and white and color to today's digital TV. It is a human challenge to the visual boundaries and a manifestation of technological excellence. Today in the 21st century, human beings continue to make breakthroughs in display technology, hoping to bring another different visual image revolution to humanity. And display technology is moving from a more vivid and detailed picture to a more realistic direction.

In the past, the display media were mainly based on two-dimensional displays. The video tube TV and the computer screen went all the way to the modern LCD screen and plasma TV. But for humans, binocular vision is more real and natural. Under such a trend, the provision of stereoscopic content will also become a necessity for digital photos in the new era. The stereoscopic image can present images of different viewpoints to the viewer's eyes. On the dual-view 3D television system, the stereoscopic vision can be presented to the audience, especially when presenting concerts, movies, sports, and scenic spots. If you want to have a three-dimensional sense, knowing the "depth" of the subject is the most important.

According to the conventional skill, it is necessary to take a stereo photo with a camera, and most of them must use a double lens and a double negative film. In the digital age, you can use a dual-lens double-sensing element. As long as the camera position is corrected to the appropriate position, you can complete the stereo photo. Filming. In order to solve the above problems, a calibration method for a machine vision measuring system for more than one camera is disclosed in U.S. Patent No. 6,959,253. In addition, U.S. Patent No. 6,781,618 discloses a method for constructing a three-dimensional scene model by first taking a scene with an unknown feature to obtain a first image, and then photographing another scene with a known feature with a second camera. Corresponding to the second image. The first and second cameras have a fixed entity relationship with each other. Its three-dimensional model can be analyzed by using the corresponding orientation and fixed entity relationships of the two cameras.

See Figure 1. It discloses a block diagram of the circuit structure of the stereoscopic image capturing device of U.S. Patent No. 6,977,674. As shown in Fig. 1, a CCD having an RGB wafer color filter 14 is used as its image forming apparatus 11. That is, the color filters applied to the apertures 22R, 22G, and 22B correspond to the color filters used by the imaging device 11. The image signal detected by the imaging device (CCD) 11 is supplied to the image processing unit 30, so that the signal is converted from the analog signal into a digital signal and then subjected to preset signal processing. The image capturing operation of the stereoscopic image capturing device and the image data recording operation of the recording medium M are all controlled by the operation of the operation switch group 34. Although only one camera lens is used to capture stereoscopic images, the camera lens should be designed in a specific and complex shape and further implemented by many restricted devices.

The above apparatus uses several calibration lenses for the purpose of capturing stereoscopic images, where these devices are larger and more complex. Therefore, image calibration is considered to achieve the goal of reconstructing three-dimensional images. The 3D image reconstruction method comprises the following steps: temporarily storing a front view image of a scene, combining the photogrammetric image and the industrial drawing of the scene to form an alignment image of the front view and the perspective view, and then from the COP image Like reconstructing 3D images. However, photogrammetry images must be taken in advance by several cameras.

A three-dimensional orientation and orientation sensing device is disclosed in U.S. Patent No. 6,724,930. Referring to FIG. 2, a block diagram of a three-dimensional orientation and orientation sensing device in accordance with U.S. Patent No. 6,724,930 is incorporated. As shown in FIG. 2, a plurality of markers 2 (abbreviated as code markers) having unique geometric features are placed on or adjacent to the object in which the three-dimensional orientation and direction is to be estimated. These code marks 2 are all taken by the image capturing device 3, and the captured image 5 is transmitted to the computer 4. In theory, the object 1 and the image capturing device 3 have their own coordinate system, and the image 5 captured by the image capturing device 3 is defined as a camera image plane. However, image 5 should be further processed by a computer. After the computer 4 receives the image 5, the computer 4 finds a candidate region corresponding to the code flag 2 from the image 5. The computer 4 analyzes the identified candidate regions in detail and then calculates the geometric features of the code corresponding to the candidate region code flag 2. Once the code is recognized, the computer temporarily stores the location in the image and code by treating the area as a marker area. Finally, by using the two-dimensional image orientation of the code mark 2 temporarily stored in the image in step 2 and the three-dimensional orientation of the code mark 2 of the object 1, the computer 4 can calculate the three-dimensional object 1 of the image capturing device 3. Direction and direction. Computer analysis has introduced many operations at the same time.

However, in practical applications, it is difficult to implement because the conventional technique requires more than one camera and many complicated calibration lenses or many computer programs to operate in three-dimensional depth capture. Therefore, it is desirable to provide a system and method for obtaining object depth by means of digital signal processing, by providing a difference vector and a camera external parameter for the depth conversion module to obtain Its depth, which simplifies the entire structure and process, and achieves the purpose of automatically obtaining the depth of the object without changing the camera itself, thus making it easier for the user to take stereoscopic images and correct the shortcomings of the prior art and solve the above problems.

Some of the features of the present invention are set forth in this section, and other exemplary embodiments that embody the features and advantages of the present invention are described in detail in the following description. It is to be understood that the invention is capable of various modifications in the various embodiments of the invention invention.

As mentioned previously, the prior art is limited by the above problems. An object of the present invention is to provide a method for processing the depth of an object through a digital signal processing method for an automatic depth of field capture system of a camera. By providing a difference vector and a camera external parameter to the depth conversion module to obtain the depth thereof, the simplification is simplified. The entire structure and process, and the purpose of automatically obtaining the depth of the object without changing the camera itself, makes it easier for the user to take stereoscopic images, and can correct the shortcomings of the prior art and solve the above problems.

According to one aspect of the present invention, an automatic depth of field capture system for a camera includes a camera lens for capturing a plurality of images, and a camera calibration device including a plurality of external polar data estimates for estimating a plurality of images. a module; and a camera external parameter estimation module for estimating position data; at least one frame buffer for temporarily storing a plurality of images; and an image correction module for correcting a plurality of external pole data Multiple shadows And a plurality of corrected images are obtained; a difference estimation module is connected to the camera calibration device and the image correction module for receiving the position data and obtaining a difference vector of the corrected image; and a difference to the depth conversion module is used to obtain a response The depth vector and the depth of the position data; and a depth image rendering module for drawing a three-dimensional image corresponding to the depth.

According to the concept of the present invention, a plurality of images can be taken by the camera in different directions.

According to the concept of the case, the plurality of external pole data are a plurality of outer pole lines and outer poles.

According to the concept of the present invention, the external pole estimation module further uses a tracking algorithm to generate a matrix that responds to a plurality of outer and outer poles.

According to the present invention, the matrix includes a relative motion vector and a relative direction vector between at least two of the plurality of images.

According to the concept of the case, the location data is the three-dimensional orientation and angle of the camera.

Another object of the present invention is to provide an automatic depth of field capture method for obtaining object depth by means of digital signal processing, which provides a depth vector and a camera external parameter to obtain depth by using a difference to the depth conversion module, which simplifies the entire structure. And process. And the purpose of automatically obtaining the depth of the object without changing the camera itself is achieved, so that it is easier for the user to take a stereoscopic image, and the shortcomings of the conventional techniques can be corrected and the above problems can be solved.

In accordance with the teachings of the present invention, an automatic depth of field capture method for a camera includes the steps of a) capturing a plurality of images; b) estimating a plurality of outer polar data of the plurality of images to obtain a matrix; c) estimating the response to the plurality of outer polarities data And the position data of the matrix; d) correcting the plurality of images corresponding to the plurality of outer pole data to obtain a plurality of corrected images; e) calculating the position data to obtain the difference vector of the corrected image; f) obtaining the response vector and the position data Depth of field; and g) draw a three-dimensional image corresponding to the depth of field.

According to the present concept, step a) is performed via a camera lens.

According to the concept of the present case, a plurality of images are taken by the camera in different orientations.

According to the present concept, step b) is performed via an external pole estimation module.

According to the concept of the present case, a plurality of external pole data are a plurality of outer pole lines and outer poles of a plurality of images.

According to the concept of the present case, the basic matrix in response to a plurality of outer and outer poles is obtained by means of a tracking algorithm.

According to the present concept, the basic matrix includes a relative motion vector and a relative direction vector between at least two of the plurality of images.

According to the present concept, step c) is performed via a camera external parameter estimation module.

According to the concept of the case, the location data is the three-dimensional orientation and angle of the camera.

According to the present concept, step d) is performed via an image correction module.

According to the present concept, step e) is performed via a difference estimation module.

According to the present concept, step f) is performed via a difference to depth conversion module.

According to the present concept, step g) is performed via a depth image rendering module.

According to the present invention, the automatic depth of field capture method further includes the step b1) providing at least one frame buffer for temporarily storing the plurality of images.

The above-mentioned aspects and advantages of the present invention will become apparent to those skilled in the art from

The invention discloses a system and method for obtaining the depth of an object by means of digital signal processing. The aspects and advantages of the present invention will become apparent to those skilled in the art from the <RTIgt; The examples described in this paragraph are illustrative of the invention, but do not limit the invention.

See Figure 3. It discloses an automatic depth of field capture system for a camera in accordance with the present invention. As shown in FIG. 3, the capture system of the present invention includes a camera lens 401 for capturing a plurality of images. A camera calibration device 41 includes an external pole estimation module 411 for estimating a plurality of outer poles of the plurality of images. And a camera external parameter estimation module 412 for estimating location data; at least one frame buffer 413 for temporarily storing a plurality of images; and an image correction module 42 for correcting corresponding to a plurality of external electrodes a plurality of images of the data and a plurality of corrected images; a difference estimation module 43 coupled to the camera calibration device 41 and the image correction module 42 to receive the position data and obtain a difference vector of the corrected image; a difference to depth conversion module 44, for obtaining a depth of field in response to the difference vector and the position data; and a depth image drawing module 45 for drawing a three-dimensional image corresponding to the depth of field.

In practical applications, multiple images are taken separately by the camera in different orientations. After the camera lens 401 captures at least two images, the images are temporarily stored in the frame buffer 413. The image stored in the image buffer 413 is again supplied to the external calibration module 411 of the camera calibration device 41. In this embodiment, the plurality of outer pole data are a plurality of outer pole lines and outer poles. The outer pole estimation module 411 generates a matrix, such as a matrix composed of a rotation matrix R and a translation matrix t in the following equations. It responds to a plurality of outer and outer poles and is obtained by means of a tracking algorithm. The matrix includes a relative motion vector and a relative direction vector between at least two of the plurality of images. For example, the basic matrix can be used as a matrix and described as: F~K ₂ ^-1 EK ₁ ^-1 , where K ₁ and K ₂ are camera parameters of two images taken at different angles. For the purpose of real-time calculation, it is possible to reverse the direction of the outer pole line by the global difference estimation, and then deduct the basic matrix by the outer pole line. This will increase efficiency. The calculated matrix will be passed to the camera external parameter estimation module 412.

After receiving the plurality of outer pole lines and the matrix, the camera external parameter estimation module 412 can generate the position data of the camera, wherein the position data is the three-dimensional orientation and angle of the camera and is related to the source of the previous image. The location data will be provided to the difference estimation module 43 and the difference versus depth conversion module 44.

The image correcting module 42 corrects at least two images corresponding to the plurality of outer pole data, and obtains a plurality of corrected images, wherein the outer lines of the two images are corrected to be the same; and the two images are moved A center point and named to capture the center point of the image. The corrected image and position data of the camera external parameter estimation module 412 are input to the difference estimation module 43. in In the present invention, the external pole estimation module 411, the camera external parameter estimation module 412, and the frame buffer 413 can be integrated into the camera calibration device 41, which is a smaller size IC device to replace the large computer.

After the difference estimation module 43 receives the corrected image and position data of the camera external parameter estimation module 412, a difference vector is generated to be transmitted to the difference-to-depth conversion module 44. The difference versus depth conversion module 44 will generate a depth that is responsive to the difference vector and the location data, and then the depth image rendering module 45 will render the 3D image corresponding to the depth.

As previously stated, the present invention can be performed via a camera. As shown in FIG. 4, it discloses an automatic depth of field capture system for a camera in accordance with the present invention. As shown in Figure 4, digital camera 50 includes the system of the present invention described above. In addition, a camera lens having an image sensor 501 is further mounted on the digital camera 50. When the user presses the shutter button 502 halfway, the digital camera 50 will begin the process of capturing the three-dimensional image. The result can be displayed on a two-dimensional or three-dimensional screen of the digital camera 50 (not shown in Figure 5). To capture more images, the digital camera 50 can be moved or rotated in several directions 60. When the user is satisfied with the result, the user can fully press the shutter button 502 to complete the shooting of the three-dimensional image, and the result will be output or stored. Figure 5 further discloses a display screen for use with the camera of the present invention. Display screen 70 may further include an annotation area 701 to define a focus area as a primary goal in implementing the capture process of the present invention.

In accordance with the above system, the present invention further provides an automatic depth of field capture method for a camera. See Figure 6. It discloses an automatic depth of field capture method for a camera in accordance with the present invention. According to FIG. 3 and FIG. 6, the method includes the following steps: a) capturing a plurality of images via the camera lens 401, such as Step S601, wherein a plurality of images are temporarily stored in the frame buffer 413; b) estimating a plurality of outer pole data of the plurality of images via the outer pole estimation module 411 to obtain a basic matrix, as shown in step S602; c) Estimating the position data of the plurality of external pole data and the basic matrix via the camera external parameter estimation module 412, as shown in step S603; d) correcting the plurality of images corresponding to the plurality of outer pole data via the image correction module 42 Obtaining a plurality of corrected images, as shown in step S604; e) calculating the position data via the difference estimation module 43 to obtain the difference vector of the corrected image, as shown in the program S605; f) obtaining the response via the difference pair depth conversion module 44 The difference vector and the depth of the position data are as shown in the program S606; and g) the three-dimensional image corresponding to the depth is drawn via the depth image drawing module 45, as shown in the program S607.

In practical applications, multiple images are taken separately by the camera in different orientations. After the camera lens 401 captures at least two images, the image will be temporarily stored in the frame buffer 413. The image stored in the image buffer 413 is again supplied to the external calibration module 411 of the camera calibration device 41. In this embodiment, the plurality of outer pole data are a plurality of outer pole lines and outer poles. The outer pole estimation module 411 generates a basic matrix responsive to the plurality of outer and outer poles in a tracking algorithm, wherein the basic matrix includes a relative motion vector and a relative direction between the at least two complex images. vector. As described above, the method of the present invention can achieve the purpose of automatically obtaining the depth of the object without changing the camera itself, and thus it is easier for the user to take a stereoscopic image.

In summary, the present invention provides an automatic depth of field capture system for obtaining camera depth by means of digital signal processing. By providing the difference vector and the camera external parameters to the depth of the depth conversion module to obtain the depth, which simplifies the entire structure and process, and achieves the purpose of automatically obtaining the depth of the object without changing the camera itself, so that it is easier for the user to shoot 3D imagery. It can correct the shortcomings of those skilled arts and solve the above problems.

The present invention has been described in detail by the above-described embodiments, and may be modified by those skilled in the art, without departing from the scope of the appended claims.

1‧‧‧ objects

2‧‧‧ code mark

3‧‧‧Image capture device

4‧‧‧ computer

5‧‧‧ images

11‧‧‧ imaging device

22R, 22G, 22B‧‧‧ color filters

30‧‧‧Image Processing Unit

34‧‧‧Operation switch group

41‧‧‧ Camera calibration device

42‧‧‧Image Correction Module

43‧‧‧Difference Estimation Module

44‧‧‧Difference to depth conversion module

45‧‧‧Deep image rendering module

401‧‧‧ camera lens

411‧‧‧External Estimation Module

412‧‧‧ Camera external parameter estimation module

413‧‧‧Frame buffer

50‧‧‧ digital camera

501‧‧‧Image Sensor

502‧‧‧Shutter button

60‧‧‧ Direction

70‧‧‧ Display screen

701‧‧‧ marked area

1 is a block diagram showing the circuit structure of a stereoscopic image capturing device according to the prior art; FIG. 2 is a block diagram showing the structure of a three-dimensional azimuth and position sensing device according to the prior art; and FIG. 3 is a view showing a camera for use in accordance with the present invention. Automatic depth of field capture system; FIG. 4 discloses an automatic depth of field capture system for a camera in accordance with the present invention; FIG. 5 further discloses a display screen for a camera of the present invention; and FIG. 6 discloses an automatic use for a camera according to the present invention Depth of field capture method.

401‧‧‧ camera lens

41‧‧‧ Camera calibration device

411‧‧‧External Estimation Module

412‧‧‧ Camera external parameter estimation module

413‧‧‧Frame buffer

42‧‧‧Image Correction Module

43‧‧‧Difference Estimation Module

44‧‧‧Difference to depth conversion module

45‧‧‧Deep image rendering module

Claims (20)

An automatic depth of field capture system for a camera, comprising: a camera lens for capturing a plurality of images; a camera calibration device comprising a plurality of external pole data external estimation modules for estimating the plurality of images, And a camera external parameter estimation module for estimating location data; at least one frame buffer for temporarily storing the plurality of images; and an image correction module for correcting the corresponding plurality of external pole data a plurality of images and obtaining a plurality of corrected images; a difference estimation module coupled to the camera calibration device and the image correction module for receiving the position data and obtaining a difference vector of the corrected image; a difference to depth conversion mode a group for obtaining a depth in response to the difference vector and the position data; and a depth image drawing module for drawing a three-dimensional image corresponding to the depth. The automatic depth of field capture system of claim 1, wherein the plurality of images are captured by the camera in different orientations. The automatic depth of field capture system of claim 1, wherein the plurality of outer pole data are a plurality of outer pole lines and outer poles. The automatic depth of field capture system of claim 3, wherein the external pole estimation module further uses a tracking algorithm to generate a matrix that is responsive to the plurality of outer and outer poles. The automatic depth of field capture system of claim 4, wherein the matrix includes a relative motion vector between at least two of the plurality of images and A relative direction vector. The automatic depth of field capture system of claim 1, wherein the location data is a three-dimensional orientation and angle of the camera. An automatic depth of field capture method for a camera, comprising the steps of: a) capturing a plurality of images; b) estimating a plurality of outer polar data of the plurality of images to obtain a matrix; c) estimating an response to the plurality of outer polar data And the position data of the matrix; d) correcting the plurality of images corresponding to the plurality of outer pole data to obtain a plurality of corrected images; e) calculating the position data to obtain a difference vector of the corrected image; f) obtaining a response The difference vector and the depth of the position data; and g) drawing a three-dimensional image corresponding to the depth. The automatic depth of field capture method of claim 7, wherein the step a) is performed via a camera lens. The automatic depth of field capture method of claim 7, wherein the plurality of images are captured by the camera in different orientations. The automatic depth of field capture method of claim 7, wherein the step b) is performed via an external pole estimation module. The automatic depth of field capture method of claim 7, wherein the plurality of outer pole data are a plurality of outer pole lines and outer poles of the plurality of images. The automatic depth of field capture method of claim 11, wherein the matrix that is responsive to the plurality of outer and outer poles is obtained by a tracking algorithm. The automatic depth of field capture method of claim 7, wherein the matrix includes a relative motion vector and a relative direction vector between the at least two of the plurality of images. The automatic depth of field capture method of claim 7, wherein the step c) is performed via a camera external parameter estimation module. The automatic depth of field capture method of claim 7, wherein the location data is a three-dimensional orientation and angle of the camera. The automatic depth of field capture method of claim 7, wherein the step d) is performed by the image correction module. The automatic depth of field capture method of claim 7, wherein the step e) is performed via a difference estimation module. The automatic depth of field capture method of claim 7, wherein the step f) is performed via a difference pair depth conversion module. The automatic depth of field capture method of claim 7, wherein the step g) is performed via a depth image rendering module. The automatic depth of field capture method of claim 7, further comprising the step b1) providing at least one frame buffer for temporarily storing the plurality of images.