KR20070089314A - Apparatus and method for providing subject movement information - Google Patents

Abstract

An apparatus and method for providing subject motion information are provided.

The apparatus and method for providing subject movement information project pixel values of a current image belonging to a window for selecting an important part in one dimension, and calculate a correlation between pixel values projected in the current projection memory and pixel values projected in the previous projection memory. A motion vector is calculated, and a window position for calculating a window position with respect to the next image is calculated using the motion vector.

Description

Apparatus and method for providing motion information of object}

1 is a view showing the concept of digital image stabilization (Digital Image Stabilization).

2 is a block diagram schematically showing a conventional imaging device apparatus including a DIS.

3 is a block diagram illustrating a configuration of an apparatus for providing subject motion information according to an exemplary embodiment of the present invention.

4 is a block diagram showing a configuration of a window position calculator according to an embodiment of the present invention.

5 is a flowchart illustrating a process of providing subject motion information according to an embodiment of the present invention.

6 is a block diagram illustrating a configuration of a digital imaging apparatus having a tracking focus function according to an embodiment of the present invention.

7 is a view showing a process for changing the position of a window according to an embodiment of the present invention.

The present invention relates to a tracking focus technique, and more particularly, to a tracking focus apparatus and method using a digital image stabilizer (DIS).

Video signal recording and reproducing devices such as Digital Video Camcoder (DVC), Digital Still Camera (DSC), and Video Tape Recorder (VTR) are equipped with various functions, allowing users to shoot subjects in various shapes as the user selects a mode. do. When shooting with a small and portable video device such as a DVC or DSC, the shaking of the device itself has a great effect on the collected images. Therefore, in order to secure a stable image, various methods of detecting a motion and performing motion compensation accordingly have been used.

Image stabilizers are classified into Electronic Image Stabilizers (EIS), Optical Image Stabilizers (OIS), and Digital Imager Stabilizers (DIS). DIS is widely used because it is easy to integrate because the video signal itself is used instead of a separate motion sensor.

1 is a view showing the concept of digital image stabilization (Digital Image Stabilization).

The DIS device performs digital image stabilization using one-dimensional projection.

The DIS device projects the previous image 112 stored in the image buffer 110 into the previous vertical projection memory 130 and the previous horizontal projection memory 151, and the current image 111 stored in the image buffer 110 is currently vertical. Projection is performed on the projection memory 120 and the current horizontal projection memory 140.

The DIS device then finds the movement in the vertical direction by calculating the correlation between the previous vertical projection memory 130 and the current image 121 projected in the current vertical projection memory 120 with respect to the projected previous image 131. Also, the DIS device calculates a correlation between the previous image 151 projected in the previous horizontal projection memory 150 and the current image 141 projected in the current horizontal projection memory 140 to find a movement in the horizontal direction.

The DIS device uses the vertical motion vector and the horizontal motion vector when the display device requests the current image 111 to stabilize the vertical and horizontal shaking when the current image is displayed. Change the address of reading the current image 111 stored in the.

2 shows a conventional imaging device apparatus including such a DIS function.

Referring to FIG. 2, a conventional imaging device apparatus includes an image buffer 210 storing an image captured by an image sensor (not shown) and a vertical projection memory 211 storing a value of projecting an image in a vertical direction. ) And a horizontal projection memory 212 storing a value of projecting the image in the horizontal direction, a motion estimator 220 estimating a motion vector of the image moved by the camera shake, and the estimated motion vector. And a field memory 250 that stores an image to be displayed.

The motion estimator 220 includes a vertical motion estimator 221 for detecting vertical motion of an image due to hand shaking and a horizontal motion estimator 222 for detecting horizontal motion of an image due to hand shaking. The motion compensator 230 generates a signal for controlling the address of the field memory 250 according to the detected motion amount.

The field memory 240 outputs, to the display apparatus, image data in which movement due to hand shake is corrected in response to the control signal.

As such, the DIS device can be applied to various devices such as DVC and DSC because digital image processing can compensate for image movement caused by camera shake. Such digital imaging apparatuses may exhibit optimal performance when the subject is correctly focused. In general, digital imaging devices use a centered focus method that focuses on a centrally located object. There are many other focus methods, but most of them focus on a stationary object. If the subject is moving, it is difficult to focus. In this case, the focus window must be changed to match the movement of the subject so that the moving object can be focused. This focusing of a moving subject is called tracking focus.

In order to implement the tracking focus, an operation of detecting a movement of a subject and changing a focal length of the digital imaging apparatus according to the movement of the subject is required. Therefore, in order to implement the tracking focus, it is necessary to calculate the movement of the subject, and separate devices for calculating the movement of the subject may increase the cost of the digital imaging apparatus. Therefore, if the implementation of tracking focus using the functions of components already included in the digital imaging device is implemented, it will help to reduce the cost and hardware size of the digital imaging device.

The present invention has been made to meet the above necessity, and an object of the present invention is to provide a method and apparatus for providing motion information of a subject using a digital image stabilization function.

According to an aspect of the present invention, there is provided a subject motion information providing apparatus comprising: a first projection memory configured to store a value obtained by projecting pixel values of a current image belonging to a window for selecting an important part in one dimension; A second projection memory storing a value of projecting pixel values of a previous image belonging to a window in one dimension, and calculating a correlation between pixel values projected in the first projection memory and pixel values projected in the second projection memory A motion vector estimator for calculating a motion vector, a window position calculator for calculating a position of the window with respect to a next image using the motion vector, and a projection position to be projected on the first and second projection memories according to the position of the window. A window position controller for controlling pixel values. The size of the window may have a size of 1/16 of an image size.

The window position controller may set the initial position of the window to the center of the image.

The first and second projection memories each include a horizontal projection memory and a vertical projection memory, the motion estimator includes a horizontal motion estimator and a vertical motion estimator, and the window position calculator calculates a horizontal motion calculator and a vertical motion. It may include wealth.

The window position calculator adds a first delay unit that delays a previously accumulated motion vector, an output of the first delay unit, and a current motion vector to obtain a currently accumulated motion vector, and calculates the currently accumulated motion vector from the first delay unit. Obtaining a current window position by adding a first adder provided to the delay unit, a second delay unit delaying a previous window position value, and outputs of the currently accumulated motion vector, the current motion vector, and the second delay unit, The second adder may provide a window position value to the second delay unit.

In order to achieve the above object, a method of providing subject motion information according to an embodiment of the present invention includes projecting pixel values of a current image belonging to a window for selecting an important part in one dimension, and projecting the projected current pixel values and the projection. Calculating a motion vector by calculating correlation of the previous pixel values, calculating a position of the window with respect to a next image using the motion vector, and the first and second projections according to the position of the window. Controlling the pixel values to be projected into the memory. The size of the window may have a size of 1/16 of an image size.

The initial position of the window may be the center of the image.

The projecting includes vertically projecting pixel values of the current image belonging to the window, and horizontally projecting pixel values of the current image belonging to the window. The calculating of the motion vector may include calculating a correlation vector between the pixel values of the vertically projected current image and the pixel values of the previous image projected vertically, and calculating the motion vector in the vertical direction. Calculating a horizontal motion vector by calculating a correlation between pixel values of the image and pixel values of a previous image projected horizontally. The step of calculating the position of the window calculates the position of the window with respect to the next image by using the vertical motion vector and the horizontal motion vector.

The calculating of the position of the window may include delaying a previously accumulated motion vector, adding the delayed previously accumulated motion vector and a current motion vector to obtain a current accumulated motion vector, and calculating a previous window position value. Delaying, and adding the current accumulated motion vector, the current motion vector, and the previous window position value to obtain a current window position.

DETAILED DESCRIPTION Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the accompanying drawings so that those skilled in the art may easily implement the technical idea of the present invention. do.

3 is a block diagram illustrating a configuration of an apparatus for providing subject motion information according to an exemplary embodiment of the present invention.

The apparatus 300 for providing motion information of the subject calculates a motion vector by using projection memories 311 and 312 storing values of projecting pixel values of the current image in one dimension and pixel values projected in one dimension. The motion vector estimator 320 includes a window position calculator 330 for calculating the position of the window using the motion vector, and a window position controller 340 for generating a control signal according to the calculated position of the window.

The projection memories 311 and 312 include a current projection memory (not shown) that stores a value obtained by projecting values of pixels belonging to a window in one dimension among pixels of the current image, and projecting pixel values of a previous image in one dimension. A previous projection memory (not shown) that stores the specified values. The window allows you to select the pixels of the important part of the image. In one embodiment, the size of the window has a size of 1/16 of the image size. In other words, when the image has an M * N size, the window has a size of (M / 4) * (N / 4). The window allows the pixel of the subject to be tracked to be selected and its position may change according to the movement of the subject.

The motion vector estimator 320 obtains a motion vector by using a correlation between pixel values projected in the current projection memory and pixel values projected in the previous projection memory.

Consider the case of estimating the motion vector using the DIS method.

The projection memory includes a vertical projection memory 311 and a horizontal projection memory 312, and the motion vector estimator 320 includes a vertical motion estimator 321 and a horizontal motion estimator 322. The vertical projection memory 311 stores pixel values projected in the first direction, and the horizontal projection memory 312 stores pixel values projected in the second direction. For example, the average value of all pixel values lying in the first direction in the vertical projection memory 311 may be represented by one value. In the case of an image of M * N, M horizontal lines are each represented by one value, and a total of M data are stored in the vertical projection memory 311. Similarly, the N horizontal lines are each represented by one value, and the total N data are stored in the horizontal projection memory 312. Both vertical projection memory 311 and horizontal projection memory 312 store projection values of current image pixel values and projection values of previous image pixel values.

Meanwhile, when storing pixel values projected in the projection memories 311 and 312 using the window, the number of projected pixels may vary according to the window size. When the size of the window is (M / 4) * (N / 4), M / 4 data is stored in the vertical projection memory 311 for one image, and N / 4 in the horizontal projection memory 312. Data is stored. In this case, each data of the vertical projection memory 311 is a value obtained by projecting N / 4 pixel values (for example, an average value of pixels), and each data of the horizontal projection memory 312 represents M / 4 pixel values. The projected value. The vertical projection memory 311 stores M / 4 data for the current image and M / 4 data for the previous image, and the horizontal projection memory 312 stores N / 4 data for the current image and the previous image. N / 4 data are stored.

The motion vector estimator 320 obtains a motion vector by using a correlation between the projected pixel values of the current image stored in the projection memories 311 and 312 and the projected pixel values of the previous image. To this end, the motion vector estimator 320 includes a vertical motion vector estimator 321 and a horizontal motion vector estimator 322. The vertical motion vector estimator 321 obtains a vertical motion vector by using correlations between current and previous pixel values projected in the vertical projection memory 311, and the horizontal motion vector estimator 322 provides the horizontal projection memory 312. The horizontal motion vector is obtained by using the correlation between the current and previous pixel values projected on.

The window position calculator 330 calculates the position of the window with respect to the next image using the motion vector. The window position calculator 330 includes a window vertical position calculator 331 and a window horizontal position calculator 332. The window vertical position calculator 331 calculates the vertical position of the window using the vertical motion vector, and the window horizontal position. The position calculator 332 calculates the horizontal position of the window using the horizontal motion vector. The window position calculator 330 will be described later with reference to FIG. 4.

The window position controller 340 controls the position of the window for selecting the pixels to be projected into the projection memories 311 and 312 according to the calculated window position value. The window position controller 340 may include a window vertical position controller 341 for controlling the vertical position of the window and a window horizontal position controller 342 for controlling the horizontal position of the window. In one embodiment, window position controller 340 sets the initial window position to the center of the image. This is because an important subject usually to be photographed is located in the center of the image captured by the digital imaging device.

The position change of the window will be described with reference to FIG. 7.

Referring to FIG. 7, the subject 710 moves in the direction of the arrow 730 in the current image. The movement direction of the subject 710 may be obtained by comparing the projection values of the pixel values included in the window 720 and the projection values of the pixel values included in the window of the previous image (not shown). When the subject 710 obtains a motion vector through the above process, the window 750 for the next image is moved by the motion vector. By moving the motion vector as described above, the window includes the subject 740.

As described above, according to the exemplary embodiment of the present invention, the motion information of the subject can be easily obtained by using the DIS method. Accordingly, the digital imaging apparatus having the tracking focus function may be implemented by using the motion information of the subject.

4 is a block diagram showing a configuration of a window position calculator according to an embodiment of the present invention.

The window position calculator 400 includes first and second adders 410 and 420 and first and second delay units 430 and 440.

The first delay unit 430 delays the accumulated motion vector (corresponding to the correction vector in the DIS method) for the previous image by one image interval (one field or one frame interval). The first adder 410 adds the output of the first delay unit 430 and the current motion vector to obtain a currently accumulated motion vector. The currently accumulated motion vector is provided to the first delay unit 430 for the next image and provided to the second adder 420 to be used to find the current window position.

The second delay unit 440 delays the position value of the previous window by one image interval. The second adder 410 obtains the current window position by adding the output of the second delay unit 440, the current accumulated motion vector, and the current motion vector. The current window position is used to calculate the motion of the next image and is provided to the second delay unit 440 to be used to calculate the next window position.

5 is a flowchart illustrating a process of providing subject motion information according to an embodiment of the present invention.

First, the window position is initialized (S510). The window is used to select pixels of the important area in the image where the subject to be tracked is located, and the window position is initially set to the center of the image.

Then, the horizontal motion vector and the vertical motion vector of the subject are calculated using the DIS method (S520). The motion vector is calculated using the DIS method as follows. For the image, the pixel values contained in the window are projected into the vertical projection memory and the horizontal projection memory, and the vertical projection values and the horizontal projection values are respectively compared with the vertical projection values and the horizontal projection values for the previous image to obtain the highest correlation. Find the motion vector and the horizontal motion vector.

When the vertical motion vector and the horizontal motion vector are obtained, the horizontal / vertical position of the window with respect to the next image is calculated (S530).

The position of the window is changed with respect to the next image (S540). When the position of the window is changed, the position on the image of the pixels to be projected in the projection memory is changed when the horizontal motion vector and the vertical motion vector of the subject are obtained by the DIS method.

The position of the window represents the position of the main subject, and thus, a digital imaging apparatus having a tracking focus function can be implemented by focusing according to the position of the window.

6 is a block diagram showing the configuration of a digital imaging device having a tracking focus function according to an embodiment of the present invention.

The digital imaging device 600 converts an optical signal for a subject collected through the lens group 610 into an electrical image signal through the image sensor 620. The analog image signal is converted into a digital image signal by an analog-to-digital converter (not shown).

The digital signal processor (DSP) 630 performs various image processing on the image signal. Such image processing includes DIS processing and processing for estimating motion information of a subject according to an embodiment of the present invention.

The field memory 640 stores an image signal to be displayed, and when the DIS operation is performed on the digital imaging device 600, the field memory 640 transmits the image signal to the display device (not shown) according to the DIS control signal. Output

The DSP 630 controls the focus correction lens motor 660 so that the image of the current subject position (window position) is well focused using the movement information of the subject. The focus correcting lens motor 660 adjusts the focal length of the lens group 610 by adjusting the position of each lens constituting the lens group 610.

In one embodiment, the DSP 630 controls the focus correcting lens motor 660 to sharpen the edges of the pixels belonging to the window. For example, when the focal length of the lens group 610 is shorter than the position of the subject, the DSP 630 controls the focus correcting lens motor 660 to increase the focal length of the lens group 610.

The apparatus and method for providing motion information of a subject according to an exemplary embodiment of the present invention may provide the motion information of the subject by using a DIS function included in the digital imaging apparatus. Therefore, when the digital imaging apparatus is implemented by using the apparatus or method for providing the movement information of the subject according to an exemplary embodiment of the present invention, since a separate hardware for the movement information of the subject is unnecessary, a digital imaging apparatus having a tracking focus function is easily provided. Can be implemented.

Although the technical idea of the present invention has been described in detail according to the above preferred embodiment, it should be noted that the above-described embodiment is for the purpose of description and not of limitation. In addition, those skilled in the art will understand that various embodiments are possible within the scope of the technical idea of the present invention.

Claims (10)

A first projection memory configured to store a value obtained by projecting pixel values of a current image belonging to a window for selecting an important part in one dimension; A second projection memory configured to store a value obtained by projecting pixel values of a previous image belonging to the window in one dimension; A motion vector estimator configured to calculate a motion vector by calculating a correlation between pixel values projected in the first projection memory and pixel values projected in the second projection memory; A window position calculator for calculating a position of the window with respect to a next image using the motion vector; And And a window position controller for controlling pixel values to be projected in the first and second projection memories according to the position of the window. The method of claim 1, And the size of the window has a size of 1/16 of an image size. The method of claim 2, And the window position controller sets the initial position of the window to the center of the image. The method of claim 1, The first and second projection memories each include a horizontal projection memory and a vertical projection memory, The motion estimator includes a horizontal motion estimator and a vertical motion estimator, The window position calculator includes a horizontal motion calculator and a vertical motion calculator. The method of claim 1, wherein the window position calculator A first delay unit for delaying a previously accumulated motion vector; A first adder which adds an output of the first delay unit and a current motion vector to obtain a currently accumulated motion vector and provides the currently accumulated motion vector to the first delay unit; A second delay unit delaying a previous window position value; And And a second adder which obtains a current window position by adding the current accumulated motion vector, the current motion vector, and the output of the second delay unit, and provides the current window position value to the second delay unit. Motion information providing device. Projecting pixel values of a current image belonging to a window for selecting an important part in one dimension; Calculating a motion vector by calculating a correlation between the projected current pixel values and the projected previous pixel values; Calculating a position of the window with respect to a next image using the motion vector; And Controlling pixel values to be projected in the first and second projection memories according to the position of the window. The method of claim 6, And the size of the window has a size of 1/16 of an image size. The method of claim 7, wherein And the initial position of the window is set to the center of the image. The method of claim 1, The projecting step includes vertically projecting pixel values of the current image belonging to the window, and horizontally projecting pixel values of the current image belonging to the window, The calculating of the motion vector may include calculating a correlation vector between the pixel values of the vertically projected current image and the pixel values of the previous image projected vertically, and calculating the motion vector in the vertical direction. Calculating a horizontal motion vector by calculating a correlation between pixel values of the image and pixel values of a previous image projected horizontally; The calculating of the position of the window may include calculating the position of the window with respect to the next image using the motion vector in the vertical direction and the motion vector in the horizontal direction. The method of claim 6, wherein calculating the position of the window Delaying a previously accumulated motion vector; Obtaining a current accumulated motion vector by adding the delayed previously accumulated motion vector and a current motion vector; Delaying the previous window position value; And And obtaining a current window position by adding the current accumulated motion vector, the current motion vector, and the previous window position value.

Images