KR102521656B1 - Method and apparatus of identifying object - Google Patents

Abstract

A method and apparatus for recognizing an object according to an embodiment determines whether blur occurs in an input image, determines a blur type of the blur using vehicle control information based on whether blur occurs, and determines the blur type according to the determined blur type. A corresponding blur removal method is selected, blur is removed from the input image using the selected blur removal method, and an object included in the input image is recognized using the blur-removed input image.

Description

Method and apparatus for recognizing objects {METHOD AND APPARATUS OF IDENTIFYING OBJECT}

The embodiments below relate to a method and apparatus for recognizing an object.

A variety of visual information augmentation is provided to assist driving of vehicles and other means of transportation. Among them, methods for recognizing various objects such as lanes, traffic lights, pedestrians, and vehicles from any input image may be used.

For example, when a lane is blocked by another vehicle, weather changes such as snow and rain, or changes in image brightness or contrast due to time changes such as day and night, the image information is unclear. It is difficult to detect objects clearly. In particular, in the case of night images, an auto exposure sensor is often used due to an illumination problem. In the case of using an automatic exposure sensor, it is not easy to accurately detect an object due to blur caused by the movement of a dynamically moving vehicle or the movement of an object.

According to one aspect, a method of recognizing an object includes determining whether blur occurs in an input image; determining a blur type of the blur using vehicle control information based on whether the blur occurs; selecting a de-blur method corresponding to the determined blur type; removing the blur from the input image using the selected blur removal method; and recognizing an object included in the input image by using the input image from which the blur is removed.

Determining whether the blur occurs may include determining whether the blur occurs based on sharpness of an edge portion of the input image.

Determining whether the blur occurs may include dividing the input image into partial images; calculating a gradient magnitude value corresponding to each of the partial images; and determining whether the blur occurs based on a comparison result between the gradient magnitude value and a threshold value.

The vehicle control information may include any one or a combination of a steering angle of the vehicle, a speed of the vehicle, and whether or not left and right direction indicators are used.

The determining of the blur type may include determining whether a steering angle of the vehicle is greater than or equal to a first threshold value, whether a speed of the vehicle is greater than or equal to a second threshold value, and left and right direction indicators of the vehicle are used. and determining whether the blur corresponds to a first blur type or a second blur type based on any one or a combination of whether or not the blur has occurred.

The control information of the vehicle may be acquired by a sensor including an odometer, a gyro sensor, lidar, radar, an IMU sensor, and an acceleration sensor.

The selecting of the blur removal method may include selecting a blur removal method using a first filter when the blur is determined to be a first blur type; and selecting a blur control method using a second filter when the blur is determined to be the second blur type.

The method of recognizing the object further includes acquiring illuminance at the time the input image is obtained by using an illuminance sensor, and determining whether or not the blur occurs depends on the illumination intensity at the time the input image is obtained. Based on the method, determining whether the blur occurs or not may be included.

According to one side, an apparatus for recognizing an object determines whether blur occurs in an input image, determines a blur type of the blur using vehicle control information based on whether blur occurs, and determines a blur type of the blur, based on whether blur occurs. A processor that selects a blur removal method corresponding to the type, removes the blur from the input image using the selected blur removal method, and recognizes an object included in the input image using the input image from which the blur is removed. includes

The processor may determine whether the blur occurs based on sharpness of an edge portion of the input image.

The processor divides the input image into partial images, calculates a gradient magnitude value corresponding to each of the partial images, and determines whether the blur occurs based on a comparison result between the gradient magnitude value and a threshold value. can

The vehicle control information may include any one or a combination of a steering angle of the vehicle, a speed of the vehicle, and whether or not left and right direction indicators are used.

The processor determines whether the steering angle of the vehicle is greater than or equal to a first threshold, whether the speed of the vehicle is greater than or equal to a second threshold, and whether left and right direction indicators of the vehicle are used. Alternatively, the blur type of the blur may be determined based on a combination thereof.

The control information of the vehicle may be acquired by a sensor including an odometer, a gyro sensor, lidar, radar, an IMU sensor, and an acceleration sensor.

The processor may select a blur removal method using a first filter when the blur is determined to be a first blur type, and select a blur control method using a second filter when the blur is determined to be a second blur type.

The apparatus for recognizing the object further includes an illuminance sensor that obtains illuminance at the time the input image is acquired, and the processor determines whether or not the blur occurs based on the illuminance at the time the input image is acquired. can

1 is a diagram for explaining blur occurring in an input image according to an exemplary embodiment;
2 is a flowchart illustrating a method of recognizing an object according to an exemplary embodiment;
3 is a flowchart illustrating a method of determining whether a blur occurs according to an exemplary embodiment;
4 is a diagram for explaining a method of determining a blur type of a blur according to an exemplary embodiment;
5 to 6 are flow charts illustrating a method of determining a blur type of a blur according to embodiments.
7 to 8 are flowcharts illustrating a method of recognizing an object according to another embodiment.
9 is a block diagram of a device for recognizing an object according to an exemplary embodiment;

Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. However, the scope of the patent application is not limited or limited by these examples. Like reference numerals in each figure indicate like elements.

Various changes may be made to the embodiments described below. The embodiments described below are not intended to be limiting on the embodiments, and should be understood to include all modifications, equivalents or substitutes thereto.

Terms used in the examples are used only to describe specific examples, and are not intended to limit the examples. Singular expressions include plural expressions unless the context clearly dictates otherwise. In this specification, terms such as "include" or "have" are intended to designate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, but one or more other features It should be understood that the presence or addition of numbers, steps, operations, components, parts, or combinations thereof is not precluded.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by a person of ordinary skill in the art to which the embodiment belongs. Terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with the meaning in the context of the related art, and unless explicitly defined in the present application, they should not be interpreted in an ideal or excessively formal meaning. don't

Embodiments to be described below display lanes in an augmented reality navigation system such as a smart vehicle, generate visual information to help steering of an autonomous vehicle, or various control information for vehicle driving. can be used to provide In addition, the embodiments can be used to help safe and comfortable driving by providing visual information to a device including an intelligent system such as a Head Up Display (HUD) installed for in-vehicle driving assistance or fully autonomous driving. there is. Embodiments may be applied to autonomous vehicles, intelligent vehicles, smart phones, and mobile devices, for example. Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Like reference numerals in each figure indicate like elements.

1 is a diagram for explaining blur occurring in an input image according to an exemplary embodiment. Referring to FIG. 1 , an image 110 with global blur and an image 130 with local blur are shown.

For example, when a vehicle rotates in a certain direction, such as a U-turn or a left or right turn, blur occurs in the entire image due to the horizontal movement of the vehicle. In this case, blur occurring in the horizontal and/or vertical directions in the entire image as in the image 110 will be referred to as 'global blur' or 'first blur type'.

Alternatively, for example, if the vehicle is stopped or is moving slowly, but the object to be photographed is moving, in the photographed image, blur occurs in an object having motion among the entire image. At this time, blur occurring in a part of the image or a local portion of the image, such as the image 130, will be referred to as a 'local blur' or a 'second blur type'. Local blur may occur when the photographing device that captures the image 130 hardly moves, but the subject (object) to be captured moves. Since global blur and local blur have different causes, a blur removal method suitable for each cause is required.

2 is a flowchart illustrating a method of recognizing an object according to an exemplary embodiment. Referring to FIG. 2 , a device for recognizing an object (hereinafter referred to as a “recognition device”) according to an exemplary embodiment determines whether blur occurs in an input image (210). One or a plurality of input images may be obtained for each frame using, for example, a photographing device mounted on the front of the vehicle. The photographing device may be fixed to a predetermined location, such as a windshield, dashboard, or rear-view mirror of the vehicle, and capture an image of the front of the vehicle. At this time, it is assumed that calibration information of the photographing device is known in advance. The photographing device may include, for example, a mono camera, a vision sensor, an image sensor, or a device performing similar functions.

The input image may be a photographing device included in the recognizing device or an image captured by a device other than the recognizing device. The input image may be, for example, the image 110 or the image 130 shown in FIG. 1 . The input image may be an image captured at night or an image captured during the day.

In step 210, the recognizing device determines whether blur occurs in the input image without distinguishing between global blur and local blur. In step 210, the recognizing device may determine whether blur occurs based on sharpness of an edge portion of the input image. For example, the recognizing device may determine that blur has occurred when sharpness of an edge portion of the input image is lower than a threshold value. Alternatively, the recognizing device may determine that blur has occurred based on a distribution ratio of an edge portion having a sharpness lower than a threshold value. Alternatively, the recognizing device may determine whether blur occurs by using a gradient magnitude value. A method for the recognizing device to determine whether or not blur occurs using the gradient magnitude value will be described in more detail with reference to FIG. 3 below.

The recognizing device determines a blur type of the blur using vehicle control information based on whether blur occurs (220). The vehicle control information may include, for example, a steering angle of the vehicle, vehicle speed, whether left and right direction indicators of the vehicle are used, and the like. Vehicle control information may be acquired by sensors including, for example, an odometer, a gyro sensor, lidar, radar, an IMU sensor, and an acceleration sensor. The detection sensors may be, for example, sensors 910 of FIG. 9 . For example, the recognizing device may determine which blur type among the first blur type and the second blur type corresponds to the blur.

In step 220, the recognizing device determines, for example, whether the steering angle of the vehicle is greater than or equal to the first threshold value, whether the speed of the vehicle is greater than or equal to the second threshold value, and left and right direction indicators of the vehicle. The blur type of the blur may be determined based on any one or a combination of whether or not the blur is used. A method for determining the blur type of blur by the recognizing device will be described in detail with reference to FIGS. 4 to 6 below.

Depending on the embodiment, the recognizing device may determine the blur type of the blur using the direction of the gradient of the input image. As described above, since the global blur is caused by a vehicle movement such as a U-turn or a left or right turn, a gradient may occur mainly in a horizontal direction (x-axis direction). In addition, since local blur is generated by the movement of a pedestrian or vehicle in front when the vehicle is slow or stopped, a gradient may occur in the entire horizontal direction (x-axis direction) and/or vertical direction (y-axis direction). Using this fact, the recognizing device may determine that the blur corresponds to the first blur type, that is, the global blur, when a gradient occurs in the horizontal direction (x-axis direction) in the input image. When a gradient occurs in the entire horizontal direction (x-axis direction) and vertical direction (y-axis direction) of the input image, the recognizing device may determine that the blur corresponds to the second blur type, that is, local blur.

Alternatively, the recognition device uses an algorithm capable of locally detecting the motion of an object, such as an optical flow in two consecutive input images or an RGB histogram of a detection box. A blur type of blur generated in an image may be determined.

The recognition device selects a de-blur method corresponding to the determined blur type (230). For example, when blur is determined to be the first blur type, the recognizing device may select a blur removal method using a first filter. The first filter is a filter that removes global blur, and may be, for example, a motion filter. Alternatively, when the blur is determined to be the second blur type, the recognizing device may select a blur control method using the second filter. The second filter is a filter that removes local blur, and may be, for example, a de-focusing filter.

The recognition device removes blur from the input image using the selected blur removal method (240). For example, in the case of the first blur type, since the blur mainly moves in the horizontal direction, the recognition device uses a 1-dimensional vector basis motion PSF (Point Spread Function) (convolution filter) to input the motion. Blur can be removed from video. Alternatively, in the case of the second blur type, since the blur moves in the horizontal and vertical directions, the recognition device may remove the blur using a Gaussian convolution filter (PSF).

In step 240, the recognition device may perform de-convolution for blur removal using the selected convolution filter. The recognition device may use various de-convolution methods such as Wiener, Lucy, Regression, and Blind. The de-convolution method can be expressed as, for example, [Equation 1] below.

는 이미지 도메인(Image domain)에서 디-컨볼루션된 이미지, 다시 말해 블러된 이미지(blurred image)를 나타내고,

는 이미지 도메인에서의 컨볼루션 필터를 나타내며,

는 이미지 도메인에서의 원본 영상을 나타낸다. G(U, V)는 푸리에 도메인(Fourier domain)에서 디-컨볼루션된 이미지(블러된 이미지)를 나타내고, H(U, V)는 푸리에 도메인에서의 컨볼루션 필터를, F(U, V)는 푸리에 도메인에서의 원본 영상을 나타낸다.here,

Represents a de-convoluted image in the image domain, that is, a blurred image,

denotes a convolution filter in the image domain,

represents the original image in the image domain. G(U, V) denotes a de-convoluted image (blurred image) in the Fourier domain, H(U, V) denotes a convolution filter in the Fourier domain, and F(U, V) denotes the original image in the Fourier domain.

The recognition device recognizes an object included in the input image by using the input image from which the blur is removed (250). At this time, the recognizing device is, for example, a Convolution Neural Network (CNN), a Deep Neural Network (DNN), and a support vector machine ( An object may be recognized from an input image from which blur is removed by using a Support Vector Machine) or the like. The convolutional neural network may be trained to determine, for example, a bounding box corresponding to an object to be detected and the type of object to be detected in the (blurred) input image.

3 is a flowchart illustrating a method of determining whether a blur occurs according to an exemplary embodiment. Referring to FIG. 3 , the recognition apparatus according to an embodiment may divide an input image into partial images (310). For example, the recognizing device may divide the input image into four partial images divided into four equal parts (height/2, width/2).

The recognition apparatus may calculate a gradient magnitude value corresponding to each of the partial images (320). The recognizing device may obtain gradient magnitude X-axis and Y-axis values from each of the four partial images divided into quarters. After sorting the values obtained here in descending order, the recognition device may define the median value of the top 0.1% of the sorted values as the blurriness measure. Accordingly, eight blurriness measures (X-axis values and Y-axis values of each quadrant) can be defined.

The recognizing device may determine whether blur occurs based on a comparison result between the gradient magnitude value and the threshold value (330). For example, if 8 blurriness measures (X-axis values and Y-axis values of each quadrant) ≤ Th1_x, Th1_y, Th2_x, Th2_y, Th3_x, Th3_y, Th4_x, Th4_y, it may be determined that blur has occurred.

As described above, global blur occurs when the entire screen is rotated mainly in the horizontal direction (X-axis direction), and local blur is a probability of change in both the X-axis and Y-axis even if it occurs only in some of the quadrants of the entire screen. is high

In order to include these properties of both global and local blur, if one representative blurriness measure that is a well-combined combination of eight values (X-axis value and Y-axis value of each quadrant) is less than the threshold value, global blur and local blur can be judged to have occurred. FIG. 4 to be described later explains a method for determining a blur type of blur using one representative blurriness measure value that is a well-combined combination of these eight values.

The recognition device may determine that no blur has occurred if eight blurriness measures (X-axis values and Y-axis values of each quadrant) > Th1_x, Th1_y, Th2_x, Th2_y, Th3_x, Th3_y, Th4_x, and Th4_y.

4 is a diagram for explaining a method of determining a blur type of a blur according to an exemplary embodiment. 4(a) is a graph illustrating a blurriness measure value and a yaw rate and velocity detected by a motion sensor when global blur occurs due to, for example, a U-turn of a vehicle. In addition, (b) of FIG. 4 is a graph showing a blurriness measure value and a yaw rate and speed detected by a motion sensor when local blur is generated by a movement of a pedestrian in front when the vehicle is stopped. The yaw rate represents, for example, the degree to which the vehicle shakes from side to side by the steering angle of the steering wheel. The speed represents a change in speed caused by operation of an accelerator or brake, for example, during a U-turn.

Referring to (a) of FIG. 4, it can be seen that changes occur in the yaw rate and speed detected by the motion sensor when it is determined to be a blur state in which blur occurs in the input image by the blurriness measure value. there is.

On the other hand, referring to (b) of FIG. 4, when it is determined that the input image is in a blur state in which blur occurs by the blurriness measure value, there is no change in the yaw rate and speed detected by the motion sensor. there is.

In one embodiment, the blur type may be determined using a measurement value of a motion sensor (or vehicle control information) using this point. For example, when a change occurs in a measured value (motion sensor data) of a motion sensor in a blur state as shown in (a) of FIG. 4, the recognition device may determine the corresponding blur as a global blur. Also, as shown in (b) of FIG. 4 , when there is no change in the measurement value (motion sensor data) of the motion sensor in the blur state, the recognition device may determine the corresponding blur as a local blur.

5 is a flowchart illustrating a method of determining a blur type of a blur according to an exemplary embodiment. Referring to FIG. 5 , the recognizing apparatus according to an exemplary embodiment may determine whether a steering angle of the vehicle is greater than or equal to a first threshold value (510). The first threshold may be, for example, 20 degrees. If it is determined in step 510 that the steering angle of the vehicle is smaller than the first threshold value, the recognizing device may determine that the blur corresponds to the second blur type (540). For example, when the steering angle is 20 degrees or less, the vehicle may be driving straight ahead rather than making a U-turn or turning left and right. In this case, blur caused by the movement of an object in front of the vehicle may occur rather than blur caused by the movement of the vehicle. The recognizing device may determine that the blur corresponds to the second type blur.

If it is determined in operation 510 that the steering angle of the vehicle is greater than or equal to the first threshold value, the recognizing device may determine whether the speed of the vehicle is greater than or equal to the second threshold value (520). The second threshold may be 15 km/s. If it is determined that the speed of the vehicle is smaller than the second threshold in step 520, the recognizing device may determine that the blur corresponds to the second type blur (540). For example, a vehicle makes a U-turn or turns left or right, but since the rotation speed is not fast, left-right movement due to rotation may hardly occur. In this case, the recognizing device may determine that the blur corresponds to the second type blur.

If it is determined in step 520 that the speed of the vehicle is greater than or equal to the second threshold, the recognizing device may determine that the blur corresponds to the first type blur (530). For example, when the vehicle makes a U-turn or makes a left or right turn, and the rotation speed is high, left-right movement due to rotation may occur. In this case, the recognizing device may determine that the blur corresponds to the first type blur.

6 is a flowchart illustrating a method of determining a blur type of a blur according to another embodiment. Referring to FIG. 6 , the recognizing apparatus according to an exemplary embodiment may determine whether a steering angle of the vehicle is greater than or equal to a first threshold value (610). The first threshold may be, for example, 20 degrees.

If it is determined in step 610 that the steering angle of the vehicle is smaller than the first threshold value, the recognizing device may determine whether the left and right direction indicators of the vehicle are used (640). If it is determined that the left and right direction indicators are not used in step 640, the recognizing device may determine that the blur corresponds to the second blur type (650). For example, if the steering angle is 20 degrees or less and direction indicators are not used, the vehicle may be in a straight driving or stopped state rather than a U-turn or a left/right turn. In this case, the recognizing device may determine that the blur corresponds to the second type blur.

If it is determined that the left and right direction indicators are used in step 640, the recognizing device may determine whether the speed of the vehicle is greater than or equal to the second threshold value (620). The second threshold may be 15 km/s.

If it is determined in step 620 that the speed of the vehicle is smaller than the second threshold, the recognizing device may determine that the blur corresponds to the second type blur (650). For example, a vehicle makes a U-turn or turns left or right, but since the rotation speed is not fast, left-right movement due to rotation may not occur. In this case, the recognizing device may determine that the blur corresponds to the second type blur. If it is determined in step 620 that the speed of the vehicle is greater than or equal to the second threshold, the recognizing device may determine that the blur corresponds to the first type blur (630). For example, if the vehicle makes a U-turn or turns left or right and the rotation speed is high, left-right movement occurs due to rotation, so the recognizing device may determine that the blur corresponds to the first blur type.

If it is determined in operation 610 that the steering angle of the vehicle is greater than or equal to the first threshold value, the recognizing device may determine whether the speed of the vehicle is greater than or equal to the second threshold value (620). If it is determined in step 620 that the speed of the vehicle is smaller than the second threshold, the recognizing device may determine that the blur corresponds to the second type blur (650). If it is determined in step 620 that the speed of the vehicle is greater than or equal to the second threshold, the recognizing device may determine that the blur corresponds to the first type blur (630).

7 is a flowchart illustrating a method of recognizing an object according to another exemplary embodiment. Referring to FIG. 7 , the recognition device according to an embodiment may obtain an input image from a camera or an image sensor (710). In addition, the recognizing device may obtain vehicle control information from the detection sensor (730). The recognizing device may obtain, for example, vehicle control information corresponding to a point in time when an input image is obtained. In the embodiment of FIG. 7 , it has been described that vehicle control information is obtained after obtaining an input image, but the timing of acquiring vehicle control information is not necessarily limited thereto.

The recognizing device may determine whether blur is detected (generated) in the input image (720). If it is determined that blur is not detected in step 720, the recognizing device may detect and recognize an object from the input image (780).

If it is determined that blur is detected in step 720, the recognizing device may determine a blur type using vehicle control information obtained in step 730 (740).

The recognizing device may determine whether the determined blur type is the first blur type (750). If it is determined in step 750 that the blur type is the first blur type, the recognizing device may remove the blur from the input image using a motion filter (760).

If it is determined in step 750 that the blur type is not the first blur type (ie, if it is the second blur type), the recognizing device may remove the blur from the input image using a defocusing filter (770).

The recognition device may detect and recognize an object from the blur-removed input image (780).

8 is a flowchart illustrating a method of recognizing an object according to another exemplary embodiment. Referring to FIG. 8 , the recognizing device according to an exemplary embodiment may obtain an illuminance at a time when an input image is obtained using an illuminance sensor (810). The recognizing device may determine whether or not blur occurs based on illuminance at a point in time when the input image is obtained. More specifically, the recognizing device may determine whether the illuminance at the time of acquiring the input image is less than the illuminance threshold (820). The illuminance threshold may be, for example, 10 Lux. According to embodiments, the recognizing device may apply the above-described object recognition method to an input image captured at night with low illumination or on a very cloudy day. The recognizing device may determine whether the input image is obtained at night or on a very cloudy day by comparing the illuminance at the time when the input image is acquired with the illuminance threshold. If it is determined in step 820 that the illuminance is greater than or equal to the illuminance threshold, the recognizing device may end its operation.

If it is determined in step 820 that the illuminance is less than the illuminance threshold, the recognizing device may determine whether blur occurs (830). Hereinafter, since the process of steps 830 to 870 is the same as the process of steps 210 to 250 of FIG. 2, reference will be made to the description of the corresponding part.

9 is a block diagram of an apparatus for recognizing an object according to an exemplary embodiment. Referring to FIG. 9 , a recognition device 900 according to an embodiment includes a processor 920 . The recognition device 900 may further include sensors 910 , a memory 930 , a communication interface 940 , and a display device 950 .

Sensors 910 , processor 920 , memory 930 , communication interface 940 , and display device 950 may communicate with each other through a communication bus 905 . The sensors 910 may include, for example, an image sensor, an accelerometer sensor, an odometer, a gyro sensor, a lidar sensor, a radar sensor, an IMU sensor, and the like. The sensors 910 may sense or acquire vehicle control information. The vehicle control information may include, for example, a steering angle of the vehicle, vehicle speed, whether left and right direction indicators of the vehicle are used, and the like. The sensors 910 may further include an illuminance sensor. The illuminance sensor may obtain illuminance at a time when the input image is obtained.

The recognition device 900 may receive sensing information of the sensors 910 through the communication interface 940 . According to embodiments, the communication interface 940 may receive an input image and sensing information from the outside of the recognizing device 900 .

The processor 920 determines whether blur occurs in the input image. The processor 920 determines a blur type of the blur using vehicle control information based on whether blur occurs. The processor 920 may determine which one of the first blur type and the second blur type the blur corresponds to by using vehicle control information.

The processor 920 selects a blur removal method corresponding to the determined blur type, and removes blur from the input image using the selected blur removal method. The processor 920 recognizes an object included in the input image by using the input image from which blur is removed.

The processor 920 may determine whether blur occurs based on the sharpness of the edge portion of the input image. The processor 920 may divide the input image into partial images and calculate a gradient magnitude value corresponding to each of the partial images. The processor 920 may determine whether blur occurs based on a comparison result between the gradient magnitude value and the threshold value.

The processor 920 may determine, for example, whether the steering angle of the vehicle is greater than or equal to a first threshold, whether the speed of the vehicle is greater than or equal to a second threshold, and whether left and right turn indicators of the vehicle are used. The blur type of the blur may be determined based on any one or a combination thereof. The processor 920 may select a blur removal method using a first filter when the blur is determined to be a first blur type, and select a blur control method using a second filter when the blur is determined to be a second blur type.

The processor 920 may determine whether or not blur occurs based on illuminance at the time when the input image is obtained.

Also, the processor 920 may perform at least one method described above with reference to FIGS. 1 to 8 or an algorithm corresponding to the at least one method. The processor 920 may execute desired operations. It may be a data processing device implemented as hardware having a circuit having a physical structure for For example, desired operations may include codes or instructions included in a program. For example, a data processing unit implemented in hardware includes a microprocessor, a central processing unit, a processor core, a multi-core processor, and a multiprocessor. , Application-Specific Integrated Circuit (ASIC), and Field Programmable Gate Array (FPGA). The processor 920 may execute a program and control the recognizing device 900 . Program codes executed by the processor 920 may be stored in the memory 930 .

The memory 930 may store an input image obtained through an image sensor. In addition, the memory 930 may store an input image on which blur is processed by the processor 920 and/or an object recognized from the blur-processed input image. The memory 930 may store various pieces of information generated in the process of processing in the processor 920 described above. In addition, the memory 930 may store various data and programs. The memory 930 may include volatile memory or non-volatile memory. The memory 930 may include a mass storage medium such as a hard disk to store various types of data.

The display device 950 may display the blur-processed input image and/or the object recognized from the blur-processed input image by the processor 920 .

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program commands recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited drawings, those skilled in the art can apply various technical modifications and variations based on the above. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

Therefore, other implementations, other embodiments, and equivalents of the claims are within the scope of the following claims.

The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. Program commands recorded on the medium may be specially designed and configured for the embodiment or may be known and usable to those skilled in computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tapes, optical media such as CD-ROMs and DVDs, and magnetic media such as floptical disks. - includes hardware devices specially configured to store and execute program instructions, such as magneto-optical media, and ROM, RAM, flash memory, and the like. Examples of program instructions include high-level language codes that can be executed by a computer using an interpreter, as well as machine language codes such as those produced by a compiler. The hardware devices described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

As described above, although the embodiments have been described with limited examples and drawings, those skilled in the art can make various modifications and variations from the above description. For example, the described techniques may be performed in an order different from the method described, and/or components of the described system, structure, device, circuit, etc. may be combined or combined in a different form than the method described, or other components may be used. Or even if it is replaced or substituted by equivalents, appropriate results can be achieved.

900: recognition device
905: communication bus
910: sensors
920: processor
930: memory
940: communication interface
950: display device

Claims (17)

determining whether blur occurs in an input image captured by the vehicle;
determining a blur type of the blur using control information indicating rotation of the vehicle based on whether the blur occurs;
selecting a de-blur method corresponding to the determined blur type;
removing the blur from the input image using the selected blur removal method; and
Recognizing an object included in the input image using the input image from which the blur is removed.
Including, how to recognize the object. According to claim 1,
The step of determining whether the blur occurs is
Determining whether the blur occurs based on sharpness of an edge portion of the input image
Including, how to recognize the object. According to claim 1,
The step of determining whether the blur occurs is
dividing the input image into partial images;
calculating a gradient magnitude value corresponding to each of the partial images; and
Determining whether the blur occurs based on a comparison result between the gradient magnitude value and a threshold value
Including, how to recognize the object. delete According to claim 1,
The step of determining the blur type is
The blur is set to a first blur type and a second blur based on whether the steering angle of the vehicle is greater than or equal to the first threshold value and whether the left and right direction indicators of the vehicle are used or a combination thereof. Determining which of the blur types corresponds to which blur type
Including, how to recognize the object. delete According to claim 1,
The step of selecting the blur removal method is
selecting a blur removal method using a first filter when the blur is determined to be a first blur type; and
selecting a blur control method using a second filter when the blur is determined to be a second blur type;
Including, how to recognize the object. delete delete Determining whether or not blur occurs in an input image captured by a vehicle, determining a blur type of the blur using control information indicating rotation of the vehicle based on whether or not blur occurs, and determining a blur type according to the determined blur type A processor for selecting a corresponding blur removal method, removing the blur from the input image using the selected blur removal method, and recognizing an object included in the input image using the blur-removed input image.
Including, the device for recognizing the object. According to claim 10,
The processor
An apparatus for recognizing an object that determines whether the blur occurs based on sharpness of an edge portion of the input image. According to claim 10,
The processor
An object that divides the input image into partial images, calculates a gradient magnitude value corresponding to each of the partial images, and determines whether the blur occurs based on a comparison result between the gradient magnitude value and a threshold value. device that recognizes it. delete According to claim 10,
The processor
An object that determines the blur type of the blur based on whether a steering angle of the vehicle is greater than or equal to a first threshold and whether or not left and right direction indicators of the vehicle are used or a combination thereof. device that recognizes. delete delete According to claim 10,
An illuminance sensor for obtaining the illuminance at the time when the input image was obtained
Including more,
The processor
An apparatus for recognizing an object that determines whether or not the blur occurs based on illuminance at a time point when the input image is acquired.

Images