JP2014235615A - Image processing apparatus, image processing circuit, image processing program, and image processing method - Google Patents

Abstract

A calculation amount for forming a distance image is reduced. In an image processing apparatus, a left distance image forming unit 11 uses a left image as a reference image and a right image as a reference image, forms a left distance image using block matching, and a right distance image forming unit 12 Each of the plurality of blocks constituting the left distance image formed by the left distance image forming unit 11 is moved by a distance corresponding to the parallax value of each block, and a right distance image is formed from each moved block. . [Selection] Figure 1

Description

  The present invention relates to an image processing apparatus, an image processing circuit, an image processing program, and an image processing method.

  When a person sees a 3D (three dimensions) image, the person may feel uncomfortable or uncomfortable. When a person views an object in real space, the person's focus is in the vicinity of the object in the central field of view. For this reason, an object outside the central visual field is recognized in a state of being out of focus. In other words, in a real space, it is rare to recognize a plurality of objects having greatly different distances from a person at the same level. For this reason, when a plurality of objects having greatly different depths (that is, distances from the viewpoint) are viewed in the image, the observer feels tired because the degree of contradiction between convergence and adjustment is large.

  Therefore, in order to suppress the degree of contradiction between congestion and adjustment in this situation, a filtering process is generally performed. In this filtering process, a distance image is used. The distance image is obtained from two images taken from two viewpoints, that is, a “left image” taken from the left viewpoint and a “right image” taken from the right viewpoint. It is done. For example, the origin of the left image and the origin of the right image are matched, and the difference (that is, parallax) between the coordinates of the reference point of the left image and the coordinates of the corresponding point of the right image corresponding to the reference point is calculated as the reference point. The image as the value is the “left-side distance image”. Conversely, the origin of the left image is aligned with the origin of the right image, and the difference (that is, parallax) between the coordinates of the reference point of the right image and the coordinates of the corresponding point of the left image corresponding to the reference point is determined as the reference point. The image with the value of “is the right distance image”. That is, when obtaining the left distance image, the left image is the reference image and the right image is the reference image. Conversely, when obtaining the right distance image, the right image is the reference image, and the left image is the reference image. When the filtering process is performed on the 3D image, the filtering process is performed on each of the left image and the right image. That is, the filtering process is performed on the left image using the left distance image, and the filtering process is performed on the right image using the right distance image. That is, when the filtering process is performed on the 3D image, two distance images of the left distance image and the right distance image are obtained.

  Here, there is “block matching” as a conventional technique used for forming a distance image. Block matching is a method of dividing the reference image into a plurality of small area areas, that is, a plurality of blocks, comparing the image of each block with the entire area of the reference image, and the position corresponding to each block on the reference image. It is a technique to search for. Hereinafter, a block that is a target of distance calculation among a plurality of blocks of the reference image may be referred to as a “target block”. In block matching, the position of the image most similar to the target block (that is, the position of the image with the smallest error) is searched for as the corresponding position on the reference image. Then, the value of the distance between the target block and the position of the image found on the reference image is written in the target block as a parallax value. The parallax is small for an object that is far from the viewpoint, and the parallax is large for an object that is close to the viewpoint. In other words, the closer the distance from the viewpoint, the larger the parallax value.

JP-A-11-102438 JP 2000-215311 A JP 2001-256482 A JP 2011-060216 A

  Block matching is performed over the entire area of the reference image for each of all blocks of the base image while sequentially shifting the target block on the base image. For this reason, the calculation amount of block matching is large. Therefore, if block matching is used for forming a distance image, the amount of calculation for forming the distance image increases, and it takes much time to form the distance image. In particular, when filtering processing is performed on a 3D image, block matching for obtaining a left distance image using the left image as a reference image and block matching for obtaining a right distance image using the right image as a reference image are performed. The amount of calculation for image formation is further increased.

  The disclosed technology has been made in view of the above, and provides an image processing device, an image processing circuit, an image processing program, and an image processing method capable of reducing the amount of calculation for forming a distance image. With the goal.

  In the disclosed aspect, each of the plurality of blocks constituting the first distance image is moved by a distance corresponding to the parallax value of each block, and the viewpoint of the first distance image is different from the plurality of blocks after the movement. A second distance image is formed.

  According to the disclosed aspect, it is possible to reduce the amount of calculation for forming a distance image.

FIG. 1 is a block diagram illustrating an example of an image processing apparatus according to the first embodiment. FIG. 2 is a diagram illustrating an example of block division according to the first embodiment. FIG. 3 is a diagram for explaining the formation of the right distance image according to the first embodiment. FIG. 4 is a flowchart for explaining processing of the image processing apparatus according to the first embodiment. FIG. 5 is a diagram illustrating an example of a left-side distance image according to the first embodiment. FIG. 6A is a diagram for explaining an example of forming a right distance image according to the first embodiment. FIG. 6B is a diagram for explaining an example of forming the right distance image according to the first embodiment. FIG. 6C is a diagram for explaining an example of forming the right distance image according to the first embodiment. FIG. 6D is a diagram for explaining an example of forming the right distance image according to the first embodiment. FIG. 7 is a diagram illustrating an example of the left image according to the first embodiment. FIG. 8 is a diagram illustrating an example of a right image according to the first embodiment. FIG. 9 is a diagram illustrating an example of a left-side distance image according to the first embodiment. FIG. 10 is a diagram illustrating an example of the right distance image according to the first embodiment. FIG. 11 is a block diagram illustrating an example of an image processing apparatus according to the second embodiment. FIG. 12 is a flowchart for explaining processing of the image processing apparatus according to the second embodiment. FIG. 13A is a diagram for explaining an example of updating the occlusion area map according to the second embodiment. FIG. 13B is a diagram for explaining an example of updating the occlusion area map according to the second embodiment. FIG. 13C is a diagram for explaining an update example of the occlusion area map according to the second embodiment. FIG. 13D is a diagram for explaining an example of updating the occlusion area map according to the second embodiment. FIG. 14 is a block diagram illustrating an example of an image processing apparatus according to the third embodiment. FIG. 15 is a flowchart for explaining processing of the image processing apparatus according to the third embodiment. FIG. 16A is a diagram for explaining an example of complementation of the occlusion area according to the third embodiment. FIG. 16B is a diagram for explaining an example of complementing the occlusion area according to the third embodiment. FIG. 17 is a diagram illustrating a hardware configuration example of the image processing apparatus.

  Hereinafter, embodiments of an image processing device, an image processing circuit, an image processing program, and an image processing method disclosed in the present application will be described with reference to the drawings. The image processing apparatus, the image processing circuit, the image processing program, and the image processing method disclosed in the present application are not limited by the following embodiments. Moreover, the same code | symbol is attached | subjected to the structure which has the same function in each Example, or the step which performs the same process, and the overlapping description is abbreviate | omitted.

  In the following, as an example, a case where a right distance image is formed from a left distance image will be described. However, by replacing “left” with “right” and “right” with “left” in the following embodiments, a left-side distance image can be formed from a right-side distance image in the same manner as in the following embodiments. . When a right distance image is formed from a left distance image, the left distance image corresponds to a first distance image, and the right distance image corresponds to a second distance image having a different viewpoint from the first distance image. Conversely, when forming the left distance image from the right distance image, the right distance image corresponds to the first distance image, and the left distance image corresponds to the second distance image having a different viewpoint from the first distance image.

[Example 1]
<Configuration example of image processing apparatus>
FIG. 1 is a block diagram illustrating an example of an image processing apparatus according to the first embodiment. In FIG. 1, the image processing apparatus 10 includes a left-side distance image forming unit 11 and a right-side distance image forming unit 12.

  The left-side image and the right-side image are input to the left-side distance image forming unit 11. The left distance image forming unit 11 uses the left image as a standard image and the right image as a reference image, and forms a left distance image using block matching.

  That is, as shown in FIG. 2, the left-side distance image forming unit 11 divides the left-side image into a plurality of M × N blocks including M blocks on the x axis and N blocks on the y axis. FIG. 2 is a diagram illustrating an example of block division according to the first embodiment. In the M × N block, if the origin block is the upper left block (0,0) and any target block is the block (m, n), the lower right block is the block (M-1, N-1). It becomes. That is, m takes any value from 0 to M-1, and n takes any value from 0 to N-1. The length of one block in the x-axis direction is X, and the length in the y-axis direction is Y. Further, the pixel position at the upper right end of the block (m, n) is (bx, by).

  The left distance image forming unit 11 uses the left image as a base image and the right image as a reference image, and the target block (m, n) is one block at a time from block (0, 0) to block (M-1, N-1). While shifting, block matching is performed on all blocks of M × N blocks. That is, the left distance image forming unit 11 searches for the position of the image most similar to the image of the target block (m, n) (that is, the position of the image with the smallest error) in the entire area of the right image. Here, the same object that exists in both the left image and the right image is photographed in the left image that is shifted to the right relative to the photographing position in the right image. Therefore, as shown in FIG. 2, when the right direction is the positive direction of the x-axis and the downward direction is the positive direction of the y-axis, the left-side distance image forming unit 11 determines the target from the position of the image found on the right-side image. The value (xmn, ymn) of the distance to the block (m, n) is written in the target block (m, n) as a parallax value.

Here, for example, SAD (Sum of Absolute Difference) is used to evaluate the similarity. Here, SAD is represented by the formula (1). In Expression (1), in represents a left image that is a base image, and ref represents a right image that is a reference image. Further, in (bx + i, by + j) is the luminance value of the pixel at the coordinates (bx + i, by + j) of the left image, and ref (bx + i + x, by + j + y) is the luminance value of the pixel at the coordinates (bx + i + x, by + j + y) of the right image. Represent.

  The left-side distance image forming unit 11 obtains SAD (x, y) for the entire area of the right image according to the equation (1) for the target block (m, n), and SAD (x, y) is minimum on the right image. , That is, a position (x, y) with the smallest error from the target block (m, n). The left-side distance image forming unit 11 obtains a distance (xmn, ymn) from the position (bx, by) on the target block (m, n) to the position (x, y) on the right image, and the distance ( xmn, ymn) is written to the target block (m, n) as the parallax value of the target block (m, n). The left-side distance image forming unit 11 calculates and writes the distance by shifting the target block (m, n) from the block (0, 0) to the block (M-1, N-1) one block at a time. A left-side distance image is formed by performing processing for all blocks. Then, the left distance image forming unit 11 outputs the formed left distance image.

  The left distance image formed by the left distance image forming unit 11 is input to the right distance image forming unit 12. The right distance image forming unit 12 forms a right distance image based on the left distance image as follows.

FIG. 3 is a diagram for explaining the formation of the right distance image according to the first embodiment. The target block (m, n) in which the distance (xmn, ymn) is written in the left distance image by the above block matching is the distance (xmn) from the position of the right image with the smallest error from the target block (m, n). , ymn). In addition, the same object that exists in both the left image and the right image is photographed in the right image that is shifted to the left relative to the photographing position in the left image. Therefore, when the right image is divided into a plurality of M × N blocks, the block (m ′, n) having the smallest error from the target block (m, n) in which the distance (xmn, ymn) of the left distance image is written. The position where ') is present in the right image is represented by equation (2).

  Also, between the two blocks with the smallest error between the left image and the right image, the distance from one block to the other block is the same as the distance from the other block to the one block. In opposite directions. Therefore, the disparity value of the block (m ′, n ′) with the smallest error from the target block (m, n) in which the distance (xmn, ymn) of the left distance image is written is (−xmn, −ymn). .

  Therefore, the right distance image forming unit 12 moves each block of the left distance image by a distance corresponding to the parallax value of each block, and forms a right distance image from each block after the movement. That is, in the M × N block, the right distance image forming unit 12 converts the left distance image block (m, n) into a distance (xmn, ymn) corresponding to the parallax value (xmn, ymn) of the block (m, n). ) Is moved in the negative direction of the x-axis and y-axis to form a right distance image block (m ′, n ′). Then, the right distance image forming unit 12 uses the parallax values (−xmn, −ymn) obtained by inverting the sign of the parallax values (xmn, ymn) of the block (m, n) as the parallax of the block (m ′, n ′). Value. The right distance image forming unit 12 moves the block and inverts the sign of the parallax value, and changes the block (m, n) from the block (0, 0) to the block (M-1, N-1) in the left distance image. A right-distance image is formed by performing all blocks of M × N blocks while shifting each block. Then, the right distance image forming unit 12 outputs the formed right distance image.

  In general, since parallax in the vertical direction (y-axis direction) is not considered in a 3D image, the processing in the y-axis direction can be omitted in each of the above processes. That is, ymn can always be 0.

<Processing of image processing apparatus>
FIG. 4 is a flowchart for explaining processing of the image processing apparatus according to the first embodiment.

  First, the right distance image forming unit 12 sets the left distance image block (m, n) as an initial position (step S41). For example, the initial position is (m, n) = (0,0).

  Next, the right distance image forming unit 12 determines the block position (m ′, n ′) of the right distance image from the parallax value of the block (m, n) according to Expression (2) (step S42).

  Next, the right distance image forming unit 12 moves the block (m, n) to the block position (m ′, n ′) (step S43).

  Next, the right distance image forming unit 12 inverts the sign of the parallax value of the block (m, n) moved to the block position (m ′, n ′) (step S44).

  Next, the right distance image forming unit 12 determines whether or not the movement of all the blocks of the left distance image has been completed (step S45). For example, the right distance image forming unit 12 determines that the movement of all the blocks of the left distance image is completed when (m, n) = (M−1, N−1). When the movement of all the blocks of the left distance image is completed (step S45: Yes), the right distance image forming unit 12 outputs the right distance image and ends the process.

  On the other hand, when the movement of all the blocks of the left distance image is not completed (step S45: No), the right distance image forming unit 12 shifts the block (m, n) by one block in the x-axis direction or the y-axis direction. (Step S46), and the process returns to Step S42.

<Example of forming a right distance image>
FIG. 5 is a diagram illustrating an example of the left distance image of the first embodiment, and FIGS. 6A to 6D are diagrams for explaining an example of forming the right distance image of the first embodiment. 5 and 6A to 6D show an example in which M = 16 and N = 8.

  The left distance image forming unit 11 forms a left distance image as shown in FIG. The shade of each block of the left distance image represents the magnitude of the parallax value of each block. That is, the darker the value is, the smaller the parallax value is, and the longer the distance from the viewpoint to the object, and the lighter the value, the larger the parallax value, the closer the distance from the viewpoint to the object. Here, it is assumed that each block of the left distance image has a parallax value of 0, 2, or 3. In addition, the parallax in the vertical direction (y-axis direction) is not considered.

  On the other hand, in the right distance image forming unit 12, the initial state of the M × N block is a state in which all the blocks of the M × N block are null as shown in FIG. 6A.

  Here, the block movement described above is performed from the block (0, 0) to the block (M-1, N-1) in the order of the blocks. However, here, in order to explain the formation of the right distance image by the block movement in an easy-to-understand manner, the block movement is performed in the order of the parallax.

  That is, as shown in FIG. 6B, the right distance image forming unit 12 moves the block of “parallax value = 0” in the left distance image (FIG. 5) by a distance according to the parallax value, that is, without moving the block. Write to the M × N block of the right distance image.

  Next, as shown in FIG. 6C, the right-side distance image forming unit 12 converts the block of “parallax value = 2” in the left-side distance image (FIG. 5) into a parallax value in the negative x-axis direction (that is, the left direction). The corresponding distance is moved, that is, it is moved by two blocks and written in the M × N block of the right distance image.

  Next, as shown in FIG. 6D, the right distance image forming unit 12 converts the block of “parallax value = 3” in the right distance image (FIG. 5) into a parallax value in the negative x-axis direction (that is, the left direction). The corresponding distance is moved, that is, it is moved by 3 blocks and written in the M × N block of the right distance image.

  Then, in the state shown in FIG. 6D in which writing of all the blocks of the left distance image to the right distance image is completed, the right distance image forming unit 12 adds the sign of the parallax value of each block except for the blocks that remain Null. Invert to "-".

<Specific examples of images>
FIG. 7 is a diagram illustrating an example of the left image of the first embodiment, and FIG. 8 is a diagram illustrating an example of the right image of the first embodiment. 7 and 8, there are a bus, a building, and a background in order from the viewpoint. The left distance image forming unit 11 forms a left distance image as shown in FIG. 9 by block matching using the left image (FIG. 7) as a standard image and the right image (FIG. 8) as a reference image. FIG. 9 is a diagram illustrating an example of a left-side distance image according to the first embodiment.

  On the other hand, the right distance image forming unit 12 uses the left distance image (FIG. 9) to form the right distance image shown in FIG. 10 by the block movement described above. FIG. 10 is a diagram illustrating an example of the right distance image according to the first embodiment. The shading in FIGS. 9 and 10 represents the magnitude of the parallax value. That is, the darker the value is, the smaller the parallax value is, and the longer the distance from the viewpoint to the object, and the lighter the value, the larger the parallax value, the closer the distance from the viewpoint to the object. In the right distance image, as shown in FIG. 10, an “occlusion region” exists. The “occlusion area” is an image area that exists in one of the left image and the right image but does not exist in the other.

  As described above, according to the present embodiment, in the image processing apparatus 10, the right distance image forming unit 12 treats each of the M × N blocks constituting the left distance image by a distance corresponding to the parallax value of each block. A right distance image having a different viewpoint from the left distance image is formed from the moved M × N block.

  That is, the right distance image forming unit 12 forms the right distance image by moving the blocks of the left distance image without performing block matching. The calculation amount of block movement is very small compared to the calculation amount of block matching. Further, the right distance image forming unit 12 does not perform SAD calculation when forming the right distance image. Therefore, according to the present embodiment, it is possible to reduce the amount of calculation for forming the distance image. As a result, the power consumption of the image processing apparatus 10 that forms a distance image can be reduced.

[Example 2]
<Configuration example of image processing apparatus>
FIG. 11 is a block diagram illustrating an example of an image processing apparatus according to the second embodiment. In FIG. 11, the image processing apparatus 20 includes an occlusion area determination unit 21.

  In addition to the processing of the first embodiment, the right distance image forming unit 12 moves the left distance image block (m, n) to the block position (m ′, n ′) of the right distance image. The occlusion area determination unit 21 is notified of the position (m ′, n ′).

  The occlusion area determination unit 21 updates the occlusion area map each time the block position (m ′, n ′) after movement is notified. Then, when the movement of all the blocks in the left distance image is completed, the occlusion area determination unit 21 refers to the occlusion area map, and in the right distance image, occludes an area in which no block after movement from the left distance image exists. Determine the area. And the occlusion area | region determination part 21 outputs the information which shows the position of the block which comprises an occlusion area | region as a determination result.

<Processing of image processing apparatus>
FIG. 12 is a flowchart for explaining processing of the image processing apparatus according to the second embodiment. In addition, since the process of step S41, S42, S44-S46 is the same as FIG. 4, description is abbreviate | omitted.

  First, in step S51, the occlusion area determination unit 21 initializes an occlusion area map (step S51). The occlusion area map is composed of a plurality of M × N blocks as in the right distance image, and all blocks are set to “0” by the initialization in step S51.

  In step S52, as in step S43 in FIG. 4, the right distance image forming unit 12 moves the block (m, n) to the block position (m ′, n ′). Further, the right distance image forming unit 12 notifies the occlusion region determining unit 21 of the block position (m ′, n ′) after the movement (step S52).

  In step S53, the occlusion area determination unit 21 changes the value of the block corresponding to the block position (m ′, n ′) notified in step S52 from “0” to “1” among the M × N blocks of the occlusion area map. To "" (step S53). The occlusion area determination unit 21 performs the process of step S53 every time the left distance image block is moved.

  When the movement of all the blocks of the left distance image is completed (step S45: Yes), the right distance image forming unit 12 outputs the right distance image and indicates that the movement of all the blocks of the left distance image is completed. Notify the determination unit 21.

  When the movement of all the blocks in the left distance image is completed (step S45: Yes), the occlusion area determination unit 21 refers to the occlusion area map and is configured by blocks whose values remain “0”. The area is determined as an occlusion area (step S54).

<Example of updating the occlusion area map>
13A to 13D are diagrams for explaining an example of updating the occlusion area map according to the second embodiment. 13A to 13D show an example where M = 16 and N = 8.

  When the right distance image is in the initial state shown in FIG. 6A, the occlusion area map is in the initial state shown in FIG. 13A. That is, in the occlusion area map, the values of all blocks are “0” in the initial state.

  Next, when the right distance image is in the state shown in FIG. 6B, the occlusion area map is updated to the state shown in FIG. 13B. That is, among the M × N blocks of the occlusion area map, the value of the block at the position where the block of “parallax value = 0” is written in the right distance image is updated from “0” to “1”.

  Next, when the right distance image is in the state shown in FIG. 6C, the occlusion area map is updated to the state shown in FIG. 13C. That is, among the M × N blocks of the occlusion area map, the value of the block at the position where the block of “parallax value = 2” is written in the right distance image is updated from “0” to “1”.

  Next, when the right distance image is in the state shown in FIG. 6D, the occlusion area map is updated to the state shown in FIG. 13D. That is, among the M × N blocks of the occlusion area map, the value of the block at the position where the block of “parallax value = 3” is written in the right distance image is updated from “0” to “1”.

  An area composed of blocks whose values remain “0” in the occlusion area map when all the blocks in the left distance image have been written to the right distance image is moved from the left distance image in the right distance image. It matches the area where no later block exists. Therefore, in the state shown in FIG. 13D in which writing of all the blocks of the left distance image to the right distance image is completed, the occlusion area determination unit 21 is configured by blocks whose values remain “0” in the occlusion area map. The area is determined as an occlusion area. That is, in the image processing apparatus 20, the occlusion area determination unit 21 determines an area in the right distance image where no block after movement from the left distance image exists as an occlusion area.

  Thereby, the occlusion area can be determined easily and in a short time.

[Example 3]
FIG. 14 is a block diagram illustrating an example of an image processing apparatus according to the third embodiment. In FIG. 14, the image processing apparatus 30 includes an occlusion area complementing unit 31.

  The right distance image is input from the right distance image forming unit 12 to the occlusion region complementing unit 31. The occlusion area complementing unit 31 receives information indicating the position of the block constituting the occlusion area from the occlusion area determining unit 21 as a determination result.

  The occlusion area complementing unit 31 complements the occlusion area with a block having the smallest parallax value among a plurality of blocks adjacent to the occlusion area in the right distance image. Then, the occlusion region complementing unit 31 outputs the right-side distance image after complementing.

<Processing of image processing apparatus>
FIG. 15 is a flowchart for explaining processing of the image processing apparatus according to the third embodiment. In addition, since the process of step S41, S42, S44-S46 and step S51-S54 is the same as FIG. 4, FIG. 12, description is abbreviate | omitted.

  When the occlusion area is determined by the occlusion area determining unit 21 (step S54), the occlusion area complementing unit 31 complements the occlusion area of the right distance image according to the determination result (step S61).

<Complementary example of occlusion area>
16A and 16B are diagrams for explaining an example of complementing the occlusion area according to the third embodiment. 16A and 16B show an example where M = 16 and N = 8.

  In FIG. 6D, the area constituted by the blocks remaining as Null is constituted by the blocks whose values remain “0” in the occlusion area map when the writing of all the blocks of the left distance image to the right distance image is completed. Matches the region being That is, the area constituted by the blocks remaining as Null in FIG. 6D is an occlusion area.

  Therefore, for example, the right distance image formed as shown in FIGS. 5 and 6A to D includes an occlusion area divided into five areas OC1 to OC5 as shown in FIG. 16A.

  Here, the occlusion region is estimated to be an invisible region where an object that is far from the viewpoint is hidden by an object that is close to the viewpoint. That is, it is estimated that an object with a long distance from the viewpoint exists in the occlusion area. Also, the parallax value for an object that is far from the viewpoint is smaller than the parallax value for an object that is close to the viewpoint.

  Therefore, the occlusion area complementing unit 31 complements the occlusion area with a block having the smallest parallax value among a plurality of blocks adjacent to the occlusion area in the right distance image.

  For example, in FIG. 16A, a block with a parallax value = 2 and a block with a parallax value = 0 are adjacent to each other in the x-axis direction in the occlusion areas OC1, OC2, and OC3. Therefore, the occlusion area complementing unit 31 complements each of the occlusion areas OC1, OC2, and OC3 with a block having a parallax value = 0.

  Also, in the x-axis direction, a block with a parallax value = 3 and a block with a parallax value = 2 are adjacent to each other in the occlusion areas OC4 and OC5. Therefore, the occlusion area complementing unit 31 complements each of the occlusion areas OC4 and OC5 with a block having a parallax value = 2.

  As a result of complementing the occlusion areas OC1 to OC5, the occlusion area complementing unit 31 obtains a right-side distance image after complementing as shown in FIG. 16B.

  As described above, according to the present embodiment, the occlusion area complementing unit 31 complements the occlusion area with the block having the smallest parallax value among the plurality of blocks adjacent to the occlusion area in the right distance image.

  As a result, the occlusion area can be complemented by the distance image that is farther from the viewpoint, so that the occlusion area can be complemented with high accuracy. In addition, since the occlusion region can be complemented with high accuracy, the accuracy of the right distance image can be improved.

[Other embodiments]
[1] The image processing apparatuses 10, 20, and 30 illustrated in the first to third embodiments do not necessarily need to be physically configured as illustrated. In other words, the specific form of distribution / integration of each unit is not limited to that shown in the figure, and all or a part thereof may be functionally or physically distributed / integrated in arbitrary units according to various loads or usage conditions. Can be configured.

  Furthermore, various processing functions performed in the image processing apparatuses 10, 20, and 30 are performed on a CPU (Central Processing Unit) (or a microcomputer such as an MPU (Micro Processing Unit) or MCU (Micro Controller Unit)). All or any part may be executed. In addition, various processing functions may be executed in whole or in any part on a program that is analyzed and executed by a CPU (or a microcomputer such as an MPU or MCU) or on hardware based on wired logic. Good.

  [2] The image processing apparatuses 10, 20, and 30 according to the first to third embodiments can be realized by the following hardware configuration, for example.

  FIG. 17 is a diagram illustrating a hardware configuration example of the image processing apparatus. As illustrated in FIG. 17, the image processing apparatus 100 includes a processor 101, a memory 102, a camera 103, and a display device 104.

  Examples of the processor 101 include a central processing unit (CPU), a digital signal processor (DSP), and a field programmable gate array (FPGA). Examples of the memory 102 include a random access memory (RAM) such as a SDRAM (Synchronous Dynamic Random Access Memory), a read only memory (ROM), and a flash memory.

  Various processing functions performed by the image processing apparatuses 10, 20, and 30 according to the first to third embodiments may be realized by the processor 101 executing programs stored in various memories such as a nonvolatile storage medium. . That is, a program corresponding to each process executed by the left distance image forming unit 11, the right distance image forming unit 12, the occlusion area determining unit 21, and the occlusion area complementing unit 31 is recorded in the memory 102, and each program is recorded. May be executed by the processor 101. Further, the left image and the right image may be taken by the camera 103. Further, the left distance image, the right distance image, and the complemented right distance image may be displayed on the display device 104.

  The processor 101 may be realized as a one-chip image processing circuit.

10, 20, 30 Image processing apparatus 11 Left distance image forming unit 12 Right distance image forming unit 21 Occlusion region determining unit 31 Occlusion region complementing unit

Claims (6)

A memory, and a processor connected to the memory,
The processor is
Each of the plurality of blocks constituting the first distance image is moved by a distance according to the parallax value of each block,
Forming a second distance image having a different viewpoint from the first distance image from the plurality of blocks after movement;
Image processing device. The processor is
In the second distance image, an area where the plurality of blocks after the movement does not exist is determined as an occlusion area.
The image processing apparatus according to claim 1. The processor is
In the second distance image, the occlusion area is complemented by a block having a minimum parallax value among a plurality of blocks adjacent to the occlusion area.
The image processing apparatus according to claim 2. A first forming unit for forming a first distance image;
Each of the plurality of blocks constituting the first distance image is moved by a distance corresponding to the parallax value of each block, and the second distance image having a different viewpoint from the first distance image is moved from the plurality of blocks after the movement. A second forming part for forming
An image processing circuit comprising: Each of the plurality of blocks constituting the first distance image is moved by a distance according to the parallax value of each block,
Forming a second distance image having a different viewpoint from the first distance image from the plurality of blocks after movement;
An image processing program for causing a processor to execute processing. Each of the plurality of blocks constituting the first distance image is moved by a distance according to the parallax value of each block,
Forming a second distance image having a different viewpoint from the first distance image from the plurality of blocks after movement;
Image processing method.

Images