JP2020021329A - Image processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing device capable of realizing a process of creating an integrated image from a modified differential image obtained by processing a differential image or a process equivalent thereto. SOLUTION: In an image processing apparatus, a differential processing unit 211 creates an X-direction partial differential image 202 and a Y-direction partial differential image 203 by performing differentiation in one-dimensional directions for each of the X direction and the Y direction. The differential image change processing unit 212 creates the X direction change partial differential image 204 and the Y direction change partial differential image 205 obtained by changing the X direction partial differential image 202 and the Y direction partial differential image 203. The path matching integration processing unit 213 creates a path matching integral image 206 in which the X direction changing partial differential image 204 and the Y direction changing partial differential image 205 are route matching integrated. [Selection diagram] Fig. 2

Description

  The present invention relates to an image processing device.

  In the field of image processing technology, when looking at the framework, for example, as described in Non-Patent Document 1, a converted image is obtained by directly processing an original image or by performing a Fourier transform, a wavelet transform, or the like on the original image. After processing the converted image, an inverse conversion is performed to obtain a processed image. When an edge image is created from an original image, the edge image is used as a material for extracting and recognizing features, or as a processed material for adding to the original image and sharpening the image.

  Various image processes are used depending on the purpose and the characteristics of the target image. For example, in order to improve the local contrast of an image, a process such as CLAHE (Contrast limited adaptive histogram equalization) described in Prior Art Document 2 is used. In CLAHE, an image is divided into small areas of 8 * 8 pixels, and adaptive processing is performed using flattening of a histogram in which contrast is limited for each small area. In addition, bilinear interpolation (interpolation) is performed in order to eliminate pseudo contours appearing at boundaries between adjacent small areas.

  Further, for example, when performing edge-preserving smoothing in which noise is reduced by preserving the edges of an image, as described in Prior Art Document 3, smoothing is performed based on a local deviation in consideration of the direction. Adaptive processing such as deciding a direction and a degree and changing parameters such as smoothing according to an edge portion or a flat portion is performed.

Yukiichi Sakai, "Basics and Application of Digital Image Processing", CQ Publishing Company "Histogram 2: Histogram flattening OpenCV-Python Tutorials 1" http: // labs. eecs. tottori-. ac. jp / sd / Member / Oyamada / OpenCV / html / py_tutorials / py_imgproc / py_histograms / py_histogram_equalization / py_histogram_equalization. html Junichi Taguchi, et al., “Direction-dependent image filter that separates edge part and flat part”, IEICE Transactions, D-II, Vol. J80-D-II, No. 9, pp. 2345-2350 (1997)

  However, in the related art, a kind of differential image of the edge image is created, but the modified differential image obtained by processing and modifying the original differential image may have a different integral value depending on a path. is there. Therefore, strictly speaking, there is no inverse transformation of the modified differential image obtained by processing and modifying the original differential image, and it is necessary to create an integral image by performing some kind of matching processing. Heretofore, there has been no known image processing having a framework for creating an integral image from a modified differential image obtained by processing an original differential image and making a modification.

  The present invention has been made in view of the above circumstances, and an object of the present invention is to provide an image processing apparatus capable of realizing a process of processing a differential image to create an integrated image from a modified differential image obtained by modification and a process equivalent thereto. Is to provide.

  To achieve the above object, an image processing apparatus according to a first aspect includes a differential processing unit that generates a differential image obtained by differentiating an input image, and a differential image changing process that generates a modified differential image obtained by changing the differential image. And a path integration processing section that creates a path integral image by path integrating the modified differential image.

  According to the present invention, it is possible to realize a process of creating an integrated image from a modified differential image obtained by processing a differential image and making a change, or a process equivalent thereto.

FIG. 1 is a block diagram illustrating a configuration of the image processing system according to the first embodiment. FIG. 2 is a block diagram showing an example of the configuration of the image processing apparatus of FIG. 1 and a processed image. FIG. 3 is a diagram illustrating an example of a direction of a processed image of the image processing apparatus of FIG. FIG. 4 is a flowchart illustrating the path integration processing of the image processing apparatus of FIG. FIG. 5 is a diagram showing a path matching integration method for the entire processed image of the image processing apparatus of FIG. FIG. 6 is a diagram showing a path matching integration method for a part of the processed image of the image processing apparatus of FIG. FIG. 7 is a flowchart illustrating a method of changing the offset of an integral image according to the second embodiment. FIG. 8 is a diagram showing a setting example of the average value calculation area in FIG. FIG. 9 is a diagram illustrating a method for creating an integral image according to the third embodiment. FIG. 10 is a flowchart illustrating a method for creating an integral image according to the third embodiment. FIG. 11 is a diagram showing an example of block division of a block division integral image according to the fourth embodiment. FIG. 12 is a diagram showing the positional relationship between the four blocks in FIG. 11 and the pixel of interest. FIG. 13 is a flowchart illustrating a method for removing noise from a block-divided integral image according to the fourth embodiment. FIG. 14 is a diagram showing an integral image according to the presence or absence of the processing of FIG. 13 with respect to the input image of FIG. FIG. 15 is a flowchart illustrating the level adjustment processing of the integral image according to the fifth embodiment. FIG. 16 is a diagram showing an integrated image according to the presence or absence of the processing of FIG. FIG. 17 is a flowchart illustrating the display processing of the integral image according to the sixth embodiment. FIG. 18 is a diagram showing an integral image according to the presence or absence of the processing of FIG. FIG. 19 is a block diagram illustrating an example of a configuration and a processed image of the image processing apparatus according to the seventh embodiment. FIG. 20 is a flowchart showing the binarization processing of the integral image of FIG. FIG. 21 is a diagram showing a binarized image of the input image and the integral image of FIG. FIG. 22 is a diagram illustrating an example of another processed image of the image processing apparatus in FIG. FIG. 23 is a flowchart illustrating a method for generating a modified differential image of the image processing device according to the eighth embodiment. FIG. 24 is a block diagram illustrating a configuration of the path matching integration processing unit of the image processing device according to the ninth embodiment. FIG. 25 is a flowchart illustrating a block image creation method of the image processing apparatus according to the tenth embodiment. FIG. 26 is a flowchart illustrating a block distortion removal method for a block image according to the eleventh embodiment. FIG. 27 is a block diagram illustrating a configuration of an image processing device according to the twelfth embodiment. FIG. 28 is a diagram illustrating a state of a gradient in an adjacent four-point mesh when a matched differential image is created by the image processing apparatus in FIG. 27. FIG. 29 is a flowchart showing a matched differential image creating process of the image processing apparatus of FIG. FIG. 30 is a diagram illustrating an example of the distance between adjacent four-point meshes when the matched differential image is created by the image processing apparatus in FIG. 27. FIG. 31 is a block diagram illustrating a hardware configuration example of the image processing apparatus in FIG.

  Embodiments will be described with reference to the drawings. It should be noted that the embodiments described below do not limit the invention according to the claims, and all the elements and combinations thereof described in the embodiments are essential for solving the invention. Not necessarily.

FIG. 1 is a block diagram illustrating a configuration of the image processing system according to the first embodiment.
1, the image processing system includes a photographing device 100, image processing devices 111, 121, 131, display devices 112, 122, 132, input devices 113, 123, 133, and storage devices 114, 124, 134.

  The image processing apparatuses 111, 121, and 131 are connected to each other via a communication network 140 such as the Internet. The image processing device 111 is connected to the imaging device 100, the display device 112, the input device 113, and the storage device 114. The image processing device 121 is connected to the display device 122, the input device 123, and the storage device 124. The image processing device 131 is connected to the display device 132, the input device 133, and the storage device 134.

  The photographing apparatus 100 photographs a subject and generates image data. The imaging device 100 is, for example, a digital camera, a camera attached to a smartphone or a mobile phone, a scanner, an X-ray photography device used in a medical field, a monitoring camera used in a monitoring site such as an MRI (Magnetic Resonance Imaging), and the like. Examples include various imaging devices that capture images capturing ultrasonic waves, infrared rays, visible light, ultraviolet rays, X-rays, γ-rays, electron beams, and the like used in various inspection sites.

  The image processing device 111 can receive image data from the imaging device 100 and perform various image processing based on input information from the input device 113. Further, the image processing device 111 can display the processing result including the processed image data on the display device 112, store the processing result in the storage device 114, or transmit the processing result to the communication network 140. Further, the image processing device 111 can receive request information from the outside and transmit various information such as image data stored in the storage device 114 to the outside. The image processing device 111 may be, for example, a general-purpose computer such as a workstation, a desktop personal computer, a notebook personal computer, a tablet terminal, or a smartphone, or may be hardware dedicated to image processing.

  As the display device 112, a display, a television, or the like can be used. As the input device 113, a keyboard, a mouse, or the like can be used. As the storage device 114, a magnetic disk device, an optical disk device, an SSD (Solid State Drive), a USB (Universal Serial Bus) memory, or the like can be used.

  Note that an image processing device 111, a display device 112, an input device 113, and a storage device 114 are integrated in a notebook computer, a publet terminal, a smartphone, and the like.

  The communication network 140 is a line capable of transmitting and receiving various information data including image data, and can be connected to all over the world. As the communication network 140, for example, the Internet can be used. The communication network 140 may also have its own dedicated line in a local area.

  The image processing device 121 can receive image data from the storage device 114 connected to the image processing device 111 and perform various image processing based on input information from the input device 123. Further, the image processing device 121 can display a processing result including the processed image data on the display device 122, store the processing result in the storage device 124, or transmit the processing result to the communication network 140. Further, the image processing device 121 can receive request information from the outside and transmit various information such as image data stored in the storage device 124 to the outside.

  The image processing device 131 can receive image data from the storage device 124 connected to the image processing device 121 and perform various image processing based on input information from the input device 133. Further, the image processing device 131 can display a processing result including the processed image data on the display device 132, store the processing result in the storage device 134, or transmit the processing result to the communication network 140.

  The image processing function of each of the image processing apparatuses 111, 121, and 131 can be implemented by installing software (program) for implementing image processing. When the image capturing apparatus 100 includes an image processing apparatus, the built-in image processing apparatus can also perform image processing, and the image processing can be performed by installing hardware dedicated to image processing. Can also.

  In this image processing system, for example, an individual can use a digital camera as the photographing device 100 and use a notebook personal computer with a built-in storage device 114 as the image processing device 111. Then, the image data taken by the individual with the digital camera is stored in the storage device 114, and the storage device 124 connected to the image processing device 121 of the information service company of the external SNS (social networking service) via the communication network 140. You can upload your images to and make them accessible to the general public. At this time, the user can view the uploaded image on the display device 132 connected to the image processing device 131 owned by the user.

  In addition, at a medical site, image data in various image capturing apparatuses such as an X-ray photographing apparatus or an MRI as the image capturing apparatus 100 is connected to these image capturing apparatuses, or an image processing apparatus incorporating these image capturing apparatuses. 111 can be sent to the image processing device 121 which is a data server in the hospital. Then, the doctor can refer to the image on the display device 132 connected to the image processing device 131.

  Here, the image processing device 111 creates a differentiated image obtained by differentiating the captured image captured by the imaging device 100, creates a modified differential image obtained by modifying the differentiated image, and integrates the modified differential image into a path. An integral image can be created. Here, by performing the change processing on the differential image, various characteristics can be given to the path integral image obtained by path integrating the changed differential image depending on the manner of the change. For example, a sharpened image with improved contrast or a noise-reduced image with reduced noise components can be obtained while preserving edges.

FIG. 2 is a block diagram showing an example of the configuration of the image processing apparatus of FIG. 1 and a processed image.
2, the image processing device 111 includes a differential processing unit 211, a differential image change processing unit 212, and a path matching integration processing unit 213.

  The differential processing unit 211 creates a differential image obtained by differentiating the input image 201. The input image 201 may be a color image or a black and white image. The input image 201 may be an image photographed by the photographing device 100 of FIG. In FIG. 2, a monochrome image having monochrome grayscale values in each pixel is taken as an example of the input image 201. When the input image 201 is a two-dimensional image, the differentiation direction can be set in two directions, the X direction and the Y direction. At this time, the differential processing unit 211 creates an X-direction partial differential image 202 and a Y-direction partial differential image 203 by performing one-dimensional differentiation in each of the X direction and the Y direction.

FIG. 3 is a diagram illustrating an example of a direction of a processed image of the image processing apparatus of FIG.
In FIG. 3, the input image 201 has two directions, that is, vertical and horizontal directions on a two-dimensional plane. The upper left point of the two-dimensional plane is defined as the coordinate origin 221, the right direction is defined as the X direction 222, and the downward direction is defined as the Y direction 223.

  At this time, if the input image 201 is described as I, the pixel value of the input image 201 at the point (pixel) at the coordinates (x, y) can be expressed as I (x, y). When the X-direction partial differential image 202 is described as Dx and the Y-direction partial differential image 203 is described as Dy, the respective differential values of the coordinates (x, y) can be given by the following equations (1) and (2).

(Equation 1)
Dx (x, y) = I (x + 1, y) -I (x, y)

(Equation 2)
Dy (x, y) = I (x, y + 1) -I (x, y)

  The above differential image is a partial differential image. Alternatively, the differential image may be an image linked to the partial differential image by conversion and inverse conversion. For example, the X direction partial differential image and the Y direction partial differential image can be converted into an absolute value image and an angle image. Further, the absolute value image and the angle image can be inversely transformed into an X direction partial differential image and a Y direction partial differential image. Since the differential image is a partial differential image or an image that can be converted to a partial differential image, line integration along a path is possible.

  In the field of general image processing, there exists a differential image different from the definition described above. For example, in Reference Document 1 below, a differential image defined by an inverse operation is introduced for an integral image (integrated image) referred to in Reference Document 2 below. The integral image (integrated image) of Reference 2 has a different definition from the integrated image of the present embodiment because line integration along the path is not used. In addition, the differential image of Reference 1 cannot be line-integrated along a path, and therefore cannot be a target differential image in the present embodiment by itself. In order to target the differential image of Reference 1 in the present embodiment, it is necessary to add some information so that path integration can be performed. As described above, the differential image targeted in the present embodiment provides information that allows line integration along the path.

  Reference 1: Kohei Inoue, Kenji Hara, Kiichi Urahama, "Integral Image-Based Differential Filters," International Journal of Computer, Electronica., International, International, and International. 8, no. 5, pp. 812-821, 2014.

  The differential image change processing unit 212 creates an X direction changed partial differential image 204 and a Y direction changed partial differential image 205 obtained by changing the X direction partial differential image 202 and the Y direction partial differential image 203. There are various ways of changing the X-direction partial differential image 202 and the Y-direction partial differential image 203. For example, in accordance with the absolute values of the pixel values of the X-direction changed partial differential image 204 and the Y-direction changed partial differential image 205, the absolute values can be scaled. At this time, when the X-direction changed partial differential image 204 is expressed by Ex and the Y-direction changed partial differential image 205 is expressed by Ey, the X-direction changed partial differential image 204 and the Y-direction changed partial differential image The change to the changed partial differential image 205 can be given by, for example, the following Expressions 3 and 4.

(Equation 3)
Ex (x, y) = k * (Dx (x, y) -lv) + lv (when Dx (x, y) ≧ lv)
= Dx (x, y) * | Dx (x, y) | / lv (when | Dx (x, y) | <lv)
= K * (Dx (x, y) + lv) -lv (when Dx (x, y) ≤-lv)

(Equation 4)
Ey (x, y) = k * (Dy (x, y) -lv) + lv (when Dy (x, y) ≧ lv)
= Dy (x, y) * | Dy (x, y) | / lv (when | Dy (x, y) | <lv)
= K * (Dy (x, y) + lv) -lv (when Dy (x, y) ≤-lv)

  Here, k and lv are predetermined values. | Dx (x, y) | is the absolute value of Dx (x, y). By the processing of Expression 3, Dx (x, y) of the X-direction changed partial differential image 204 has a smaller absolute value in a portion where the absolute value is smaller than lv according to the square equation when the absolute value is smaller than lv. When the absolute value is larger than lv, the value of the part where the absolute value is larger than lv is multiplied by k.

  The path matching integration processing unit 213 creates a path matching integration image 206 by performing path matching integration of the X direction changed partial differential image 204 and the Y direction changed partial differential image 205. The path matching integration processing unit 213 refers to the input image 201 when setting the brightness of the initial point at which the integration is started to be the brightness of the input image 201. In addition, the path matching integration processing unit 213 refers to the input image 201 also when it is desired to divide and integrate into blocks and to match the average brightness in the block with the corresponding average brightness in the block of the input image 201. Further, the path matching integration processing unit 213 can also refer to the input image 201 when setting the display level of the path matching integration image 206.

  Here, the path integral of the present embodiment is a line integral along a path from one point on the image to another point. It is a different concept from path integral when dealing with quantum mechanical state transitions used in the field of physics. In the path integration according to the present embodiment, line integration is performed by adding and subtracting pixel values on a path set in an image. At this time, the path matching integration processing unit 213 refers to the X-direction changed partial differential image 204 when performing path integration in the X direction, and refers to the Y-direction changed partial differential image 205 when performing path integration in the Y direction. . When the differential image is path-integrated without changing the differential image, even when there are a plurality of paths from a certain pixel to another pixel, the line integral values of the plurality of paths do not differ.

  On the other hand, when the path of the modified differential image obtained by modifying the differential image is integrated, if there are a plurality of paths from a certain pixel to another pixel, the line integrated values of the plurality of paths may be different. Here, an integral including a process of calculating one integral value based on a plurality of line integral values obtained by path integral of the modified differential image is referred to as path matching integral. As a method of calculating one integral value, an average value of a plurality of line integral values may be obtained, or one integral value may be obtained based on a plurality of line integral values using a neural network. Is also good. Alternatively, the line integral value of any one of the paths may be selected as the integral value, or a value obtained by adding an appropriate value to the line integral value may be used as the integral value. For example, the maximum value may be selected from a plurality of line integral values, or the maximum value may be selected.

  Note that in the field of general image processing, there is an integrated image different from the definition of the integrated image of the present embodiment described above. For example, in Reference Document 2 below, for high-speed calculation, an image is created in which the sum of the pixel values in a rectangle surrounded by the upper left point of the image and the point of interest is a pixel value, and this is called an integral image. .

  Reference 2: Paul Viola, Michael Jones, "Rapid Object Detection using a Boosted Cascade of Simple Features", "CONFERENCE ON COMPUTER VISION CONTROL UPDATE PATH CONTROL UPDATE PATH CONTROL UPDATE PATH CONTROL UPDATE PATH CONTROL UPDATE PATH CONTROL UPDATE PATH CONTROL ACCESS PART PAGE. 511-518.

  In Reference Document 3 below, a summed-area table (image) in which the sum of pixel values in a rectangle surrounded by a lower left point and a point of interest is calculated for high-speed calculation. The integral image of Reference 2 and the summed area table of Reference 3 differ from each other only in the way of taking a square, but have the same principle of speeding up. Neither the integral image nor the summed area table in Reference 2 described above is an integrated image of the present embodiment.

  Reference 3: Franklin C. et al. Crow, "Summed-Area Tables for Texture Mapping," Computer Graphics Volume 18, Number 3, pp. 147-64. 207-212, July 1984.

  When software (program) for implementing the above-described procedure is created and the software is installed in a computer, the computer implements the differential processing unit 211, the differential image change processing unit 212, and the path matching integration processing unit 213. Can be. The differential processing unit 211, the differential image change processing unit 212, and the path matching integration processing unit 213 may be realized by dedicated hardware.

FIG. 4 is a flowchart illustrating the path integration processing of the image processing apparatus of FIG.
In step S11 in FIG. 4, an initial value and an initial point such as a map are set. At this time, a map M having the same size as the input image 201 and an image F into which an integral value is substituted are prepared. When the integral value has been substituted into a predetermined range of the image F by a later process, the path matching integral image 206 is obtained.

  Here, the size of the input image 201 in the X direction is nx, and the size in the Y direction is ny. Further, the predetermined range for calculating the integral value is x1 or more and x2 or less in the X direction, and y1 or more and y2 or less in the Y direction. When x1 = 0, x2 = nx-1, y1 = 0, and y2 = ny-1, an integrated value of the entire size of the input image 201 is obtained.

  The initial point at which the integration starts is (x0, y0). The initial point (x0, y0) is registered in the 0 adjacent point array as an adjacent point (0 adjacent point) whose distance from the initial point is 0. The number of arrays of the 0 adjacent point array is 1. That is, the value of k (k is an integer of 0 or more) of the k adjacent point array is 0, and the number of arrays is 1. The k adjacent points are adjacent points at a distance k from the initial point. The integral value of the image F is set as the value of the input image 201. At this time, the integral value of the image F can be given by Expression 5 below.

(Equation 5)
F (x0, y0) = I (x0, y0)

  Note that the initial value of the number of arrays of k adjacent point arrays when k is other than 0 is 0. Further, the value of the unreferenced confirmation flag of the k-adjacent-point array indicating that the k-adjacent point has not been focused on is set to true. The map value of the map M is 0 in a predetermined range for obtaining the integral value, and -1 outside the range. However, the map value of the initial point is 2. That is, M (x0, y0) = 2.

  Next, in step S12, a point of interest Pa is set. The k-neighboring points in the k-neighboring point array are sequentially set as the point of interest Pa. That is, when the value of the unreferenced flag of the k-neighboring point array is true, the point stored at the first position of the k-neighboring point array is regarded as the point of interest Pa, and the value of the unreferenced flag of the k-neighboring point array is false. I do. If the value of the unreferenced flag in the k-neighboring point array is false, the k-neighboring point stored at the position next to the position of the k-neighboring point that has been focused on is determined from the k-neighboring point array. And

  Next, in step S13, a point of interest Pb is set. At this time, four adjacent points of the point of interest Pb of the k adjacent points are sequentially set as the point of interest Pb. The point of interest Pb has four adjacent points, left, right, upper and lower. For example, the point of interest Pb is determined in this order.

  Next, in step S14, it is determined whether the point of interest Pb should be integrated. If the map value of the position of the point of interest Pb is 0 or 1, the process proceeds to step S15. If the map value of the position of the point of interest Pb is neither 0 nor 1, the process proceeds to step S16.

  In step S15, processing such as integration of the point of interest Pb is performed. In processing such as integration, path matching integration is performed. At this time, if the map value of the position of the point of interest Pb is 0, the map value is set to 1, and the integrated value of the position is integrated with the integrated value of the point of interest Pa, the X-direction changed partial differential image 204, and the Y-direction changed partial differential image. The value of the point of interest Pb of the image F is obtained by line integration with reference to 205. Then, the point is added to the k + 1 adjacent point array, and the number of k + 1 adjacent point arrays is increased by one.

  If the map value of the position of the point of interest Pb is 1, the map value is set to 2, and the integrated value of the position is referred to the integrated value of the point of interest Pa, the X-direction changed partial differential image 204, and the Y-direction changed partial differential image 205. And calculate by line integration. Then, it is averaged with the value of the point of interest Pb of the image F, which is the integration value of the position already obtained, and the average value is newly set as the value of the point of interest Pb of the image F.

  Next, in step S16, it is determined whether the point of interest Pb is the last point adjacent to the point of interest Pa. If the target point Pb is the last point adjacent to the target point Pa, the process proceeds to step S17. If the target point Pb is not the last point adjacent to the target point Pa, the process returns to step S13.

  In step S17, it is determined whether the point of interest Pa is the last point of the k adjacent points. If the point of interest Pa is the last point of the k adjacent points, the process proceeds to step S18. If the point of interest Pa is not the last point of the k adjacent points, the process returns to step S12.

  Next, in step S18, it is determined whether there is a k + 1 adjacent point. If the number of arrangements of the (k + 1) adjacent points is not 0, there is a (k + 1) adjacent point, and the process proceeds to step S19. If the number of arrays of the k + 1 adjacent points is 0, there is no k + 1 adjacent point, and the path matching integration process ends. At this time, the integral value is substituted into a predetermined range of the image F, and the path matching integral image 206 of FIG. 2 is obtained.

  Next, in step S19, k + 1 reference is performed. At this time, the location where the map value is 1 is set to 2. Then, the process returns to step S12 by referring to the next k + 1 adjacent point. That is, in the processing after step S12, the k + 1 adjacent point is replaced with the k adjacent point, and the value of the unreferenced flag of the k adjacent point array is set to true. The processing from step S12 to step S19 in FIG. 4 is referred to as S10.

FIG. 5 is a diagram showing a path matching integration method for the entire processed image of the image processing apparatus of FIG.
5, the entire image 231 having the same size as the input image 201 is sequentially integrated based on the path matching integration processing in FIG. 4, thereby creating the path matching integration image 206 in FIG. The path matching integrated image 206 created based on the path matching integration processing in FIG. 4 is referred to as an equal distance path average integrated image 206a.

  At this time, the center of the image 231 was set as the initial point P0 at which the integration was started. That is, x0 = nx / 2 and y0 = ny / 2. The display level of the same distance path average integral image 206a is displayed in the range from the minimum value 3 to the maximum value 244 of the input image 201.

  Regarding the vicinity 233 of the initial point P0 at which the integration is started, a path at a distance of 1 from the initial point P0 is shown at 233a, and a path at a distance of 2 from the initial point P0 is shown at 233b. There are points P1 to P4 on the route 233a whose distance from the initial point P0 is 1. There are points P5 to P12 on the route 233b whose distance from the initial point P0 is 2. Here, for example, the path from the initial point P0 to the point P11 includes two paths L1 and L2. At this time, for example, by taking the average of the integral value on the route L1 and the integral value on the route L2, the route matching integral value at the point P11 can be obtained.

  Specifically, the path matching integral values from the initial point P0 to the points P1 to P4 can be given by the following equations 6 to 9.

(Equation 6)
F (x0-1, y0) = F (x0, y0) -Ex (x0-1, y0)

(Equation 7)
F (x0 + 1, y0) = F (x0, y0) + Ex (x0, y0)

(Equation 8)
F (x0, y0-1) = F (x0, y0) -Ey (x0, y0-1)

(Equation 9)
F (x0, y0 + 1) = F (x0, y0) + Ey (x0, y0)

  The path matching integral value from the initial point P0 to the points P5 to P12 can be given by the following Expressions 10 to 17.

(Equation 10)
F (x0-2, y0) = F (x0-1, y0) -Ex (x0-2, y0)

(Equation 11)
F (x0 + 2, y0) = F (x0 + 1, y0) + Ex (x0 + 1, y0)

(Equation 12)
F (x0, y0-2) = F (x0, y0-1) -Ey (x0, y0-2)

(Equation 13)
F (x0, y0 + 2) = F (x0, y0 + 1) + Ey (x0, y0 + 1)

(Equation 14)
F (x0-1, y0-1) = (F (x0-1, y0) -Ey (x0-1, y0-1)
+ F (x0, y0-1) -Ex (x0-1, y0-1)) / 2

(Equation 15)
F (x0 + 1, y0-1) = (F (x0 + 1, y0) -Ey (x0 + 1, y0-1)
+ F (x0, y0-1) + Ex (x0, y0-1)) / 2

(Equation 16)
F (x0 + 1, y0 + 1) = (F (x0 + 1, y0) + Ey (x0 + 1, y0)
+ F (x0, y0 + 1) + Ex (x0, y0 + 1)) / 2

(Equation 17)
F (x0-1, y0 + 1) = (F (x0-1, y0) + Ey (x0-1, y0)
+ F (x0, y0 + 1) -Ex (x0-1, y0 + 1)) / 2

  Expressions 6 to 13 correspond to a case where there is only one shortest path for performing line integration. Equations 14 to 17 are for the case where there are two shortest paths for performing line integration. At this time, the average value of the integral values of these two routes was taken as the path matching integral value.

  For example, in the case of Equation 17, the integral F (x0-1, y0) + Ey (x0-1, y0) along the first path and the integral F (x0, y0 + 1)-along the second path The average value of Ex (x0-1, y0 + 1) was defined as a path matching integral value.

  In the same distance path average integrated image 206 a of FIG. 5, path matching integration is performed for the entire image 231 having the same size as the input image 201. In the same distance path average integrated image 206a, the portion near the center of the image, that is, the initial point P0 at which the integration starts, is an image close to the value of the input image 201, but generally moves away from the value of the input image 201 toward the end. Will be. At this time, the whole balance may be disturbed, deviating from the value of the input image, and an unnatural oblique strip-shaped portion Z1 may be generated. This is because the X-direction changed partial differential image 204 and the Y-direction changed partial differential image 205 obtained by changing the X-direction partial differential image 202 and the Y-direction partial differential image 203 are generally not changed in a well-balanced manner in the entire image, and the overall balance is not changed. This is because the influence of the portion changed by breaking the line becomes larger and more conspicuous as the distance from the initial point P0 of the integration is increased. In particular, in the diagonal direction in which the average of the same distance route is averaged, more contradictory changes are imposed on the integrated image, and thus a diagonally unnaturally-appearing band Z1 may appear.

FIG. 6 is a diagram showing a path matching integration method for a part of the processed image of the image processing apparatus of FIG. 1 similarly to FIG. 4 and 5 have different integration ranges. That is, the integration range in FIG. 5 is the same size as the input image 201, and the integration range in FIG. 6 is a region 234 near the center smaller than the input image.
In FIG. 6, based on the path matching integration processing of FIG. 4, the range of the small area 234 near the center in the image 231 having the same size as the input image 201 is sequentially integrated to obtain the same distance path average integrated image 206b. It was created.
The same distance path average integral image 206b is path-matched integrated in a small area 234 near the center. Therefore, an unnatural band-like portion Z1 that stands out toward the end as seen in FIG. 5 is not seen. The extent to which the unnatural upper part of the band is seen depends on the size of the originally undesirable change in the X-direction changed partial differential image 204 and the Y-direction changed partial differential image 205 and the size of the range 234 for obtaining the integral value.

  In the above description, the X direction partial differential image 202 and the Y direction partial differential image 203 are taken as examples of differential images, but differential images created by other methods may be used. In addition to the Y direction, partial differential images in oblique directions may be created, and the partial differential images in these four directions may be used as differential images. In this case, distances are defined for vertical, horizontal, vertical, and diagonal connections. For example, vertical and horizontal distances are defined as 1, diagonal distances are defined as 2, and so on. You can measure the distance you go.

  At this time, when a plurality of shortest paths based on the definition of the discrete distance of the point for which the integration value is to be calculated from the initial point P0 appear, the average value of the integration values of the plurality of paths is calculated by integrating the point. It can be a value. For example, there are three diagonal routes to the upper right point, a diagonal route, a route from the top to the right point, and a route from the right to the upper point. The average value of the integral values can be used as the integral value of the point on the upper right. However, it is also possible to perform weighted averaging in which the path on the diagonally upper side is given a larger weight. In this specification, a weighted average is also included in the average.

  As described above, according to the first embodiment described above, the integral value of the initial point P0 of the integral image to be created is set to a predetermined value. Next, for a point adjacent to the initial point P0 (in the case of partial differentiation in two directions of the X direction and the Y direction, as shown in Expressions 6 to 9), the integral value of the initial point P0 and the modified differential image By performing line integration by a predetermined addition / subtraction operation with reference to the value at the corresponding position, an integrated value at a point adjacent to the initial point P0 is obtained.

  By sequentially performing a line integration by a predetermined addition / subtraction operation with respect to a point next to the point for which the integral value has already been obtained, by referring to the integral value of the point for which the integral value has already been obtained and the value of the corresponding position of the differential image, The integral value of a point next to the point whose integral value has already been obtained is obtained. At this time, if there are a plurality of paths having the same discrete distance between the point for which the integration value is to be obtained and the initial point, the average value of the values obtained by performing line integration for each of the plurality of paths is defined as the integration value of the point. The above processing is sequentially repeated up to a predetermined range, and a line-integrated value of all points in the predetermined range is obtained, thereby creating an average integrated image of the same distance path within the predetermined range.

  Accordingly, even when the integral value of the modified differential image obtained by modifying the differential image has a different value depending on the path, the integral value of the modified differential image can be uniquely determined. Therefore, even when the differential image is changed, an integral image can be created from the changed differential image that has been changed.

  Note that, as shown in Expression 5, the integral value F (x0, y0) of the initial point P0 (x0, y0) is set to the value I (x0, y0) of the input image 201. In the path matching integration, addition, subtraction, and averaging are performed starting from the initial point P0 (x0, y0). Therefore, if the value of the integral value F (x0, y0) of the initial point P0 (x0, y0) is increased by a, the integral value increases by a at all points. That is, an image to which the offset is added by a is obtained. Therefore, when the integral value of the initial point P0 (x0, y0) is set to 0, an obtained integrated image is obtained in which the offset differs by -I (x0, y0).

  Therefore, when the average value of the predetermined range of the integrated image is set to a predetermined value, the integration value of the initial point P0 (x0, y0) is not set to I (x0, y0), but any value including 0 or a random value is used. Even if it is set to the predetermined value, the integrated image in which the average value in the predetermined range as the final result is set to the predetermined value does not change.

FIG. 7 is a flowchart illustrating a method of changing the offset of the integral image according to the second embodiment, and FIG. 8 is a diagram illustrating an example of setting the average value calculation area in FIG.
In FIG. 8, in order to change the offset of the integral image, an average value calculation area 236 can be provided in the area 234 for calculating the integral value. The same distance path average integrated image 206c with the offset changed was created from the same distance path average integrated image 206b of FIG. The display level of the same distance route average integrated image 206c is displayed in accordance with the minimum value 3 to the maximum value 244 of the input image 201.

  The process S20 in FIG. 7 includes a process S21 and a process S23. The process S21 is a process for obtaining an integral value. In process S21, the value of the integral value F (x0, y0) at the initial point P0 at which the integration is started is set to 0 instead of I (x0, y0). The processing S21 includes steps S22 and S10. The process S23 is a process of changing the offset of the integral image. The process S23 executes an offset adjustment for setting the average value in the average value calculation area 236 to a predetermined value. The process S23 includes steps S24 to S27.

  In step S22, an initial value such as a map and an initial point 0 are set. At this time, the integral value F (x0, y0) at the initial point where the integration is started is set to zero. Others are the same as the processing of step S11 in FIG. Next, a process similar to the process of step S10 in FIG. 4 is performed.

  Next, in step S24, the average value of the input image 201 is calculated. At this time, an average value h0 in the average value calculation area 236 in FIG.

  Next, in step S25, the average value of the integral image is calculated. At this time, the average value h1 in the average value calculation area 236 is obtained for the integrated image obtained in step S21.

  Next, in step S26, an offset value is calculated. At this time, the difference h0−h1 between the average values h0 and h1 is set as the offset change amount.

  Next, in step S27, the offset of the integral image is changed. At this time, the offset change amount is added to the integral image obtained in step S21, and an offset-changed integral image is created.

  In step S21 of FIG. 7, the integral value F (x0, y0) of the initial point P0 at which the integration is started is set to 0. However, as shown in the first embodiment, the value I (x0, The integrated image obtained in step S27 does not change whether it is set to y0) or a random value. Therefore, in step S21, the integration value of the initial point P0 may be set to any value.

  In the above-described second embodiment, a method has been described in which the average value of the input image 201 and the average value of the path matching integral image 206 are matched in the average value calculation area 236. The target to be averaged is not limited to the input image 201, and may be an image obtained by processing the input image 201. Further, the target to be averaged may be a uniform image having a value such as 0 or 128.

  As described above, according to the above-described second embodiment, the offset adjustment for changing the average value in the average value calculation area 232 to the predetermined value is performed on the path matching integrated image.

  Here, when the average value of the integral image obtained in step S10 is matched with the average value of the input image 201, the path matching integral image 206 in which the average value in the average value calculation area 236 does not change from that of the input image 201 is obtained. Can be. When matching the average value of the integrated image obtained in step S10 with the average value of the processed image of the input image 201, the path matching integrated image 206 in which the average value in the averaged value calculation area 236 does not change with the processed image of the input image 201 Can be obtained. When the average value of the integrated image obtained in step S10 is adjusted to a uniform image, the path matching integrated image 206 in which the average value in the average value calculation area 236 becomes a uniform value can be obtained.

  In FIG. 5, when the entire image 231 having the same size as the input image 201 is sequentially integrated, as described above, there is a case where the obliquely unnaturally-looking band-like portion Z1 that becomes larger and becomes more prominent as the distance from the initial point P0 of the integration is increased. is there. On the other hand, as shown in FIG. 6, when only the small area 234 near the center is integrated, the obliquely unnatural band-like portion Z1 becomes inconspicuous. Therefore, since this processing needs to be limited to a small area, it is necessary to divide the image 231 into small blocks in order to obtain an integrated image of the entire image 231 having the same size as the input image 201. In the following third embodiment, the image 231 is divided into blocks, and path matching integration is performed for each block. An image obtained by performing path matching integration for each block is referred to as a block division integrated image.

FIG. 9 is a diagram illustrating a method for creating an integral image according to the third embodiment.
In FIG. 9, for example, the image 231 is divided into blocks of a small size such as 16 * 16. Then, the processing order 235 of the target block 240 is determined from each block, and the path matching integration is executed for each target block 240 according to the order 235. At this time, a block division integrated image 206d was obtained for the input image 201 of FIG. In the method of executing the path matching integration for each block of interest 240, the band Z1 in FIG. 5 can be made inconspicuous at the block end. The display level of the block division integrated image 206d is displayed in accordance with the minimum value 3 to the maximum value 244 of the input image 201, and this range is displayed.

FIG. 10 is a flowchart illustrating a method for creating an integral image according to the third embodiment.
The process S30 in FIG. 10 includes steps S31, S20, and S32. In step S31, the image 231 is divided into blocks. Then, the block of interest 240 is determined in order from each block. Next, in step S20, the processing 20 of FIG.

  When the process 20 of FIG. 7 is performed for each block, the average value calculation area 236 is set to a predetermined area in each block of interest 240. For example, assuming that the image size of the block of interest 240 is sx * sy and the block size of the average value calculation area 236 is hx * hy, sx = 16, sy = 16, hx = 8, hy = 8. The center and the center of the average value calculation area 236 can be matched.

  Next, in step S32, if the block of interest 240 is the last block of the image 231, the process ends. If the block of interest 240 is not the last block of the image 231, the process returns to step S31. When the process proceeds to the last block, a block division integrated image 206d having the same size as the input image 201 is obtained.

  Note that, in the above-described third embodiment, an example has been described in which sx = 16, sy = 16, hx = 8, and hy = 8, but other values may be set. For example, sx = 16, sy = 16, hx = 16, hy = 16, or sx = 32, sy = 32, hx = 16, hy = 16.

  Also, in the example of FIG. 9, the size of the image 231 is an integral multiple of the block size, and all the divided block sizes have the same size. If the size of the image 231 is not an integral multiple of the block size, a block having a smaller block size occurs at the right end or lower end. For this small block as well, the process S30 of FIG. 10 is executed similarly to the other blocks, and a block division integral image can be created.

  The target for which the average value is adjusted for each block is not limited to the input image, but may be an image obtained by processing the input image, or may be a uniform image having a value such as 0 or 128.

  As described above, according to the above-described third embodiment, the image 231 is divided into predetermined blocks, and a block division integrated image is created for all the divided blocks. Further, offset adjustment for changing the average value in the predetermined average value calculation area 236 to a predetermined value is performed for all the divided blocks.

  This makes it possible to obtain an integral image of the entire image 231 having the same size as the input image 201 without making the band Z1 in FIG.

  Since the average value calculation area 236 is set in a predetermined area in each target block 240, block distortion Z2 may occur at a connection portion between the blocks. In order to reduce the block distortion Z2, in the following fourth embodiment, a weighted average of the block division integral image is taken. The block-divided integral image obtained by taking the weighted average will be referred to as a block-divided average image.

FIG. 11 is a diagram showing an example of block division of a block division integral image according to the fourth embodiment.
In FIG. 11, a division method for creating four block division integrated images 250a to 250d having different division methods is determined. The block division integrated image 250a is an image obtained by the same division method as the image 231 in FIG. The block division integral image 250b is an image obtained by dividing the block division integral image 250a by a half block in the −X direction in FIG. The block division integral image 250c is an image obtained by dividing the block division integral image 250a by a half block in the −Y direction in FIG. The block division integral image 250d is an image obtained by dividing the integral image 250a by a half block in the −X direction and the −Y direction in FIG.

  Although the left and right ends of the block division integral image 250b have only half blocks, a block division integral image can be created from these half blocks. However, for simplicity, the value of the corresponding position of the block division integral image 250a can be substituted for only this half block.

  Although the upper and lower ends of the block division integral image 250c have only half blocks, a block division integral image can be created from this half block. However, for simplicity, the value of the corresponding position of the block division integral image 250a can be substituted for only this half block.

  Although the left and right ends and upper and lower ends of the block division integrated image 250d have only half blocks, a block division integral image can be created from these half blocks. Although the four corners of the block division integral image 250d have only 1/4 blocks, a block division integral image can be created from these quarter blocks. However, for simplicity, the values of the corresponding positions of the block division integrated image 250c can be substituted for only the left and right half blocks. Also, the values of the corresponding positions of the block division integrated image 250b can be substituted for only the upper and lower half blocks. In addition, for only the quarter blocks at the four corners, the average value of the value of the corresponding position of the block division integrated image 250b and the value of the corresponding position of the block division integration image 250c can be substituted.

  Focusing on a certain pixel 251, the pixel 251 belongs to each of the four blocks 252a to 252d of the block division integrated images 250a to 250d. The position of the pixel 251 of interest determines which of the block divided integral images 250a to 250d the four blocks 252a to 252d to which the pixel 251 belongs.

FIG. 12 is a diagram showing the positional relationship between the four blocks in FIG. 11 and the pixel of interest.
In FIG. 12, when blocks 252a to 252d are overlapped so as to include these four blocks 252a to 252d, block 253 is obtained. At this time, the weights of the blocks 252a to 252d are obtained based on the positional relationship between the target pixel 251 and the centers 254a to 254d of the blocks 252a to 252d, and the weighted average of the four blocks 252a to 252d is obtained.

  Here, the point of interest 251 is separated from the center 254a by a distance a in the X direction and a distance b in the Y direction. At this time, assuming that the block division integrated images of the blocks 252a to 252d are F0, F1, F2, and F3, the size of each of the blocks 252a to 252d in the X direction is sx, and the size in the Y direction is sy, the weighted average image G Can be given by the following Expressions 18 to 22.

(Equation 18)
G (x, y) = w0 * F0 (x, y) + w1 * F1 (x, y) + w2 * F2 (x, y)
+ W3 * F3 (x, y)

(Equation 19)
w0 = (sx / 2-a) * (sy / 2-b) / (sx * sy / 4)

(Equation 20)
w1 = a * (sy / 2-b) / (sx * sy / 4)

(Equation 21)
w2 = (sx / 2−a) * b / (sx * sy / 4)

(Equation 22)
w3 = a * b / (sx * sy / 4)

FIG. 13 is a flowchart illustrating a method for removing noise from a block-divided integral image according to the fourth embodiment.
In step S41 of FIG. 13, a division method for creating four block division integrated images 250a to 250d having different division methods by half blocks is determined.

  Next, in step S30a, the block division integral image 250a is created by executing the process S30 of FIG. 10 according to the method of division Ba for creating the block division integral image 250a.

  Further, in step S30b, the block division integrated image 250b is created by executing the process S30 of FIG. 10 according to the method of division Bb for creating the block division integral image 250b.

  Further, in step S30c, the block division integral image 250c is created by executing the processing S30 of FIG. 10 according to the method of division Bc for creating the block division integral image 250c.

  In addition, in step S30d, the block division integrated image 250d is created by executing the process S30 of FIG. 10 according to the method of division Bd for creating the block division integral image 250d.

  Next, in step S42, a weight is determined according to the distance between the center 254a to 254d of each of the corresponding blocks 252a to 252d of the four block division integrated images 250a to 250d to which the focused pixel 251 belongs and the focused pixel 251. A weighted average of the four block division integrated images 250a to 250d is performed. An image obtained by taking a weighted average of the four block division integral images 250a to 250d is referred to as a block average integral image.

  In the above-described fourth embodiment, weighted averaging is performed by creating four block division integrated images 250a to 250d having different division methods for each half block. Alternatively, weighted averaging can be performed by creating 16 block division integrated images having different division methods for each quarter block. As for the weighting method, in Expressions 19 to 22, the weight is determined according to the distance between the pixel 251 of interest and the centers 254a to 254d of the block division blocks 252a to 252d in the X and Y directions. did. In addition, the weight can be determined according to the Euclidean distance between the focused pixel 251 and the centers 254a to 254d of the block division blocks 252a to 252d.

  Further, in the above-described fourth embodiment, each block of one block division integrated image does not overlap. Here, overlapping block division is performed at the end of each block, an average is calculated at the overlapped portion, the difference from the average is determined as an error at the end of each block, and a portion other than the end of the block is determined from the error at the end of the block. Can be obtained by interpolation, and by subtracting this error, an integrated value obtained by correcting a portion other than the end of the block can be obtained.

  In this case, a discrete integrated image in which an integral value is obtained in a discrete block in which an overlapping portion of a block is skipped is created, and a plurality of discrete images are created. Interpolating the inner part of the block with the difference from the average as an error at the overlapping end portion is equivalent to performing a weighted average of the error according to the distance. Therefore, by this processing, a weighted average integrated image in which the method of weighted averaging is changed can be created.

  As described above, according to the above-described fourth embodiment, a plurality of block division integrated images having different division methods are created, and the center of each block of the plurality of block division integral images to which the target pixel belongs and the target pixel Weighted averaging according to the distance is performed to create a block average integrated image 206e.

  Thereby, the path matching integral image of the entire image 231 having the same size as the input image 201 can be created from the modified differential image while reducing the band-shaped portion Z1 in FIG. 5 and the block distortion Z2 in FIG.

FIG. 14 is a diagram showing an integral image according to the presence or absence of the processing of FIG. 13 with respect to the input image of FIG.
In FIG. 14, a block distortion Z2 occurs in the block division integrated image 206d obtained from the input image 201. In the block average integral image 206e created for the block division integral image 206d, the block distortion Z2 is reduced. The block distortion Z2 is also reduced in the block average integral image 206f created by changing parameters for the block average integral image 206e. Note that the display levels of the block division integral image 206d and the block average integral images 206e and 206f are displayed in accordance with the minimum value 3 to the maximum value 244 of the input image 201 in this range.

  The block average integral image 206e was created by setting the parameters in the above equations 3 and 4 to lv = 10 and k = 2. By setting lv to 10, small luminance fluctuations are suppressed. For example, clouds reflected on the upper side of the airplane are flattened. Within the range not exceeding, the improvement of the contrast almost twice as compared with the input image 201 could be achieved.

  The block average integral image 206f was created with lv = 0 and k = 3. Since lv is set to 0, even a portion having a small density change is not flattened, and since k is set to 3, local contrast is improved within a range not exceeding a display level. For example, a change in density of a cloud portion is also emphasized. Have been.

The change from the X-direction partial differential image 202 and the Y-direction partial differential image 203 to the X-direction modified partial differential image 204 and the Y-direction modified partial differential image 205 can be performed by various methods other than the methods using the mathematical formulas 3 and 4. There is a way. For example, the change from the X direction partial differential image 202 and the Y direction partial differential image 203 to the X direction changed partial differential image 204 and the Y direction changed partial differential image 205 may be given by the following Expressions 23 to 25.
(Equation 23)
D (x, y) = sqrt (Dx (x, y) * Dx (x, y) + Dy (x, y)
* Dy (x, y))

  Note that sqrt () is an operation that takes a square root.

(Equation 24)
Ex (x, y) = k * (Dx (x, y) -lv) + lv (when D (x, y) ≧ lv and Dx (x, y) ≧ 0)
= Dx (x, y) * D (x, y) / lv (when D (x, y) <lv)
= K * (Dx (x, y) + lv) -lv (when D (x, y) ≧ lv and Dx (x, y) <0)

(Equation 25)
Ey (x, y) = k * (Dy (x, y) -lv) + lv (when D (x, y) ≧ lv and Dy (x, y) ≧ 0)
= Dy (x, y) * D (x, y) / lv (when D (x, y) <lv)
= K * (Dy (x, y) + lv) -lv (when D (x, y) ≧ lv and Dy (x, y) <0)

  When the process of changing the differential image by Expressions 23 to 25 is performed, the change can be performed by an isotropic conversion formula regardless of the direction of the edge of the image. In this changing process, various effects can be obtained by changing the parameters.

  For example, if lv = 0, the pixel value of the differential image is changed to k times. This change means that the local shading change is multiplied by k times. For this reason, when k is larger than 1, an integrated image in which the local contrast of the input image is improved can be obtained.

  If lv is set to an appropriate value obtained by evaluating the level of the noise level and k is set to 1, the absolute value of the value lower than the value lv obtained by evaluating the level of the noise level is reduced to a value larger than lv. Edge changes can be preserved. Therefore, by this change, it is possible to perform noise removal while preserving the edges of the differential image and flattening while suppressing minute changes, and it is possible to obtain an edge-preserving smoothed image with noise removal and flattening. If lv is set to a predetermined value and k is set to a value larger than 1, the absolute value of the value lower than lv can be reduced and the value higher than lv can be changed to k times.

  If lv is set to a value obtained by evaluating the degree of the noise level, an integrated image in which noise removal and local contrast improvement are performed simultaneously can be obtained. This change is a process including a process of reducing a value whose absolute value is lower than lv. This change includes a process of multiplying the value of the differential image by a constant, since the value having an absolute value higher than lv is multiplied by a constant.

  In addition to the method using the above formulas 23 to 25, for example, a case where D (x, y) exceeds lv2 is provided in the formulas 23 to 25, and when D (x, y) exceeds lv2, Is set, for example, when Dx (x, y)> lv2, it can be changed as k2 * (Dx (x, y) -lv2) + lv2, and the other can be similarly changed.

  At this time, if lv is set to a small value obtained by evaluating the level of the noise level, k is set to a value larger than 1, and k2 is set to 1, the block average integrated image has a noise reduction in which a change in a small change in the noise level is reduced. Thus, a local contrast improvement is achieved in which the change in the portion where the change is smaller than lv2 is increased by a change larger than the noise level, and the portion where the change is larger than lv2 has a local contrast faithful to the change. be able to. If lv2 is set to a value smaller than 1, local contrast in a portion where the change is larger than lv2 can be suppressed.

  There are many other ways to change the differential image according to the purpose. For example, when D (x, y) is smaller than lv, noise removal by simply performing a process of setting the value to 0 can be performed. In addition, Dx (x, y) * D (x, y) in Expression 24 is converted to sqrt (| Dx (x, y) |) * sgn (Dx (x, y)), and Dy (x, y) in Expression 25 * D (x, y) can be changed to sqrt (｛Dy (x, y) |) * sgn (Dy (x, y)). Note that sgn () is a function that returns 1 when the value is positive, −1 when the value is negative, and 0 when the value is 0. In this case, a change with a low absolute value can be emphasized using a square root function.

  In addition, there are various ways of changing the differential image, which can be changed using an arbitrary function. Equations 26 and 27 show the case of averaging in a 3 * 3 area around the point of interest.

(Equation 26)
Ex (x, y) = (Dx (x-1, y-1) + Dx (x, y-1) + Dx (x + 1, y-1) + Dx (x-1, y) + Dx (x, y) + Dx ( x + 1, y) + Dx (x-1, y + 1) + Dx (x, y + 1) + Dx (x + 1, y + 1)) / 9

(Equation 27)
Ey (x, y) = (Dy (x-1, y-1) + Dy (x, y-1) + Dy (x + 1, y-1) + Dy (x-1, y) + Dy (x, y) + Dy ( x + 1, y) + Dy (x-1, y + 1) + Dy (x, y + 1) + Dy (x + 1, y + 1)) / 9

  An integrated image obtained by integrating the differential images subjected to the averaging process is an image very similar to the averaged image. Equations 28 and 29 show the case where the image is sharpened.

(Equation 28)
Ex (x, y) = Ex (x, y)-(Dx (x-1, y-1) + Dx (x, y-1) + Dx (x + 1, y-1) + Dx (x-1, y) + Dx (X, y) + Dx (x + 1, y) + Dx (x-1, y + 1) + Dx (x, y + 1) + Dx (x + 1, y + 1)) / 9

(Equation 29)
Ey (x, y) = Ey (x, y)-(Dy (x-1, y-1) + Dy (x, y-1) + Dy (x + 1, y-1) + Dy (x-1, y) + Dy (X, y) + Dy (x + 1, y) + Dy (x-1, y + 1) + Dy (x, y + 1) + Dy (x + 1, y + 1)) / 9

  An integrated image obtained by integrating the above-described differential image subjected to the sharpening process is an image very similar to an image obtained by performing image sharpening.

  In addition to the above, for example, the processing of Non-Patent Document 3 can be performed on a differential image instead of an input image. At that time, in the differential image, the absolute value (square root of the sum of squares) of the partial differential image in the X direction and the partial differential image in the Y direction is calculated, and the minimum change direction in eight directions is found for the absolute value image. The directional partial differential image can be processed by the X-directional partial differential image, and the Y-directional partial differential image can be processed by the Y-directional partial differential image. Furthermore, referring to the absolute value image, it is possible to perform adaptive processing in which the way of changing the value of the differential image is changed according to the processing result of evaluating whether the point of interest is an edge portion or a flat portion.

  Edge-preserving processing including the processing of Non-Patent Document 3 can be performed. In the edge-preserving processing, an adaptive processing that changes the value of the differential image can be performed according to the processing result of evaluating whether the point of interest of the differential image is an edge portion or a flat portion. This adaptive processing is effective for filter processing for preserving an edge portion and smoothing a flat portion.

  In the above description, the input image 201 is an image having a black and white gray scale. However, in the case of a color image, the process of changing the differential image can be performed in the same manner. For example, the RGB colors are sequentially represented by a color number c, the input image 201 is represented by I [c], and Ex [c], Ey [c], Dx [c], and Dy [c] are each described above. The expression is obtained by adding [c] to each image, and can be independently calculated. However, since Expression 23 is the respective addition, it can be given by Expression 30 below in which the addition of the color numbers 0 to 2 is performed.

(Equation 30)
D (x, y) = sqrt (Dx [0] (x, y) * Dx [0] (x, y) + Dy [0] (x, y) * Dy [0] (x, y) + Dx [1 ] (X, y) * Dx [1] (x, y) + Dy [1] (x, y) * Dy [1] (x, y) + Dx [2] (x, y) * Dx [2] ( (x, y) + Dy [2] (x, y) * Dy [2] (x, y))

  In the above description, the case where the colors are three colors of RGB is taken as an example. However, in a multispectral or hyperspectral image, there are more colors than three colors. In this case, processing can be performed in the same manner as in the case of three colors.

  In the above description, the case where the image is a two-dimensional image is taken as an example, but a three-dimensional image may be obtained by MRI, X-ray CT (Computed Tomography), or the like. When a three-dimensional image is represented by I (x, y, z), there are three differentiating directions of X, Y, and Z, and Dz (x, y, z) = I (x, y, z + 1) -I ( x, y, z). At this time, similarly to the two-dimensional image, the path matching integral value can be an average value of the line integral values of the path that is the shortest path.

  For example, when moving from the point (x, y, z) to the point (x, y + 1, z + 1), the movement is a two-dimensional movement in the yz plane, and is an average of two paths. However, moving from the point (x, y, z) to the point (x + 1, y + 1, z + 1) is a three-dimensional movement in the xyz space. Therefore, the line integral value of three points (x, y + 1, z + 1), (x + 1, y, z + 1) and (x + 1, y + 1, z) is obtained, and from these three points, a point of (x + 1, y + 1, z + 1) is obtained. Line integrals can be obtained for each of the three routes to which the vehicle travels, and the average can be used as the integral. In the block division, a two-dimensional image is divided into planar blocks, but a three-dimensional image can be divided into cubic blocks.

  In the two-dimensional case, four block division integral images having different divisions are generated for each half block. In the three-dimensional case, eight block division integral images having eight different divisions are generated for each cube half block. In the two-dimensional case, the block average integral image is obtained based on Expressions 18 to 22, but in the three-dimensional case, the block average integral image is created based on a symmetric expression obtained by adding the z direction to Expressions 18 to 22. The fourth and subsequent dimensions can be similarly created.

  Further, in the above description, an example in which the input image 201 is prepared as the image input is shown, but another input image may be prepared in addition to the input image 201. Here, a differential image of each of these input images is created, and an image obtained by integrating the modified differential image obtained by adding both is equal to an image obtained by adding two input images.

  At this time, another input image is cut out along the contour of the subject and is smaller than the input image 201. In this case, in a portion where the differential image of another input image is superimposed on the differential image of the input image 201, the value of the overlapping portion of the differential image of the input image 201 may be set to 0, and the differential image of another input image may be added. it can. By integrating this modified differential image, a vivid composite image is obtained in which the luminance gradient of the original input image 201 is ignored at the overlapping portion.

  Further, the value of the differential image of the input image 201 of the overlapping portion is added to the closest contour portion, the overlapping portion is processed to be 0, and a differential image of another input image is added. A block average integral image with another input image added can be created without impairing the overall luminance of the original input image 201.

FIG. 15 is a flowchart illustrating the level adjustment processing of the integral image according to the fifth embodiment.
In step S50 of FIG. 15, the image processing apparatus 111 creates the path matching integral image 206, and then adjusts the level of the path matching integral image 206 based on the input image 201, so that the level adjusted path matching integral image 271 is created. In the level adjustment, for example, the pixel value of the path matching integral image 206 exceeding the maximum value of the input image 201 is set to the maximum value of the input image 201, and the pixel value of the path matching integral image 206 smaller than the minimum value of the input image 201 is set. Is set to the minimum value of the input image 201. In addition, when the image level is expressed in a predetermined value range such as 0 to 255, the pixel value of the path matching integral image 206 is set to 0 (minimum predetermined value) when the pixel value is smaller than 0 (minimum predetermined value). When it is larger than 255 (the maximum predetermined value), it can be set to 255 (the maximum predetermined value).

  Accordingly, even when the maximum value or the minimum value of the luminance of the path matching integrated image 206 is different from the maximum value or the minimum value of the input image 201, the path matching integrated image 206 can be displayed at an appropriate level. For this reason, even when the display level of the path matching integral image 206 is different from the display level of the input image 201, the path matching integral image 206 is displayed in this range based on the maximum value and the minimum value of the path matching integral image 206. In addition, the path matching integral image 206 can be displayed in this range based on the maximum value and the minimum value of the input image 201, and unnatural display of the path matching integral image 206 can be prevented.

FIG. 16 is a diagram showing an integrated image according to the presence or absence of the processing of FIG.
In FIG. 16, the block average integrated image 206 e displays this range by adjusting the display level from the minimum value 3 to the maximum value 244 of the input image 201. On the other hand, the block average integrated image 206e2 displays this range by adjusting the display level from the minimum value -128 to the maximum value 356. Although the block average integral images 206e and 206e2 are created with the same parameters, the appearance such as contrast differs depending on the display level.

  As another method for displaying the path matching integral image 206 in FIG. 2 at an appropriate level, information on the default display level of the path matching integral image 206 is generated, and the default display level is set in the image display processing. And displaying the path matching integral image 206 by referring to the above, and changing the default display level by a user who has viewed the displayed image, adjusting the display level according to the change, and displaying the path matching integral image 206. Can be.

FIG. 17 is a flowchart illustrating the display processing of the integral image according to the sixth embodiment.
In step S60 in FIG. 17, a default display level of the path matching integral image 206 is determined based on the input image 201. At this time, a default value of the display level is calculated or set, and an attribute-added integral image 282 added as attribute data to the path matching integral image 206 is created and stored. For example, when the image processing device 121 in FIG. 1 performs this process, the attribute-added integrated image 282 is stored in the storage device 124. ,

  The default value of the display level has an upper limit value indicating a value above the display level and a lower limit value indicating a value below the display level. For example, the upper limit can be set to the maximum value of the input image 201, and the lower limit can be set to the minimum value of the input image 201. Alternatively, the upper limit may be set to 255 and the lower limit to 0.

  Next, in step S61, image display processing of the attributed integrated image 282 is executed. At this time, the attribute-added integral image 282 stored in step S60 is read and displayed on the display device. For example, when the image processing device 121 performs this process, the integrated image 282 with attributes is read from the storage device 124 and displayed on the display device 122.

  The attribute-added integral image 282 gives default upper and lower display levels as attribute data together with the path matching integral image 206. The image processing device 121 displays the path matching integral image 206 in the range between the upper limit and the lower limit.

FIG. 18 is a diagram showing an integral image according to the presence or absence of the processing of FIG.
In FIG. 18, when the default display level is changed, for example, an upper limit value or a lower limit value of the display level can be designated from the input device 123 in FIG. For example, the block average integral image 206e whose display level is set from the minimum value 3 to the maximum value 244 of the input image 201 can be displayed at the default display level. Further, a block average integrated image 206e3 in which the display level is set from the minimum value 100 to the maximum value 244 of the input image 201 can be displayed.

FIG. 19 is a block diagram illustrating an example of a configuration and a processed image of the image processing apparatus according to the seventh embodiment.
In FIG. 19, the image processing apparatus includes a differential processing unit 311, a differential image change processing unit 312, and a path matching integration processing unit 313.

  The differential processing unit 311 creates a differential image obtained by differentiating the input image 301. When the input image 301 is a two-dimensional image, the differentiation processing unit 311 performs differentiation in the one-dimensional direction in each of the X direction and the Y direction, thereby creating an X-direction partial differential image 302 and a Y-direction partial differential image 303. . The input image 301 is not limited to a photographed image photographed by the photographing device 100 of FIG. 1, but may be a computer graphic image created by an image processing device, or may be a copy image or a scanner image.

  As the input image 301, a character image in which a gradual shading occurs in the background is taken as an example. The gradual shading of the background of the input image 301 may be caused by the degree of light when the imaging device 100 is a camera. The image size of the input image 301 is 256 * 256, and when the pixel (x, y) is the background portion, the pixel value is 128 + ((128-x) + (128-y)) * 0.5, and the pixel ( When (x, y) is a character portion, the pixel value is set to a value 50 lower than that of the background portion. At this time, the pixel values of the X-direction partial differential image 302 and the Y-direction partial differential image 303 are 0.5 in the background portion, and the pixel value of the edge of the character portion has a square root of the sum of squares of both values of 49. To 51, corresponding to the angle of the character, and the pixel value inside the character portion becomes 0.

  The differential image change processing unit 312 creates an X direction changed partial differential image 304 and a Y direction changed partial differential image 305 obtained by changing the X direction partial differential image 302 and the Y direction partial differential image 303. At this time, the differential image changing process 312 sets the value to 0 if the absolute value of the pixel value of the X-direction partial differential image 302 is equal to or smaller than the threshold lv3, and sets the absolute value of the pixel value of the Y-direction partial differential image 203 to the threshold. If it is less than lv3, the value can be set to 0. At this time, the threshold value lv3 is set to a value that makes the pixel value due to the gentle inclination of the background portion zero.

  In the example of FIG. 19, the threshold value lv3 is set to 1 because the input image 301 is created by changing the luminance of the background portion by 0.5 in both the X and Y directions per pixel in the background portion. At this time, in the X-direction changed partial differential image 304 and the Y-direction changed partial differential image 305, the background portion becomes 0, the edge of the character portion hardly changes, and the inside of the character portion remains 0.

  The path matching integration processing unit 313 creates a path matching integration image 306 by performing path matching integration of the X direction changed partial differential image 304 and the Y direction changed partial differential image 305. When setting the display level of the path matching integrated image 306, the path matching integration processing unit 313 can refer to the uniform image 307.

  Here, the path matching integration processing unit 313 creates a block average integration image as the path matching integration image 306 by the processing in FIG. In this case, in each block of the block average integral image, a block division integral image is created by the processing of FIG. The average value of the block division integral image of each block is adjusted to the average value of the uniform image 307 (that is, a predetermined uniform value).

  In the example of FIG. 19, the image size of each block of the block division integral image is set to half the image size of the input image 301. That is, since the image size of the input image 301 is 256 * 256, the block size of the block division integrated image is 128 * 128. The value of the uniform image 307 was set to an intermediate value of 128.

  Here, the X-direction partial differential image 302 and the Y-direction partial differential image are obtained by differentiating the input image 301 to reduce the value of the background portion where the gradation of the input image 301 changes gradually while preserving the edges of the characters. 303 can be created. At this time, the value of the background portion of the X-direction partial differential image 302 and the Y-direction partial differential image 303 can be larger than the value of the character edge portion of the X-direction partial differential image 302 and the Y-direction partial differential image 303.

  Further, by performing a process of changing the small value of the background portion of the X direction partial differential image 302 and the Y direction partial differential image 303 to 0, the gradation of the input image 301 changes gently while preserving the edge of the character. An X-direction changed partial differential image 304 and a Y-direction changed partial differential image 305 with the value of the background portion set to 0 can be created.

  Here, by setting the value of the background portion to 0, the value of the background portion can be maintained at 0 even when the value of the background portion is integrated. Therefore, by integrating the X-direction changed partial differential image 304 and the Y-direction changed partial differential image 305, it is possible to remove the gradual shading of the background portion while preserving the edges of the characters of the input image 301. The path matching integral image 306 in which the density of the portion is uniform can be created.

  Further, in the input image 301, the characters are bright in a bright portion of the background, and the characters are dark in a dark portion of the background. For example, in the input image 301, the letter A is brighter than the letter F. Here, by adjusting the average value of the block division integrated image of each block to the average value of the uniform image 307, the shading between characters can be made uniform. For example, in the path matching integral image 306, the brightness of the character A and the brightness of the character F are substantially equal.

FIG. 20 is a flowchart showing the binarization processing of the path matching integral image 306 of FIG.
In step S70 in FIG. 20, the image processing apparatus 111 creates a binarized image 322 by binarizing the path matching integral image 306 with respect to the path matching integral image 306 created by the processing in FIG. I do. The binarization process is a process of setting the pixel value to 0 when the pixel value of the image is equal to or smaller than the threshold, and setting the pixel value to 1 when the pixel value is higher than the threshold. The binarized image is not limited to the case where the pixel value takes a binary value of 0 and 1, and the pixel value may be multiplied by 255 to take the binary value of 0 and 255. Here, it is assumed that the value of the binarized image takes two values of 0 and 255.

FIG. 21 is a diagram showing a binarized image of the input image 301 and the path matching integral image 306 of FIG.
In FIG. 21, a binarized image 308 is created by performing a binarization process on an input image 301 in which the density of the background is gently inclined. At this time, the binarization threshold was set to 103. In the binarized image 308, in a portion with a bright background, the character and the background become white, and the character and the background cannot be distinguished. In a portion with a dark background, the characters and the background are black, and the characters and the background cannot be distinguished.

  On the other hand, the path matching integral image 306 is created from the input image 301 by the processing of FIG. Then, when a binarized image 309 obtained by binarizing the path matching integral image 306 is created, it is possible to make only the background white and only the characters black. At this time, when the binarization threshold was set to 98 or more and 135 or less, a binary image 309 in which the background and the character were separated was obtained. Here, the input image 301 is created so as to be in the range of a luminance level from 0 to 255, and the path matching integral image 306 is adjusted to have a maximum value of 255 and a minimum value of 0.

  As described above, according to the above-described seventh embodiment, even when the threshold value of the binarization that can separate the background and the character cannot be set because the gradient of the background of the input image 301 has a gentle slope. In addition, a path matching integral image 306 can be created in which the edges of the characters of the input image 301 are preserved and the gradual gradient of the density of the background is removed, and a threshold for binarization that can separate the characters from the background is set. can do.

FIG. 22 is a diagram illustrating an example of another processed image of the image processing apparatus in FIG.
In FIG. 22, it is assumed that the input image 301c has been input to the image processing apparatus in FIG. The background of the input image 301c is given a density of 0.1 per pixel from the right to the left, with the center at 128. In addition, a uniform square having a density of 128 was created at the center of the input image 301c. In the case of this example, the differential image changing process 312 executes a changing process using Expressions 3 and 4, and sets lv to 1. The input image 301c displays image values from 115 to 142 at the minimum luminance to the maximum luminance. The square placed at the center of the input image 301c has a uniform density, but has a gradation in the background. For this reason, even if the density of the inside of the square is originally uniform, the illusion of a human being appears as if a gradation is applied to the inside of the square.

  Here, in the block average integral image 306b in which the average value of the block division integral image of each block created from the input image 301c is adjusted to the average value of the uniform image 307, a gradation appears inside a square located at the center. The background also shows a slight change in shading. For example, the background becomes dark around a bright portion inside the rectangle, and the background becomes bright around a dark portion inside the rectangle.

  In this block average integral image 306b, for example, by setting the value of the background other than the central square to a constant value of 128, the block average integral image 306c can be obtained. In the block average integral image 306c, a change in shading of the background can be removed while preserving the gradation of the square in the center, and an image reproducing the illusion of a human can be obtained.

FIG. 23 is a flowchart illustrating a method for generating a modified differential image of the image processing device according to the eighth embodiment.
In FIG. 23, this image processing apparatus includes differential image change processing units 400a and 400b. The differential image change processing unit 400b includes a convolutional neural network 401. The convolutional neural network 401 includes a convolution layer 402 and a neural net layer 403. The differential image change processing unit 400a includes a convolution layer 402.

  The convolution layer 402 performs convolution of the input image. The input image can be an edge image created by subtracting the image from a locally averaged image. The convolution layer 402 performs convolution of a small area such as 3 * 3 or 5 * 5 on the entire image, and stacks the images to extract features of the images.

  The neural network layer 403 performs calculations for recognizing the features captured by the convolution layer 402. The neural network layer 403 can improve the recognition rate by deep learning in which deep calculations are performed over many layers. The neural network layer 403 outputs a recognition result 404.

  Further, by performing the recognition object extraction process 421 on the output of the neural network layer 403, the recognition object is displayed by surrounding it with a square, the outline of the recognition object is displayed, or only the outline of the recognition object is displayed. Instead, the interior can be painted black.

The network structure of the convolutional neural network 401 and the method of deep learning are described in, for example, the following Reference 4. As described in Reference 4, the method of learning the convolution layer 402 and the neural network layer 403 includes a method of performing pre-learning and a method of performing deep learning.
Reference 4: Takayuki Okaya, "Deep Learning", Kodansha

  When the differential image change processing units 400a and 400b are used instead of the differential image change processing unit 212 in FIG. 2, a differential image is used as an input to the convolutional neural network 401. Since there are two differential images, the X-direction partial differential image 202 and the Y-direction partial differential image 203, the number of input data is twice as large as usual and the network capacity is twice as large as usual. However, the differential image has reproducible information except for the offset of the original image, and the original image can be reproduced by giving some information regarding the offset or limiting the image to an image that can be calibrated internally.

  When the convolution layer 402 performs the convolution of the input image, the differential image change processing unit 400a performs the simplification processing 411 on the data appearing in the convolution layer 402. In the simplification processing 411, sparse processing is performed to limit the data appearing in the convolution layer 402 to a convolution having a maximum value for each pixel.

  Next, the differential image change processing unit 400a performs the image reconstruction processing 412 on the sparse data. In the image reconstruction processing 412, learning is performed on sparse data so that the output image is as close as possible to the input image. By obtaining a reconstructed image in which the features of the image appear by performing the sparse processing, an output image in which the features of the input image appear well can be created, and can be used for noise removal and creation of a deformed image.

  The differential image change processing unit 400a sets the images obtained by performing such image reconstruction processing 412 as an X-direction changed partial differential image 204a and a Y-direction changed partial differential image 205a. At this time, as the X-direction changed partial differential image 204a and the Y-direction changed partial differential image 205a, an output image with less noise expressing the characteristics of the input image is obtained. Can be obtained.

  On the other hand, the differential image change processing unit 400b learns the extraction processing 421 of the recognized object recognized via the neural network layer 403. For learning, a deep learning method using an input image for learning, a recognition flag of an object, and data indicating an extraction area is widely known.

  The differential image change processing unit 400b performs the simplification processing 422 when the recognition object extraction processing 421 is performed on the output of the neural network layer 403. In the simplification processing 422, based on the information of the object extracted in the recognition object extraction processing 421, the convolutional neural network 401 is traced backward, and the selection and the sparse processing of the part related to the object extracted in the convolution layer 402 are performed. I do.

  Next, the differential image change processing unit 400b performs the image reconstruction processing 423 on the selected sparse data. In the image reconstruction processing 423, learning is performed on the selected sparse data so that the output image is as close as possible to the input image.

  The differential image change processing unit 400b sets the images obtained by performing the image reconstruction processing 423 as an X-direction changed partial differential image 204b and a Y-direction changed partial differential image 205b. At this time, since an output image with less noise that well expresses the features of the object extracted from the input image can be obtained as the X-direction changed partial differential image 204b and the Y-direction changed partial differential image 205b, noise is included as an integral image. It is possible to obtain an output image in which the form of the object extracted from the input image is reduced and the shape of the object appears well.

FIG. 24 is a block diagram illustrating a configuration of the path matching integration processing unit of the image processing device according to the ninth embodiment.
24, the image processing apparatus includes a neural network layer 431 and a parameter update amount calculation unit 432. The neural network layer 431 creates a path matching integral image 433 based on the input image 201, the X-direction changed partial differential image 204, and the Y-direction changed partial differential image 205. The parameter update amount calculation unit 432 calculates the parameter update amount of the neural network layer 431 based on the path matching integral images 206 and 433, and updates the parameters of the neural network layer 431.

  At this time, the parameter update amount calculation unit 432 can use, for example, a block average integral image as the path matching integral image 206 as a reference data image for learning. Then, the parameter update amount calculation unit 432 can make the neural network layer 431 learn so that the path matching integral image 433 output from the neural network layer 431 approaches the reference data image as much as possible. In this learning, for the difference image between the path matching integrated images 206 and 433, the sum of squares of each pixel is used as an evaluation value, and the update amount of a parameter for reducing the evaluation value is obtained. The calculation for obtaining the parameter update amount can be performed by deep learning such as a back propagation method for reducing the evaluation value.

  By learning the neural network layer 431 in this way, the neural network layer 431 can have a function of creating an image substantially equivalent to the path matching integral image 206. The neural network layer 431 trained in this way has some personality depending on the reference data image for learning and the net structure, but is similar to the path matching integral that created the path matching integral image 206 used for learning. Machine learning can have the function of making calculations. That is, the neural network layer 431 trained in this way can execute path matching integration or make a slight change by performing path matching integration, and is used as the path matching integration processing unit 213 in FIG. be able to. The neural network layer 431 trained in this way can function as a unit that performs calculation including path matching integration by itself.

  In addition, the neural network layer 431 can create a path matching integral image based on the image obtained by adding noise to the input image 201, the X-direction modified partial differential image 204, and the Y-directional modified partial differential image 205. . Then, the parameter update amount calculation unit 432 uses the path matching integral image 206 as a reference data image for learning, and causes the neural network layer 431 to learn so that the output image of the neural network layer 431 approaches the reference data image as much as possible. it can. At this time, the neural network layer 431 can perform a process in which a noise removing function is added to the integration process when the road matching integral image 206 used for learning is created. The neural network layer 431 thus learned can perform path matching integration when the path matching integration image 206 is created, and can be used as the path matching integration processing unit 213 in FIG.

  In the above-described embodiment, a differential image obtained by differentiating an input image can be created, a modified differential image obtained by changing the differential image can be created, and a path integral image can be created by integrating the changed differential image.

  However, an image obtained by multiplying the differential image by k (k is a real number greater than 1) has the same line integral value regardless of the path. At this time, a simple process for obtaining the same mathematically equivalent result without performing a line matching integration in which a line integral value is obtained for each path of an image obtained by multiplying the differential image by k and an average value is used as an integral value is obtained. Exists. That is, if the differential image is limited to an image multiplied by k, the path matching integral image is equal to the image obtained by simply multiplying the original image by k except for the offset.

  A simple process that obtains the same result mathematically equivalent to path matching integration in which a line integral value is calculated for each path of an image obtained by multiplying the differential image by k and the average value is an integrated value will be described.

FIG. 25 is a flowchart illustrating a block image creation method of the image processing apparatus according to the tenth embodiment.
25 includes steps S81, S82, S83, and S88. In step S81, the image 231 is divided into blocks. Then, the block of interest 240 is determined in order from each block.

  Next, in step S82, the image of the block of interest 240 is multiplied by a constant (for example, k times) with the input image 201 as an input.

  Next, in step S84, the average value of the input image 201 is calculated. At this time, the average value in the average value calculation area 236 in FIG.

  Next, in step S85, an average value of the constant-multiplied image is calculated. At this time, an average value in the average value calculation area 236 is obtained for the constant-multiplied image obtained in step S82.

  Next, in step S86, an offset value is calculated. At this time, the difference between the average value obtained in step S84 and the average value obtained in step S85 is set as the offset change amount.

  Next, in step S87, the offset of the constant multiple image is changed. At this time, the offset change amount is added to the constant-multiplied image obtained in step S22 to create an offset-changed constant-multiplied image.

  Next, in step S88, if the block of interest 240 is the last block of the image 231, the process ends. If the block of interest 240 is not the last block of the image 231, the process returns to step S81. When the process proceeds to the last block, an image 206j of the same size as the input image 201 is obtained.

  By the process of creating the block-divided constant-multiplied image 206j, it is possible to obtain a result mathematically equivalent to the process of creating the block average integral image 206j from the k-times differential image while reducing the amount of calculation.

  When the input image 201 is used as a reference image for matching the average value of the blocks, the block average constant multiple image 206j can be displayed at the same display level as when the input image 201 is displayed. At this time, since the image of the block average constant multiplied image 206j is multiplied by k times in the block, the contrast can be locally made k times larger than that of the input image 201.

  For example, in Formulas 23 to 25, after performing a process of increasing the differential image by k times with lv = 0, and integrating for each block, a block-divided integrated image in which the local density change is increased by a factor of k is obtained. The process of creating a block-divided integral image involves differentiation of the input image and integration of each block. In the process of creating a block-divided constant-multiplied image, the differentiation of the input image and integration of each block are performed by a constant for each block. By substituting the doubling process, it is possible to obtain a result mathematically equivalent to the process of creating the block division integral image.

  This is because an image obtained by multiplying the differential image by k and performing path matching integration is equal to an image obtained by multiplying the input image before differentiation by k, except for the offset. In the block division integral image, offset adjustment is performed to match the average value in the block to the input image 201, the uniform image 307, and the like. Similarly, the image in the block is multiplied by k, and the average value in the block is adjusted to the input image 201, the uniform image 307, and the like, so that mathematically equivalent processing including the offset is performed. Can be realized.

  However, if the average value is adjusted to the input image 201 by multiplying the image by k in the block, the local contrast in the block is increased by k times, but block distortion occurs due to discontinuity in density between the blocks. At this time, according to an eleventh embodiment of FIG. 26 described later, a plurality of block division constant multiplied images can be created to eliminate discontinuity in density between blocks. At that time, since the gradation correction has been performed with a slight gentle inclination, the k-times local contrast slightly changes to become almost k times the local contrast.

  In addition, since the brightness exceeding the upper and lower limits of the display level of the image is set to the brightness of the upper and lower limits, if the density exceeding the upper and lower limits is also desired to be expressed, the display level of the image may be changed or the input image 201 may be displayed. This can be dealt with by processing the image in which the minimum value or the maximum value of the average luminance in the block has been changed as a reference image for matching the average value.

FIG. 26 is a flowchart illustrating a block distortion removal method for a block image according to the eleventh embodiment.
In step S41 of FIG. 26, a division method for creating four block division constant multiplied images having different division methods for each half block is determined.

  Next, in step S80a, the first divisional multiplied image is created by executing the process S80 of FIG. 25 in accordance with the first division Ba for creating a multiplied block division constant image.

  Further, in step S80b, a second block division constant image is created by executing the processing S80 of FIG. 25 in accordance with the second division Bb for creating a block division constant image.

  Further, in step S80c, a third divided-multiple image is created by executing the process S80 in FIG. 25 in accordance with the third division Bc for producing a multiplied block-divided image.

  In addition, in step S80d, the process S80 of FIG. 25 is executed in accordance with the fourth division Bb for producing a multiplied block division constant image, thereby generating a fourth multiplication block division constant image.

  Next, in step S42, the weight is determined according to the distance between the target pixel and the center of each of the corresponding blocks of the four block division constant multiplied images to which the pixel of interest belongs, and the weighted average of the four block division constant multiplied images is determined. Is performed to create a block average constant-multiplied image 206k.

  In the above processing, the example in which the input image 201 is used as the reference image for matching the average value of the blocks has been described. However, the reference image for matching the average value may be the uniform image 307, or the input image 201 is processed. It may be an image.

  Also, a method has been described in which four block division constant multiplied images having different division methods are created in order to take a weighted average. However, more block division constant multiplied images may be created, or the above-described method may be used. Other weighted averaging methods may be used.

  As described above, according to the above-described eleventh embodiment, the image is divided into blocks, the image in each block is multiplied by a constant, and the average value in the average value calculation area of each block is set to a desired value. Average value matching processing to obtain an image multiplied by a block division constant.

  Next, a plurality of block division constant multiplied images having different methods of division are created, and a weighted average according to the distance between the center of each block of the plurality of block division constant multiplied images to which the pixel of interest belongs and the pixel of interest is calculated. Create an average constant multiple image.

  This makes it possible to increase the local contrast in a block by k times while reducing the calculation amount, and to eliminate block distortion due to discontinuity in density between blocks. For example, even in an image having a large amount of information, such as a four-dimensional CT image in which time information is added to a three-dimensional CT image, local contrast can be emphasized while reducing the time required for image processing. Ability can be improved.

  In the process of creating a block average constant-multiplied image, a process of multiplying the image in each block by a constant is performed, but offset adjustment is performed in each block. Therefore, if the processing for multiplying the image in each block by a constant is performed, an image with a block average constant multiplied can be created except for the offset of the image. In the process of adjusting the offset by multiplying the image in the block by a constant, there are many mathematically equivalent processes.

  For example, an image obtained by Fourier-transforming an image in a block may be multiplied by a constant to perform an inverse Fourier transform, or an image obtained by transforming an image in a block by a wavelet may be multiplied by a constant to perform an inverse wavelet transform. Mathematically, if it is a linear transformation, the inverse transformation can be performed by multiplying the transformed image by a constant. Here, the process of multiplying the image by a constant includes a process of multiplying the converted image by a constant and performing inverse conversion, and a process of multiplying the image by a constant except for offset.

  In the above-described embodiment, the method of calculating one integral value based on the plurality of line integral values when the line integral values of the plurality of paths obtained by the path integral of the modified differential image are different has been described. In addition, mathematically, the modified differential image is matched so that the line integral value of the modified differential image is uniquely determined regardless of the path, and the integrated modified differential image is line-integrated along an arbitrary path, thereby obtaining a mathematical expression. May be obtained.

  In the following, a description will be given of a process of matching the changed differential images so that the line integral value of the changed differential image is uniquely determined regardless of the path, and performing line integration of the changed changed differential images.

FIG. 27 is a block diagram illustrating a configuration of an image processing device according to the twelfth embodiment.
27, the image processing apparatus includes a differential processing unit 211, a differential image change processing unit 212, a matched differential image creation processing unit 214, and a matched differential image integration processing unit 215. The differential processing unit 211 and the differential image change processing unit 212 have the same configuration as in FIG.

  The matched differential image creation processing unit 214 creates an X direction matched partial differential image 207 and a Y direction matched partial differential image 207 in which the X direction changed partial differential image 204 and the Y direction changed partial differential image 205 are matched. The X direction matched partial differential image 207 and the Y direction matched partial differential image 207 are images in which the line integral value is uniquely determined regardless of the path.

  The matched differential image integration processing unit 215 creates a path integral image 209 obtained by integrating the X-direction matched partial differential image 207 and the Y-direction matched partial differential image 207. When setting the display level of the path integral image 209, the matched differential image integration processing unit 215 can refer to the input image 201. The process of creating the path integral image 209 is mathematically equivalent to the process of creating the path matching integral image 206 in FIG.

  The matched differential image integration processing unit 215 obtains an integrated value by performing line integration with reference to the X direction matched partial differential image 207 and the Y direction matched partial differential image 207 with respect to successive points starting from the initial point. In the X-direction matched partial differential image 207 and the Y-direction matched partial differential image 207, even if there are a plurality of paths to a point at the same distance, the value obtained by line integration does not change, so that any path may be followed. For example, the route can be narrowed down to one and the line integration can be performed, for example, a route is determined as an order in which left and right are given priority over top and bottom. When the integral values of all the desired points are obtained and the values are substituted into the path integral image 209, the process ends.

  Hereinafter, a method of creating a matched differential image in which the modified differential image has been matched will be described. First, the modified differential image E is substituted for the differential image E '. At this point, the differential image E 'has not been matched, and the matching differential image is finally obtained by sequentially performing the following matching processes. At the time of the first substitution, the value of the X-direction modified partial differential image 204 is substituted for the X-direction differential image Ex ', and the value of the Y-direction partial differential image 205 is substituted for Ey' of the Y-direction differential image.

  Next, matching processing is performed on the differential image E ′ of the adjacent four-point mesh including the initial point. Matching means changing the value of the differential image E 'so that the line integral value is the same regardless of the path.

FIG. 28 is a diagram illustrating a state of a gradient in an adjacent four-point mesh when a matched differential image is created by the image processing apparatus in FIG. 27.
In FIG. 28, it is assumed that line integration along the path from the initial point (x0, y0) to the point (x0 + 1, y0 + 1) is performed. Here, attention is paid to an adjacent four-point mesh having the vertices of four points (x0, y0), (x0 + 1, y0), (x0 + 1, y0 + 1), and (x0, y0 + 1). At this time, the route from the initial point (x0, y0) to the point (x0 + 1, y0 + 1) includes a route R1 of (x0, y0) → (x0 + 1, y0) → (x0 + 1, y0 + 1) and a route (x0, y0). There is a route R2 of → (x0, y0 + 1) → (x0 + 1, y0 + 1).

  The path matching integration of the twelfth embodiment can be executed using the following equations 31 to 34. Here, F is a path matching integral image, and the value of (x0, y0) of the input image 201 is substituted for F (x0, y0).

(Equation 31)
e1 = Ex ′ (x0, y0) + Ey ′ (x0 + 1, y0)

(Equation 32)
e2 = Ey '(x0, y0) + Ex' (x0, y0 + 1)

(Equation 33)
e = (e1 + e2) / 2 = (Ex ′ (x0, y0) + Ey ′ (x0 + 1, y0) + Ey ′ (x0, y0) + Ex ′ (x0, y0 + 1)) / 2

(Equation 34)
F (x0 + 1, y0 + 1) = F (x0, y0) + e

  At this time, by using Expressions 31 to 33, the values of the X-direction differential image Ex 'and the Y-direction differential image Ey' are matched by the following Expressions 35 and 36.

(Equation 35)
Ey ′ (x0 + 1, y0) = e−Ex ′ (x0, y0)

(Equation 36)
Ex ′ (x0, y0 + 1) = e−Ey ′ (x0, y0)

  When the matching process is performed as described above, the line integral values of both the routes R1 and R2 become the same as shown in the following Expressions 37 and 38. Can be uniquely determined.

(Equation 37)
F (x0 + 1, y0 + 1) = F0 (x0, y0) + Ex ′ (x0, y0) + Ey ′ (x0 + 1, y0)
= F0 (x0, y0) + e

(Equation 38)
F (x0 + 1, y0 + 1) = F0 (x0, y0) + Ey '(x0, y0)
+ Ex '(x0, y0 + 1)
= F0 (x0, y0) + e

  The matching process is also performed on the adjacent four-point mesh including the other initial points (x0, y0). In the adjacent four-point mesh, there are two paths from the point at a distance of 1 to the point at a distance of 2 from the point at a distance of 1 based on the initial point (x0, y0). For this reason, in the matching process, the average value of the line integral values of the respective paths is set as a new line integral value so that the line integral values of the two paths become the same value, and the distance from the point at a distance of 1 is determined. The value of the differential image E ′, which is the gradient toward the point whose distance is 2 away, is changed.

  If the point at a distance k from the initial point (x0, y0) is the point closest to the initial point (x0, y0) among the points of the adjacent four-point mesh, the distance is k + 1 in the adjacent four-point mesh. There are two routes to a point at a distance k + 2 from the distant point. For this reason, the differential image E ′ is a gradient of a point at a distance of k + 2 from a point at a distance of k + 1 so that the average value of the line integrals along each path becomes a new line integral. You can change the value.

  The matching process is started from the adjacent four-point mesh including the initial point (x0, y0), the matching process is performed on the adjacent four-point meshes that are successively separated from each other, and the matching of all the adjacent four-point meshes in the area for which the integral value is to be obtained is performed. After the conversion process, a matched differential image can be obtained as the differential image E ′.

FIG. 29 is a flowchart showing a matched differential image creating process of the image processing apparatus of FIG.
In step S91 in FIG. 29, the matched differential image creation processing unit 214 in FIG. 27 calculates the distance from the initial point P0 for each of the four adjacent meshes of the image 231 having the same size as the input image 201. The distance between the nearest point to the initial point P0 among the four points on the adjacent four-point mesh is defined as the distance between the adjacent four-point mesh and the initial point P0.

  Next, in step S92, the matched differential image creation processing unit 214 assigns the changed differential image E to the differential image E '. At this time, the value of the X direction modified partial differential image 204 is substituted for the X direction differential image Ex ', and the value of the Y direction partial differential image 205 is substituted for Ey' of the Y direction differential image. At the time of this first substitution, the differential image E 'has not been matched yet.

  Next, in step S93, the matched differential image creation processing unit 214 performs the matching process on the differential images E 'in ascending order of the distance between adjacent four-point meshes. When the matching process is completed for the adjacent four-point mesh including all the points desired to be integrated, the differential image E 'becomes a matched differential image.

  In the matching processing, matching is performed between two adjacent four-point meshes of interest that have two routes from a point closest to the initial point P0 to a point farthest from the initial point P0. For example, assuming that a point closest to the initial point P0 is (x, y) and a point farthest from the initial point P0 is (x + α, y + β) in the four adjacent meshes of interest, the following equations 39 to 43 are obtained. And use for matching. Here, α and β are 1 or −1.

(Equation 39)
e1 ′ = α * Ex ′ (x + (α−1) / 2, y)
+ Β * Ey ′ (x + α, y + (β−1) / 2)

(Equation 40)
e2 ′ = β * Ey ′ (x, y + (β−1) / 2)
+ Α * Ex ′ (x + (α−1) / 2, y + β)

(Equation 41)
e '= (e1' + e2 ') / 2

(Equation 42)
Ey '(x + α, y + (β-1) / 2)
= Β (e'-α * Ex '(x + (α-1) / 2, y))

(Equation 43)
Ex '(x + (α-1) / 2, y + β)
= Α (e′−β * Ey ′ (x, y + (β−1) / 2))

  In the above matching process, as shown in Expression 41, the value of the differential image is changed such that the average value of the line integral values of the two paths in the adjacent four-point mesh becomes a new line integral value. There are various other matching methods. For example, in Equation 41, e ′ is set to the average value (e1 ′ + e2 ′) / 2. The value may deviate from the value.

  As described above, after performing the process of matching the values of the differential image so that the line integral values of different paths match, the process of performing the line integration is performed on the different line integral values of the different image paths that are not matched. This is substantially equivalent to the process of using the average value as the integral value. Therefore, the process of creating a differential image matched by averaging Expression 41 and performing line integration is included in the same distance path average integration.

  It should be noted that, when the integrated differential image matched using the average value obtained by Expression 41 is linearly integrated, the image becomes the same distance path average integrated image. However, when the same distance path average integrated image is differentiated, the average obtained by Expression 41 is obtained. It becomes a matched differential image matched using the values. For this reason, both are images connected by conversion and inverse conversion.

  Since the differentiated image obtained by differentiating the path matching integral image is a matched differential image, the modified differential image (the X direction partial differential image 204 and the Y direction partial differential image 204) is obtained so as to obtain the matched differential image. 205) is also included in the matching process.

  In the example of FIG. 27, the configuration is shown in which the differential processing unit 211, the differential image change processing unit 212, the matched differential image creation processing unit 214, and the matched differential image integration processing unit 215 are provided in the image processing apparatus. The differential processing unit 211, the differential image change processing unit 212, the matched differential image creation processing unit 214, and the matched differential image integration processing unit 215 are realized by separate programs or provided in separate devices, and operate separately. You may be able to. For example, the matched differential image integration processing unit 215 that receives the output of the matched differential image creation processing unit 214 as an input is realized by another program or provided in another device, and the matched differential image of the matched differential image is provided by another program or another device. Integration processing may be performed.

FIG. 30 is a diagram illustrating an example of the distance between adjacent four-point meshes when the matched differential image is created by the image processing apparatus in FIG. 27.
In FIG. 30, a value obtained by calculating the distance from the initial point P0 is assigned to each adjacent four-point mesh. For example, there are four adjacent 4-point meshes having a distance of 0 including the initial point P0, 8 adjacent 4-point meshes having a distance of 1 adjacent thereto, and 12 adjacent 4-point meshes having a distance of 2 adjacent thereto. I know there is. Although FIG. 30 shows only a total of 64 adjacent 4-point meshes, in actuality, adjacent 4-point meshes including all points for which it is desired to obtain an integral value in an image 231 having the same size as the input image 201 are shown. Distances can be assigned to point meshes.

  For the process of matching the differential images, in addition to the method described above, the methods described in References 5 and 6 below can be used.

  Reference 5: Vishal M. Patel, Ray Maleh, Anna C.E. Gilbert, and Rama Chellappa, "Gradient-Based Image Recovery Methods from Incomplete Complete Fourier Measurements", IEEE TRANSACTIONS ON IMAGE PROGRAM. 21, NO. 1, JANUARY 2012.

  Reference 6: Tal Simchony, Rama Chellappa, M.S. Shao, “Direct Analytical Methods for Solving Poisson Equations in Computer Vision Problems”, TRANSACTION on ANALYSIS and MACHINE INTERLIGENCE. 12, NO. 5 MAY 1990.

  In the above-mentioned Reference 5, the imaging data is X-ray CT or MRI data, and a matching process is performed on a differential image created from such data. The matching process of Reference 5 is described in “V. IMAGE RECONSTRUCTION FROM GRADIENTS” in the document.

  Reference 6 described above is to produce a differential image showing irregularities in the direction perpendicular to the photographing surface of the subject from the shadows and rays of the photographed image, and to perform a matching process of the differential images showing irregularities of the differential image.

  Therefore, the above-mentioned References 5 and 6 are different from the present embodiment in the overall configuration, and are characterized by “the differential image of the input image is obtained, the differential image is changed, The integration process is performed on the differential image to perform line integration or path matching integration.

FIG. 31 is a block diagram illustrating a hardware configuration example of the image processing apparatus in FIG.
31, the image processing apparatus 111 includes a processor 11, a communication control device 12, a communication interface 13, a main storage device 14, and an external storage device 15. The processor 11, the communication control device 12, the communication interface 13, the main storage device 14, and the external storage device 15 are mutually connected via an internal bus 16. The main storage device 14 and the external storage device 15 are accessible from the processor 11.

  Further, an input device 20 and an output device 21 are provided outside the image processing device 111. The input device 20 and the output device 21 are connected to the internal bus 16 via the input / output interface 17.

  The input device 20 is, for example, a keyboard, a mouse, a touch panel, a card reader, a voice input device, or the like. The output device 21 is, for example, a screen display device (a liquid crystal monitor, an organic EL (Electro Luminescence) display, a graphic card, or the like), an audio output device (such as a speaker), or a printing device.

  The processor 11 is hardware that controls the operation of the entire image processing apparatus 111. Note that the processor 11 may be a general-purpose processor or a dedicated processor specialized in image processing. The main storage device 14 can be composed of, for example, a semiconductor memory such as an SRAM or a DRAM. The main storage device 14 can store a program being executed by the processor 11 or provide a work area for the processor 11 to execute the program.

  The external storage device 15 is a storage device having a large storage capacity, for example, a hard disk device or an SSD. The external storage device 15 can hold executable files of various programs and data used for executing the programs. The external storage device 15 can store an image processing program 15A. The image processing program 15A may be software that can be installed in the image processing device 111, or may be incorporated in the image processing device 111 as firmware.

  The communication control device 12 is hardware having a function of controlling communication with the outside. The communication control device 12 is connected to a network 19 via a communication interface 13. The network 19 may be a WAN (Wide Area Network) such as the Internet, a LAN (Local Area Network) such as WiFi, or a mixture of a WAN and a LAN.

  The input / output interface 17 converts data input from the input device 20 into a data format that can be processed by the processor 11, and converts data output from the processor 11 into a data format that can be output by the output device 21. .

  The processor 11 reads the image processing program 15A into the main storage device 14, executes the image processing program 15A, creates a differentiated image obtained by differentiating the input image, creates a modified differentiated image obtained by changing the differentiated image, A path integral image can be created by path integrating the modified differential image.

  At this time, the image processing program 15A can realize the functions of the differential processing unit 211, the differential image change processing unit 212, and the path matching integration processing unit 213 in FIG.

  The execution of the image processing program 15A may be shared by a plurality of processors and computers. Alternatively, the processor 11 may instruct a cloud computer or the like to execute all or part of the image processing program 15A via the network 19 and receive the execution result.

100 photographing device, 111, 121, 131 image processing device, 112, 122, 132 display device, 113, 123, 133 input device, 114, 124, 134 storage device, 140 communication network

Claims (14)

A differentiation processing unit that creates a differential image by differentiating the input image;
A differential image change processing unit that creates a changed differential image obtained by changing the differential image,
A path integration processing unit that creates a path integration image by path integration of the modified differential image.   The image processing device according to claim 1, wherein the differential processing unit generates an X-direction partial differential image of the input image and a Y-direction partial differential image of the input image when the input image is a two-dimensional image. . The differential image change processing unit creates an X direction changed partial differential image obtained by changing the X direction partial differential image and a Y direction changed partial differential image obtained by changing the Y direction partial differential image,
The path integration processing unit refers to the X-direction changed partial differential image when performing path integration in the X direction, and refers to the Y-direction changed partial differential image when performing path integration in the Y direction. The image processing apparatus according to any one of the preceding claims.   The image processing apparatus according to claim 1, wherein the path integral is a path matching integral that calculates one integral value based on a plurality of integral values obtained by path integral of the modified differential image. In the path integral,
An adjacent value adjacent to the first point of interest is calculated based on a line integration by an addition / subtraction operation of an integral value of the first point of interest and a pixel value of the adjacent point adjacent to the first point of interest in the modified differential image. Find the integral value of the point,
Addition / subtraction operation of the integral value of the second point of interest and the pixel value of the adjacent point adjacent to the second point of interest in the modified differential image at the position corresponding to the adjacent point for which the integral value has been obtained as the second point of interest The image processing apparatus according to claim 4, wherein an integral value of an adjacent point adjacent to the second point of interest is obtained based on the line integration by: In the path matching integration,
When there are a plurality of paths having the same discrete distance between the point for obtaining the integral value of the path integral image and the initial point, the average of the values obtained by performing the line integration for each of the plurality of paths is defined as the integral value of the point. ,
6. The image processing according to claim 5, wherein a process of setting an average value of the values obtained by performing the line integration for each path as an integral value of the point is sequentially repeated up to a predetermined range to obtain integrated values of all points in the predetermined range. apparatus. The path integration processing unit,
Dividing the image on which the path integration is performed into blocks,
Create a path integral image by path integrating the image in the block,
The image processing apparatus according to claim 1, wherein an offset of the path integral image is adjusted. The path integration processing unit,
Determine the division method to create a plurality of path integral images with different division methods,
The image processing apparatus according to claim 7, wherein a weighted average according to a distance between the center of each block and the pixel of interest is performed on a plurality of blocks of the path integral image to which the pixel of interest belongs.   The image processing device according to claim 1, wherein the differential image change processing unit multiplies a pixel value of the differential image by a constant.   The image processing device according to claim 1, wherein the differential image change processing unit reduces an absolute value of a pixel value of the differential image that is lower than a predetermined value. A differentiation processing unit that creates a differential image by differentiating the input image;
A differential image change processing unit that creates a changed differential image obtained by changing the differential image,
Image processing comprising: a matched differential image creating unit that creates a matched differential image by changing a pixel value of the changed differential image so that an integral value of the path integral of the changed differential image is uniquely determined regardless of the path. apparatus.   The image processing apparatus according to claim 11, further comprising a path integration processing unit configured to generate a path integration image obtained by performing path integration on the matched differential image. Divide the input image into blocks,
Generate a constant-multiplied image obtained by multiplying the pixel value in the block by a constant,
An image processing device for adjusting an offset of the constant-multiplied image. Determine the division method to create multiple constant multiple images with different division methods,
14. The image processing apparatus according to claim 13, wherein a weighted average according to a distance between the center of each block and the pixel of interest is performed on a plurality of blocks of the constant-multiplied image to which the pixel of interest belongs.

Images