JP2017005660A - Image processing apparatus and image processing method - Google Patents

Abstract

A large amount of calculation is involved in performing a filter convolution operation represented by edge enhancement processing, and the processing time becomes enormous.
An image processing apparatus extracts an edge component of a pixel having a fixed period from pixels included in an input image. Then, the edge component of the target pixel that has not been extracted is calculated by interpolation using the extracted edge component of the pixel adjacent to the target pixel.
[Selection] Figure 2

Description

  The present invention relates to a technique for obtaining an edge component included in an image.

  Conventionally, edge enhancement processing is performed by obtaining an edge component from an image and adding the obtained edge component. By performing the edge enhancement processing, the readability of a character image is increased, and if it is a photograph, the photograph becomes sharp and the image quality can be improved. As processing for obtaining the edge component, it is common to use a convolution operation by a filter. The image is converted into an image in which the contrast near the edge is enhanced by the edge enhancement process. Specifically, the bright part is brighter at the position where the bright area and the dark area are in contact with each other, and the dark part is converted to darker so that the boundary portion is exaggerated, and it looks sharp.

  Many techniques have been published so far for edge enhancement techniques for color images.

  Improve readability by scanning the original image with a black character, performing edge emphasis on the black character, and generating the black character when printing the black character. There is technology. For example, color image data is separated into data indicating luminance and data indicating saturation, a black edge degree is obtained from the edge amount and saturation calculated from the luminance, and enhancement processing is performed on the edge.

  In addition, the amount of sharpness enhancement is selectively controlled to sharpen solid borders with clear edges, such as letters and logos, and to suppress sharpness in areas where the pattern is not clear, based on the relationship between the brightness around the pixel of interest and the threshold. The process to perform is also disclosed (see Patent Document 1).

JP 7-274004 A

  However, when the edge enhancement process is performed using software, the processing time for the convolution calculation may become enormous depending on the size of the input image and the CPU resource to be processed. In the convolution operation, usually, a process of multiplying each pixel in the processing window used for the convolution operation by a coefficient at a corresponding position in the filter and accumulating the result is performed. In other words, when the size of the window is 5 × 5, 25 multiplications and additions are required to process one pixel, and the weight occupying the edge enhancement processing is the largest. In addition to the edge enhancement by simple convolution calculation as described above, the calculation amount per pixel becomes enormous if the accompanying edge enhancement amount correction process is included.

  An image processing apparatus according to the present invention includes an extraction unit that extracts an edge component of a pixel having a fixed period among pixels included in an input image, and an edge component of a pixel of interest that has not been extracted by the extraction unit. And calculating means for calculating by interpolation using the edge component extracted by the extracting means for the pixels adjacent to the pixel.

  According to the present invention, it is possible to reduce processing time when performing convolution operation processing on an image.

1 is a block diagram illustrating a schematic configuration of an image forming apparatus according to an embodiment of the present invention. It is a flowchart of the edge emphasis process which concerns on Embodiment 1 of this invention. It is a figure of the edge part before and behind edge emphasis concerning Embodiment 1 of the present invention. It is a figure of a mode that the filter window concerning Embodiment 1 of this invention was shifted. It is a flowchart of the edge emphasis process which concerns on Embodiment 2 of this invention. It is a figure of a mode that the filter window concerning Embodiment 2 of this invention was shifted. It is the figure which showed the frequency response which concerns on Embodiment 2 of this invention.

  Hereinafter, embodiments for carrying out the present invention will be described with reference to the drawings. The configurations shown in the following embodiments are merely examples, and the present invention is not limited to the illustrated configurations.

[Embodiment 1]
[Configuration of Image Forming Apparatus]
FIG. 1 is a block diagram illustrating a configuration of an image processing apparatus having a function of performing edge enhancement according to the present exemplary embodiment. As illustrated in FIG. 1, the image processing apparatus includes an image reading unit 101, an image processing unit 102, a storage unit 103, a CPU 104, an image output unit 105, a user interface (UI) unit 106, and a transmission / reception unit 107. The image processing apparatus can be connected to a server that manages image data, a personal computer (PC) that instructs execution of printing, and the like via a network.

  The image reading unit 101 reads an image of a document and outputs image data. The image processing unit 102 converts print information including image data input from the outside from the image reading unit 101 and the transmission / reception unit 107 into intermediate information (hereinafter referred to as “object”) and stores it in an object buffer of the storage unit 103. To do. Further, the image processing unit 102 generates bitmap data based on the buffered object and stores it in the buffer of the storage unit 103. The image processing unit 102 performs color conversion processing, edge enhancement processing, and the like in the processing for generating bitmap data. Details will be described later.

  The storage unit 103 includes, for example, a ROM, a RAM, and a hard disk (HD). The ROM stores various control programs and image processing programs executed by the CPU 104. The RAM is used as a reference area or work area in which the CPU 104 stores data and various types of information. The RAM and HD are used for storing the object buffer. Further, processing parameters necessary for image processing are also stored. Image data is stored on the RAM and HD, the pages are sorted, the originals over the sorted pages are stored, and a plurality of copies are printed out.

  The image output unit 105 forms and outputs a color image on a recording medium such as recording paper. The UI unit 106 performs an operation for instructing the apparatus to perform image processing type and level adjustment in the image processing unit. For example, the adjustment amount of the image processing described above is set. The transmission / reception unit 107 receives image data for printing from the outside, saves it in the storage unit 103, and outputs it to the image output unit 105. In addition, the image data stored in the storage unit 103 is transmitted and output outside the device.

  In the present embodiment, the image processing apparatus will be described by taking an example of an apparatus having a configuration as shown in FIG. 1, for example, an MFP (Multifunction Peripheral), but the image processing apparatus is not limited to this. Absent. That is, any device that handles image data may be used, and for example, an information processing device such as a PC or an imaging device may be used.

  Next, image edge enhancement processing performed in the image processing unit 102 of the present embodiment will be described in detail with reference to FIG. The processing shown in the subsequent flow is executed by the image processing unit 102 based on a command from the CPU 104. The image data input to the image processing unit 102 is composed of three colors of each pixel RGB, and is configured as a pixel array in the horizontal direction (row direction) and the vertical direction (column direction).

First, in step S201, the image processing unit 102 performs color conversion processing on the input RGB image data. For example, the image processing unit 102 performs a process of converting a pixel value from an RGB color space to a luminance and color difference color space. Here, color conversion from RGB components to YCbCr is performed. The color conversion formula is shown below.
Y = 0.2990 * R + 0.5870 * G + 0.1140 * B
Cb = -0.1687 * R-0.3313 * G + 0.5000 * B
Cr = 0.5000 * R-0.4187 * G-0.0813 * B ... Equation (1)

  In Expression (1), the Red signal value (for example, a value of 0 to 255 for an 8-bit signal) of the input RGB image is represented by R, the Green is represented by G, and the Blue is represented by B. Expression (1) shows a process of converting the input RGB component into Y (value of 0 to 255) Cb (value of −128 to 127) Cr (value of −128 to 127). By this conversion, the input RGB image is separated into luminance Y and color differences Cb and Cr. In step S201, the image processing unit 102 performs color conversion processing on all the pixels of the input RGB image data. That is, conversion is performed from data in which each pixel has RGB values to data in which each pixel has YCbCr values.

  Next, in step S <b> 202, the image processing unit 102 determines whether the position of the pixel of interest that is the pixel being processed is an even row or an odd row in the pixel array. That is, it is determined whether the pixel of interest is located in an even row or an odd row. If it is determined that the line is an even line such as the 0th line, the 2nd line, the 4th line,..., The process proceeds to the next step S203, and if not, the process skips to step S205.

  Next, in step S203, the image processing unit 102 cuts out the pixels after the color conversion processing in step S201 by the filter window. The pixels after the color conversion include the luminance information Y and color difference information CbCr signals as described above. The image processing unit 102 cuts out pixels corresponding to a window having a certain size from the color-converted image. Specifically, in this embodiment, a 5 × 5 window having 5 pixels in the row direction and 5 pixels in the column direction is used. In the present embodiment, pixels (a total of 25 pixels) corresponding to a 5 × 5 window size centered on the target pixel are cut out. In this case, the target pixel is located in an even number line in the image data. Although a 5 × 5 window has been described as an example here, the window size is not limited to this.

  In step S204, the image processing unit 102 calculates an edge component by convolving a filter matrix with the luminance Y signal of each pixel in the window cut out in step S203. As a general edge enhancement technique, a technique is used in which an edge component corresponding to the second derivative is calculated using a Laplacian filter and the value is added to the signal value of the original image. In step S204, first, processing up to the calculation of the edge component is performed for each pixel. The calculation of the secondary differential component is performed using, for example, the following filter matrix.

  This filter matrix is a matrix having a size of 5 × 5, and each element includes a filter coefficient. The brightness value of each pixel in the window cut out in step S203 is convolved using this filter matrix. That is, the luminance value of the pixel located at the upper left in the window cut out in step S303 is multiplied by the coefficient (−0.09) located at the upper left in the filter matrix. Further, the luminance value of the pixel shifted to the right from the upper left in the window is multiplied by the coefficient shifted to the right from the upper left in the filter matrix. Such multiplication is performed for all 25 pixels, and all the obtained values for 25 pixels are added. The added value obtained in this way is extracted as the edge component of the target pixel (here, the center pixel in the 5 × 5 window).

  In this way, the center of the 5 × 5 filter matrix (in this example, the position multiplied by 1.08) is the target pixel of this window, and the output result at this position. The size of the filter matrix needs to be equal to or smaller than the size of the window cut out in step S203. This is because if the size of the filter matrix is larger, pixels that are not calculated by convolution appear, so that the edge component cannot be obtained correctly. The convolution process in step S204 is performed only for the luminance Y and not for the color difference CbCr. In this way, only the brightness is enhanced, and the color change at the edge portion can be suppressed to some extent. The luminance edge component thus obtained is treated as dY hereinafter. The obtained luminance edge components are stored in corresponding positions of the target pixel, for example, by providing a data array having the same number of pixels as the number of pixels of the image data in the storage unit 103.

  In step S204, the image processing unit 102 determines whether all pixels included in the image data have been processed. When there is an unprocessed pixel, the process returns to step S202, and the process is repeated by changing the target pixel to another pixel.

  FIG. 3 is a two-dimensional graph showing the edge component of the luminance of each pixel obtained as a result of the convolution process and the luminance change of each pixel when this edge component is added to the luminance value of the original image. is there. In FIG. 3, the coordinates of each pixel are represented in one dimension on the horizontal axis. A graph indicated by a broken line 301 is a graph of the signal value of the input luminance Y. A solid line 302 is a graph of the edge component dY calculated and output according to the input luminance value in each pixel. A solid line 303 indicates a luminance value as a result when the edge component indicated by the solid line 302 is added to the input luminance value indicated by the broken line 301 for each pixel. As indicated by a solid line 303, in a place where the luminance change occurs, the dark part is darker and the bright side is brighter and the contrast appears. In other words, the edge component dY can be taken as a difference signal before and after the edge enhancement processing. Note that FIG. 3 illustrates an example in which the edge component is added to the input luminance, but the edge component has not yet been added at the time of the processing in step S205. In the subsequent processing, edge components are added. This will be described from the following step S206.

  In step S <b> 206, the image processing unit 102 determines whether the position of the target pixel being processed is an even row or an odd row in the pixel array, and switches processing between the even row and the odd row. If it is an even line, the following step S207 is performed. If it is an odd row, the processing of step S208 and step S209 is performed.

  When the position of the target pixel is an even row, in step S207, the image processing unit 102 adds the edge component dY of the target pixel obtained in step S204 to the luminance signal Y of the target pixel obtained in the process of step S201. Then, the luminance signal Y ′ after edge enhancement is generated.

On the other hand, even when the position of the target pixel is an odd row, the image processing unit 102 originally uses the target pixel luminance signal Y obtained in step S201 in the same manner as in step S207. I want to add the edge component dY. However, the image processing unit 102 has not obtained the edge component dY at the position of the target pixel in step S204. For this reason, in step S208, the image processing unit 102 generates an edge component dY at the position of the target pixel from the pixels in the upper and lower rows of the row including the target pixel by interpolation. For example, when the horizontal coordinate of the target pixel being processed is x, and the vertical coordinate is y (y is an odd number), the dY value is expressed as dY (x, y). y) is obtained by the following equation (2).
dY (x, y) = (dY (x, y-1) + dY (x, y + 1)) / 2 ... Equation (2)

  As a result, the respective values obtained by convolving the filter matrix in the step S204 with respect to the upper and lower pixels by one pixel are further added and divided by 2 as shown in the equation (2). Become. This will be described with reference to FIG. Consider a 5 × 7 window 402 centered on the position 401 of the pixel of interest being processed. Here, the result of the convolution of the pixel position 403 processed in the row before the target pixel (the pixel position one level up in FIG. 4) can be considered as follows. That is, it is equivalent to the result of convolution of the following filter B with the bottom two rows 0 added to the pixels in the 5 × 5 window 404 indicated by a broken line with the pixel position 403 as the center.

  The portion obtained by adding 0 to the lower two rows is not used for the calculation as an edge component, and thus is the same as the result using the 5 × 5 matrix filter described in step S304.

  Similarly, the result of convolution of the pixel position 405 processed in the row after the pixel position 401 of interest (the pixel position one lower in the figure) is considered as follows. That is, this is equivalent to the result of convolution of the following filter C with the upper two rows 0 added to the pixels in the 5 × 5 window 406 indicated by the broken line with the pixel position 405 as the center.

  As described above, the interpolation result at the position 401 of the target pixel is obtained by adding the results of the convolution using the two filters and dividing the result by 2. This is equivalent to the result of convolution of the following 5 × 7 filter D, which is obtained by adding the above two filter matrices and dividing by 2 as long as a real number operation is performed, into pixels in the 5 × 7 window.

  The frequency response of the filter D is substantially equivalent to the convolution result of the filter A used in the original step S204 in the horizontal direction. Also, the amount of calculation at this time is only required to add edge components for two pixels and divide by two, so that the amount of computation is reduced to a level that can be ignored compared to the case of multiplying and adding 25 pixels of 5 × 5.

  As described above, in the edge component interpolation processing in step S208, the edge component dY is obtained according to the equation (2). In step S209, as in step S207, the image processing unit 102 adds the edge component dY obtained in step S208 to the luminance signal Y obtained in step S201 to obtain the luminance signal Y ′ after edge processing enhancement. Generate.

Next, in step S210, the image processing unit 102 performs color conversion from the luminance / color difference color space to the RGB color space. This calculation is based on the following equation for the inverse matrix calculation of the color conversion performed in step S201.
R = Y '+ 1.4020 * Cr
G = Y '-0.3441 * Cb-0.7141 * Cr
B = Y '+ 1.7720 * Cb… (3)

  In Expression (3), the luminance signal Y ′ after edge enhancement is used as the luminance signal. In step S211, it is determined whether all pixels have been processed. If there is an unprocessed pixel, the target pixel is changed, and the process returns to step S206.

  In this way, edge enhancement is performed on even-numbered pixels by filter convolution. On the other hand, for the pixels in the odd rows, edge enhancement is performed by interpolating the edge components obtained in the even rows. According to such processing, the amount of calculation can be reduced to about half compared to the convolution calculation performed on all the pixels constituting the image data.

  In this embodiment, the luminance color difference color space YCbCr is used as the color space for edge enhancement, but the same can be realized in other color spaces such as L * a * b * as long as the color space is a luminance color difference system. Further, the effect can be obtained by performing processing on each of the RGB signal values as they are without performing color conversion. Although RGB is used as an input, the effect is not impaired even with a Gray image or a CMYK image. Further, an example has been described in which a convolution operation is performed on pixels having a fixed period in the pixel array of image data, and an interpolation operation is performed on pixels that do not correspond to the fixed period. That is, the example in which the convolution calculation is performed on the even-numbered rows and the interpolation is performed on the odd-numbered rows has been described. Similarly, it is possible to switch between convolution calculation and interpolation not in units of rows but in units of columns. It is also possible to perform a convolution operation by thinning out both matrices.

  In the present embodiment, the edge component is calculated, and the result is added to the original signal to obtain the edge enhanced image. However, the present invention is not limited to this, and if the signal value difference before and after edge enhancement can be calculated ( It may be applied to another process (if dY can be calculated). For example, the present invention can be applied not only to edge enhancement but also to smoothing processing.

  In this embodiment, the orthodox edge enhancement processing using the filter convolution operation has been described. However, the present invention is not limited to this as long as the signal value difference before and after the processing can be calculated as described above. For example, the present embodiment can be applied to a process of determining a black character and switching an edge enhancement parameter.

  Note that the odd-numbered row interpolation result obtained by the processing described in this embodiment is not necessarily equivalent to the even-numbered filter convolution result. As described above, the frequency response in the horizontal direction is almost the same, but the response in the vertical direction is greatly different. As a result, depending on the image, stripes with an unnatural period may occur. However, if an image with high resolution is input, there is little concern.

  When connected to an external device via a network or the like, the image data received from the transmission / reception unit 107 may be image data with various resolutions depending on the original data of the transmission source. The image processing unit 102 enlarges or reduces the resolution to a print output resolution.

  Therefore, when it is determined that the resolution of those input images is low enough to cause stripes after processing, the processing may be switched in accordance with the input resolution so that the filter convolution operation is uniformly performed in even rows and odd rows. . That is, the image processing unit 102 determines whether or not the resolution of the input image is greater than a predetermined threshold value, and performs edge enhancement processing according to the flow illustrated in FIG. On the other hand, when the resolution of the input image is equal to or less than a predetermined threshold value, the filter convolution operation is uniformly performed for both, instead of switching the processing depending on whether the row is an even row or an odd row. If the resolution of the input image is less than or equal to the predetermined threshold, the number of pixels is reduced, and the amount of calculation required for processing is also reduced. That is, since the total processing time does not change, the situation where the filter convolution calculation becomes enormous does not occur in the first place, and thus the above-described problem does not occur.

[Embodiment 2]
In the first embodiment, switching between a pixel that is subjected to filter convolution calculation and a pixel that is generated by interpolation is performed between even and odd lines of an image. However, as described above, the output image obtained as a result is different from the result obtained by performing the filter convolution operation on all the pixels, and depending on the image, a texture such as a stripe may appear by switching between the even and odd lines. Can happen. This is due to the fact that the frequency response of the filter D, which is substantially used in the interpolation processing in step S208 as described above, has different responses in the vertical direction and the horizontal direction.

  Therefore, in the second embodiment, a configuration that reduces the amount of calculation necessary for filter convolution calculation in edge enhancement while reducing the frequency response difference between the vertical and horizontal of the filter that is substantially used in the interpolation processing. explain. Note that the description regarding the configuration of the image processing apparatus, which is the same as that of the first embodiment, and the description of the overlapping flows are omitted, and the description will focus on the points that are important in this embodiment.

  The edge enhancement processing for an image in the image processing unit 102 according to the present embodiment will be described in detail with reference to FIG. The processing shown in the subsequent flow is processed by the image processing unit 102 based on a command from the CPU 104.

  Similarly to the first embodiment, the input image data is composed of three colors of each pixel RGB, and is configured as an array in the horizontal direction (row direction) and the vertical direction (column direction).

  First, in step S501, the image processing unit 102 performs color conversion processing on input RGB image data, and performs color conversion from a luminance and color-difference color space, here, RGB components to YCbCr. Detailed description is omitted because it is the same as step S201.

  In step S <b> 502, the image processing unit 102 determines whether the position of the target pixel being processed is an even-numbered row and an even-numbered column, an odd-numbered row and an odd-numbered column, or any other number in the image array. Specifically, when the horizontal coordinate is x and the vertical coordinate is y, both x and y are even or both x and y are odd, and in other cases, that is, x is even and y is The determination for switching the process is performed when the odd number or x is an odd number and y is an even number. If both x and y are even or odd, the process proceeds to the subsequent step S503. Otherwise, the process skips to step S505. In other words, processing is skipped in a staggered pattern.

  Next, the processing performed by the image processing unit 102 in step S503 and step S504 is the same as that described in step S203 and step S204, respectively.

  The processing up to step S504 is performed for all the pixels included in all the image data. With these processes, the processing in steps S503 and S504 is performed in a staggered pattern on the image, and the edge components are calculated in the staggered pixels. After that, the process of adding edge components will be described from the subsequent step S506.

  In step S506, the image processing unit 102 determines whether the position of the pixel of interest being processed is an even-numbered row and an even-numbered column, an odd-numbered row and an odd-numbered column, or other than that in step S502. Judge and switch processing. If it is an even-numbered row and an even-numbered column, or an odd-numbered row and an odd-numbered column, the following step S507 is performed. Otherwise, the processing of steps S508 and S509 is performed.

  In step S507, the image processing unit 102 adds the edge component dY of the target pixel to the luminance signal Y of the target pixel obtained in the process of step S501, and generates a luminance signal Y ′ after edge enhancement.

  On the other hand, also in the step S508, the image processing unit 102 wants to add the edge component dY of the target pixel to the luminance signal Y of the target pixel obtained in the process of step S501 as in the process of step S506. However, the image processing unit 102 has not obtained the edge component dY at the position of the target pixel in step S504. For this reason, in step S508, the image processing unit 102 interpolates and generates an edge component dY at the position of the target pixel from the upper and lower and left and right pixels of the target pixel. In the present embodiment, since edge components are obtained in a staggered pattern, not only interpolation from the upper and lower pixels described in step S208 of the first embodiment but also interpolation using left and right pixels is possible. The value of the edge component dY when the horizontal coordinate of the target pixel being processed is x and the vertical coordinate is y is expressed as dY (x, y). That is, dY (x, y) is unknown, and edge components dY (x-1, y), dY (x + 1, y), dY (x, y-1), dY (x, y + 1) of pixels adjacent to it are unknown. Are already processed in step S504 and are already known. An unknown edge component dY (x, y) of the target pixel is interpolated and estimated using known edge components of the upper, lower, left, and right pixels of the target pixel.

The interpolation formula is, for example,
dY (x, y) = (dY (x, y-1) + dY (x, y + 1) + dY (x-1, y) + dY (x + 1, y)) x α / 2 ... (Four)
Ask for. Here, the value of α is a parameter that adjusts the filter gain as a positive real number. This value is a parameter for bringing the result closer to the filter shown in the original step S204. Naturally, multiplication is involved, which affects the processing time. In equation (4), when interpolation is performed from four known edge components, division is performed by 2 instead of 4. This is because an edge is composed of a direction in which similar values are continuous and a direction that is a boundary in which values are not continuous. For example, considering vertical line edges, similar values are continuous for pixels in the vertical direction. On the other hand, the values of the pixels in the left-right direction are different. Dividing by 4 is to obtain a value considering both pixels that contribute to edge enhancement and pixels that do not contribute to edge enhancement. However, it is essentially desirable to interpolate only from pixels that contribute to edge enhancement. Therefore, in this embodiment, division by 2 is performed for the purpose of performing interpolation from pixels that contribute to edge enhancement. That is, according to the process of the present embodiment, the relevance of pixels that do not contribute to edge enhancement can be weakened as a result. Of course, this is a probability problem, and division by 2 does not solve all the relevance between pixels that contribute to edge enhancement and pixels that do not contribute to edge enhancement. Therefore, in this embodiment, the filter gain is adjusted by multiplying α as a parameter.

  In the present embodiment, the following description will be made assuming that α is 1. When α is 1, as a result, 5 × 5 pixels on the top, bottom, left and right of the target pixel are convolved with a matrix in which the filter matrix shown in step S204 is shifted up, down, left and right by one pixel of the target pixel. Then, the respective convolved values are added according to Equation (4) and divided by 2. This will be described with reference to FIG. The result of convolution of the pixel position 603 processed in the row before the target pixel position 601 is considered as follows. That is, it is equivalent to the result of convolving the following filter E in which the lower two rows 0 and the left and right columns are filled with 0 in the pixel in the 5 × 5 window 604 indicated by a broken line with the pixel position 603 as the center. is there,

  Similarly, the result of convolution of the pixel position 605 processed in the row after the target pixel position 601 is considered as follows. That is, this is equivalent to the result of convolving the following filter F in which the upper two rows 0 and the left and right 0s are filled with respect to the pixels in the 5 × 5 window 606 centered on the pixel position 605.

  Similarly, the result of convolution of the pixel position 607 processed in the left column of the target pixel position 601 is considered as follows. That is, it is equivalent to the result of convolution of the following filter G in which the right two columns 0 and the upper and lower 0 are filled in the pixels in the 5 × 5 window 608 centered on the pixel position 607.

  Similarly, the convolution result of the pixel position 609 processed in the right row of the target pixel position 601 is considered as follows. That is, it is equivalent to the result of convolving the following filter H in which the left two columns 0 and the upper and lower 0 are filled in the pixels in the 5 × 5 window 610 centered on the pixel position 609.

  Thus, Expression (4) is obtained by adding the result of convolution to the pixels in the corresponding window using the filters at the positions of the upper, lower, left and right pixels, and dividing the result by 2.

  In other words, the 7 × 7 window 602 centered on the target pixel position 601 being processed is added to the pixels of the window 602 using the following filter I, which is obtained by adding the filters E to H and dividing by 2 as long as the real number calculation is performed. Equivalent to the result of convolution.

  The frequency response of the filter I becomes a uniform response in the vertical and horizontal directions, and the amount of calculation is reduced to a level that can be ignored with respect to the multiplication and addition of 5 × 5 25 pixels. Here, if α shown in the equation (4) described in the previous step S508 is a value other than 1, for example, β, this is equivalent to all the filter matrices multiplied by the value β.

  An example of the frequency response of the filter I is shown using FIG. Since the filter matrix of the filter I has completely the same value in the horizontal direction and the vertical direction, the result of the FFT is the same. FIG. 7 shows the one-dimensional frequency in the horizontal direction and the gain at that time. The filter matrix of the 5 × 5 filter A used in step S504 (that is, described in step S204 of the first embodiment) shows a frequency response like a solid line 701. On the other hand, the frequency response of the filter matrix of the 7 × 7 filter I shown above is as shown by a broken line 702. In this case, the peak frequency shifts to the low frequency side as the matrix increases, and the gain at the peak also changes greatly. A line 703 shows a frequency response when α shown in the equation (4) is less than 1, for example, about 0.8. By controlling α, the peak gain can be adjusted. By using this value, the frequency band is shifted, but the peak gain can be controlled in the same manner as the filter matrix of the original 5 × 5 filter A.

  In step S509, the image processing unit 102 adds the edge component dY of the pixel of interest obtained in this way to the luminance signal Y of the pixel of interest obtained in the processing of step S501 in the same manner as in step S507, and luminance after edge enhancement A signal Y ′ is generated.

  Next, in step S510, the image processing unit 102 performs color conversion from the luminance and color difference color space to the output RGB color space. Since this calculation is the same as that described in step S210, the description is omitted. In step S511, it is determined whether all pixels have been processed. If there is an unprocessed pixel, the target pixel is changed, and the process returns to step S506.

  Thus, in the present embodiment, edge enhancement is performed on the image in a staggered pattern by filter convolution, and the remaining positions are interpolated by simple calculation. By such processing, it is possible to suppress the calculation amount to about half compared to performing the convolution calculation for all the pixels constituting the image data.

  In addition, the vertical and horizontal frequency responses that were not uniform with respect to the previous embodiment 1 are aligned, and as shown in FIG. It is possible to suppress the occurrence of natural stripes.

[Embodiment 3]
In the first and second embodiments, the method of reducing the amount of calculation by thinning out the filter calculation processing by using a simple interpolation calculation has been described. However, the output image obtained as a result is different from the result obtained by performing the filter convolution operation on all the pixels, and depending on the image, there may be a stripe or a staggered texture. This is because the vertical and horizontal frequency response differences in step S208 of the first embodiment do not match the frequency bands emphasized in step S508 of the second embodiment.

  Therefore, in the third embodiment, a configuration for reducing the amount of calculation necessary for the filter convolution calculation in edge enhancement while switching between using the vertical interpolation and the horizontal interpolation according to the image will be described. Note that description regarding the configuration of the image processing apparatus, which is the same as in the first and second embodiments, and description of overlapping flows are omitted, and the description will focus on points that are important in the present embodiment.

  An edge enhancement process for an image in the image processing unit 102 according to the present embodiment will be described in detail. Since the flowchart is the same as that of the second embodiment, the description thereof is omitted, and the edge component interpolation in step S508, which is greatly different in the present embodiment, will be described in detail.

  In step S508 in the present embodiment, the image processing unit 102 generates an edge component dY at the position of the target pixel by interpolation from the pixels in the upper and lower rows of the target pixel row being processed or the pixels in the left and right columns. The value of the edge component dY when the horizontal coordinate of the pixel of interest is x and the vertical coordinate is y is expressed as dY (x, y) as in the second embodiment. That is, the edge component dY (x, y) of the pixel of interest is unknown as in the second embodiment. On the other hand, the edge components dY (x-1, y), dY (x + 1, y), dY (x, y-1), and dY (x, y + 1) of the upper, lower, left, and right pixels of the target pixel have been processed in step S504. It is already known. In the present embodiment, the edge component dY (x, y) of this unknown target pixel is interpolated and estimated by a method different from the method described in the second embodiment.

  When the image being processed is a vertical line, the edge components dY (x, y−1) and dY (x, y + 1) of pixels located above and below the target pixel take close values and are positioned on the left and right sides of the target pixel. The edge components dY (x−1, y) and dY (x + 1, y) of the pixels to be processed have significantly different values. Conversely, in the case of a horizontal line, the edge components dY (x−1, y) and dY (x + 1, y) of the pixels located on the left and right of the target pixel take close values, and the edge components dY of the pixels positioned above and below the target pixel. (X, y-1) and dY (x, y + 1) are greatly different values.

  This will be described using FIG. 3 again. In the case of this figure, it is expressed in one dimension, but the change of the solid line 302 changes greatly with respect to the direction in which the edge is located at the edge portion.

  In the present embodiment, it is assumed that the coordinate direction in FIG. 3 is the horizontal direction x, and the image is a vertical line continuing in the y direction. When the image is a vertical line, there is no change in the y direction. Therefore, if the image is a vertical line, assuming that FIG. 3 is a diagram (not shown) in which the y direction is the coordinate direction of the horizontal axis, the solid line 302 has the same shape continuously in the y direction. That is, when the image is a vertical line, the difference between the edge components dY (x−1) and dY (x + 1) of the left and right pixels of the target pixel is large. On the other hand, the difference between the edge components dY (y−1) and dY (y + 1) of the upper and lower pixels is small.

  Using this feature, in the present embodiment, whether the image included in the window being processed is a vertical edge or a horizontal edge is determined from the difference between the edge components dY, and the result is interpolated from above and below, Switch between interpolating the results from the left and right.

If the absolute value of (dY (x, y−1) −dY (x, y + 1)) is smaller than the absolute value of (dY (x−1, y) −dY (x + 1, y)) The image in the window including the pixel is determined as a vertical line. The result of the edge component dY (x, y) of the target pixel uses a value derived from the edge component obtained for the upper and lower pixels considered to have a similar frequency response by the following equation (5).
dY (x, y) = (dY (x, y-1) + dY (x, y + 1)) / 2 ... Equation (5)

Conversely, if the absolute value of (dY (x-1, y) −dY (x + 1, y)) is smaller than the absolute value of (dY (x, y−1) −dY (x, y + 1)). Judged as a horizontal line. The result of the edge component dY (x, y) of the target pixel uses a value derived from the edge component obtained for the left and right pixels considered to have a similar frequency response by the following equation (6).
dY (x, y) = (dY (x-1, y) + dY (x + 1, y)) / 2 ... Equation (6)

  By doing this, the horizontal frequency response when the image is a vertical line is almost the same as the result made by interpolation, and the vertical frequency response when the image is a horizontal line is also almost the same. .

  With such a flow, edge emphasis is performed on the image in a staggered pattern by filter convolution, and the remaining positions are interpolated using simple determination. As a result, it is possible to reduce the calculation amount to about half compared to performing the convolution calculation for all the pixels constituting the image data.

  In addition, by switching using the image feature for the previous embodiment, the frequency response of the thinned pixels and the pixels that are not the same are aligned, and generation of unnatural stripes and staggered textures can be suppressed. Is possible.

<Other embodiments>
In the second and third embodiments, the edge components are calculated in a staggered pattern, and the interpolation calculation is performed on the pixels that have not been subjected to the edge enhancement processing using the edge components that have been processed on the upper, lower, left, and right pixels. explained. However, the present invention is not limited to this example, and any form may be used as long as the interpolation calculation is performed using the edge components that have been processed for the surrounding pixels for the pixels that have not been subjected to the edge enhancement processing. Further, depending on the mode of interpolation, interpolation may be performed using pixels located diagonally, instead of using upper, lower, left, and right pixels.

  In the processing of each embodiment, an example has been described in which processing for adding an edge component to each pixel is performed after the edge components have been calculated for all pixels. However, a form in which edge enhancement processing is sequentially performed for each pixel due to the memory capacity may be employed. For example, an example of the first embodiment will be described. First, edge enhancement processing is performed for the 0th and 2nd rows. Then, the edge component for each pixel in the 0th row and the 2nd row is calculated from the difference between the pixel value after the edge enhancement processing and the pixel value before the edge enhancement processing. Thereafter, for each pixel in the first row, interpolation processing using the edge components calculated for the 0th row and the second row and edge enhancement processing for adding the interpolated edge components are performed. In this way, similar processing may be performed by sequential processing.

  The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

Claims (13)

An extracting means for extracting an edge component of a pixel having a constant period among the pixels included in the input image;
An image processing apparatus comprising: a calculation unit that calculates an edge component of a pixel of interest that has not been extracted by the extraction unit by interpolation using an edge component extracted by the extraction unit of a pixel adjacent to the pixel of interest.   The image processing apparatus according to claim 1, wherein the extraction unit extracts the edge component by a filter convolution operation.   2. The image processing according to claim 1, wherein the extraction unit extracts a difference between a value of a pixel before performing edge enhancement processing and a value of a pixel after performing edge enhancement processing as the edge component. apparatus. The fixed period is a row unit of the pixel array of the input image,
The image according to any one of claims 1 to 3, wherein the calculation means performs an interpolation operation using the edge components extracted by the extraction means for pixels located above and below the target pixel. Processing equipment. The fixed period is a column unit of the pixel array of the input image,
The image according to any one of claims 1 to 3, wherein the calculation means performs an interpolation operation using the edge components extracted by the extraction means for pixels located on the left and right of the target pixel. Processing equipment. The fixed period is a period of a staggered lattice of the pixel array of the input image,
The said calculation means performs the interpolation calculation using the edge component extracted by the said extraction means of the pixel located in the upper, lower, right, and left of the said attention pixel, The one of Claim 1 to 3 characterized by the above-mentioned. Image processing device.   The calculating means uses the edge components extracted by the extracting means for pixels located above and below the pixel of interest, or the edge components extracted by the extracting means for pixels located on the left and right of the pixel of interest. The image processing apparatus according to claim 6, wherein whether to use is switched for each pixel.   When the difference between the edge components extracted by the extraction unit for the pixels located above and below the target pixel is larger than the difference between the edge components extracted by the extraction unit for the pixels located on the left and right sides of the target pixel, The image processing apparatus according to claim 7, wherein the calculation unit performs an interpolation operation using the edge components of pixels located on the left and right of the target pixel.   When the difference between the edge components extracted by the extraction unit for the pixels located above and below the target pixel is smaller than the difference between the edge components extracted by the extraction unit for the pixels located on the left and right sides of the target pixel, The image processing apparatus according to claim 7, wherein an interpolation operation is performed using the edge components of pixels located above and below the pixel.   Depending on the resolution of the input image, the extraction unit extracts edge components of pixels of a certain period included in the input image, or the extraction unit extracts edge components of all pixels included in the input image. The image processing apparatus according to claim 1, further comprising a switching unit that switches whether or not to perform.   2. The image processing apparatus according to claim 1, further comprising an output unit that outputs an image in which the edge component extracted by the extraction unit or the edge component obtained by the calculation unit is added to the value of each pixel of the input image. The image processing apparatus according to claim 10. An extraction step of extracting an edge component of a pixel having a certain period among the pixels included in the input image;
An image processing method comprising: calculating an edge component of a pixel of interest that has not been extracted in the extraction step by an interpolation operation using the edge component extracted in the extraction step of a pixel adjacent to the pixel of interest.   The program for functioning a computer as each means of the image processing apparatus as described in any one of Claims 1-11.