JP2005340900A - Signal processing apparatus and method, recording medium, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To smooth a portion other than an edge without impairing continuity and correlation of pixels when smoothing a portion other than the edge.
A horizontal processing direction component pixel extractor 31 includes a target pixel of the input image signal S _I, extracts the horizontal processing direction component pixels. Vertical processing direction component pixel extractor 34, and the target pixel of the input image signal S _I, extracts the vertical processing direction component pixels. The non-linear smoothing processing unit 32 performs non-linear smoothing processing on the target pixel in the horizontal direction using the horizontal processing direction component pixels, and generates an image signal _SFH . The flat rate calculation unit 35 calculates the flat rate of the target pixel in the vertical direction using the vertical processing direction component pixels. The mixing unit 33 mixes the input image signal S _I and the image signal S _FH that has been subjected to the non-linear smoothing process in the horizontal direction using the flat rate, and the image signal that has been subjected to the horizontal direction smoothing process on the target pixel. S Output _NL-H .
[Selection] FIG.

Description

  The present invention relates to a signal processing apparatus and method, a recording medium, and a program, and in particular, when smoothing a portion other than the edge while retaining a sharp edge of a pixel constituting an image, The present invention relates to a signal processing apparatus and method, a recording medium, and a program that can smooth a portion other than the edge without impairing the correlation.

  Conventionally, in a video camera, the contrast (brightness difference) and sharpness (brightness of the boundary) of an image captured by an image sensor such as a CCD (Charge Coupled Device) or CMOS (Complementary Metal-Oxide Semiconductor) are improved. As a method, a contrast enhancement method by gradation conversion and a high frequency component enhancement method for enhancing the contrast of a high frequency component in an image are considered.

  As a contrast enhancement method, tone curve adjustment for converting each pixel level of an image with a function having a predetermined input / output relationship (hereinafter referred to as a level conversion function), or frequency distribution of pixel levels A method called histogram equalization has been proposed in which the level conversion function is adaptively changed according to the above.

  As a high frequency component enhancement method, a method called an unsharp mask that performs edge enhancement that extracts edges from an image and emphasizes the extracted edges has been proposed.

  However, in contrast enhancement methods, in addition to the problem that the contrast can be improved only in a part of the luminance range in the entire dynamic range (difference between the maximum level and the minimum level) of the image, in addition, in the case of tone curve adjustment However, there is a problem that the contrast is lowered in the brightest and darkest parts of the image, and in the case of histogram equalization, in the vicinity of the luminance region where the frequency distribution is small. Further, in the high frequency component enhancement method, only the contrast of the high frequency component of the image is enhanced, thereby causing a problem that the vicinity of the edge of the image is unnaturally enhanced and the image quality is deteriorated.

  Therefore, conventionally, an image signal processing device configured as shown in FIG. 1 is used to amplify a portion other than the edge by amplifying a portion other than the edge while preserving an edge having a sharp change in pixel value. There is a method for emphasizing other parts (for example, Patent Document 1).

In the image signal processing apparatus shown in FIG. 1, the input image signal (pixel value) is input to the ε filter 1 and the subtraction unit 2. The ε filter 1 has an edge larger than a predetermined threshold ε ₁ , for example, an edge as shown in FIG. 2B when an image signal slightly fluctuating across a steep edge as shown in FIG. 2A is input. Is converted into an extracted image signal and output to the subtracting unit 2 and the adding unit 4.

Specific processing of the ε filter 1 will be described with reference to FIGS. 3 and 4. The ε filter 1 sequentially determines each pixel of the input image as the target pixel C, and, as shown in FIG. 3, a plurality of neighboring pixels (in this case, four pixels L2) that are continuous in the horizontal direction with the target pixel C as the center. , L1, R1, R2), and the pixel values of the target pixel C and the plurality of neighboring pixels are set to tap coefficients (eg, {1, 2, 3, 2, 1}) to obtain a conversion result C ′ corresponding to the target pixel C.
C ′ = (1 × L2 + 2 × L1 + 3 × C + 2 × R1 + 1 × R2) / 9
... (1)

However, as shown in FIG. 4, for a neighboring pixel (a neighboring pixel R2 in the case of FIG. 4) whose difference from the pixel value of the target pixel C is larger than a predetermined threshold ε ₁ , the pixel value of the target pixel C is changed. Replace it with something to calculate. That is, in the case of FIG. 4, the following equation (2) is calculated.
C ′ = (1 × L2 + 2 × L1 + 3 × C + 2 × R1 + 1 × C) / 9
... (2)

  The image signal smoothed by the ε filter 1 is also referred to as a structural component of the image signal.

  Returning to FIG. The subtracting unit 2 subtracts the image signal input from the ε filter 1 from the image signal input from the previous stage (the same as the input to the ε filter 1), thereby slightly changing the components other than the structural components of the image signal. The extracted image signal is extracted and output to the amplifying unit 3. The amplification unit 3 amplifies the output of the subtraction unit 2 and outputs the amplified output to the addition unit 4. Note that the component of the image signal that slightly varies other than the structural component is also referred to as an amplitude component of the image signal. The adder 4 adds the image signal in which a part other than the structural component output from the amplifier 3 is amplified and the structural component of the image signal input from the ε filter 1. This addition result is an image signal in which a portion other than the edge is amplified while a steep edge is held.

By the way, in the ε filter 1 of the image signal processing device shown in FIG. 1, for example, as shown in FIG. 5, even if there is a steep edge in the change of the pixel value, the size is a predetermined threshold value ε _1. Is smaller than that, the converted image signal output from the ε filter 1 is smoothed as shown in FIG. 6 and the sharp edge disappears. There has been a problem that image quality deteriorates remarkably in pattern images and the like.

  Therefore, a signal that is continuously arranged is sequentially designated as a signal of interest, and a plurality of signals that are continuously arranged in the horizontal direction or the vertical direction with respect to the designated signal of interest as a reference. Determining a neighborhood signal, computing a difference between the signal of interest and each neighborhood signal, computing a smoothed signal by weighted averaging the neighborhood signal having the difference smaller than the first threshold, and the signal of interest; A difference from the neighboring signal is calculated, and a small edge is found in the vicinity of the signal of interest depending on whether or not there is a value larger than the second threshold value compared to the difference and the second threshold value smaller than the first threshold value. It has been proposed to determine whether or not it exists and to select one of a smoothed signal or a signal of interest based on the determination result.

JP 2001-298621 A

However, in the above method, processing is performed independently for the direction component in the horizontal direction or the vertical direction, and thus it may be difficult to extract an appropriate structural component. For example, as in the image shown in FIG. 7, an image in which a vertical thick line (upward left diagonal area) is placed on a horizontal thick line (upward diagonal hatched area). That is, when non-linear smoothing processing is performed on the image signal as shown in FIG. 7 independently in the horizontal direction as described above (using a signal made up of a pixel that is adjacent to the target pixel in the horizontal direction. For example, in the case of the non-linear smoothing process, in the broken line Line1, if the target pixel is indicated by a cross mark in FIG. 8, the change in the pixel value is as shown in FIG. That is, in the left part of FIG. 8, the difference between the pixel values of neighboring pixels does not fall within the threshold value ε ₁ in the vertical thick line part. The process is not performed. Therefore, the pixel value of the pixel on the vertical thick line does not change. The lower right part of FIG. 8 shows the amplitude component of the input signal, which is an image signal in which the sum of the amplitude component and the structural component is input.

On the other hand, in the broken line Line2, as shown in the left part of FIG. 9, the difference between the pixel values of neighboring vertical lines (pixels on the horizontal thick line) is the threshold ε. _Since it falls within ₁ , the smoothing process is performed by treating it as the amplitude component shown in the lower right part of FIG. Therefore, the pixel value of the pixel on the vertical thick line changes. For this reason, the pixel value of the pixel on the vertical thick line, which should have a constant pixel value, may vary due to the pixel value of the pixel in the horizontal direction. The grid-like area to be displayed should be shown as a thick line composed of the same pixel value in the vertical direction, but there is a problem that it is emphasized slightly black. Note that the upper right part of FIG. 9 shows the structural components of the input image signal.

  The present invention has been made in view of such a situation, and in particular, when smoothing a part other than the edge while retaining the steep edge of the pixel constituting the image, the continuity and correlation of the pixel are achieved. This makes it possible to smooth the portion other than the edge without impairing the properties.

  The signal processing apparatus according to the present invention sequentially arranges continuously arranged signals in the first direction with reference to the designation means for sequentially designating the attention signal and the attention signal designated by the designation means. Based on the comparison result between the processing signal extraction means for extracting a plurality of neighboring signals as processing signals from the processed signals, the processing difference that is the difference between the target signal and each processing signal, and the first threshold value, The smoothing signal and the signal of interest are calculated on the basis of the comparison result between the computing means for calculating the smoothed signal by weighted average of the signal of interest and the plurality of processed signals, and the processing difference and the second threshold value smaller than the first threshold value. A first mixing ratio calculating means for calculating the first mixing ratio and the first mixing ratio calculated by the first mixing ratio calculating means to mix the smoothed signal and the signal of interest. , First mixing means for generating a first mixed signal A reference signal extracting means for extracting a plurality of neighboring signals as reference signals from signals continuously arranged in the second direction with reference to the attention signal designated by the designation means; the attention signal and each reference A second mixing ratio calculating unit that calculates a second mixing ratio between the mixed signal and the signal of interest based on a reference difference that is a difference from the signal; and a second mixing ratio calculating unit that calculates the second mixing ratio calculating unit. And a second mixing unit configured to mix the first mixed signal and the signal of interest and generate a second mixed signal based on the mixing ratio.

  The signal may be a pixel value of a pixel constituting the image.

  According to the signal processing method of the present invention, a signal that is continuously arranged is sequentially designated in the first direction, with the designation step sequentially designating the signal of interest and the signal of interest designated in the processing of the designation step as a reference. A processing signal extraction step for extracting a plurality of neighboring signals as processing signals from among the signals arranged in the signal, a processing difference that is a difference between the signal of interest and each processing signal, and a comparison result with the first threshold value A smoothing signal based on a comparison result between a calculation step of calculating a smoothed signal by weighted averaging the target signal and a plurality of processed signals, and a processing difference and a second threshold smaller than the first threshold A first mixing ratio calculation step for calculating a first mixing ratio with the signal of interest, a smoothing signal based on the first mixing ratio calculated in the processing of the first mixing ratio calculation step, and the attention Mix the signals, the first mixed signal A plurality of neighboring signals are extracted as reference signals from signals continuously arranged in the second direction with reference to the first mixing step for generating the signal of interest specified in the processing of the specifying step. A reference signal extraction step, a second mixing ratio calculation step for calculating a second mixing ratio of the mixed signal and the signal of interest based on a reference difference that is a difference between the signal of interest and each reference signal, A first mixing signal and a second mixing step of mixing the signal of interest and generating a second mixing signal based on the second mixing ratio calculated in the processing of the mixing ratio calculation step. It is characterized by.

  The recording medium program of the present invention is based on a designation control step for sequentially controlling a signal that is continuously arranged as a signal of interest, and a signal of interest designated in the processing of the designation control step. A processing signal extraction control step for controlling extraction of a plurality of neighboring signals as processing signals from among signals continuously arranged in the first direction, and a processing difference that is a difference between the target signal and each processing signal And a calculation control step for controlling the calculation of a smoothed signal obtained by weighted averaging the signal of interest and the plurality of processing signals based on the comparison result with the first threshold, and a processing difference and a second smaller than the first threshold. The first mixture ratio calculation control step for controlling the calculation of the first mixture ratio between the smoothed signal and the signal of interest based on the comparison result with the threshold value of the first and the first mixture ratio calculation control step. First mixed The first mixing control step for controlling the generation of the first mixed signal by mixing the smoothed signal and the target signal based on the ratio, and the target signal specified in the processing of the specified control step as a reference A reference signal extraction control step for controlling extraction of a plurality of neighboring signals as reference signals from among signals continuously arranged in the second direction, and a reference difference that is a difference between the signal of interest and each reference signal A second mixing ratio calculation control step for controlling the calculation of the second mixing ratio between the mixed signal and the signal of interest based on the second mixing ratio, and the second mixing ratio calculated in the processing of the second mixing ratio calculation control step And a second mixing control step for controlling generation of the second mixed signal by mixing the signal of interest based on the first mixed signal.

  The program according to the present invention includes a designation control step for controlling a process of sequentially designating signals that are continuously arranged as a signal of interest, and a signal of interest designated in the process of the designation control step. A processing signal extraction control step for controlling extraction of a plurality of neighboring signals as processing signals from among the signals continuously arranged in the direction of, a processing difference that is a difference between the target signal and each processing signal, A calculation control step for controlling the calculation of a smoothed signal obtained by weighted averaging the signal of interest and a plurality of processed signals based on a comparison result with a threshold of 1, a processing difference and a second threshold smaller than the first threshold; The first mixing ratio calculation control step for controlling the calculation of the first mixing ratio between the smoothed signal and the signal of interest based on the comparison result of the first and the first mixing ratio calculation control step calculated by the processing of the first mixing ratio calculation control step Based on a mixing ratio of 1 The first mixing control step for controlling the generation of the first mixed signal by mixing the smoothed signal and the target signal, and the second signal based on the target signal specified in the processing of the specified control step. A reference signal extraction control step for controlling extraction of a plurality of neighboring signals as reference signals from among signals continuously arranged in the direction of the reference signal, and a reference difference that is a difference between the target signal and each reference signal The second mixing ratio calculation control step for controlling the calculation of the second mixing ratio between the mixed signal and the signal of interest, and the second mixing ratio calculated in the processing of the second mixing ratio calculation control step. And a second mixing control step for controlling generation of a second mixed signal by mixing the first mixed signal and the signal of interest.

  In the signal processing apparatus and method, and the program of the present invention, continuously arranged signals are sequentially designated as attention signals, and continuously arranged in the first direction with reference to the designated attention signals. A plurality of neighboring signals are extracted as processed signals from the processed signals, and based on the result of comparison between the processing difference that is the difference between the target signal and each processed signal and the first threshold, The processed signal is weighted and averaged to calculate a smoothed signal. Based on the comparison result between the processed difference and the second threshold value smaller than the first threshold value, the first mixing ratio of the smoothed signal and the signal of interest is calculated. Based on the calculated first mixing ratio, the smoothed signal and the signal of interest are mixed to generate a first signal, and the second signal is generated in the second direction with reference to the specified signal of interest. Of continuously arranged signals A plurality of neighboring signals are extracted as reference signals from the reference signal, a second mixing ratio between the mixed signal and the signal of interest is calculated based on a reference difference that is a difference between the signal of interest and each reference signal, and the calculated second Based on the mixing ratio, the first mixed signal and the signal of interest are mixed to generate a second mixed signal.

  The signal processing apparatus of the present invention may be an independent apparatus or a block that performs signal processing.

  According to the present invention, when the portion other than the edge is smoothed while retaining the sharp edge of the pixel constituting the image, the portion other than the edge is smoothed without impairing the continuity and correlation of the pixel. It becomes possible to do.

  Embodiments of the present invention will be described below. The correspondence relationship between the invention described in this specification and the embodiments of the invention is exemplified as follows. This description is intended to confirm that the embodiments supporting the invention described in this specification are described in this specification. Therefore, although there is an embodiment which is described in the embodiment of the invention but is not described here as corresponding to the invention, it means that the embodiment is not It does not mean that it does not correspond to the invention. Conversely, even if an embodiment is described herein as corresponding to an invention, that means that the embodiment does not correspond to an invention other than the invention. Absent.

  Further, this description does not mean all the inventions described in this specification. In other words, this description is for the invention described in the present specification, which is not claimed in this application, that is, for the invention that will be applied for in the future or that will appear and be added by amendment. It does not deny existence.

  That is, the signal processing apparatus of the present invention is designated by designation means (for example, the horizontal direction smoothing processing unit 22 in FIG. 12) and designation means for sequentially designating continuously arranged signals as attention signals. Processing signal extraction means (for example, a horizontal processing direction component pixel extraction unit in FIG. 12) that extracts a plurality of neighboring signals as processing signals from signals continuously arranged in the first direction with reference to the target signal. 31) and an arithmetic operation for calculating a smoothed signal by weighted averaging of the target signal and the plurality of processing signals based on a comparison result between the processing difference that is a difference between the target signal and each processing signal and the first threshold value. The first mixing ratio of the smoothed signal and the signal of interest is calculated based on the comparison result between the means (for example, the non-linear filter 51 in FIG. 14) and the processing difference and the second threshold value smaller than the first threshold value. First mixing ratio calculating means ( For example, the smoothed signal and the signal of interest are mixed based on the first mixture ratio calculated by the mixture ratio detection unit 53) and the first mixture ratio calculation means in FIG. The first mixing unit (for example, the mixing unit 52 in FIG. 14) that generates the signal and a plurality of signals from among the signals continuously arranged in the second direction on the basis of the attention signal specified by the specifying unit Based on a reference signal extraction unit (for example, the vertical reference direction component pixel extraction unit 34 in FIG. 12) that extracts a neighborhood signal as a reference signal, and a reference difference that is a difference between the attention signal and each reference signal, the mixed signal and the attention Based on the second mixing ratio calculated by the second mixing ratio calculating means (for example, the flat rate calculating unit 35 in FIG. 12) for calculating the second mixing ratio with the signal and the second mixing ratio calculating means. The first mixed signal and the signal of interest Combined, a second mixing means for generating a second mixed signal (e.g., mixing section 33 in FIG. 12), characterized in that it comprises a.

  The signal processing method of the present invention is specified by a designation step (for example, the processing in step S11 in the flowchart of FIG. 16) and the designation step in which signals that are continuously arranged are sequentially designated as signals of interest. A processing signal extraction step (for example, the processing in step S12 in the flowchart of FIG. 16) for extracting a plurality of neighboring signals as processing signals from signals continuously arranged in the first direction with reference to the signal of interest; An arithmetic step (for example, calculating a smoothed signal by weighted averaging the target signal and the plurality of processed signals based on a comparison result between the processing difference that is a difference between the target signal and each processed signal and the first threshold value. , Based on the comparison result between the processing difference and the second threshold value smaller than the first threshold value, and the smoothed signal and the attention signal. Based on the first mixture ratio calculated in the first mixture ratio calculation step (for example, the process of step S33 in the flowchart of FIG. 18) and the first mixture ratio calculation step of calculating the first mixture ratio. The first mixing step (for example, the processing in step S36 in the flowchart of FIG. 18) that mixes the smoothed signal and the signal of interest to generate the first mixed signal and the processing in the designation step are specified. A reference signal extraction step (for example, the process of step S13 in the flowchart of FIG. 16) for extracting a plurality of neighboring signals as reference signals from signals continuously arranged in the second direction on the basis of the signal of interest; A second mixing ratio calculation step (for example, FIG. 5) that calculates a second mixing ratio between the mixed signal and the signal of interest based on a reference difference that is a difference between the signal of interest and each reference signal. 6) and the second mixture ratio calculated in the second mixture ratio calculation step, the first mixed signal and the signal of interest are mixed, and the second And a second mixing step for generating a mixed signal (for example, the process of step S17 in the flowchart of FIG. 16).

  FIG. 11 emphasizes the contrast and sharpness of other areas while leaving the edge portion of the image to which the present invention is applied, and further suppresses unnatural enhancement processing that occurs according to the processing direction. It is a figure which shows the structure of one Embodiment of a processing apparatus.

  The buffer 21 temporarily stores the input image signal and supplies it to the subsequent horizontal smoothing processing unit 22. The horizontal direction smoothing processing unit 22 performs non-linear smoothing processing on the target pixel in the horizontal direction using the neighboring pixel and the target pixel arranged in the horizontal direction with respect to the target pixel, and supplies the target pixel to the buffer 23. To do. The buffer 23 temporarily stores the image signal supplied from the horizontal direction smoothing processing unit 22 and sequentially supplies the image signal to the vertical direction smoothing processing unit 24. The vertical direction smoothing processing unit 24 performs non-linear smoothing processing on the pixel of interest using neighboring pixels and the pixel of interest arranged in the vertical direction with respect to the pixel of interest, and supplies the result to the buffer 25. The buffer 25 temporarily stores an image signal composed of pixels that are nonlinearly smoothed in the vertical direction, supplied from the vertical direction smoothing processing unit 24, and outputs the image signal to a subsequent device (not shown).

  Next, a detailed configuration of the horizontal direction smoothing processing unit 22 will be described with reference to FIG.

  The horizontal processing direction component pixel extraction unit 31 sequentially sets a target pixel from each pixel of the image signal stored in the buffer 21 and extracts a pixel necessary for the non-linear smoothing process corresponding to the target pixel. The data is output to the smoothing processing unit 32. More specifically, the horizontal processing direction component pixel extraction unit 31 extracts two pixels adjacent to the left and right in the horizontal direction as horizontal processing direction component pixels from the target pixel, and extracts the extracted four pixels and the target pixel. Each pixel value is supplied to the nonlinear smoothing processing unit 32. Note that the number of horizontal processing direction component pixels to be extracted is not limited to two adjacent pixels on the left and right with respect to the target pixel, but may be any pixel adjacent in the horizontal direction. Three pixels adjacent to the left and right may be provided, or one pixel adjacent to the target pixel in the left direction and three pixels adjacent in the right direction may be used.

  The non-linear smoothing processing unit 32 performs non-linear smoothing on the target pixel using the target pixel supplied from the horizontal processing direction component pixel extracting unit 31 and the horizontal processing direction component pixels that are two pixels adjacent to the left and right of the target pixel. Process and supply to the mixing unit 33. The configuration of the non-linear smoothing processing unit 32 will be described later with reference to FIG. Here, the non-linear smoothing process in the horizontal direction is a process in which the target pixel is non-linearly smoothed by a plurality of pixels adjacent to the target pixel in the horizontal direction. Similarly, the non-linear smoothing process in the vertical direction, which will be described later, is a process in which the target pixel is non-linearly smoothed by a plurality of pixels adjacent to the target pixel in the vertical direction.

  The vertical reference direction component pixel extraction unit 34 sequentially sets a pixel of interest from each pixel of the image signal stored in the buffer 21 and arranges pixels necessary for nonlinear smoothing processing corresponding to the pixel of interest. Pixels adjacent in the vertical direction different from the direction are extracted and output to the flat rate calculation unit 35. More specifically, the vertical reference direction component pixel extraction unit 34 extracts, as vertical reference direction component pixels, two adjacent pixels in the vertical direction with respect to the target pixel, and extracts each of the extracted four pixels and the target pixel. Are supplied to the flat rate calculator 35. The number of vertical reference direction component pixels to be extracted is not limited to two pixels vertically adjacent to the target pixel, but may be any pixel adjacent in the vertical direction. Three pixels may be adjacent to each other in the vertical direction, or one pixel adjacent to the target pixel in the upward direction and three pixels adjacent in the downward direction may be used.

  The flat rate calculation unit 35 calculates the absolute difference between the pixel values of the target pixel supplied from the vertical reference direction component pixel extraction unit 34 and the vertical reference direction component pixel, and calculates the maximum value of the difference absolute value. Is supplied to the mixing unit 33 as a flat rate. Here, the flat rate in the vertical direction indicates a change in the absolute value of the difference between the pixel values of the target pixel and the vertical reference direction component pixel. When the flat rate is large, the pixel value of the pixel near the target pixel is Indicates a non-flat image with large change and small correlation between pixels in the vertical direction (non-flat image with a large change in pixel value). Conversely, when the flat rate is small, the pixel near the target pixel It is shown that the image is a flat image (a flat image with a small change in pixel value) with a small change in pixel value and a large correlation between pixels in the vertical direction.

  Based on the vertical flat rate supplied from the flat rate calculation unit 35, the mixing unit 33 mixes the pixel values of the target pixel that has been subjected to the nonlinear smoothing process and the unprocessed target pixel, and performs the horizontal direction smoothing process. The resulting pixel is output to the subsequent buffer 23.

  Next, a detailed configuration of the vertical direction smoothing processing unit 24 will be described with reference to FIG.

  The vertical direction smoothing processing unit 24 basically replaces the horizontal processing and the vertical processing in the configuration of the horizontal direction smoothing processing unit 22 described above. That is, the vertical processing direction component pixel extraction unit 41 sequentially sets a target pixel from each pixel of the image signal stored in the buffer 23 and extracts a pixel necessary for the nonlinear smoothing process corresponding to the target pixel. To the non-linear smoothing processing unit 42. More specifically, the vertical processing direction component pixel extraction unit 41 extracts vertical processing direction component pixels each including two pixels vertically adjacent to the target pixel in the vertical direction, and the extracted four pixels and the target pixel. Are supplied to the non-linear smoothing processing unit 42. Note that the number of vertical processing direction component pixels to be extracted is not limited to two adjacent pixels above and below the target pixel, but may be any pixel adjacent in the vertical direction. Three pixels may be adjacent to each other in the vertical direction, or one pixel adjacent to the target pixel in the upward direction and three pixels adjacent in the downward direction may be used.

  The non-linear smoothing processing unit 42 uses the target pixel supplied from the vertical processing direction component pixel extraction unit 41 and the vertical processing direction component pixels that are two adjacent pixels above and below the target pixel in the vertical direction. Non-linear smoothing processing is performed and the result is supplied to the mixing unit 43. The configuration of the nonlinear smoothing processing unit 42 is the same as that of the nonlinear smoothing processing unit 32, and details thereof will be described later with reference to FIG.

  The horizontal reference direction component pixel extraction unit 44 sequentially sets a pixel of interest from each pixel of the image signal stored in the buffer 23 and arranges pixels necessary for nonlinear smoothing processing corresponding to the pixel of interest. Pixels adjacent in the horizontal direction different from the direction are extracted and output to the flat rate calculation unit 45. More specifically, the horizontal reference direction component pixel extraction unit 44 extracts two pixels adjacent to the left and right in the horizontal direction with respect to the target pixel, and sets the pixel values of the extracted four pixels and the target pixel to Flat. This is supplied to the rate calculator 45. Note that the number of pixels to be extracted is not limited to two pixels horizontally adjacent to the target pixel, but may be any pixel adjacent in the horizontal direction. For example, three pixels adjacent to the target pixel horizontally Alternatively, one pixel adjacent to the target pixel in the left direction and three pixels adjacent in the right direction may be used.

  The flat rate calculation unit 45 calculates the absolute difference between the pixel value of the pixel of interest supplied from the horizontal reference direction component pixel extraction unit 44 and the two pixels adjacent to the left and right of the pixel of interest, The maximum value of the absolute difference is supplied to the mixing unit 43 as a flat rate.

  Based on the horizontal flat rate supplied from the flat rate calculation unit 45, the mixing unit 43 mixes the pixel values of the target pixel that has undergone nonlinear smoothing processing and the unprocessed target pixel, and performs horizontal direction smoothing processing. The resulting pixel is output to the subsequent buffer 25.

  Next, a detailed configuration of the nonlinear smoothing processing unit 32 will be described with reference to FIG.

The non-linear filter 51 of the non-linear smoothing processing unit 32 holds a steep edge whose size is larger than the threshold value ε ₁ among the fluctuations of the pixels constituting the input image signal S _I, and a portion other than the edge , And the smoothed image signal S _LPF-H is output to the mixing unit 52.

The mixing ratio detection unit 53 calculates the mixing ratio based on a minute change among the fluctuations of the pixels constituting the input image signal S _I and supplies it to the mixing unit 52.

Mixing unit 52, an image signal S _I that is input that is not the image signal S _LPF-H and smoothing is smoothed, based on the mixture ratio supplied from the mix rate detector 53, were mixed, nonlinear A smoothed image signal S _FH is output.

An LPF (Low Pass Filter) 61 of the non-linear filter 51 is based on a control signal supplied from the control signal generator 62, and a horizontal processing direction component pixel that is two pixels adjacent to the left and right in the horizontal direction. Is used to smooth the target pixel and output the smoothed image signal S _LPF-H to the mixing unit 52. The control signal generator 62 calculates an absolute difference between pixel values of the target pixel and the horizontal processing direction component pixel, generates a control signal for controlling the LPF 61 based on the calculation result, and supplies the control signal to the LPF 61. As the nonlinear filter 51, for example, the above-described conventional ε filter 1 may be used.

  Next, image enhancement processing by the enhancement processing device 11 of FIG. 11 will be described with reference to the flowchart of FIG.

  In step S <b> 1, the horizontal direction smoothing processing unit 22 performs horizontal direction smoothing processing using image signals that are sequentially stored in the buffer 21.

  Here, with reference to the flowchart of FIG. 16, the horizontal direction smoothing process by the horizontal direction smoothing process part 22 is demonstrated.

  In step S11, the horizontal processing direction component pixel extraction unit 31 of the horizontal direction smoothing processing unit 22 sets a pixel of interest in the raster scan order. At the same time, the vertical reference direction component pixel extraction unit 34 similarly sets the target pixel in the raster scan order. The target pixel may be set in an order other than the raster scan order, but is set by the target pixel set by the horizontal processing direction component pixel extraction unit 31 and the vertical reference direction component pixel extraction unit 34. It is necessary to set so that the pixel of interest is the same.

  In step S12, the horizontal processing direction component pixel extraction unit 31 includes a total of five pixels including the horizontal processing direction component pixels which are neighboring pixels adjacent to the target pixel by two pixels in the horizontal direction (left and right direction) together with the target pixel. Pixel values are extracted from the buffer 21 and output to the nonlinear smoothing processing unit 32. For example, in the case shown in FIG. 17, the pixels L2, L1, C, R1, and R2 are extracted as the target pixel and the horizontal processing direction component pixel. In FIG. 17, the pixel C is the target pixel, the pixels L2 and L1 are two horizontal processing direction component pixels adjacent to the left side of the target pixel C, and the pixels R1 and R2 are the target pixel C. These are two horizontal processing direction component pixels adjacent to the right side.

  In step S13, the vertical reference direction component pixel extracting unit 34, together with the target pixel, includes a total of five pixels including vertical reference direction component pixels which are neighboring pixels adjacent to the target pixel by two pixels in the vertical direction (vertical direction). Pixel values are extracted from the buffer 21 and output to the flat rate calculator 35. For example, in the case shown in FIG. 17, the pixels U2, U1, C, D1, and D2 are extracted as the target pixel and the vertical reference direction component pixel. In FIG. 17, the pixel C is the target pixel, the pixels U2 and U1 are two vertical reference direction component pixels adjacent to the upper side of the target pixel C, and the pixels D1 and D2 are the target pixel C. These are vertical reference direction component pixels of two pixels adjacent to the lower side.

  In step S <b> 14, the nonlinear smoothing processing unit 32 performs nonlinear smoothing processing on the target pixel based on the target pixel and the horizontal processing direction component pixel supplied from the horizontal processing direction component pixel extraction unit 31.

  Here, with reference to the flowchart of FIG. 18, the nonlinear smoothing process by the nonlinear smoothing process part 32 is demonstrated.

  In step S31, the control signal generation unit 62 of the nonlinear filter 51 calculates a difference absolute value of pixel values between the target pixel and the horizontal processing direction component pixel. That is, in the case of FIG. 17, the control signal generator 62 determines the absolute difference value of pixel values of the target pixel C and the horizontal processing direction component pixels L2, L1, R1, and R2 that are neighboring pixels adjacent in the horizontal direction | C-L2 |, | C-L1 |, | C-R1 |, | C-R2 | are calculated.

In step S32, the low-pass filter 61 compares each difference absolute value calculated by the control signal generator 62 with a predetermined threshold value ε _2, and nonlinearly corresponds to the input image signal S _{I according} to the comparison result. Apply filtering processing. More specifically, the low-pass filter 61 corresponds to the target pixel C by performing a weighted average of the pixel values of the target pixel C and the horizontal processing direction component pixel using a tap coefficient, for example, as in Expression (1). The conversion result C ′ to be output is output to the mixing unit 52 as a smoothed image signal S _LPF-H . However, for a horizontal processing direction component pixel whose difference absolute value from the pixel value of the target pixel C is larger than the predetermined threshold value ε ₂ , the pixel value is replaced with the pixel value of the target pixel C and weighted average is performed. (For example, calculation is performed as shown in Expression (2)).

  In step S33, the mixture ratio detection unit 53 executes a minute edge determination process to determine whether or not a minute edge exists.

  Here, the minute edge determination processing will be described with reference to the flowchart of FIG.

In step S41, the mixture ratio detection unit 53 calculates a difference absolute value of the pixel value between the target pixel and each horizontal processing direction component pixel, and all the difference absolute values are smaller than the threshold value ε ₃ (<< ε ₂ ). It is determined whether or not the edge is small, and it is determined whether or not a minute edge exists based on the determination result.

That is, for example, as illustrated in FIG. 17, the mixture ratio detection unit 53 determines the absolute value of the difference between the pixel value of the target pixel C and the horizontal processing direction component pixels L2, L1, R1, and R2 adjacent in the horizontal direction. Is calculated, and it is determined whether all the difference absolute values are smaller than the threshold ε _{3, and when} it is determined that all the difference absolute values are all smaller than the threshold ε ₃ , the pixel values of the neighboring pixel and the target pixel In step S44, it is determined that no minute edge exists in the vicinity of the target pixel.

On the other hand, in step S41, among the calculated absolute difference value, if it is determined that even one has a threshold epsilon ₃ or more of, the process proceeds to step S42, the mix rate detector 53, one of the left and right of the pixel of interest absolute difference value between the target pixel and the horizontal processing direction component pixels on the side is smaller than all the threshold epsilon _3, and the difference between the horizontal processing direction component pixels and the pixel of interest in the left and right on the other side of the target pixel absolute values are all the threshold a is epsilon ₃ or more, and determines whether the sign of the difference between the horizontal processing direction component pixels and the pixel of interest in the left and right on the other side of the pixel of interest matches.

That is, the horizontal processing direction component pixels on the left and right sides of the target pixel C are, for example, the pixels L2 and L1 in FIG. 17, and the horizontal processing direction component pixels on the left and right sides of the target pixel C are the pixels in FIG. In the case of R2 and R1, the mixture ratio detection unit 53 has all the absolute differences between the horizontal processing direction component pixels on the left and right sides of the target pixel C and the target pixel C smaller than the threshold ε ₃ , and the target pixel a is C in the horizontal processing direction component pixels R1 of the right and left on the other side, R2 the difference absolute value between the target pixel C are all threshold epsilon ₃ or more and the horizontal processing direction component pixels of the left and right on the other side of the pixel of interest C It is determined whether or not the difference between R1 and R2 and the target pixel C is the same.

  For example, when it is determined that the above condition is satisfied, in step S43, the mixture ratio detection unit 53 determines that a minute edge exists in the vicinity of the target pixel.

  On the other hand, when it is determined in step S42 that the above condition is not satisfied, in step S44, the mixture ratio detection unit 53 determines that a minute edge does not exist in the vicinity of the target pixel.

For example, when the relationship between the target pixel C and the horizontal processing direction component pixels L2, L1, R1, and R2 is as shown in FIG. 20, the absolute difference value | L2− between the target pixel C and the left horizontal processing direction component pixels L2 and L1. C |, | L1-C | is smaller than the threshold epsilon _3, and the difference absolute value of the horizontal processing of the target pixel C and the right direction component pixels R1, R2 | R1-C | , | R2-C | is threshold epsilon _Since the sign of the difference (R1-C) and (R2-C) between the target pixel C and the right-side horizontal processing direction component pixels R1, R2 is equal (3 in this case, both) It is determined that a minute edge exists in the vicinity of the pixel C.

Also, for example, when the relationship between the target pixel C and the horizontal processing direction component pixels L2, L1, R1, and R2 is as shown in FIG. 21, the difference absolute value between the target pixel C and the left horizontal processing direction component pixels L2 and L1 | L2-C |, | L1- C | is smaller than the threshold epsilon _3, and the difference absolute value of the horizontal processing of the target pixel C and the right direction component pixels R1, R2 | R1-C | , | R2-C | is Although not less than the threshold value ε ₃ , the signs of the differences (R1−C) and (R2−C) between the target pixel C and the right horizontal processing direction component pixels R1 and R2 do not match (in this case, positive, Negative), it is determined that there is no minute edge in the vicinity of the target pixel C.

Furthermore, for example, when the relationship between the target pixel C and the horizontal processing direction component pixels L2, L1, R1, and R2 is as shown in FIG. 22, the target pixel C and the horizontal processing direction component pixel are on either side of the target pixel C. Are not all smaller than the threshold value ε ₃ , it is determined that no minute edge exists in the vicinity of the target pixel C.

  Thus, after determining whether or not a minute edge exists in the vicinity of the target pixel, the process returns to step S34 in FIG.

When the process of step S33 ends, in step S34, the mixture ratio detection unit 53 determines whether or not the determination result by the minute edge determination process in step S33 is “a minute edge exists in the vicinity of the target pixel C”. Determine. For example, when the determination result by the minute edge determination process is “a minute edge is present in the vicinity of the target pixel C”, in step S <b> 35, the mixture ratio detection unit 53 performs the horizontal non-linear filtering process on the image signal. the mix rate Mr _-H mixture ratios of S _LPF-H and the image signal is input S _I output to the mixing unit 52 as the maximum mix rate Mr _{-H max.} The maximum Mix rate Mr _{-H max} is the maximum value of the Mix rate Mr _-H, i.e., the difference absolute value between the maximum value and the minimum value of the dynamic range of pixel values.

In step S36, the mixing unit 52, based on the Mix rate Mr _-H supplied from the mix rate detector 53, the image signal is non-linearly smoothed by the image signal S _I and the non-linear filter 51 to be input S _{LPF- H} is mixed and output to the buffer 23 as a non-linearly smoothed image signal _SFH . More specifically, the mixing unit 52 calculates the following equation (3), mixing the image signal S _LPF-H which is nonlinear smoothing the image signal S _I and the non-linear filter input.

S _FH = S _I × Mr _−H / Mr _{−H max} + S _LPF−H × (1−Mr _−H / Mr _{−H max} )
... (3)
Here, Mr _-H is the Mix rate, and Mr _{-H max} is the maximum value of the Mix rate Mr _-H , that is, the absolute difference between the maximum value and the minimum value of the pixel values.

As shown in Expression (3), if the mix rate Mr _−H is large, the weight of the image signal S _LPF-H processed by the nonlinear filter 51 becomes small, and the input unprocessed image signal S _I The weight increases. Conversely, if the mix rate Mr- _H is small, that is, the smaller the absolute value of the pixel value difference between adjacent pixels in the horizontal direction, the weight of the image signal S _LPF-H processed by the nonlinear filter increases. The weight of the input unprocessed image signal is reduced.

Therefore, if the minute edge is detected, the Mix rate Mr _-H is maximized Mix rate Mr _{-H max,} substantially inputted image signal S _I is to be output as it is.

On the other hand, if it is determined in step S34 that “a minute edge does not exist”, in step S37, the mixture ratio detection unit 53 calculates the absolute value of the difference between the pixel value of the target pixel and each horizontal processing direction component pixel. The maximum value among the calculated absolute values of the differences is obtained as the mixing rate, which is the mix rate Mr _−H , and is output to the mixing unit 52, and the process proceeds to step S36.

That is, in the case of FIG. 17, the mixture ratio detection unit 53 determines the absolute difference values | C−L2 | and | C−L1 of the pixel values between the target pixel C and the horizontal processing direction component pixels L2, L1, R1, and R2. |, | C−R1 |, | C−R2 | are calculated, and the maximum value among the calculated absolute values of the differences is obtained as a mix rate Mr _−H that is a mixing ratio, and is output to the mixing unit 52.

That is, when there is no minute edge, the image signal S _LPF-H that has been subjected to nonlinear filtering according to the maximum value of the absolute difference of the pixel value between the target pixel and each horizontal processing direction component pixel, and the input image The image signal S _FH subjected to the nonlinear smoothing process is generated by mixing with the signal S _I, and when there is a minute edge, the input image signal S _I is output as it is.

As a result, in the non-linear smoother 32, since minute edge is to be detected the threshold epsilon ₃ as a reference, portions fine edge exists, as well as such is not performed the non-linear smoother, Even in the part where there is no edge, the pixel value that has been subjected to the nonlinear smoothing process according to the magnitude of the absolute value of the difference is mixed with the input image signal. It is possible to suppress a situation in which the image quality is significantly deteriorated by a simple pattern image or the like configured.

  Now, the description returns to the flowchart of FIG.

  In step S <b> 15, the flat rate calculation unit 35 calculates a difference absolute value of the pixel value between the target pixel and each vertical reference direction component pixel adjacent in the vertical direction to the target pixel. That is, in the case of FIG. 17, the flat rate calculation unit 35 calculates the absolute difference value | C−U2 | of the pixel value between the target pixel C and the vertical reference direction component pixels U2, U1, D1, and D2 adjacent in the vertical direction. , | C-U1 |, | C-D1 |, | C-D2 |.

In step S <b> 16, the flat rate calculation unit 35 obtains the absolute difference value that is the maximum value among the absolute difference values between the target pixel and each vertical reference direction component pixel adjacent in the vertical direction to the target pixel. This is supplied to the mixing unit 33 as a flat rate Fr- _V .

In step S17, the mixing unit 33 on the basis of the Flat rate Fr _-V supplied from the Flat rate calculation unit 35, an image signal S _I and the non-linear smoother 32 non-linear smoother by image signal input S _FH is mixed and output to the buffer 23 as an image signal S _NL-H subjected to horizontal processing and smoothing processing. More specifically, the mixing unit 33 calculates the following equation (4), mixing the image signal S _FH which is non-linearly smoothed by the image signal S _I and the non-linear smoother 32 is inputted .

S _NL-H = S _FH x Fr _-V / Fr _{-H max} + S _I x (1-Fr _-V / Fr _{-V max} )
... (4)
Here, Fr _-V is the flat rate in the vertical direction, and Fr _{-V max} is the maximum value of the vertical flat rate Fr _-V , that is, the absolute difference between the maximum value and the minimum value of the dynamic range of the pixel value. Value. Since the flat rate Fr- _V is the maximum value of the absolute difference between the vertical reference direction component pixel and the target pixel, the smaller the value, the vertical reference direction component pixel that is adjacent to the target pixel in the vertical direction. In this area, since the change in pixel value is small and the change in color is small visually, it can be said that it is a flat state (flat state). On the other hand, a large flat rate Fr- _V means that there is a large change between the pixels in the target pixel and the vertical reference direction component pixel area that is adjacent to the target pixel in the vertical direction, and the appearance is not flat. (Non-Flat state).

For this reason, as shown in the equation (4), if the flat rate Fr _−V is large, the weight of the image signal S _FH subjected to the nonlinear smoothing processing by the nonlinear smoothing processing unit 32 increases, and the input processing is performed. The weight of the non-image signal S _I becomes small. On the contrary, if the flat rate Fr- _V is small, that is, the absolute value of the pixel value difference between the pixels in the vertical direction is small, the weight of the image signal S _FH subjected to the non-linear smoothing process by the non-linear smoothing unit 32 is increased. decreases, the weight of the image signal S _I increases not treated inputted.

  In step S18, the horizontal processing direction component pixel extraction unit 31 determines whether all the pixels have been processed as the target pixel, that is, whether or not there is an unprocessed pixel. For example, all the pixels are determined as the target pixel. If it is determined that there is no processing, that is, there is an unprocessed pixel, the processing returns to step S11. If it is determined in step S18 that all pixels have been processed as the target pixel, that is, it is determined that there is no unprocessed pixel, the process ends, and the process in step S1 in FIG. 15 ends. Similarly, the vertical reference direction component pixel extraction unit 34 determines whether all the pixels have been processed as the target pixel, that is, whether or not there is an unprocessed pixel. The process may be terminated only when it is determined that it does not exist.

As a result, when processing is performed on the target pixel on the vertical thick line on the broken line Line2 in FIG. 7, the difference between the pixel values of the vertical reference direction component pixels adjacent to the target pixel in the vertical direction is absolute The image signal S _FH that has been subjected to nonlinear smoothing processing in the horizontal direction and the input image signal S _I are mixed in accordance with the flat rate Fr _-V in the vertical direction obtained from the value, thereby correlating in the vertical direction. strong, i.e., the vertical direction of the Flat rate Fr _-V is small, when the correlation in the vertical direction is strong, to increase the weight of the input image signal S _I, conversely, large vertical Flat rate Fr _-V When the correlation in the vertical direction is weak, by increasing the weight of the image signal S _FH that has been subjected to nonlinear filtering in the horizontal direction, the edges are conscious and the corresponding pixels are used according to the processing direction (neighboring pixels used for nonlinear smoothing processing). Is water for the pixel of interest Whether it is a pixel adjacent in the direction, or, depending on whether the pixels adjacent in the vertical direction) can be suppressed unnatural process. Therefore, for example, it is possible to suppress the occurrence of an unnatural image signal such as a grid-like region generated by the smoothing process shown in FIG. 10 on a vertical vertical line.

In the above description, the example of multiplying the pixel value by using the flat rate Fr _-V as a weighting factor as it is when mixing is described. However, the image signal S _FH that has been subjected to nonlinear filtering processing with other weighting factors according to the flat rate. it may be mixed by multiplying the respective input image signals S _I and. That is, for example, as shown in FIG. 23, the weighting factors W ₁ and W ₂ set according to the flat rate Fr _−V may be used to perform mixing using the following equation (5). .

S _NL-H = S _I × W ₁ + S _FH × W ₂
... (5)

Here, W ₂ is the weight coefficient of the image signal S _FH which is non-linear filtering processing in the horizontal direction, W ₁ is the weight coefficient of the image signal S _I that is input. Further, (W ₁ + W ₂ ) is the maximum value W _max (= 1) of the weight coefficient.

That is, in FIG. 23, the Flat rate Fr _-H is Fr1 smaller range (Fr _-V <Fr1), the weighting factor W ₁ is the maximum value W _max of the weighting factor, the weighting factor W ₂ is 0 . In the range where the flat rate Fr _-V is higher than Fr1 and lower than Fr2 (Fr1 ≤ Fr _-V ≤ Fr2), the weighting factor W ₁ decreases in proportion to the flat rate Fr _-V , and the weighting factor W ₂ Is increased in proportion to the flat rate Fr− _V , and (W ₁ + W ₂ ) is set to be the maximum value W _max (= 1) of the weight coefficient. Furthermore, the Flat rate Fr _-V is Fr2 larger range (Fr2 ≦ Fr _-V), the weight coefficient W ₁ is 0, the weighting factor W ₂ is the maximum value W _max of the weighting factor.

As a result, it is possible to nonlinearly smooth the image while accurately conscious of the presence or absence of edges. In the case where the Fr1 = Fr2, Flat, rate Fr _-V is, Fr1 (= Fr2) as the threshold when a image signal is outputted, whether an input image signal S _I, or, the non-linear smoother One of the image signals S _FH that has been switched is output after being switched.

  Here, it returns to the flowchart of FIG.

As described above, in step S _<b > 1, the horizontal direction smoothing processing unit 22 sequentially stores the image signal S _NL-H generated by the horizontal direction smoothing process in the buffer 23.

In step S _<b> 2, the vertical direction smoothing processing unit 24 performs vertical direction smoothing processing using the image signals S _NL-H that are sequentially stored in the buffer 23 and subjected to horizontal direction smoothing processing. The vertical direction smoothing process is a process in which the horizontal direction process and the vertical direction process in the horizontal direction smoothing process are interchanged, and the process content itself is the same.

  That is, in step S51, the vertical processing direction component pixel extraction unit 41 of the vertical direction smoothing processing unit 24 sets a pixel of interest in the raster scan order. At the same time, the horizontal reference direction component pixel extraction unit 44 similarly sets the target pixel in the raster scan order. The target pixel may be set in an order other than the raster scan order, but is set by the target pixel set by the vertical processing direction component pixel extraction unit 41 and the horizontal reference direction component pixel extraction unit 44. It is necessary to set so that the pixel of interest is the same.

  In step S52, the vertical processing direction component pixel extraction unit 41 includes a total of five pixels including the horizontal reference direction component pixels which are adjacent pixels adjacent to the target pixel by two pixels in the vertical direction (vertical direction) together with the target pixel. Pixel values are extracted from the buffer 23 and output to the nonlinear smoothing processing unit 42. For example, in the case shown in FIG. 17, the pixels U2, U1, C, D1, and D2 are extracted as the target pixel and the horizontal reference direction component pixel.

  In step S53, the horizontal reference direction component pixel extracting unit 44, together with the target pixel, includes a total of five pixels including horizontal reference direction component pixels which are neighboring pixels adjacent to the target pixel by two pixels in the horizontal direction (left and right direction). Pixel values are extracted from the buffer 23 and output to the flat rate calculation unit 45. For example, in the case shown in FIG. 17, the pixels L2, L1, C, R1, and R2 are extracted as the target pixel and the horizontal reference direction component pixel.

In step S54, the nonlinear smoothing processing unit 42 performs nonlinear smoothing processing on the target pixel based on the target pixel and the vertical processing direction component pixel supplied from the vertical processing direction component pixel extracting unit 41. Note that the non-linear smoothing process in step S54 is the same as the non-linear smoothing process in step S14 in FIG. 16 except that the relationship between the horizontal direction and the vertical direction is interchanged, and the description thereof is omitted. It shall be. Therefore, by this processing, the non-linear smoothing processing unit 42 outputs the image signal S _FV subjected to non-linear smoothing processing in the vertical direction to the mixing unit 43.

  In step S55, the flat rate calculation unit 45 calculates the absolute value of the difference between the pixel value of the target pixel and each horizontal reference direction component pixel adjacent to the target pixel in the horizontal direction. That is, in the case of FIG. 17, the flat rate calculation unit 45 calculates the difference absolute value | C−L2 | of the pixel value between the target pixel C and each horizontal reference direction component pixel L2, L1, R1, R2 adjacent in the horizontal direction. , | C-L1 |, | C-R1 |, | C-R2 |.

In step S56, the flat rate calculation unit 45 obtains the absolute difference value that is the maximum value among the absolute difference values between the target pixel and each horizontal reference direction component pixel that is adjacent to the target pixel in the horizontal direction. This is supplied to the mixing unit 43 as a flat rate Fr- _H .

In step S57, the mixing unit 43, based on the flat rate Fr- _H supplied from the flat rate calculation unit 45, receives the image signal S _NL that has been subjected to horizontal non-linear smoothing processing by the input horizontal direction smoothing processing unit 22. by _-H and non-linear smoother 42 by mixing the image signal S _FV which is nonlinear smoothing processing, and outputs an image signal S _O smoothed with the vertical direction of adjacent pixels to the buffer 25. More specifically, the mixing unit 43 calculates the following expression (6), and inputs the image signal S _NL-H that has been subjected to horizontal non-linear smoothing processing and the non _- linear smoothing processing unit 42 in the vertical direction. The image signal S _FV subjected to nonlinear smoothing is mixed.

S _O = S _FV × Fr _−H / Fr _{−H max} + S _NL−H × (1−Fr _−H / Fr _{−H max} )
... (6)
Here, Fr _-H is the flat rate in the horizontal direction, and Fr _{-H max} is the maximum value of the flat rate Fr _-H , that is, the absolute value of the difference between the maximum value and the minimum value of the dynamic range of the pixel value. . The flat rate Fr- _H is the maximum value of the absolute difference between each horizontal reference direction component pixel adjacent in the horizontal direction and the target pixel, so the smaller the value, the adjacent to the target pixel and the target pixel in the horizontal direction. In the neighboring pixel region, the change in pixel value is small and the change in color is small visually, so it can be said that it is a flat state (Flat state). On the other hand, a large flat rate Fr- _H means that there is a large change between the pixels in the target pixel and the horizontal reference direction component pixel area that is adjacent to the target pixel in the vertical direction, and it is not even in appearance. (Non-Flat state).

Therefore, as shown in Expression (6), if the flat rate Fr _−H is large, the weight of the image signal S _FV that has been nonlinearly smoothed in the vertical direction by the nonlinear smoothing processing unit 42 is increased, and horizontal smoothing is performed. The weight of the processed image signal S _NL-H is reduced. On the contrary, if the flat rate Fr- _H is small, that is, the difference absolute value of the pixel value between the pixels in the horizontal direction is small, the image signal S _FV subjected to nonlinear smoothing processing in the vertical direction by the nonlinear smoothing processing unit 32. And the weight of the input image signal S _NL-H that has been subjected to nonlinear smoothing processing in the horizontal direction is increased.

  In step S58, the vertical processing direction component pixel extraction unit 41 determines whether all pixels have been processed as the target pixel, that is, whether or not there is an unprocessed pixel. For example, all the pixels are set as the target pixel. If it is determined that there is no processing, that is, there is an unprocessed pixel, the processing returns to step S51. If it is determined in step S58 that all the pixels have been processed as the target pixel, that is, it is determined that there is no unprocessed pixel, the process ends, and the process in step S2 in FIG. 15 ends. Note that the horizontal reference direction component pixel extraction unit 44 also determines whether all pixels have been processed as the target pixel in the same processing, that is, whether or not there is an unprocessed pixel. The process may be terminated only when it is determined that there is no pixel.

As a result, the image signal S _FV smoothed in the vertical direction is input in accordance with the flat rate Fr _-H obtained from the difference in pixel value with the horizontal reference direction component pixel adjacent to the target pixel in the horizontal direction. When the generated image signal S _NL-H is mixed, the horizontal direction has a strong correlation, that is, the horizontal flat rate Fr _-H is small and the horizontal direction has a strong correlation. When the weight of the smoothed image signal S _NL-H is increased, the horizontal flat rate Fr _-H is large, and the horizontal correlation is weak, the weight of the image signal S _FV subjected to nonlinear filtering in the vertical direction By increasing the size, the edge pixel is conscious and the neighboring pixel used in the non-linear smoothing process is adjacent to the target pixel in the horizontal direction or adjacent in the vertical direction. Whether the pixel is Flip was) it is possible to suppress the unnatural process.

In the above description, the example of multiplying the pixel value by using the flat rate Fr _-H as a weighting factor as it is when mixing is described, but an image obtained by smoothing the weighting factor corresponding to the other flat rate Fr _-H The signal S _FV and the input image signal S _NL-H subjected to the horizontal direction smoothing may be multiplied and mixed. That is, as shown in FIG. 23 in the horizontal direction smoothing process described above, the following equation (7) is used, similarly to the case where the weighting factors W ₁ and W ₂ set according to the flat rate Fr _−H are used. In this manner, the image signal S _O subjected to the vertical direction smoothing process may be obtained.

S _O = S _NL-H × W ₁₁ + S _FV × W ₁₂
... (7)

Here, W ₁₂ is the weight factor of the image signal S _FV which is smoothed in a vertical direction, W ₁₁ is the weight coefficient of the image signal S _NL-H which is processed horizontally smoothed input. Further, (W ₁₁ + W ₁₂ ) is the maximum weighting factor value W _max (= 1).

  As a result, it is possible to non-linearly smooth an image generated with an accurate awareness of the presence or absence of edges.

  Now, the description returns to the flowchart of FIG.

  When the vertical direction smoothing process is executed in step S2, it is determined in step S3 whether or not the next image has been input. If it is determined that the next image has been input, the process is performed in step S1. Return to, and the subsequent processing is repeated. If it is determined in step S3 that the next image has not been input, that is, the image signal has ended, the processing ends.

  By the above processing, the unnatural smoothing processing according to the processing direction is obtained in each of the horizontal direction smoothing processing and the vertical direction smoothing processing, the flat rate is obtained, and the non-linear smoothing is performed according to the flat rate. Since the image signal and the image signal that has not been subjected to the nonlinear smoothing process are mixed, the edge of the image is left and the unnatural enhancement processing that occurs according to the processing direction is suppressed, while the contrast of the region other than the edge And sharpness can be emphasized.

  In the above description, the example in which the vertical direction smoothing process is performed after the horizontal direction smoothing process has been described. However, the horizontal direction smoothing process is performed after the vertical direction smoothing process. It may be. In addition, even if only the horizontal direction smoothing process or the vertical direction smoothing process is performed on the image signal, the flat rate corresponding to the processing direction is obtained in each process. It is possible to enhance the contrast and sharpness of the region other than the edge while leaving the edge portion of the image and suppressing the unnatural enhancement processing that occurs according to the processing direction.

  In the nonlinear smoothing processing unit 32 described above, an example in which nonlinear smoothing processing is performed using a pixel of interest and a horizontal processing direction component pixel for the pixel of interest has been described. For example, in the nonlinear smoothing processing, A vertical reference direction component pixel may be used together with the processing direction component pixel. FIG. 25 shows a configuration of the horizontal direction smoothing processing unit 22 that uses vertical reference direction component pixels together with horizontal processing direction component pixels. 25, components corresponding to those in FIG. 12 are denoted by the same reference numerals, and description thereof is omitted as appropriate.

25 differs from the horizontal direction smoothing processing unit 22 in FIG. 12 in that a non-linear smoothing processing unit 71 is provided instead of the non-linear smoothing processing unit 32 and the mixing unit 33 is deleted. Thus, the flat rate Fr _−V calculated by the flat rate calculation unit 35 is supplied to the non-linear smoothing processing unit 71. The non-linear smoothing processing unit 71 basically has the same function as that of the non-linear smoothing processing unit 32. Furthermore, in the non-linear smoothing processing, the vertical reference direction component pixel is used together with the horizontal processing direction component pixel. The difference is that the vertical reference direction component pixels supplied from the extraction unit 34 are used.

  Next, a detailed configuration of the nonlinear smoothing processing unit 71 will be described with reference to FIG. In the configuration of the non-linear smoothing processing unit 71 in FIG. 26, the same reference numerals are given to the configuration corresponding to the configuration of the non-linear smoothing processing unit 32 in FIG. 14, and the description thereof will be omitted as appropriate. To do.

The configuration of the nonlinear smoothing processing unit 71 in FIG. 26 differs from the configuration of the nonlinear smoothing processing unit 32 in FIG. 14 in that a control signal generating unit 81 is provided instead of the control signal generating unit 62. The basic function of the control signal generation unit 81 is the same as that of the control signal generation unit 62, but in addition to the horizontal processing direction component pixels, the flat rate Fr- _V supplied from the flat rate calculation unit 35 is used. The difference is that a control signal is generated.

  Next, the non-linear smoothing process by the horizontal direction smoothing process part 22 of FIG. 25 is demonstrated with reference to the flowchart of FIG. Note that the processing of steps S73 to S77 in the flowchart of FIG. 27 is the same as the processing of steps S33 to S37 in the flowchart of FIG.

  In step S71, the control signal generation unit 62 of the nonlinear filter 51 calculates a difference absolute value of pixel values between the target pixel and the horizontal processing direction component pixel.

In step S72, the low-pass filter 61 compares the difference absolute value between the target pixel C calculated by the control signal generation unit 81 and the pixel values of the plurality of horizontal processing direction component pixels with a predetermined threshold ε ₂ . Corresponding to this comparison result, if the threshold value ε ₂ is greater than the threshold value ε ₂ , the first processing to be replaced with C is performed, and then supplied to the respective pixels on which the first processing has been performed by the flat rate calculation unit 35. taking into account the coming Flat rate Fr-v, performing nonlinear filtering processing on the image signal S _I.

More specifically, for example, when the image signals of the left and right pixels centering on the target pixel C are the signals L2, L1, C, R1, and R2, the LPF 61 controls the control signal generated by the control signal generator 81. When the difference from the pixel value of the target pixel C is larger than the threshold value, a process of replacing with C is performed, and two horizontal signals L2 ′, L1 ′, C, Assume that R1 'and R2' are generated. Further, the LPF 61 performs a process considering the flat rate Fr- _V on each of the signals L2 ′, L1 ′, C, R1 ′, and R2 ′.

That is, for example, in the case of the signal L2 ′, the signal L2 ″ obtained by the processing considering the flat rate Fr _−V can be obtained by calculating the following equation (8).

L2 ”＝ W _C × C + W _L2 × L2 '
... (8)

Here, in Expression (8), W _L2 and W _C are the same as the weights in Expression (5) described above, and W _{C corresponds} to W ₁ and W _L2 corresponds to W ₂ (W _L2 + W _C = 1). Therefore, when the flat rate Fr-v is small, the weight W _C is the maximum value (= 1), and the signal L2 ″ is C (L2 ″ = C).

  In addition, since the same processing is performed on the pixel signal, each of the signals L1 ′, R1 ′, and R2 ′ is calculated by the following equations (9) to (11). Is done.

L1 ”＝ W _C × C + W _L1 × L1 '
... (9)

R1 ”＝ W _C × C + W _R1 × R1 '
(10)

R2 ”＝ W _C × C + W _R2 × R2 '
(11)

The weights obtained here are substantially the same as W _L2 = W _L1 = W _R2 = W _R1 since the flat rate Fr _{−V in} the vertical direction of the target pixel C is used. Therefore, when the flat rate Fr-v is smaller than the predetermined threshold, L1 "= C, R1" = C, R2 "= C.

  As a result, in such a case, the LPF 61 performs the horizontal direction smoothing process by calculating the following expression (12) using the expression (1).

C ′ = (1 × C + 2 × C + 3 × C + 2 × C + 1 × C) / 9
= C
(12)

  As a result, when the flat rate Fr-v is small, the input image signal is output as it is. On the other hand, when the flat rate Fr-v is large, a value subjected to filtering processing based on the value of the pixel in the horizontal direction as shown in Expression (1) is used.

As described above, since it is possible to perform processing in consideration of the flat rate in the non-linear smoothing process, it is possible to delete the subsequent mixing unit 33 (as a result, from the non-linear smoothing processing unit 71 Output signal S _FH = signal S _NL-H output from the horizontal direction smoothing processing unit 22).

  The above processing realizes non-linear filtering processing that considers not only the horizontal correlation of the pixel of interest but also the vertical correlation, so that unnatural processing according to the processing direction can be performed while being aware of edges. It becomes possible to suppress.

  In FIG. 26, as an example of the configuration of the non-linear smoothing processing unit 71, an example in which the control signal generating unit 81 performs non-linear filtering processing in consideration of vertical reference direction component pixels in addition to horizontal processing direction component pixels will be described. However, when detecting a minute edge, a vertical reference direction component pixel may be used in addition to a horizontal processing direction component pixel.

FIG. 28 shows the configuration of the non-linear smoothing processing unit 71 that uses vertical reference direction component pixels in addition to horizontal processing direction component pixels when calculating the mix rate Mr _−H . In the configuration of the non-linear smoothing processing unit 71 in FIG. 28, the same reference numerals are given to the configuration corresponding to the configuration of the non-linear smoothing processing unit 32 in FIG. 14, and the description thereof will be omitted as appropriate. To do.

The configuration of the non-linear smoothing processing unit 71 in FIG. 28 is different from the configuration of the non-linear smoothing processing unit 32 in FIG. 14 in that a mixing ratio detecting unit 91 is provided instead of the mixing ratio detecting unit 53. The basic function of the mixing ratio detection unit 91 is the same as that of the mixing ratio detection unit 53, but in addition to the horizontal processing direction component pixels, the flat rate Fr _-V supplied from the flat rate calculation unit 35 is used. Find the mix rate Mr'- _H .

  Next, the minute edge determination processing by the nonlinear smoothing processing unit 71 in FIG. 28 will be described with reference to the flowchart in FIG. Note that the processing of steps S81 to S86 in the flowchart of FIG. 29 is the same as the processing of steps S31 to S36 in the flowchart of FIG.

In step S87, the mixture ratio detection unit 91 calculates a difference absolute value of each pixel value between the target pixel and each horizontal processing direction component pixel, and calculates the maximum value among the calculated difference absolute values. After _obtaining the mix rate Mr _−H (the mix rate Mr _−H is obtained by the same method as the processing in steps S35 and S37), the flat rate Fr _−V is used to calculate the mix rate Mr ′ _−H . Obtained and output to the mixing unit 52.

More specifically, the mixing ratio detection unit 91 calculates the following formula (13) to obtain the mix rate Mr ′ _−H using the flat rate Fr _−V .

Mr ' _-H = W _A × Mr _-Hmax + W _B × Mr _-H
... (13)

Here, the weights W _A and W _B correspond to the weights W ₁ and W ₂ in the equation (5), respectively. The mix rate Mr _-Hmax is the maximum value of the mix rate Mr _-H . When the flat rate Fr- _V is small, the mix rate _Mr' - _H = Mr- _{H max} , and when it is large, Mr-H _' = Mr-H. Therefore, for example, when the flat rate Fr- _V is small, the signal S _FH = S _{I is obtained} from the result of the equation (3), and the input signal S _I is output as it is.

With the above processing, it is possible to add vertical information at the time of the mixing unit of the nonlinear smoothing processing unit, and consider not only the horizontal processing direction component pixel corresponding to the target pixel but also the vertical reference direction component pixel. Therefore, it is possible to set the mix rate Mr _-H, and it is possible to mix the image signal S _LPF-H subjected to the non-linear filtering process more naturally and the input image signal S _I, and the result As a result, it is possible to suppress unnatural processing according to the processing direction while being aware of the edge.

  Similarly, the vertical direction smoothing processing unit 24 may use not only the vertical processing direction component pixels but also the horizontal reference direction component pixels. FIG. 30 shows a configuration of the vertical direction smoothing processing unit 24 that uses not only the vertical processing direction component pixels but also the horizontal reference direction component pixels. In the configuration of the vertical direction smoothing processing unit 24 in FIG. 30, the same reference numerals are given to the configuration corresponding to the configuration of the nonlinear smoothing processing unit 24 in FIG. 13, and the description thereof will be omitted as appropriate. And

30 differs from the vertical direction smoothing processing unit 24 in FIG. 13 in that a non-linear smoothing processing unit 101 is provided instead of the non-linear smoothing processing unit 42, and the mixing unit 33 is deleted. Thus, the flat rate Fr _−H calculated by the flat rate calculation unit 45 is supplied to the nonlinear smoothing processing unit 101. The non-linear smoothing processing unit 101 basically has the same function as the non-linear smoothing processing unit 42, but further, in the non-linear smoothing processing, together with the vertical processing direction component pixel, the flat rate calculation unit 45. The difference is that the flat rate Fr- _H supplied more is used.

  The configuration of the non-linear smoothing processing unit 101 is the same as that of the non-linear smoothing processing unit 71 shown in FIGS. 26 and 28 except that the horizontal processing is vertical processing. Omitted. The processing is the same except that the processing in the horizontal direction and the processing in the vertical direction are interchanged, and a description thereof will be omitted.

  According to the signal processing apparatus, method, and program of the present invention, signals that are continuously arranged are sequentially designated as attention signals, and are continuously arranged in the first direction with reference to the designated attention signals. A plurality of neighboring signals are extracted as processed signals from the processed signals, and based on a comparison result between the processing difference that is a difference between the target signal and each processed signal and the first threshold, A smoothed signal is calculated by weighted averaging the processed signals, and a first mixing ratio between the smoothed signal and the signal of interest is calculated based on a comparison result between the processed difference and a second threshold value smaller than the first threshold value. Based on the calculated first mixing ratio, the smoothed signal and the signal of interest are mixed to generate a first mixed signal, and continuously in the second direction based on the specified signal of interest Multiple neighboring signals from among the signals placed in Based on the reference difference that is the difference between the signal of interest and each reference signal, the second mixture ratio of the mixed signal and the signal of interest is calculated, based on the calculated second mixture ratio, Since the first mixed signal and the signal of interest are mixed to generate the second mixed signal, the portion other than the edge is smoothed while maintaining the sharp edge of the pixels constituting the image. At this time, it is possible to smooth portions other than the edges without impairing the continuity and correlation of the pixels.

  The series of processes described above can be executed by hardware, but can also be executed by software. When a series of processes is executed by software, a program constituting the software may execute various functions by installing a computer incorporated in dedicated hardware or various programs. For example, it is installed from a recording medium in a general-purpose personal computer or the like.

  FIG. 31 shows a configuration of an embodiment of a personal computer when the electrical internal configuration of the enhancement processing device 11 of FIG. 11 is realized by software. The CPU 201 of the personal computer controls the overall operation of the personal computer. Further, when a command is input from the input unit 206 such as a keyboard or a mouse from the user via the bus 204 and the input / output interface 205, the CPU 201 is stored in a ROM (Read Only Memory) 202 correspondingly. Run the program. Alternatively, the CPU 201 reads a program read from the magnetic disk 221, the optical disk 222, the magneto-optical disk 223, or the semiconductor memory 224 connected to the drive 210 and installed in the storage unit 208 into a RAM (Random Access Memory) 203. To load and execute. Thereby, the function of the above-described enhancement processing device 11 of FIG. 11 is realized by software. Further, the CPU 201 controls the communication unit 209 to communicate with the outside and exchange data.

  As shown in FIG. 31, the recording medium on which the program is recorded is distributed to provide the program to the user separately from the computer, and the magnetic disk 221 (including the flexible disk) on which the program is recorded, By a package medium composed of an optical disk 222 (including compact disc-read only memory (CD-ROM), DVD (digital versatile disk)), a magneto-optical disk 223 (including MD (mini-disc)), or a semiconductor memory 224 In addition to being configured, it is configured by a ROM 202 on which a program is recorded, a hard disk included in the storage unit 208, and the like provided to the user in a state of being incorporated in a computer in advance.

  In this specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in time series in the order described, but of course, it is not necessarily performed in time series. Or the process performed separately is included.

It is a block diagram which shows the structural example of the image signal processing apparatus which emphasizes parts other than an edge in the state which preserve | saved the steep edge in an image. It is a figure which shows the image signal input into the epsilon filter of FIG. 1, and the output image signal. It is a figure which shows an example of the tap used with the epsilon filter of FIG. It is a figure for demonstrating operation | movement of the epsilon filter of FIG. It is a figure for demonstrating the problem of the epsilon filter of FIG. It is a figure for demonstrating the problem of the epsilon filter of FIG. It is a figure for demonstrating the problem of the epsilon filter of FIG. It is a figure for demonstrating the problem of the epsilon filter of FIG. It is a figure for demonstrating the problem of the epsilon filter of FIG. It is a figure for demonstrating the problem of the epsilon filter of FIG. It is a block diagram which shows the structure of one Embodiment of the emphasis processing apparatus to which this invention is applied. It is a block diagram which shows the structure of the horizontal direction smoothing process part of FIG. It is a block diagram which shows the structure of the vertical direction smoothing process part of FIG. It is a block diagram which shows the structure of the nonlinear smoothing process part of FIG. It is a flowchart explaining an image enhancement process. It is a flowchart explaining the horizontal direction smoothing process by the image enhancement apparatus of FIG. It is a figure explaining the horizontal direction smoothing process by the image enhancement apparatus of FIG. It is a flowchart explaining the nonlinear smoothing process by the image enhancement apparatus of FIG. 12 is a flowchart for explaining minute edge determination processing by the image enhancement apparatus in FIG. 11. It is a figure explaining the fine edge determination process by the image enhancement apparatus of FIG. It is a figure explaining the fine edge determination process by the image enhancement apparatus of FIG. It is a figure explaining the fine edge determination process by the image enhancement apparatus of FIG. It is a figure explaining the other method of the setting method of the weight by the image enhancement apparatus of FIG. It is a figure explaining the vertical direction smoothing process by the image enhancement apparatus of FIG. It is a block diagram which shows the other structure of the horizontal direction smoothing process part of FIG. It is a block diagram which shows the structure of the nonlinear smoothing process part of FIG. It is a flowchart explaining the nonlinear smoothing process by the image enhancement apparatus provided with the nonlinear smoothing process part of FIG. It is a block diagram which shows the other structure of the nonlinear smoothing process part of FIG. It is a flowchart explaining the nonlinear smoothing process by the image enhancement apparatus provided with the nonlinear smoothing process part of FIG. It is a block diagram which shows the other structure of the vertical direction smoothing process part of FIG. It is a figure explaining a medium.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 11 Enhancement processing apparatus, 22 Horizontal direction smoothing process part, 24 Vertical direction smoothing process part, 31 Horizontal process direction component pixel extraction part, 32 Nonlinear smoothing process part, 33 Mixing part, 34 Vertical reference direction component pixel extraction part, 35 Flat rate calculation unit, 41 Vertical processing direction component pixel extraction unit, 42 Non-linear smoothing processing unit, 43 Mixing unit, 44 Horizontal reference direction component pixel extraction unit, 45 Flat rate calculation unit, 51 Non-linear filter, 52 Mixing unit, 53 Mixing ratio detection unit, 61 LPF, 62 control signal generation unit, 71 nonlinear smoothing processing unit, 81 control signal generation unit, 91 mixing ratio detection unit, 101 nonlinear smoothing processing unit

Claims (5)

In a signal processing device that adjusts the level of a signal that is continuously arranged,
Designating means for sequentially designating the signals arranged continuously as a signal of interest;
Processing signal extraction means for extracting a plurality of neighboring signals as processing signals from among the signals continuously arranged in the first direction with reference to the signal of interest designated by the designation means;
Calculation means for calculating a smoothed signal by weighted averaging the signal of interest and the plurality of processed signals based on a comparison result between a processing difference that is a difference between the signal of interest and each processed signal and a first threshold value. When,
First mixing ratio calculation means for calculating a first mixing ratio between the smoothed signal and the signal of interest based on a comparison result between the processing difference and a second threshold value smaller than the first threshold value; ,
First mixing means for mixing the smoothed signal and the signal of interest based on the first mixing ratio calculated by the first mixing ratio calculating means to generate a first mixed signal; ,
Reference signal extraction means for extracting a plurality of neighboring signals as reference signals from among the signals continuously arranged in the second direction on the basis of the attention signal designated by the designation means;
Second mixing ratio calculating means for calculating a second mixing ratio between the mixed signal and the signal of interest based on a reference difference that is a difference between the signal of interest and each reference signal;
Based on the second mixing ratio calculated by the second mixing ratio calculation means, the first mixing signal and the signal of interest are mixed to generate a second mixing signal. And a signal processing apparatus. The signal processing apparatus according to claim 1, wherein the signal is a pixel value of a pixel constituting an image. In a signal processing method for adjusting the level of a continuously arranged signal,
A designating step of sequentially designating the signals arranged continuously as a signal of interest;
A processing signal extraction step of extracting a plurality of neighboring signals as processing signals from among the signals continuously arranged in the first direction on the basis of the target signal specified in the processing of the specifying step;
An operation step of calculating a smoothed signal by weighted averaging the attention signal and the plurality of processing signals based on a comparison result between a processing difference that is a difference between the attention signal and each processing signal and a first threshold value. When,
A first mixture ratio calculating step for calculating a first mixture ratio between the smoothed signal and the signal of interest based on a comparison result between the processing difference and a second threshold value smaller than the first threshold value; ,
Based on the first mixing ratio calculated in the processing of the first mixing ratio calculation step, the smoothing signal and the signal of interest are mixed to generate a first mixing signal. Steps,
A reference signal extraction step for extracting a plurality of neighboring signals as reference signals from among the signals continuously arranged in the second direction on the basis of the attention signal designated in the processing of the designation step;
A second mixing ratio calculating step of calculating a second mixing ratio between the mixed signal and the signal of interest based on a reference difference that is a difference between the signal of interest and each reference signal;
A second mixing signal is generated by mixing the first mixing signal and the signal of interest based on the second mixing ratio calculated in the processing of the second mixing ratio calculation step. A signal processing method comprising the steps of: A program for adjusting the level of continuously arranged signals,
A designation control step for controlling a process of sequentially designating the signals arranged continuously as a signal of interest;
A processing signal for controlling extraction of a plurality of neighboring signals as processing signals from among the signals continuously arranged in the first direction with the attention signal specified in the processing of the specifying control step as a reference An extraction control step;
Computation for controlling computation of a smoothed signal obtained by weighted averaging the attention signal and the plurality of processing signals based on a comparison result between a processing difference that is a difference between the attention signal and each processing signal and a first threshold value. Control steps;
First mixing ratio calculation for controlling calculation of a first mixing ratio between the smoothed signal and the signal of interest based on a comparison result between the processing difference and a second threshold value smaller than the first threshold value Control steps;
A first control unit that controls generation of a first mixed signal by mixing the smoothed signal and the signal of interest based on the first mixing ratio calculated in the processing of the first mixing ratio calculation control step; 1 mixing control step;
A reference signal for controlling extraction of a plurality of neighboring signals as reference signals from among the signals continuously arranged in the second direction with the attention signal designated in the processing of the designation control step as a reference An extraction control step;
A second mixing ratio calculation control step for controlling calculation of a second mixing ratio between the mixed signal and the target signal based on a reference difference that is a difference between the target signal and each reference signal;
Control generation of the second mixed signal by mixing the first mixed signal and the signal of interest based on the second mixing ratio calculated in the processing of the second mixing ratio calculation control step. A recording medium on which a computer-readable program is recorded, comprising: a second mixing control step. A program for adjusting the level of continuously arranged signals,
A designation control step for controlling a process of sequentially designating the signals arranged continuously as a signal of interest;
A processing signal for controlling extraction of a plurality of neighboring signals as processing signals from among the signals continuously arranged in the first direction with the attention signal specified in the processing of the specifying control step as a reference An extraction control step;
Computation for controlling computation of a smoothed signal obtained by weighted averaging the attention signal and the plurality of processing signals based on a comparison result between a processing difference that is a difference between the attention signal and each processing signal and a first threshold value. Control steps;
First mixing ratio calculation for controlling calculation of a first mixing ratio between the smoothed signal and the signal of interest based on a comparison result between the processing difference and a second threshold value smaller than the first threshold value Control steps;
A first control unit that controls generation of a first mixed signal by mixing the smoothed signal and the signal of interest based on the first mixing ratio calculated in the processing of the first mixing ratio calculation control step; 1 mixing control step;
A reference signal for controlling extraction of a plurality of neighboring signals as reference signals from among the signals continuously arranged in the second direction with the attention signal designated in the processing of the designation control step as a reference An extraction control step;
A second mixing ratio calculation control step for controlling calculation of a second mixing ratio between the mixed signal and the target signal based on a reference difference that is a difference between the target signal and each reference signal;
Control generation of the second mixed signal by mixing the first mixed signal and the signal of interest based on the second mixing ratio calculated in the processing of the second mixing ratio calculation control step. A program that causes a computer to execute processing including the second mixing control step.

Images