JP6350362B2 - Computer program and image processing apparatus - Google Patents

Description

  The present specification relates to image processing for executing correction processing on image data generated by optically reading a document.

  Japanese Patent Application Laid-Open No. 2004-228561 discloses a technique for performing a process of removing a background on image data obtained by reading a document using a scanner. In this technique, it is determined whether or not the background is to be removed based on the document type and the instruction from the user. If it is determined that the background is to be removed, processing for removing the background is executed.

JP 2008-205652 A

  However, in the above technique, when the process of removing the background is executed on the image data, there is a possibility that the value of the pixel in the object such as a character different from the background is changed. In this case, for example, there may be a problem that an unintended object is removed from the image by the process of removing the background. Such a problem is not limited to the process of removing the background, but is a problem common to the correction process of changing the value of the pixel in the image.

  This specification is based on the fact that an unintended pixel value in an object is changed when a correction process for changing the pixel value is performed on image data generated by optically reading a document. Disclosed is a technology that can reduce the malfunctions.

  The technology disclosed in this specification can be implemented as the following application examples.

[Application Example 1] A first object including a plurality of pixels having gradation values within a specific range, and a second object different from the first object and having gradation values outside the specific range. An original image data representing an original image including the second object including a plurality of pixels, and an acquisition function for acquiring the original image data generated by optically reading a document; A correction function for generating corrected image data representing a corrected image including the second object by executing a specific correction process on the original image data, wherein the specific correction process includes: Using the correction function and the original image data, which is a process of changing the value of a pixel having a gradation value within a specific range to a specific gradation value, the first object in the original image Contour An extraction function for extracting, and a generation function for generating output image data representing an output image using the corrected image data, wherein the output image includes a contour image indicating a contour of the first object; A computer program that causes a computer to realize the generation function including the second object in the corrected image and an output function for outputting the output image data.

  According to the above configuration, output image data representing an output image obtained by combining the contour image indicating the contour of the first object including the pixel having the gradation value within the specific range and the corrected image is output. . As a result, it is possible to provide the user with an output image that allows the user to distinguish and recognize the first object whose color is changed by the specific correction process and the second object included in the corrected image. As a result, it is possible to reduce inconvenience caused by a change in the value of a pixel in an unintended object by a specific correction process.

  The technique disclosed in the present specification can be realized in various forms, for example, in the form of a recording medium on which a computer program is recorded, an image processing apparatus, an image reading apparatus control apparatus, an image processing method, and the like. Can be realized.

1 is a block diagram illustrating configurations of a computer 200 and a multifunction peripheral 100 as image processing apparatuses in an embodiment. It is a flowchart of the scanning process of 1st Example. It is a flowchart of the scanning process of 1st Example. It is a figure which shows an example of the instruction | indication input screen WI1 of 1st Example. It is a figure which shows an example of the image used by a present Example. It is a figure which shows an example of the image used by a present Example. It is a figure which shows an example of the histogram of R component. It is a figure which shows an example of UI screen WI2. It is a flowchart of the color correction process of 2nd Example. It is a figure which shows an example of the instruction input screen WI3 of 2nd Example.

A. First embodiment:
A-1: Configuration of Image Processing Device Next, an embodiment will be described based on examples. FIG. 1 is a block diagram illustrating a configuration of a computer 200 and an MFP 100 as image processing apparatuses in the embodiment.

  The computer 200 is, for example, a personal computer, and includes a CPU 210 as a controller of the computer 200, a nonvolatile storage device 220 such as a hard disk drive, a volatile storage device 230 such as a RAM, and an operation unit 260 such as a mouse and a keyboard. , A display unit 270 such as a liquid crystal display, and a communication unit 280. The computer 200 is communicably connected to an external device such as the multifunction peripheral 100 via the communication unit 280.

  The volatile storage device 230 provides a buffer area 231 that temporarily stores various intermediate data generated when the CPU 210 performs processing. The nonvolatile storage device 220 stores a computer program CP. In this embodiment, the computer program CP is a scanner driver program for controlling the scanner installed in the multifunction peripheral 100, and is provided in a form downloaded from a server. Instead, the computer program CP may be provided in a form stored on a DVD-ROM or the like. The CPU 210 executes a scan process to be described later by executing the computer program CP.

  The multi-function device 100 includes a scanner unit 120 that generates scan data by optically reading a document using an image sensor, a printer unit 130 that prints an image using a predetermined method (for example, inkjet or laser), and a scanner unit. 120 and a control unit 110 including a CPU and a memory for controlling the printer unit 130.

A-2: Scan Process FIGS. 2 and 3 are flowcharts of the scan process of the first embodiment. This scan process is started based on a user start instruction when the computer program CP is executed and the scan driver is activated in the multi-function device 200, for example.

  In S <b> 10, the CPU 210 displays an instruction input screen WI <b> 1 for inputting an instruction regarding scan processing on the display unit 270. FIG. 4 is a diagram illustrating an example of the instruction input screen WI1. The instruction input screen WI4 in FIG. 4 includes a check box CB and three buttons BT1 to BT3. The check box CB is an input element for inputting an instruction indicating whether or not to perform background removal processing on scan data to be generated. When the detailed setting button BT3 is pressed, a screen (not shown) for inputting detailed instructions regarding scanning processing such as the designation of resolution, storage file format, and compression method is displayed. The CPU 210 acquires an instruction related to the scan process including an instruction to execute the background removal process via the instruction input screen WI1.

  For example, the user places a document on the scanner unit 120 of the multifunction peripheral 100 and presses the start button BT1 on the instruction input screen WI1. When the start button BT1 is pressed, the CPU 210 proceeds to S15, and when the cancel button BT2 is pressed, the process is interrupted.

  In S <b> 15, the CPU 210 generates scan data indicating a document using the multifunction device 100 by transmitting a scan command to the multifunction device 100, and acquires the generated scan data from the multifunction device. The acquired scan data is stored in the buffer area 231. The image represented by the scan data acquired in S15 is also referred to as an original image. The scan data includes a value indicating each color of a plurality of pixels in the image. The scan data of the present embodiment is RGB image data that indicates a color for each pixel with an RGB value. The gradation value of each color component of the RGB value is, for example, a value of 256 gradations that takes any value from 0 to 255.

  5 and 6 are diagrams illustrating examples of images used in the present embodiment. The original image SI in FIG. 5A is an example of an image represented by scan data. The original image SI includes a background BG, a drawing G1, characters T1 and T2, and solid areas B1 and B2. The background BG includes a portion UE having a color close to white but having a color different from white. The part UE is generated due to, for example, the texture of the document, the characteristics of the image sensor or the light source of the scanner unit 120, and the like. The drawing G1 is an image showing a pie chart, and the color of the portion PG having a color (for example, light gray) which is different from the color of the background BG but relatively close to the background BG and the color of the background BG is relatively different. And a portion (a portion other than PG). The solid areas B1 and B2 are monochromatic areas and have a color (for example, light gray) that is different from the background BG but is relatively close to the background BG. The character T1 has a white color and is arranged in the solid region B1. The character T2 has a color (for example, red or black) that is relatively different from the color of the background BG. A color that is relatively close to the background BG is a color within the background color range RA, which will be described later, and a color that is relatively different from the color of the background BG is a color outside the background color range RA.

  In S20, it is determined whether or not an instruction to execute background removal processing is input by the user. Specifically, when the check button CB is checked when the start button BT1 is pressed on the instruction input screen WI1, it is determined that the background removal process execution instruction is input, and the check is performed. If it has not been input, it is determined that no background removal processing execution instruction has been input.

  When the execution instruction for the background removal process is not input (S20: NO), in S25, the CPU 210 outputs scan data and ends the scan process. For example, the scan data is output to the nonvolatile storage device 220 and stored in the nonvolatile storage device 220.

  When an instruction to execute background removal processing is input (S20: YES), in S30, CPU 210 performs background removal processing on the scan data to generate corrected image data. The generated corrected image data is stored in the buffer area 231 separately from the scan data.

  The background removal process will be described. The CPU 210 generates a histogram of scan data. The color component histogram is data obtained by classifying each pixel of the original image data into a plurality of classes according to the gradation value of the color component of the pixel. In the present embodiment, a histogram is generated for each of the three color components of the RGB values, with each of the 256 gradation values as one class. FIG. 7 is a diagram illustrating an example of an R component histogram. The horizontal axis in FIG. 7 indicates the gradation value of the R component, and the vertical axis indicates the number of pixels having each gradation value.

  CPU210 determines the background color value which shows the color of background BG using a histogram. Normally, the background BG is more uniform and wider than the objects such as the characters T1 and the drawing G1, so the peak corresponding to the background BG is the highest peak in the histogram of each color component. . For this reason, in this embodiment, the highest peak of the histogram of each color component, that is, the color values (Rm, Gm, Bm) indicated by the mode values Rm, Gm, Bm of the RGB color components are the background color values. As determined. FIG. 7 shows the mode value Rm of the R component.

  The CPU 210 changes the RGB value of the pixel having the RGB value in the background color range RA based on the background color value among the plurality of pixels in the original image SI to an RGB value indicating white. Then, the CPU 210 maintains the original RGB values without changing the RGB values of the pixels having the RGB values outside the background color range RA among the plurality of pixels in the original image SI. As a result, corrected image data representing the corrected image is generated. Here, the background color range RA satisfies (Rm−ΔV) ≦ R ≦ (Rm + ΔV), (Gm−ΔV) ≦ G ≦ (Gm + ΔV), and (Bm−ΔV) ≦ B ≦ (Bm + ΔV). It is a range. The white value is, for example, an RGB value of (255, 255, 255). FIG. 7 illustrates a range RAr for the R component in the background color range RA. ΔV is, for example, a value of about 10 to 20 when the range of gradation values of each color component is 0 to 255. In the modification, for example, (Rm−ΔV1) ≦ R ≦ (Rm + ΔV2), the R, G, and B gradation values Rm, Gm, and Bm indicating the background color value to the upper limit and the lower limit. The width may be different. In this embodiment, the value of the pixel having the RGB value outside the background color range RA is not changed. Instead, the RGB value outside the background color range RA is closer to the background color range RA. You may change the value of a pixel so that it may approach white. In this case, it is possible to suppress the appearance of an unnatural boundary line between the white region and the color region different from white in the corrected image.

  FIG. 5B shows an example of the corrected image AI. The background BGa of the corrected image AI has become a white monochrome region by the background removal process. That is, the portion UE having a color different from white is removed from the original image SI by the background removal process. The corrected image AI includes the character T2 of the original image SI and the drawing G1a obtained by removing the portion PG from the drawing G1, but does not include other objects. That is, the solid regions B1 and B2, the character T1b, and the portion PG of the drawing G1 are removed from the original image SI by the background removal process. This is because the RGB values of the pixels constituting the solid regions B1 and B2 and the portion PG of the drawing G1 are included in the background color range RA, and thus are changed to white values by the background removal process. Thus, the object may be removed from the image against the user's intention by the background removal process.

  In S35 of FIG. 2, the CPU 210 performs edge extraction processing for extracting edge pixels on the original image data, thereby generating first edge image data indicating edge pixels in the original image SI. The first edge image data is a binary image indicating one of a plurality of pixels in the original image SI, that is, ON indicating that the pixel is an edge pixel and OFF indicating that the pixel is not an edge pixel. It is data. Specifically, the CPU 210 performs gray conversion processing on the original image data to generate gray image data of the original image SI. The gray conversion process is a process of converting RGB values of a plurality of pixels in an image into a 256-gradation luminance value Y indicating luminance using a known formula. The CPU 210 calculates the edge intensity of each pixel by applying a Sobel operator to the value of each pixel of the gray image data of the original image SI. Then, the CPU 210 generates a first edge image data by determining a pixel having an edge strength equal to or higher than a predetermined threshold as an edge pixel and determining a pixel having an edge strength lower than the predetermined threshold as a non-edge pixel. To do.

  In the edge extraction process described above, the edge pixel detection accuracy can be improved by performing the gray conversion process. As a modification, the CPU 210 generates edge image data of each color component of RGB by applying a Sobel operator to the value of each color component of RGB without performing gray conversion processing. Also good. Then, the CPU 210 may generate the first edge image data by taking a logical sum (OR) of the edge image data of the RGB color components. In the edge extraction process described above, other edge detection operators such as a Prewitt operator and a Roberts Cross operator may be used instead of the Sobel operator. .

  In S <b> 40 of FIG. 2, the CPU 210 generates second edge image data indicating edge pixels in the corrected image AI by performing the above-described edge extraction processing on the corrected image data. The second edge image data is a binary value that indicates one of a plurality of pixels in the corrected image AI, ie, ON indicating that the pixel is an edge pixel and OFF indicating that the pixel is not an edge pixel. Image data.

  5C and 5D show an example of the first edge image EI1 represented by the first edge image data and the second edge image EI2 represented by the second edge image data. It is shown. Black lines in the edge images EI1 and EI2 indicate edges constituted by a plurality of edge pixels, and white areas indicate areas constituted by a plurality of pixels that are not edge pixels. The first edge image EI1 includes a drawing G1 in the original image SI, characters T1 and T2, and edges G1e, T11e, T12e, T2e, B1e, and B2e of the solid areas B1 and B2. Edges G1e, T11e, T12e, T2e, B1e, and B2e indicate the contours of the corresponding objects in the original image. The second edge image EI2 includes the drawing G1a and the edges G1ae and T2e of the character T2 in the corrected image AI. In other words, the second edge image EI2 includes the drawing G1a that is not removed by the background removal process and the edges G1ae and T2e of the character T2, but the part PG of the drawing G1 that is removed by the background removal process, the character T1, It does not include the edges of the solid areas B1 and B2.

  In S50 of FIG. 2, the CPU 210 generates difference image data representing a difference image indicating a difference between the first edge image EI1 and the second edge image EI2. Specifically, the CPU 210 keeps the pixels corresponding to the edge pixels in the second edge image among the plurality of edge pixels (that is, ON pixels) in the first edge image as edge pixels. To do. Then, the CPU 210 changes a pixel corresponding to a non-edge pixel (that is, an OFF pixel) in the second edge image from a plurality of edge pixels in the first edge image from the edge pixel to the non-edge pixel. Change, that is, change the value of the pixel from ON to OFF. Thereby, difference image data is generated.

  As can be seen from the above description, the original image SI, the corrected image AI, the first edge image EI1, the second edge image EI2, and the difference image DI are images having the same size, that is, the number of pixels in the vertical direction. The images have the same number of pixels in the horizontal direction, and the pixels of these images have a one-to-one correspondence.

  FIG. 6A shows an example of the difference image DI represented by the difference image data. The difference image DI includes the edge PGe of the part PG of the drawing G1, the edge T1e of the character T1, and the solid areas B1, B2 among the edges G1e, T11e, T12e, T2e, B1e, B2e in the first edge image EI1. Edge B1e, B2e. In other words, the difference image DI includes the portion PG of the drawing G1 to be removed by the background removal process, the character T1, and the edges PGe, T11e, T12e, B1e, and B2e of the solid areas B1 and B2, but the background removal process Does not include the edge of the drawing G1a and the character T2 that are not removed by. Thus, the difference image DI shows the position of the edge pixel that forms the contour of the object to be removed by the background removal process.

  In S55, the CPU 210 generates composite image data using the original image data acquired in S15, the corrected image data generated in S30, and the difference image data generated in S50. . Specifically, the CPU 210 corresponds the RGB value of the pixel corresponding to the edge pixel (that is, the ON pixel) of the difference image DI among the plurality of pixels in the corrected image AI in the original image SI. Change to the RGB value of the pixel. Further, the CPU 210 does not change the RGB value of the pixel corresponding to the non-edge pixel (that is, the OFF pixel) of the difference image DI among the plurality of pixels in the corrected image AI. Thereby, composite image data is generated.

  As a modification, after performing a correction process (for example, a known expansion process) for thickening the contour of the object indicated in the differential image DI on the differential image data, the differential image data after the correction process is processed. Used to generate composite image data. In this way, it is possible to make it easier to visually recognize the contour image in a composite image CI described later.

  FIG. 6B shows an example of the composite image CI represented by the composite image data. As described above, since the difference image DI indicates the position of the edge pixel constituting the contour of the object to be removed by the background removal process, the composite image CI indicates the contour of the object to be removed by the background removal process. The contour image is an image synthesized with the corrected image AI. Accordingly, the composite image CI includes the drawing G1a and the character T2 in the corrected image AI, and the outline PGd, T11d, T12d, B1d, and B2d of the drawing PG, the character T1, and the solid areas B1 and B2 in the original image SI. And an outline image including Each of the plurality of pixels constituting the contours PGd, T1d, B1d, and B2d has the RGB value of the corresponding pixel in the corrected image AI. Accordingly, the contours PGd, T11d, T12d, B1d, and B2d have the colors of the corresponding objects in the original image SI, that is, the colors of the portion PG of the drawing G1, the character T1, and the solid areas B1 and B2.

  In S60 of FIG. 2, the CPU 210 performs a labeling process on the difference image data. Specifically, the CPU 210 assigns one identifier to a group of pixels including a continuous series of edge pixels, and assigns each identifier group to a plurality of pixel groups each including a plurality of edge pixels separated from each other. Assign a different identifier. A pixel group to which one identifier is assigned is specified as one outline. Thereby, in the difference image DI of FIG. 6A, edges PGe, B1e, B2e, T11e, and T12e are specified as contours. As a result, five contours PGd, T11d, T12d, B1d, and B2d are specified in the composite image CI.

  In S65, the CPU 210 converts the labeled differential image data, which is raster data, into vector data. This is because vector data generally has a smaller amount of data. This step may be omitted when the capacity of the buffer area 231 has a sufficient margin.

  In S70, the CPU 210 displays the UI screen WI2 including the composite image CI on the display unit 270. FIG. 8 is a diagram illustrating an example of the UI screen WI2. The UI screen WI2 includes a composite image CI, identifiers ID1 to ID5 of five contours PGd, T11d, T12d, B1d, and B2d in the composite image CI, and five contours PGd, T11d, T12d, B1d, and B2d, respectively. And pull-down menus PM1 to PM5 corresponding to, and buttons BT4 and BT5. Each of the pull-down menus PM1 to PM5 is an input element for the user to input an instruction for each corresponding contour. Options that can be selected from each of the pull-down menus PM1 to PM5 include three instructions, that is, an instruction for contour display, an instruction for restoration, and an instruction for removal.

  Since the composite image CI includes a contour image including five contours PGd, T11d, T12d, B1d, and B2d, the user unintentionally removes the background image by viewing the composite image CI. Recognizable objects. The user can input an instruction by operating the corresponding pull-down menus PM1 to PM5 for each of the five contours PGd, T11d, T12d, B1d, and B2d. The contour display instruction is an instruction to display only the contour. The removal instruction is an instruction to remove the object and the contour. The restoration instruction is an instruction to restore the object.

  When the print button BT4 or the save button BT5 is pressed on the UI screen WI2, the CPU 210 advances the process to S75.

  In S75, the CPU 210 obtains an input instruction for each of the plurality of contours PGd, T11d, T12d, B1d, and B2d in the composite image CI. That is, the CPU 210 acquires the instructions selected in the pull-down menus PM1 to PM5 when the print button BT4 or the save button BT5 is pressed.

  In S80, the CPU 210 selects one target contour from the plurality of contours PGd, T11d, T12d, B1d, and B2d in the composite image CI.

  In S85, CPU 210 determines whether the instruction input for the target contour is a removal instruction, a restoration instruction, or a contour display instruction.

  If the input instruction is a removal instruction, in S90, CPU 210 removes the contour of interest. Specifically, the CPU 210 specifies a plurality of pixels constituting the target contour in the composite image CI using vector data indicating the labeled difference image DI. The CPU 210 changes the RGB values of the plurality of specified pixels to the RGB values of the corresponding pixels of the corrected image AI. As a result, the synthesized image CI is in a state where the target contour is not synthesized.

  If the input instruction is a restoration instruction, in S95, the CPU 210 restores an object in the original image SI (also referred to as a noticed object) having a noticed outline in the composite image CI. Specifically, the CPU 210 specifies a restoration area where the object of interest is to be restored in the composite image CI, and converts the RGB values of a plurality of pixels in the restoration area to the RGB values of the corresponding pixels in the original image SI. By changing the target object, the target object is restored in the composite image CI.

  The restoration area is identified as follows, for example. First, the CPU 210 specifies a plurality of pixels constituting the target contour in the composite image CI using vector data indicating the labeled difference image DI. Then, the CPU 210 specifies a plurality of pixels included in the circumscribed rectangle of the contour to be noticed and not constituting the noticed contour as non-contour pixels. Then, the CPU 210 specifies a region that is formed by a continuous series of non-contour pixels and that does not contact the four sides of the circumscribed rectangle of the target contour as a restoration region. However, when the contour of interest does not form a closed loop like the arc-shaped contour PGd of the composite image CI in FIG. 6B, the CPU 210 cannot identify the restoration area by the above method. . For this reason, when the restoration area cannot be specified by the above method, the CPU 210 determines that the target contour does not form a closed loop, and specifies the restoration area by the following method, for example. First, the CPU 210 calculates the color of the adjacent region adjacent to both sides of the target contour using the values of a plurality of corresponding pixels in the original image SI. The CPU 210 determines that an area in which the color of the area is close to the background color value (Rm, Gm, Bm) of the original image SI among the adjacent areas on both sides of the contour is the background area, and restores the opposite area. It is determined that the region should be the power side. Using the edge image EI1, the CPU 210 identifies a region including a region to be restored and configured by a continuous series of non-edge pixels as a restoration region. The specified restoration area is an area surrounded by a plurality of edge pixels forming a closed loop in the edge image EI1. As a modification, for example, when the restoration area cannot be specified, the CPU 210 displays on the UI screen WI2 that the restoration area cannot be automatically specified, and receives an instruction to specify the restoration area from the user. An area may be specified. For example, the user can input an instruction to designate a restoration area on the composite image CI in the UI screen WI2 using a pointing device such as a mouse.

  If the input instruction is contour display, the CPU 210 skips S90 and S95, and proceeds to S100. This is because the composite image CI already includes an outline, and in this case, there is no need to change the composite image CI.

  In S100, the CPU 210 determines whether all contours have been processed. If there is an unprocessed contour (step S100: NO), the CPU 210 returns to S80 and selects the unprocessed contour as a newly focused contour. When all the contours have been processed (step S100: YES), the CPU 210 advances the process to S105. At this time, processed composite image data representing the processed composite image ACI that has been processed for the contour in the composite image CI based on the user's instruction is generated.

  FIG. 6C shows an example of the processed composite image ACI. In this processed composite image ACI, for example, the contour PGd of the composite image CI drawn is maintained based on the contour display instruction. In the processed composite image ACI, the objects T1 and B1 in the original image SI having the contours T11d, T12d, and B1d of the composite image CI are restored based on the restoration instruction. In the processed composite image ACI, the outline B2d of the composite image CI is erased based on the erase instruction. As described above, since appropriate processing is executed for each object based on the user's instruction, the CPU 210 can generate processed composite image data representing the processed composite image ACI in accordance with the user's intention. .

  In S105, the CPU 210 outputs processed composite image data in accordance with an instruction input via the UI screen WI2 of FIG. Specifically, when the print button BT4 on the UI screen WI2 is pressed, the CPU 210 generates print data using the processed composite image data, and transmits the print data to the multifunction peripheral 100. As a result, the processed composite image ACI printed on the sheet by the multifunction peripheral 100 is output. When the save button BT5 on the UI screen WI2 is pressed, the CPU 210 converts the processed composite image data into an image file having a predetermined format, for example, an image file having a JPEG (Joint Photographic Experts Group) format. To the non-volatile storage device 220. As a result, the processed composite image data is stored in the nonvolatile storage device 220.

  According to the first embodiment described above, the difference image data representing the difference image DI is generated using the original image data (S35 to S50 in FIG. 2), so that the object (which is removed by the background removal process) ( The contours PGe, T11e, T12e, B1e, and B2e (FIG. 6A) of the object including a plurality of pixels having RGB values within the background color range RA are extracted. Then, using the corrected image data, the contour image showing the contours PGd, T11d, T12d, B1d, and B2d of the first object and the object in the corrected image AI, that is, the object (first object that is not removed by the background removal process) Composite image data representing a composite image CI including images indicating G1a and T2 (also referred to as object 2) is generated (55 in FIG. 2). It can be said that the second object is an object including a plurality of pixels having RGB values outside the background color range RA. By using the generated composite image data, the composite image CI is output (that is, displayed) to the display unit 270 by including the composite image CI in the UI screen WI2 (S70 in FIG. 3). As a result, the composite image CI, which is an output image that allows the user to distinguish and recognize the first object whose color is changed and removed by the background removal process, and the second object in the corrected image AI, is Can be provided.

  For example, even if the corrected image AI is provided to the user, since the first object removed from the original image SI is not clearly shown in the corrected image AI, the user cannot recognize that there is the first object. there is a possibility. Further, even if the difference image DI is provided to the user, the second object remaining in the corrected image AI is not clearly shown in the difference image DI, and therefore the user may not be able to recognize the second object. . In addition, even if the first edge image EI1 is provided to the user, the first object and the second object are not distinguished in the edge image EI1 because the first object is not distinguished from the second object. Cannot be recognized separately. In the composite image CI, the first object and the second object are clearly distinguished and can be used to reduce problems caused by changing the pixel values in the unintended object by the background removal process. For example, by leaving the outline, an output image acceptable to the user can be provided to the user. Alternatively, since the user can recognize that the first object is removed, the user can take a countermeasure such as inputting a restoration instruction to restore the first object.

  More specifically, in S35, first edge image data is generated using the original image data, and in S40, second edge image data is generated using the corrected image data. The contour of the first object is extracted using the first edge image data and the second edge image data. As a result, the outline of the first object can be extracted with high accuracy using the first edge image data and the second edge image data.

  In S50, by generating difference data representing the difference image DI, no corresponding edge pixel in the second edge image EI2 exists among the plurality of edge pixels in the first edge image EI1. A region constituted by pixels is extracted as the outline of the first object. As a result, the region where the contour image is to be synthesized can be easily specified using the first edge image data and the second edge image data.

  Further, in the first embodiment, the contours PGd, T11d, T12d, B1d, and B2d of the first object in the composite image CI have the same color as the first objects PG, T1, B1, and B2 in the original image SI. Have. As a result, the color in the original image SI of the first object is expressed in the composite image CI. As a result, the user viewing the composite image CI can recognize the color of the first object before being changed by the background removal process.

  Furthermore, in the first embodiment, the second objects G1a and T2 that are not removed by the background removal process have the same color as the second objects G1a and T2 in the corrected image AI. As a result, the user can confirm the second objects G1a and T2 expressed in the corrected image AI by looking at the composite image CI.

  Furthermore, in the first example, the CPU 210 displays the UI screen WI2 including the composite image CI on the display unit 270, and acquires an instruction regarding the contour image from the user (S75 in FIG. 3). Then, when the removal instruction is acquired in S75, the CPU 210 sets each value of the plurality of pixels constituting the contour of the instruction target in S90 to the corresponding pixel in the corrected image AI. When the restoration instruction is acquired in S75, a plurality of pixels in the object area (that is, the restoration area described above) specified by the contour of the instruction target are obtained in S95. Each value is changed to the value of the corresponding pixel in the original image SI. As a result, it is possible to appropriately execute the removal of the outline of the first object and the restoration of the first object in accordance with a user instruction. As a result, processed composite image data representing an appropriate processed composite image ACI in accordance with the user's intention can be output.

  Furthermore, in the first embodiment, when an instruction for contour display is acquired, the CPU 210 does not change the values of a plurality of pixels constituting the contour to be instructed. It is also possible to leave the contour of a specific object in the composite image ACI. Therefore, a more appropriate processed composite image ACI can be output in accordance with a user instruction.

  Note that, like the contour PGd of the first object in the composite image CI, the contour of the first object is a partial contour of the first object PG in the original image SI and is the first object PG. Contain a contour that is not the entire contour. That is, the contour of the first object may not be a closed loop-shaped contour that surrounds the entire first object.

  Further, in the first embodiment, the CPU 210 obtains an instruction from the user for each of the outlines PGd, T11d, T12d, B1d, and B2d of the plurality of first objects PG, T1, B1, and B2 in the original image SI. (S75 in FIG. 3). CPU210 changes the value of a pixel about each of several outline based on the instruction | indication from a user (S85-S95 of FIG. 3). Therefore, processed composite image data that has been subjected to appropriate processing can be output for each of the plurality of first objects to be removed by the background removal processing. Therefore, the processed composite image ACI more in line with the user's intention can be provided to the user.

B. Second Embodiment FIG. 9 is a flowchart of color correction processing according to the second embodiment. In the second embodiment, the color correction process of FIG. 9 is performed instead of the scan process of FIG. This color correction process is similar to the scan process of FIG. 2 in that output (specifically, display) of composite image data representing the composite image CI and output (specifically) processed composite image data representing the processed composite image ACI. Specifically, this is a process for performing printing and storage. The color correction process is executed, for example, when the user inputs a start instruction to the scanner driver activated in the computer 200.

  In S <b> 205, the CPU 210 acquires the original image data and stores it in the buffer area 231. The CPU 210 acquires original image data by reading a document using the scanner unit 250 and generating scan data as original image data, for example. Instead, the CPU 210 stores one piece of scan data selected based on a user's designation from a plurality of scan data generated in advance and stored in the nonvolatile storage device 220. The original image data may be acquired by reading out the original image data from 220.

  In S210, the CPU 210 displays the instruction input screen WI3 on the display unit 270. FIG. 10 is a diagram illustrating an example of the instruction input screen WI3. The instruction input screen WI3 in FIG. 10 includes radio buttons RB1 and RB2 for inputting an instruction to select a specific type of correction processing to be performed on the original image data, and an instruction regarding outline display for the image to be output. Radio buttons RB3 to RB5 for inputting and buttons BT8 to BT10 are included.

  The user can input an instruction to select one of the background removal process and the color drop process by selecting one of the radio buttons RB1 and RB2. The background removal process is the same as the background removal process of the first embodiment. The color drop process is a process of changing the RGB value of a pixel having an RGB value within a specified color range RAs based on a color value indicating a specified color specified by the user to an RGB value indicating white. The color drop process is a process for removing an object that the user determines to be unnecessary from among the objects in the image. When the button BT8 is pressed, the CPU 210 displays an input screen (not shown) for inputting designation of the designated color on the display unit 270. The user can specify a specified color via this input screen. For example, the input screen includes the original image SI, and the user can input designation of the designated color by designating a position having the designated color in the original image SI using a pointing device such as a mouse.

  The user selects one of the radio buttons RB3 to RB5 to display a display instruction for displaying the outline, a non-display instruction for not displaying the outline, and an automatic determination instruction for automatically determining whether or not to display the outline. Any one of the instructions can be input.

  For example, the user selects any one of the radio buttons RB1 and RB2, selects any one of the radio buttons RB3 to RB5, and presses the start button BT9 on the instruction input screen WI3. When the start button BT9 is pressed, the CPU 210 proceeds to S220, and when the cancel button BT10 is pressed, the process is interrupted.

  In S215, the CPU 210 acquires a selection instruction for a type of a specific correction process to be executed and an instruction related to contour display. That is, when the start button BT9 is pressed, the CPU 210 acquires an instruction corresponding to the selected radio button among the radio buttons RB1 to RB5. When the selection instruction for the specific correction process type is an instruction to select the color drop process, a color value indicating the designated color designated by the user is also acquired.

  In S220, the CPU 210 executes a specific correction process based on a user's selection instruction on the original image data, in this embodiment, one of a background removal process and a color drop process, Generate corrected image data. When the specific correction process is the background removal process, the background removal process is executed as in S30 of FIG. When the specific correction process is the background removal process, the CPU 210 converts the RGB value of the pixel having the RGB value in the specified color range RAs among the plurality of pixels in the original image SI to RGB indicating white. Change to a value. Then, the CPU 210 maintains the original RGB values without changing the RGB values of the pixels having the RGB values outside the specified color range RAs among the plurality of pixels in the original image SI. As a result, corrected image data representing the corrected image is generated. Here, the designated color range RAs is a range based on color values (Rs, Gs, Bs) indicating the designated color designated by the user. For example, (Rs−ΔV) ≦ R ≦ (Rs + ΔV) and ( Gs−ΔV) ≦ G ≦ (Gs + ΔV) and (Bm−ΔV) ≦ B ≦ (Bm + ΔV).

  In S225, the CPU 210 determines whether or not the instruction regarding the contour display acquired in S215 is an automatic determination instruction. When the instruction regarding the contour display is not the automatic determination instruction (S225: NO), in S230, the CPU 210 determines whether or not the instruction regarding the contour display is a display instruction. When the instruction regarding the outline display is a display instruction (S230: YES), the CPU 210 advances the process to S250. If the instruction regarding the contour display is not a display instruction, that is, if it is a non-display process (S230: NO), the CPU 210 advances the process to S245.

  When the instruction regarding the contour display is an automatic determination instruction (S225: YES), in S235, the CPU 210 determines whether the executed specific correction process is the background removal process or the color drop process. To do. If the executed specific correction process is the background removal process, the CPU 210 advances the process to S250.

  If the executed specific correction process is a color drop process, in S240, the CPU 210 determines whether or not the original image SI is a photograph. Whether or not the object of the original image SI is a photograph can be determined by a known method. For example, the CPU 210 calculates the number of colors C of the original image data. For example, a luminance value histogram of the original image data is generated, and the number of luminance values of a predetermined number or more of the 256 gradation luminance values is calculated as the number of colors C using the luminance value histogram. The The CPU 210 determines that the object of the original image SI is a photograph when the calculated color number C is greater than or equal to the threshold value THc, and when the calculated color number C is less than the threshold value THc, Determine that the object is not a photo. Objects different from photographs include, for example, characters and computer graphics.

  When the original image SI is a photograph (S240: YES), the CPU 210 advances the process to S245. If the original image SI is not a photograph (S240: NO), the CPU 210 advances the process to S250.

  In S245, the CPU 210 outputs (for example, saves or prints) the corrected image data generated in S220, that is, corrected image data representing a corrected image in which no contour image is synthesized.

  In S250, the CPU 210 executes the processes of S35 to S105 in FIGS. As a result, output of composite image data (specifically, display of composite image CI in S70 of FIG. 3) and output of processed composite image data (specifically, S105 of FIG. 3) are performed.

  According to the second embodiment described above, the type of specific correction processing for the original image SI is determined (S235), and the specific correction processing removes the background color of the original image SI (ie, background removal processing). If it is, the composite image data representing the composite image CI including the contour image and the processed composite image data representing the processed composite image ACI are output (S250). When the specific correction process is a process for removing an object in the original image SI (that is, a color drop process), corrected image data representing a corrected image that does not include a contour image can be output ( S245). The color drop process is a process for removing an object that the user determines to be unnecessary. Therefore, the object removed by the color drop process may not be an object removed against the user's intention but an object removed according to the user's intention. For this reason, when the specific correction process is a color drop process, it may be preferable to output corrected image data that does not include the contour image, instead of the composite image data that represents the composite image CI that includes the contour image. . For this reason, in this embodiment, when the specific correction process is a color drop process, the CPU 210 outputs corrected image data on the condition that the original image SI is a photograph. On the other hand, since the background removal process is a process for removing the background color, an object removed by the background removal process is highly likely to be an object removed against the user's intention. For this reason, in this embodiment, when the specific correction process is the background removal process, the CPU 210 outputs the composite image data and the processed composite image data. As a result, appropriate image data can be output according to the type of the original image SI.

  In the second embodiment, when the original image SI is the first type image (specifically, an image showing a photograph) (S240: YES), corrected image data is output (S245). When the original image SI is the second type of image (specifically, an image showing an object different from the photograph) (S240: NO), composite image data and processed composite image data are output (S250). ). In this way, appropriate image data can be output according to the type of the original image. More specifically, when color drop processing is performed on an image showing a photograph, the user often wants to remove an object of a specific color in the photograph. In this case, when an outline is displayed in a photograph, an unnatural image is often obtained. Therefore, in this case, it is considered appropriate to output corrected image data that does not include a contour image. On the other hand, when color drop processing is performed on an object different from a photograph, for example, an image showing characters or computer graphics, the user may want to remove the color of a solid area in the image. Many. In this case, it may be possible to appropriately represent the object in the image by displaying the outline of the removed solid area. For example, for the drawing G1 of the original image SI in FIG. 5A, it is considered preferable that the contour PGd of the drawing G1 is displayed in the composite image CI in FIG. 6B. Therefore, in this case, it is considered appropriate to output composite image data including the outline of the first object and processed composite image data.

  Furthermore, according to the second embodiment, the user can input an instruction as to whether or not to display a contour (FIG. 10), and the CPU 210 receives a contour display instruction from the user. (S230: YES), the composite image data including the outline of the first object and the processed composite image data are output (S250), and when a non-display instruction is input (S230: NO), the first object The corrected image data that does not include the contour is output (S245). As a result, an image according to the user's intention can be output.

C. Variations:
(1) In the above embodiment, the contour of the first object to be removed by the background removal process by generating the difference image data using the first edge image data and the second edge image data. Are extracted (S35 to S50 in FIG. 2). Instead, the outline of the first object may be extracted by another method. For example, the outline of the first object may be extracted using only the first edge image data. In the first edge image EI1 represented by the first edge image data, the contour of the first object that is removed by the background removal process and the contour of the second object that is not removed by the plurality of edge pixels, Both have been extracted. For this reason, when the first edge image data is generated, the first object contour and the second object contour cannot be distinguished without using the second edge image data. It can be said that the outline of the object has already been extracted from the original image SI. Therefore, for example, in S55, the CPU 210 may generate composite image data using the corrected image data and the first edge image data. Specifically, the CPU 210 calculates the RGB value of the pixel corresponding to the edge pixel of the first edge image EI1 among the plurality of pixels in the corrected image AI, and the RGB value of the corresponding pixel in the original image SI. And the RGB values of the pixels corresponding to the non-edge pixels of the edge image EI1 are not changed. Thereby, the composite image data may be generated. In this case, although the value of the pixel constituting the contour of the second object in the corrected image AI is also changed to the value of the corresponding pixel in the original image SI, the pixel constituting the contour of the second object Is the same value in the corrected image AI and in the original image SI, so the result is hardly affected. For this reason, even if composite image data is generated by the method of this modification, it is possible to generate composite image data that is substantially the same as in the above-described embodiment.

  Further, by generating image data indicating a difference between the original image data and the corrected image data, and performing edge extraction processing on the image data, edge image data indicating the contour of the first object is generated. May be. Also by this method, the contour of the first object can be extracted.

(2) In the above embodiment, the outline of the first object included in the composite image CI has the color of the first object in the original image SI. Instead, the color of the outline of the first object included in the composite image CI may be another color different from the background color (white in this embodiment), for example, black or gray. When the color of the first object in the original image SI is very close to the background color, the CPU 210, for example, when the color difference between the average color of the first object and the background color is less than the reference value. The color of the outline of the first object in the composite image CI may be set to black or gray regardless of the color of the first object in the original image SI. In this way, it is possible to prevent the outline color of the first object from becoming difficult to visually recognize in the composite image CI. For the color difference between the average color of the first object and the background color, for example, the Euclidean distance in the RGB color space of color values indicating two colors is used.

(3) In the first and second embodiments, the processing after S60 in FIG. 2 may be omitted, and the composite image data generated in S55 may be output (for example, printed or saved).

(4) In the second embodiment, when the original image SI is an image showing a photograph, corrected image data not including the outline of the first object is output, and an object whose original image SI is different from the photograph is displayed. In the case of the image shown, corrected image data including the outline of the first object is output. However, the present invention is not limited to this, and the image data to be output may be switched according to the type of the original image SI classified by another classification method. For example, in the case where the original image SI includes an image showing a photograph, if the original image SI is an image that includes only a photograph and does not include an object different from the photograph, the correction that does not include the outline of the first object When the completed image data is output and the original image SI is an image including both a photograph and an object different from the photograph, the corrected image data including the outline of the first object is output. good. When the color drop process is performed on an image including only a photograph, there is a high possibility that the user intentionally removes some objects in the photograph as described above. In this case, as described above, the necessity of displaying the contour of the removed object is relatively low. On the other hand, when color drop processing is performed on an image that includes both a photo and an object that is different from the photo, it can be done to remove all or part of the object that is different from the photo. High nature. In this case, since some objects in the photograph may be removed against the user's intention, the necessity of displaying the outline of the removed object is relatively high.

(5) In each of the above embodiments, the original image data is scan data generated by the scanner unit 250. The present invention is not limited to this, and various image data representing an optically read image can be employed. For example, it may be image data generated by optically reading a document by photographing with a digital camera.

(6) In each of the above embodiments, the CPU 210 does not change the value of the pixel having the RGB value outside the background color range RA in the background removal process, but instead, it is outside the background color range RA. The pixel value may be changed so that the RGB value is closer to white as the background color range RA is approached. In this case, it is possible to suppress the appearance of an unnatural boundary line between the white region and the color region different from white in the corrected image. In this case, the second object in the original image SI and the second object in the corrected image AI are not completely the same, and the colors may be slightly different.

(7) The scan processing of FIGS. 2 and 3 and the color correction processing of FIG. 9 executed by the CPU 210 of the computer 200 in each of the above embodiments may be executed by the control unit 110 of the MFP 100, for example. In this case, the computer 200 is not necessary, and the multifunction peripheral 100 only needs to execute these processes alone.

(8) Further, the scan processing of FIGS. 2 and 3 and the color correction processing of FIG. 9 may be executed by a server that is communicably connected to the multifunction device 100 or the computer 200. The server may be managed by the manufacturer of the multifunction device 100, for example. In this case, for example, the server transmits the screen data indicating the UI screen WI2 including the composite image CI to the multifunction device 100 or the computer 200, so that the UI screen WI2 is displayed on the display unit of the multifunction device 100 or the computer 200. Display. Then, the server obtains an instruction from the user via the multifunction peripheral 100 or the computer 200, and generates processed image data representing the processed composite image ACI. The generated composite image data shown is output from the server to the multifunction peripheral 100 or the computer 200. The server may be a single computer or a computer system including a plurality of computers that can communicate with each other.

(9) In each of the above embodiments, a part of the configuration realized by hardware may be replaced with software, and conversely, part or all of the configuration realized by software is replaced with hardware. You may do it.

  As mentioned above, although this invention was demonstrated based on the Example and the modification, Embodiment mentioned above is for making an understanding of this invention easy, and does not limit this invention. The present invention can be changed and improved without departing from the spirit and scope of the claims, and equivalents thereof are included in the present invention.

  DESCRIPTION OF SYMBOLS 100 ... MFP, 110 ... Control part, 120 ... Scanner part, 130 ... Printer part, 200 ... Computer, 210 ... CPU, 220 ... Nonvolatile storage device, 230 ... volatile storage device, 231 ... buffer area, 250 ... scanner unit, 260 ... operation unit, 270 ... display unit, 280 ... communication unit

Claims (14)

A first object including a plurality of pixels having gradation values within a specific range, and a plurality of pixels that are second objects different from the first object and have gradation values outside the specific range An original image data representing an original image including the second object, and an acquisition function for acquiring the original image data generated by optically reading a document;
A correction function for generating corrected image data representing a corrected image including the second object by executing a specific correction process on the original image data, wherein the specific correction process includes: The correction function, which is a process of changing the value of a pixel having a gradation value within a specific range to a specific gradation value;
An extraction function for extracting an outline of the first object in the original image using the original image data;
A generation function for generating output image data representing an output image using the corrected image data, wherein the output image includes a contour image indicating a contour of the first object, and the corrected image in the corrected image. A second object; and the generation function,
An output function for outputting the output image data;
A computer program that causes a computer to realize The computer program according to claim 1, further comprising:
The extraction function is
Generating first edge image data representing a first edge image by performing an edge extraction process for extracting edge pixels on the original image data;
A computer program for extracting an outline of the first object using the first edge image data. The computer program according to claim 2, wherein the extraction function is:
Generating second edge image data representing a second edge image by performing the edge extraction process on the corrected image data;
A computer program for extracting an outline of the first object by using the first edge image data and the second edge image data. A computer program according to claim 3,
The extraction function is configured to determine, from the plurality of edge pixels in the first edge image, an area constituted by pixels having no corresponding edge pixel in the second edge image. A computer program that extracts contours. A computer program according to any one of claims 1 to 4,
A computer program, wherein an outline of the first object in the output image has the same color as an outline of the first object in the original image. A computer program according to any one of claims 1 to 5,
The computer program, wherein the second object in the output image has the same color as the second object in the corrected image. A computer program according to any one of claims 1 to 6, further comprising:
Causing the computer to implement a first determination function for determining the type of the original image;
The output function outputs the output image data when the original image is a first type image, and outputs the corrected image data when the original image is a second type image. A computer program. A computer program according to claim 7,
The first type image is an image showing an object different from a photograph,
The computer program in which the second type of image is an image showing a photograph. A computer program according to any one of claims 1 to 8, further comprising:
Causing the computer to realize a second determination function for determining the type of the specific correction processing for the original image;
The output function outputs the output image data when the specific correction process is a process for removing a background color of the original image, and the specific correction process performs the first correction process in the original image. A computer program for outputting the corrected image data in the case of processing for removing one object. A computer program according to any one of claims 1 to 9,
The computer program, wherein the outline of the first object includes the outline of a part of the first object and not the outline of the whole of the first object. A computer program according to any one of claims 1 to 10, further comprising:
An instruction acquisition function for displaying the output image on a display unit and acquiring an instruction regarding the contour image from a user;
When the first instruction is acquired, the value of each of the plurality of pixels constituting the specific contour of the contour image of the output image is changed to the value of the corresponding pixel in the corrected image. When the second instruction is acquired, the change is made to change the value of each of the plurality of pixels in the object region specified by the specific contour to the value of the corresponding pixel in the original image. Function and
A computer program that causes a computer to realize A computer program according to claim 11,
The change function is a computer program that does not change the values of a plurality of pixels constituting the specific contour in the output image when a third instruction is acquired. A computer program according to claim 11 or claim 12,
The contour image of the output image includes a contour of each of the plurality of first objects in the original image,
The instruction acquisition function acquires an instruction from a user for each of a plurality of contours,
The said change function is a computer program which changes the value of a pixel based on the instruction | indication from a user about each of several outline. A first object including a plurality of pixels having gradation values within a specific range, and a plurality of pixels that are second objects different from the first object and have gradation values outside the specific range An original image data representing an original image including the second object, and an acquisition unit that acquires the original image data generated by optically reading a document;
A correction unit that generates corrected image data representing a corrected image including the second object by executing a specific correction process on the original image data, and the specific correction process includes: The correction unit, which is a process of changing the value of a pixel having a gradation value within a specific range to a specific gradation value;
An extraction unit that extracts an outline of the first object in the original image using the original image data;
A generation unit that generates output image data representing an output image using the corrected image data, wherein the output image includes a contour image indicating a contour of the first object, and the corrected image in the corrected image. A second object; and
An output unit for outputting the output image data;
An image processing apparatus comprising:

Images