JP2018156274A - Image processing apparatus, image processing method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image processing apparatus, an image processing method, and a program for cutting out image data of a shape close to a rectangle as image data of a medium.SOLUTION: An image processing apparatus comprises: a reading unit for reading an area including a medium to acquire image data; an edge extraction unit for extracting edge pixels included in the image data; a contour extracting unit for extracting a closed contour corresponding to a contour of the medium among edges represented by a set of the edge pixels; a feature point extraction unit including a first feature point extraction unit for extracting a first feature point on the closed contour, a contour dividing unit for deleting edge pixels located around the first feature point on the closed contour to divide the closed contour into a plurality of contours, and a second feature point extraction unit for extracting a second feature point which is an intersection on an extended line of the divided contours; and a cutout unit for cutting out an area obtained by connecting the second feature points.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program.

  There are known methods for extracting feature points of a closed figure that forms an area such as a rectangle or a circle, a combination of open figures that do not form an area such as a straight line, or a figure that is formed of a single piece.

  A feature point is a point that becomes an end point of each straight line when the contour of a target graphic is more simply expressed by a plurality of straight lines. One of the merits of extracting feature points is a reduction in the required memory capacity. For example, by extracting feature points from a figure that draws an arc as shown in FIG. 15 (a) and connecting the feature points, the portion where the original arc is drawn as shown in FIG. 15 (b) is simplified. Can be obtained, and the required memory capacity can be reduced.

  A DP method (Douglas-Peucker algorithm) (Non-Patent Document 1) is known as a method for further simplifying a figure after extracting feature points. In the DP method, the distance between a straight line drawn from two feature points of interest and the feature point farthest from the straight line between the two feature points is compared with a threshold value, and the distance is equal to or less than the threshold value. In this method, the figure is simplified by recursively performing the process of thinning out the feature points and connecting the remaining feature points. The thinning method using the DP method will be described in detail in a later paragraph.

  As an example of application of the DP method to an MFP (Multi Function Printer), a medium such as a business card placed on a platen is scanned with an image reading device, and the outline of the medium is simplified by the DP method and then subjected to rotation conversion. There is a technique for storing the image data cut out along the line as image data. Normally, when a rectangle is cut out from scanning data read by the MFP, the coordinates of the four corner vertices of the medium included in the scanning data are extracted, and rotation conversion is performed on the medium area to read the closed contour line of the medium. A general image data format in which one of the main scanning direction and the sub-scanning direction in the reading direction of the apparatus corresponds to the x-axis and the y-axis when the other is the y-axis and is cut out. Cut out into a rectangular shape so that Usually, since a medium such as a business card has a rectangular shape, the coordinates of the vertices located at the four corners are extracted as feature points by the DP method.

  If the interior angles of the medium to be extracted are all right angles, the coordinates of the vertices at the four corners of the medium can be extracted by, for example, the Harris algorithm without using the DP method. However, if the corner of the medium is rounded or bent, the Harris algorithm will extract a plurality of feature points for that location. When a plurality of feature points are extracted at at least one corner, as a result, five or more feature points are extracted from the entire medium, and are not recognized as an area to be cut out. The reason why the region where five or more feature points are extracted is not recognized as a region to be cut out is that the angle between adjacent contours is not necessarily a right angle, so that the contour line that matches the x axis and the y axis is not determined. This is because highly accurate rotation conversion cannot be performed.

  Therefore, the DP method is applied to a medium with rounded corners or a bent medium, the outline is simplified, and four feature points instead of the vertices are extracted, so that it is recognized as an area to be cut out. It becomes possible.

However, the user's intention when scanning with the MFP is not to scan a simplified medium but to scan the entire medium so that it does not fall out. When four feature points are extracted from a medium with rounded corners as a vertex of the medium by DP method, one point on the rounded outline is extracted as a feature point for the rounded part. The If any one of the four feature points on the outline of a rounded corner is included, the area of the medium protrudes from the area connecting the four feature points, and all the original areas of the medium are included. Cannot extract the area.
An object of the present invention is to cut out image data having a shape close to a rectangle as image data of a medium.

  In order to solve the above-described problem, the image processing apparatus according to the present invention is represented by an edge extraction unit that extracts edge pixels included in image data obtained by reading an area including a medium, and a set of edge pixels. Of the edges, a contour extraction unit that extracts a closed contour corresponding to the contour of the medium, a first feature point extraction unit that extracts a first feature point on the closed contour, and a first feature on the closed contour The edge pixel located around the point is deleted, and a contour dividing unit that divides the closed contour into a plurality of contours, and a second feature point that is an intersection point on the extension line of each divided contour line, respectively. A second feature point extracting unit for extracting; and a cutout unit for cutting out a region obtained by connecting the second feature points.

  According to the present invention, it is possible to cut out image data having a shape close to a rectangle as image data of a medium.

It is a figure which shows an example of the flow of DP method with respect to the open figure which concerns on this embodiment. It is a figure which shows an example of the flow of DP method with respect to the closed figure which concerns on this embodiment. It is a figure which shows an example of the application example of DP method with respect to the open figure which concerns on this embodiment. It is a figure which shows an example of the hardware constitutions of the image processing apparatus which concerns on this embodiment. It is a figure which shows an example of a function structure of the image processing apparatus which concerns on this embodiment. It is a figure which shows an example of a function structure of the feature point extraction part which concerns on this embodiment. It is a figure explaining the feature point which concerns on this embodiment. It is a figure explaining the optimality of the feature point which concerns on this embodiment. It is a figure explaining an example of the division | segmentation method of the closed outline of the medium which concerns on this embodiment. It is a figure explaining the optimal feature point which concerns on this embodiment. It is a figure explaining the optimality of the optimal feature point which concerns on this embodiment. It is a figure which shows an example of a function structure of the post-processing part which concerns on this embodiment. It is a figure which shows an example of the operation screen which concerns on this embodiment. It is a figure which shows an example of a series of processing flows concerning this embodiment. It is a figure which shows an example which extracts a feature point and simplifies a figure.

  The DP method will be described in detail using a flowchart. The DP method is a process applied to a figure for which feature points existing on the contour line have been extracted, and the following flow will be described assuming that the feature points have already been extracted. The flow described in this embodiment is an example of thinning out feature points using the DP method, and thinning out feature points using the DP method is not limited to the flow in this embodiment.

  FIG. 1 is an example of a flow when thinning out feature points by the DP method for an open figure. The open figure in the present embodiment is a figure having two or more end points. In FIG. 1, a figure having two end points is taken as an example. In S101, a straight line connecting both end points of the open figure is drawn. This is because the DP method has a property of recursively processing a feature point located between two feature points, and therefore, if both end points are not selected, the end point is higher than the two feature points. This is because the DP method cannot be applied to the feature points located in the part. Since the feature points at both end points of the open figure are not thinned out, they become feature points after thinning out. In the DP method, feature points after thinning are sequentially determined, and the final figure after thinning is obtained by connecting the feature points after thinning.

  In S <b> 102, a feature point that is located between a diagram that connects two selected feature points and that is farthest from a straight line that connects the two feature points is extracted. In S103, the distance between the feature point extracted in S102 and the straight line connecting the two feature points is calculated and compared with a predetermined threshold value. If the distance between the feature point extracted in S102 and the straight line connecting the two feature points is larger than the threshold, the process proceeds to S104, and the extracted feature point is determined as one of the feature points after thinning. In S105, the feature point newly extracted in S104 and one of the original two feature points are newly selected as two feature points, a straight line connecting them is drawn, and the process returns to S102. At this time, the straight line connecting the two original feature points is deleted. The thinning out of the feature points by the DP method depends on the threshold value, and if the threshold value is set large, the thinning out is easily performed, and a more simplified figure can be easily obtained. If the threshold value is set small, it is difficult to perform thinning, and it becomes easier to obtain a figure that is closer to the original figure. If the distance between the feature point extracted in S102 and the straight line connecting the two feature points is smaller than the threshold value, the process proceeds to S106, and an end flag is assigned to the section between the two feature points. The feature points located in the section to which the end flag is assigned are not set as the feature points after the thinning, and are determined as the thinning section when the end flag is given.

  If the end flag is assigned in S106, the process proceeds to S107, and it is determined whether or not the end flag is assigned to all the sections. If the end flag is assigned to all the sections (S107, Yes), the flow ends. If there is still a section to which the end flag is not assigned (S107, No), the process proceeds to S108. In S108, a straight line connecting both end points of the section to which no end flag is assigned is drawn, and the process returns to S102. The above process is repeated, and when it is determined in S107 that end flags are assigned to all the sections, the flow ends. The feature points and straight lines extracted after the end of the flow become an open figure after thinning by the DP method.

  FIG. 2 shows an example of a flow when thinning out feature points by the DP method for a closed figure. The closed figure in the present embodiment is a figure having no end point. The basic flow is the same as in the case of the open figure, but differs from the case of the open figure in that the figure is first divided. In S201, the closed figure is divided into two open figures. As an example of the dividing method, two points located at a certain distance are extracted, such as a feature point at the upper left corner of the figure and a feature point at the lower right corner. If it is divided into two open figures in S201, one of them is selected and the process proceeds to S202. Since the flow from S202 to S209 is the same as the flow from S101 to S108, redundant description is omitted here.

  When the end flag is assigned to all the sections in S208 (S208, Yes), the process proceeds to S210, and it is determined whether or not the end flag is assigned to all the open figures. When the end flag is not assigned to all the open figures (S210, No), the process returns to S202, and the flow of S202 to S209 is applied to the open figure to which no end flag is assigned. When the end flag is assigned to all the open figures (S210, Yes), the process proceeds to S211. In S211, a process of connecting the end points obtained in S201 is performed. The feature points and straight lines extracted after the end of the flow become a closed figure after thinning by the DP method.

  An example to which the flow of FIG. 1 is applied will be described with reference to the drawings. In the following, the flow corresponding to FIG. 1 is shown in parentheses. FIG. 3A shows an open figure represented by a solid line and feature points a to g before applying the DP method. The feature points a to g before applying the DP method are obtained by, for example, the Harris algorithm. The DP method is applied to this open figure along the flow of FIG. First, feature points a and g, which are end points, are connected by a straight line (S101), and a feature point e that is located between the two feature points and is farthest from the connected straight line is extracted (S102). Next, as shown in FIG. 3B, the distance α1 between the straight line ag and the feature point e is compared with a threshold value (S103). If the distance α1 is larger than the threshold value, the feature point e is thinned out. It is determined as a feature point (a feature point remaining after thinning) (S104). Here, it is assumed that the distance α1 is larger than the threshold (S103, Yes), and the feature point e is set as one of the feature points after thinning.

  Next, as shown in FIG. 3C, the extracted feature point e and the feature point g positioned at one end of the previous straight line ag are selected, and the straight line eg is drawn (S105). The distance α2 between the feature point f that is between g and farthest from the straight line eg and the straight line eg is compared with a threshold value. Here, it is assumed that the distance α2 is smaller than the threshold value (S103, No), and an end flag is assigned to the section between the feature point e and the feature point g (S106). The feature point f located in the section to which the end flag is assigned is deleted. Next, it is determined whether or not an end flag has been assigned to all the sections (S107). Here, since the end flag is not given to the section from the feature point a to the feature point e (S107, No), as shown in FIG. 3D, it is the end point of the section where the end flag is not given. A straight line ae that connects the feature point a and the feature point e is drawn (S108), and the section between the feature point a and the feature point e is similarly thinned by the DP method. Finally, as shown in FIG. 3E, a figure obtained by thinning out the feature point c and the feature point f is obtained. This completes the description of the method for thinning out feature points by the DP method.

  FIG. 4 is a diagram illustrating an example of a hardware configuration of the image processing apparatus 100 according to the present embodiment. In the image processing apparatus 100 according to this embodiment, a CPU 101, a RAM 102, a ROM 103, an image reading device 104 (reading unit), a storage device 105, an external I / F 106, and an input / output I / F 107 are connected via a bus 108. .

  The CPU 101 is an arithmetic processing unit in the image processing apparatus 100, and controls the operation of the entire image processing apparatus 100 by executing a control program. The RAM 102 is a volatile storage medium for reading and writing information at high speed, and functions as a work area when the CPU 101 executes a control program. The ROM 103 is a read-only nonvolatile storage medium in which a control program is stored.

  The image reading device 104 converts an image signal obtained by reading an object set on a document table with a line sensor into R (red), G (green), and B (blue) image data (RGB data). Output. The RGB data represents the R, G, and B values of each pixel as integer values of 0 to 255, respectively. The image reading device 104 has a default reading size. For example, even when a business card size medium smaller than the default reading size is set on the platen, the input / output I / F 107 If there is no special instruction, the image reading apparatus 104 reads the medium with a default reading size. The default reading size is generally set to A4 size. The image reading device 104 reads the pressure plate for a portion where nothing is placed on the document table, and an RGB value representing white is given to a pixel corresponding to the pressure plate portion.

  The storage device 105 is an HDD (Hard Disk Drive), for example, and is a large-capacity nonvolatile storage medium capable of reading and writing information, and stores a control program and image data. The storage device 105 may be another type of storage medium such as an SSD (Solid State Drive).

The external I / F 106 is a communication interface for connecting to a network such as a LAN, and is connected to an external terminal such as a PC (Personal Computer) or a smartphone. The external I / F 106 has a reception function and a transmission function, and receives image data stored in the storage device 105 in addition to receiving information transmitted from an external terminal.
The input / output I / F 107 is an interface that inputs an instruction, setting information, and the like to the image processing apparatus 100 from the outside and outputs information indicating the state of the image processing apparatus 100 to the outside. A mouse, keyboard, touch panel display, and the like are connected to the input / output I / F 107.

  The CPU 101 controls a target device by reading out a corresponding control program from the ROM 103 based on information input by the user via the input / output I / F 107, developing it in the RAM 102, and executing it. The CPU 101 can also receive information from an external terminal such as a PC or a smartphone through the external I / F 106 and read a corresponding control program from the ROM 103 to control a target device. Further, some functions of the CPU 101 may be executed by using a circuit such as an ASIC (Application Specific Integrated Circuit) instead. The CPU 101 may be constituted by a plurality of hardware (multi-CPU).

  FIG. 5 is a diagram illustrating an example of a functional configuration of the image processing apparatus 100 according to the present embodiment. The image processing apparatus 100 includes an image processing unit 201 that processes RGB data input from the image reading apparatus 104 and outputs clipped image data. The image processing unit 201 executes RGB data processing according to a command from the CPU 101. The image processing unit 201 includes an image conversion unit 202, an edge extraction unit 203, an outline extraction unit 204, a feature point extraction unit 205, and a post-processing unit 206, which will be described later.

  The image conversion unit 202 converts RGB data read by the image reading device 104 into C (cyan), M (magenta), Y (yellow), and K (black) image data (CMYK data). The CMYK data represents the amount of color material used in each pixel when the pixel value represented by the RGB data is expressed by printing, and each CMYK color material amount used for the target pixel is an integer from 0 to 255. Represented by Conversion to CMYK data is performed using a conversion table stored in the storage device 105. CMYK values corresponding to RGB values are input to the conversion table, and these values are updated by periodic calibration.

The edge extraction unit 203 extracts edges included in the CMYK image data by filtering processing. As filter coefficients used for the filtering process, different coefficients corresponding to the mode to be used are prepared in addition to the default coefficients normally used. The edge extraction unit 203 performs a filtering process on the target pixel and its surrounding pixels, and gives information indicating that the pixel is an edge pixel to the target pixel determined to form an edge. The edge pixels extracted by the edge extraction unit 203 include pixels corresponding to the boundary between characters and figures included in the medium and the background portion, and pixels at the boundary between the medium and the pressure plate. In the present embodiment, an edge is a collection of pixels formed by continuous edge pixels.

  The contour line extraction unit 204 uses the method described in Non-Patent Document 2 to extract the outermost edge of the edges extracted by the edge extraction unit 203 as a closed contour line of the medium. When a plurality of media are placed on the document table, a closed contour line of each medium is extracted. When there are a plurality of regions surrounded by the closed contour line, the subsequent processing is performed for each region.

  The feature point extraction unit 205 extracts feature points corresponding to the corners of the medium from the closed contour line extracted by the contour line extraction unit 204. In the present embodiment, a rectangle or a medium close to a rectangle is a detection target, and the feature point extraction unit 205 extracts four feature points that are optimal positions for the corners of the medium. The extraction in the present embodiment is a process for specifying the position of a point to be extracted, and an example is a process for storing the position coordinates of a point to be extracted in the RAM 102 or the storage device 105. The feature point that is the optimum position is a position where the medium area is included in the square area obtained by connecting the four feature points, and the area of the square area is the smallest. Means a feature point located at the coordinates. The feature points extracted by the feature point extraction unit 205 are used for processing in the post-processing unit 206 described later.

  Note that the image processing apparatus 100 in the present embodiment is intended to extract a rectangle or a medium close to a rectangle, and the image processing apparatus 100 cuts out the entire medium, so that the cut-out shape is a general image data. Rotation conversion and contour extraction are performed so that the shape is a rectangle. In the present embodiment, the term “cutting out the medium” means that image data of a region to be cut out is selectively output from the image data having the size output by the image reading device 104. Since the medium in which the number of feature points extracted by the feature point extraction unit 205 is four is generally a rectangle such as a business card or a medium close to a rectangle, the post-processing unit 206 is a detection target. A medium is determined and processing is performed on a quadrangle obtained by connecting four feature points. A medium in which the number of feature points extracted by the feature point extraction unit 205 is 5 points or more, or 3 points or less is different from a medium such as a business card, which is generally rectangular or close to a rectangle. The processing unit 206 determines that the medium is not a detection target. The feature points are extracted by the DP method and extracted as vertices when the target graphic is simply expressed. In this embodiment, an object that is close to a rectangle, such as a medium with rounded corners or a bent part of a corner, is also targeted for extraction. These media extract four feature points by thinning out the feature points using the DP method. Is done.

  FIG. 6 is a diagram illustrating an example of a functional configuration of the feature point extraction unit 205. The feature point extracting unit 205 obtains an optimum position by the contour line dividing unit 208 and the second feature point extracting unit 209 after the first feature point extracting unit 207 extracts temporary feature points for the closed contour line 4. Extract feature points of points. Each process will be described in detail.

  The first feature point extraction unit 207 performs the DP method on the closed contour line extracted by the contour line extraction unit 204, and extracts feature points for simplifying the contour of the medium. FIG. 7A is an example of a diagram illustrating the feature point R1 extracted by the first feature point extraction unit 207 when a rectangular medium is read, and FIG. 7B is a diagram when a medium with rounded corners is read. 6 is an example of a diagram illustrating a feature point R1 extracted by a first feature point extraction unit 207. FIG. In the figure, one of the corners of the medium is enlarged, and one area surrounded by a square is a pixel extracted as an edge pixel by the edge extraction unit. Among the edge pixels, the pixel indicated by diagonal lines is the feature point R1 extracted by the first feature point extraction unit 207. In this embodiment, the closed contour line is treated as a collection of edge pixels, and the feature points are extracted by applying the DP method to the collection of points formed by the edge pixels. In this way, by using the point that is an edge pixel as it is as the feature point, it is possible to save time and labor for determining a point corresponding to the feature point again.

Here, the optimality of the feature points after being thinned out by the first feature point extraction unit 207 will be described.
FIG. 8A shows feature points R01 to R04 after the DP method is applied to a rectangular medium and a region X connecting them, and FIG. 8B shows a medium having rounded corners. The characteristic points R11 to R14 after the DP method and the region X connecting them are shown. Also, the hatched portion in the figure indicates the difference area between the medium area and the area connecting the feature points. The dotted line portion in the figure shows the outline of the medium, and the solid line portion shows the outline when the feature points are connected.
In the case of a rectangular medium, it can be said that the optimum feature points are extracted because the area X includes all the medium areas and the area is the smallest.
On the other hand, in the case of a medium with rounded corners, since the feature points after thinning out by the DP method are points on the edge, the medium region does not fit in the region X, and the optimum feature points are extracted. I can't say. In short, when a region to be cut out is determined based on a feature point determined only by the DP method, information on the difference region is lost for a medium with rounded corners. Further, since the region X is not rectangular, the post-processing unit 206 uses the x-axis when the main scanning direction and the sub-scanning direction of the image reading device 104 are the x-axis and the other is the y-axis. It is impossible to accurately rotate and convert the image along the y axis.

  Therefore, the contour line dividing unit 208 and the second feature point extracting unit 209 are cases where the four feature points thinned out by the first feature point extracting unit 207 are temporary feature points and the medium has rounded corners. Also, four new feature points are extracted so that the extracted feature points become optimum.

The contour dividing unit 208 deletes edge pixels and temporary feature points located around the temporary feature points, and divides the closed contour lines into four contour lines S1 to S4. In the present embodiment, the vicinity of the temporary feature point indicates an area of a predetermined length from the feature point or an n × n pixel region centered on the feature point. Moreover, the division | segmentation of the closed outline in this embodiment means the state which recognizes each of the four outlines as an individual thing.
FIG. 9 is a diagram for explaining an example of a method for dividing the closed contour line of the medium when the contour dividing unit 208 has rounded corners.
The contour dividing unit 208 deletes edge pixels that constitute the closed contour line that are located around the temporary feature points extracted by the first feature point extracting unit 207. By deleting the edge pixels around the temporary feature point, the edge pixels corresponding to the rounded corners are deleted. In the figure, edge pixels with no color are edge pixels to be deleted, edge pixels painted with a halftone dot pattern are edge pixels remaining after deletion, and a contour line in which a set of these edge pixels is divided It corresponds to. In the figure, the contour on the upper side corresponds to S1 and the contour on the right side corresponds to S2. Although not shown in FIG. 9, the contour on the lower side is S3 and the contour on the left side is S4. In the figure, edge pixels included in an area of 17 × 17 pixels centered on a temporary feature point are deleted, but the area to be deleted can be adjusted as appropriate. For example, by increasing the area to be deleted, edge pixels located at rounded corners can be surely deleted. However, if the medium to be read is small, pixels other than the edge pixels located at the corners can also be deleted. Therefore, there is a possibility that the closed contour cannot be accurately divided into four. Conversely, by reducing the area to be deleted, it becomes possible to accurately divide the closed contour line into four even for a small medium, but it is not possible to delete edge pixels located at rounded corners. there is a possibility. Adjustment to a value other than the default value is performed by the user through the input / output I / F 107. The deletion of edge pixels in the present embodiment means that information indicating edge pixels is deleted so that they are not recognized as edge pixels. Further, the additional information may be added to the deletion target edge pixel so that the deletion target edge pixel is not recognized as the edge pixel in the subsequent processing.

  The second feature point extraction unit 209 derives the linear equations of the four divided contour lines S1 to S4, respectively, and calculates the coordinates of the intersection R1 ′ on the extension line of the contour line S1 and the contour line S2, as shown in FIG. To derive. In the figure, the intersection R1 ′ is represented by vertical hatching, which corresponds to the optimum feature point in the present embodiment. The linear equation is derived by randomly selecting some of the edge pixels included in one of the divided edges and using the least square method for the coordinates of those edge pixels. The four intersections R1 ′ to R4 ′ on the extension lines of the four divided contour lines S1 to S4 are the optimum four feature points. Since some errors occur in the linear equation, the contour lines S1 to S4 do not necessarily intersect in the vertical direction, but may be anything as long as they intersect in a substantially vertical direction.

  FIG. 11 is a diagram illustrating the optimality of the feature points R1 ′ to R4 ′ extracted by the second feature point extraction unit 209 for a medium with rounded corners. In the figure, a region surrounded by a dotted line indicates a medium region, and a region surrounded by a solid line indicates a region X ′ surrounded by feature points R1 ′ to R4 ′. This area includes all of the medium area, and the area of the area is at or near the minimum. Since the linear equation has some errors, the feature points R1 ′ to R4 ′ are regarded as the optimum feature points even when the medium region slightly protrudes from the region X ′. As described above, the feature point extraction unit 205 in the present embodiment sets the feature points obtained by thinning out using the DP method as temporary feature points, and extracts the optimum feature points therefrom. Based on the coordinates of the optimum feature points R1 ′ to R4 ′ extracted by the feature point extraction unit 205, the post-processing unit 206 performs region interpolation, rotation conversion, and clipping.

  FIG. 12 is a diagram for explaining an example of the functional configuration of the post-processing unit 206. The post-processing unit 206 performs processing on a quadrangle obtained by connecting four feature points of a medium in which the number of extracted feature points is four. First, the determination unit 210 determines a medium to be processed, the region complementation unit 211 supplements pixels for a specific region in the region X ′, and the skew correction unit 212 performs rotation transformation by affine transformation, A region including the medium region corrected by the cutout unit 213 is cut out and output as cut image data. Complementation in the present embodiment means replacing the pixel value of the target pixel with another pixel value. Further, the complement in this embodiment includes a case where the pixel value after replacement is the same as the pixel value before replacement. Each process will be described in detail.

The determination unit 210 extracts a medium having four feature points. The area complementation unit 211 targets the medium extracted by the determination unit 210, is in a rectangular area obtained by connecting the optimum feature points extracted by the feature point extraction unit 205, and does not overlap the medium area Perform region completion on. Examples of the method of complementing the difference area by the area complementing unit 211 include a method of applying pixel values of pixels located in the medium area and around the difference area to pixels included in the difference area, There is a method in which a pixel value indicating a transmissive region is applied to pixels included in the region to make the difference region a transmissive region.
When complementing a difference area as a transparent area, a predetermined pixel value is allocated to pixels included in the difference area. The pixel value complementation by the transmissive region may be performed by selecting one of a plurality of pixel values stored in advance. Other than that, interpolation may not be performed. The interpolation method of the difference area by the area complementing unit 211 may be set in the ROM 103 in advance, or the user may select an arbitrary method.
FIG. 13 is an example of an operation screen that allows the user to select a difference region complementing method. The area complementing unit 211 can allow the user to select a difference area complementing method. The user can perform a predetermined process on the difference area by selecting a desired process.

  The skew correction unit 212 performs rotational transformation by affine transformation on the region X ′ obtained by connecting the four feature points extracted by the feature point extraction unit 205, and performs the main scanning direction and the sub-scanning direction in the image reading device 104. The two sides of the rectangle are made to coincide with the x-axis and the y-axis when one of the x-axis and the other is the y-axis. However, the angle of the four vertices in the area enclosed by connecting the four feature points is assumed to be slightly off from the right angle, so the two sides do not necessarily match, but they match within a predetermined error range. Anything to do. That is, the skew correction unit 212 uses two adjacent sides of the region X ′ as one of the two adjacent sides of the rectangular image data output from the image reading device 104 as the x axis and the other as the y axis. Rotation conversion is performed on the region X ′ so as to be substantially parallel to the x-axis and the y-axis.

  The cutout unit 213 cuts out the region X ′ from the image data subjected to the rotation conversion by the skew correction unit 212 and outputs it as cutout image data. When there are a plurality of areas to be cut out, all of them are output as separate cut-out image data. The output destination is an external terminal connected through the storage device 105 or the external I / F 106.

  As described above, the quadrangular area whose rotation is converted by the skew correction unit 212 has four corners at right angles or substantially right angles, and the medium area is not partially deleted by the cutout unit 213 after the rotation conversion. In addition, since the optimum feature point is extracted by the second feature point extraction unit 209, the media region is not partially deleted by the cutout unit 213 when the closed contour line is simplified by the DP method.

  FIG. 14 is a diagram illustrating an example of a series of processing flows in the present embodiment. In step S <b> 301, the edge extraction unit 203 extracts edge pixels included in the image data acquired by the image reading device 104. In S302, the contour line extraction unit 204 extracts a set of edge pixels corresponding to the contour of the medium from the edge pixels extracted in S301 as a closed contour line. In S303, the first feature point extraction unit 207 extracts the feature points of the target closed contour line. In S304, the outline dividing unit 208 divides the closed outline into a plurality of outlines. In S305, the second feature point extraction unit 209 extracts optimal feature points based on the contour lines divided in S304. In S306, the region complementing unit complements the difference region of the region surrounded by the straight line connecting the medium region and the optimum feature point with a predetermined pixel. In S307, the skew correction unit 212 performs rotation conversion on the region surrounded by the straight line connecting the optimum feature points, and the cutout unit 213 cuts out the region after the rotation conversion, and extracts the cut image data. Output. In S308, it is determined whether or not the cutout unit 213 has cut out the area formed by all the closed contour line sets. If the cutout has been carried out (S308, Yes), the flow ends. When the cutout has not been performed (No in S308), the process returns to S303, and the flow from S303 to S307 is performed on the closed contour line set that has not been cut out.

  As described above, according to the image processing apparatus 100 according to the present embodiment, the edge extraction unit 203 extracts edge pixels from the image data acquired by the image reading apparatus 104, and the contour line extraction unit 204 extracts the acquired edge pixels. Among them, a set of edge pixels corresponding to the contour of the medium is extracted as a closed contour line, the first feature point extracting unit 207 extracts temporary feature points of the closed contour line, and the contour dividing unit 208 extracts the closed contour line. Dividing into a plurality of contour lines, the second feature point extracting unit 209 extracts the optimum feature points based on the divided contour lines, the region complementing unit 211 complements the region, and the skew correcting unit 212 performs the skew correction. By doing so, even a medium with rounded corners or a medium with rounded corners can be stored as image data without damaging the original medium area.

  While specific embodiments of the present invention have been described above, the above-described embodiments are examples of the present invention. The present invention is not limited to the above-described embodiments, and can be embodied with various modifications and changes without departing from the scope of the invention in the implementation stage.

A program that realizes each function by being executed by each device of the above embodiment is a file in an installable or executable format, such as a CD-ROM, a CD-R, a memory card, a DVD (Digital Versatile Disk), The program is stored in a computer-readable storage medium such as a flexible disk (FD).
Further, the program may be provided by being stored on a computer connected to a network such as the Internet and downloaded via the network. The program may be provided or distributed via a network such as the Internet. Further, the program may be provided by being incorporated in advance in a ROM or the like.

[Summary of Embodiments, Actions, and Effects of the Present Invention]
The image processing apparatus 100 according to the first embodiment of the present invention is represented by an edge extraction unit 203 that extracts edge pixels included in image data obtained by reading an area including a medium, and a set of edge pixels. Among the edges, a contour extraction unit 204 that extracts a closed contour corresponding to the contour of the medium, a first feature point extraction unit 207 that extracts a first feature point on the closed contour, and a first on the closed contour An edge pixel located around the feature point of the contour is deleted, and a contour dividing unit 208 that divides the closed contour into a plurality of contours, and a second feature that is an intersection on the extended line of each divided contour A second feature point extraction unit 209 that extracts points respectively, and a cutout unit 213 that cuts out an area obtained by connecting the second feature points to each other.
Here, a typical example of the medium is a rectangular business card.
According to this embodiment, it is possible to cut out image data having a shape close to a rectangle as image data of a medium even if the medium has rounded corners or a medium in which some of the corners are bent. In other words, it is unlikely that the rectangular medium has a significantly inclined side other than the corner even when the corner is deformed. Therefore, the plurality of contour lines obtained by deleting the edge pixels located around the first feature point on the closed contour line represent contour lines extending along the side of the medium excluding the deformed corner portion. As a result, the region surrounded by the straight line including each divided contour line has a shape close to a rectangle. According to this aspect, rectangular image data can be obtained as the image data of the area.

In the image processing apparatus 100 according to the second embodiment of the present invention, the first feature point extraction unit 207 extracts a first feature point by performing a thinning process on a set of edge pixels constituting the closed contour line. To do.
By performing a thinning process on the set of edge pixels, an appropriate point representing the corner of the medium can be mechanically determined.

The image processing apparatus 100 according to the third embodiment of the present invention is a pixel complementing unit that complements pixels in a difference area that is inside an area obtained by connecting the second feature points and does not overlap with a medium area ( An area complementing unit 211) is further provided.
Here, the complement means that the pixel value of the target pixel is replaced with the same or different pixel value. Note that the pixel value includes a pixel value indicating a transmissive region. In the case of a medium with rounded corners, the reading unit (image reading device 104) cannot acquire image data of the medium for the difference area. In this mode, the pixels in the difference area are complemented with pixel values that match the usage mode.

In the image processing apparatus 100 according to the fourth embodiment of the present invention, the pixel complementing unit (region complementing unit 211) uses the pixel value relating to the pixel in the region of the medium located around the difference region, and uses the pixel value related to the difference. Complement the pixel values of the pixels in the region.
According to this aspect, as the cut-out image data output from the cut-out unit 213, it is possible to obtain image data having no sense of incongruity in the hue of the difference area.

The image processing apparatus 100 according to the fifth embodiment of the present invention further includes a rotation conversion unit (skew correction unit 212) that performs rotation conversion on a region obtained by connecting the second feature points. The output part 213 cuts out the area | region rotationally converted by the rotation conversion part.
Since the area obtained by connecting the second feature points is a substantially rectangular area, the medium area is not partially deleted by the rotation conversion. According to this aspect, it is possible to cut out the medium in accordance with the user's intention without partially deleting the medium area.

The image processing method according to the sixth embodiment of the present invention includes an edge extraction step (S301) for acquiring image data obtained by reading an area including a medium and extracting edge pixels included in the image data; A contour extraction step (S302) for extracting a contour line corresponding to the contour of the medium from the edges represented by the set of pixels, and a first feature point extraction step (S303) for extracting the first feature point on the contour line. ), The edge pixel located around the first feature point on the contour line is deleted, and a contour line dividing step (S304) for dividing the closed contour line into a plurality of contour lines, and each of the divided contour lines A second feature point extracting step (S305) for extracting second feature points that are intersections on the extension line, and a cutting step (S307) for cutting out an area obtained by connecting the second feature points to each other; Have .
This aspect has the same operations and effects as the first embodiment.

The image processing apparatus 100 according to the seventh embodiment of the present invention is a program for causing a computer to execute the image processing method according to the sixth embodiment.
This aspect has the same operations and effects as the first embodiment.

  DESCRIPTION OF SYMBOLS 100 Image processing apparatus 101 CPU 102 RAM 103 ROM 104 Image reading apparatus (reading part) 105 Storage apparatus 106 External I / F 107 Input I / F 108 Bus 201 Image processing part 202 Image conversion part 203 Edge extraction part 204 Contour line extraction part 205 Feature Point Extraction Unit 206 Post-Processing Unit 207 First Special Vertex Extraction Unit 208 Contour Line Division Unit 209 Second Feature Point Extraction Unit 210 Judgment Unit 211 Region Completion Unit 212 Skew Correction Unit 213 Cutout Unit

Shin-Ting Wu; Adler C. G. da Silva; Mercedes R. G. Marquez, The Douglas-peucker algorithm, J. Braz. Comp. Soc. Vol.9 no.3 Campinas Apr. 2004 Satoshi Suzuki; Keiichi Abe, Topological structural analysis of digitized binary images by border following, Computer Vision Graphics and Image Processing 30 (1): 32-46 March 1985

Claims (7)

An edge extraction unit that extracts edge pixels included in image data obtained by reading an area including a medium;
A contour extracting unit that extracts a closed contour corresponding to the contour of the medium among the edges represented by the set of edge pixels;
A first feature point extraction unit for extracting a first feature point on the closed contour line;
An edge line dividing unit that deletes edge pixels located around the first feature point on the closed outline and divides the closed outline into a plurality of outlines;
A second feature point extraction unit that extracts second feature points that are intersection points on the extended lines of the divided contour lines, and
A cutout portion for cutting out an area obtained by connecting the second feature points;
An image processing apparatus comprising:   The image processing apparatus according to claim 1, wherein the first feature point extraction unit extracts the first feature point by performing a thinning process on a set of edge pixels constituting the closed contour line.   The image processing apparatus according to claim 1, further comprising a pixel complementing unit that complements pixels in a difference area that is inside an area obtained by connecting the second feature points and does not overlap with the area of the medium.   The image processing device according to claim 3, wherein the pixel complementing unit complements pixel values of pixels in the difference area using pixel values relating to pixels in the medium area located around the difference area. A rotation conversion unit that performs rotation conversion on a region obtained by connecting the second feature points;
The image processing apparatus according to claim 1, wherein the cut-out unit cuts out an area rotationally converted by the rotation conversion unit. An edge extraction step of acquiring image data obtained by reading an area including a medium and extracting edge pixels included in the image data;
A contour extraction step for extracting a contour corresponding to the contour of the medium from the edges represented by the set of edge pixels;
A first feature point extracting step of extracting a first feature point on the contour line;
An edge dividing step for deleting edge pixels located around the first feature point on the outline and dividing the closed outline into a plurality of outlines;
A second feature point extracting step for extracting each second feature point that is an intersection point on an extension line of each of the divided contour lines;
A step of cutting out a region obtained by connecting the second feature points;
An image processing method.   A program for causing a computer to execute the image processing method according to claim 6.

Images