JP4715888B2 - Image processing apparatus and computer program - Google Patents

Description

  The present invention relates to an image processing apparatus and a computer program for extracting a rectangular area including a contour of a subject.

2. Description of the Related Art Conventionally, there has been known an imaging apparatus having a contour rectangle extraction function that extracts a rectangular region including a contour of a subject in order to enable image processing such as coordinate transformation to be performed on a subject image included in a captured image. Yes. Such an imaging apparatus detects a plurality of straight lines constituting the contour of the subject from the edge image indicating the contour of the subject using the Hough transform, and detects a straight line forming a rectangular region from the detected plurality of straight lines. By specifying, a rectangular region is extracted (see Patent Document 1).
JP 2005-267457 A

  The conventional imaging device calculates the number of pixels of the edge image existing on the straight line for each of the detected plurality of straight lines, and identifies the straight line forming the rectangular region based on the order of the straight lines having the largest number of calculated pixels. However, according to such a configuration, when the size of the rectangular area to be extracted is not clear or when a plurality of subject images exist in one image, an unsuitable rectangular area may be extracted.

  SUMMARY An advantage of some aspects of the invention is that it provides an image processing apparatus and a computer program capable of presenting to a user the likelihood of an extracted outline rectangle.

An image processing apparatus according to the present invention comprises a line segment detection means for detecting vertical and horizontal line segments from an image, and a rectangular area from the vertical and horizontal line segments detected by the line segment detection means. A plurality of combinations of the opposite-side candidate creation unit that creates the opposite-side candidate in the vertical direction and the lateral direction, and the opposite-side candidate and the opposite-side candidate created in the opposite-side candidate creation unit. Rectangle candidate creation means for creating, as a rectangle candidate, a rectangular area whose vertex is the intersection of the opposite-side candidate and the lateral opposite-side candidate ; Scoring means for calculating the probability of each rectangular candidate based on the ratio of the lengths of the opposite side candidates located at .

A computer program according to the present invention includes a line segment detection process that detects vertical and horizontal line segments from an image, and a vertical area that forms a rectangular area from the vertical and horizontal line segments detected by the line segment detection process. A plurality of combinations of the opposite-side candidate creation process for creating the opposite-side candidate in the horizontal direction and the opposite-side candidate and the opposite-side candidate created by the opposite-side candidate creation process, and the vertical opposite side for each combination A rectangular candidate creation process that creates a rectangular area having a vertex at the intersection of the candidate and the opposite side candidate as a rectangle candidate , and on the outer circumference of the rectangular candidate with respect to the outer circumference of the rectangular candidate created by the rectangular candidate creation process The computer executes a scoring process for calculating the probability of each rectangular candidate based on the ratio of the length of the opposite side candidate located .

  According to the image processing apparatus and the computer program according to the present invention, it is possible to present the probability of the extracted outline rectangle to the user.

  Hereinafter, the configuration of a digital still camera according to an embodiment of the present invention will be described in detail.

〔overall structure〕
First, an overall configuration of a digital still camera 1 according to an embodiment of the present invention will be described with reference to FIGS. 1 (a) and 1 (b).

  As shown in FIG. 1A, a digital still camera 1 according to an embodiment of the present invention includes a photographing lens 3 and a self-timer lamp 4 on the front surface of a substantially rectangular thin plate body (hereinafter abbreviated as a body). , Finder window 5, strobe light emitting unit 6, and microphone unit 7. A power key 8 and a shutter key 9 are provided on the right end side (for the user) of the upper surface of the main body 2. The taking lens 3 has a zoom function and an AF (AutoFocus) function for steplessly changing the focal length, and retracts inside the main body 2 when the power is off and in the reproduction mode. The power key 8 is a key that is operated every time the power is turned on / off, and the shutter key 9 instructs the photographing timing in the photographing mode.

  As shown in FIG. 1B, on the back of the main body 2, a shooting mode (R) key 10, a playback mode (P) key 11, an electronic viewfinder (EVF) 12, a speaker unit 13, a macro key 14, a strobe A key 15, a menu (MENU) key 16, a ring key 17, a set (SET) key 18, and a liquid crystal display unit 19 are provided. The shooting mode key 10 is automatically turned on by operating from a power-off state to shift to a still image shooting mode, while being repeatedly operated from the power-on state to circulate through still image and moving image shooting modes. To set. In this embodiment, the still image shooting mode includes a single shooting mode in which a normal shooting operation is performed with a predetermined exposure time, and a subject is continuously shot with an exposure time shorter than that in the single shooting mode. A continuous shooting mode in which image frames are combined to generate a single image.

  When the playback mode key 11 is operated from the power-off state, the playback mode key 11 is automatically turned on to enter the playback mode. The EVF 12 is an eyepiece type finder that uses a liquid crystal screen, and displays a through image on the liquid crystal screen in the photographing mode, and reproduces and displays a selected image and the like in the reproduction mode. The macro key 14 is operated when switching between normal shooting and macro shooting in the still image shooting mode. The strobe key 15 is operated when switching the light emission mode of the strobe light emitting unit 6. The menu key 16 is operated when selecting various menu items. The ring key 17 is integrally formed with item selection keys in the up, down, left, and right directions, and the set key 18 located in the center of the ring key 17 sets the item selected at that time. To operate.

  The liquid crystal display unit 19 is composed of a color liquid crystal panel with a backlight, and displays a through image on the monitor in the photographing mode, and reproduces and displays the selected image and the like in the reproduction mode. Although not shown, a USB (Universal Serial Bus) is provided on the bottom surface of the digital still camera 1 as a memory card slot for attaching / detaching a memory card used as a recording medium or a serial interface connector for connecting to an external personal computer or the like. Connectors etc. are provided.

[Configuration of imaging system and control system]
Next, with reference to FIGS. 2 and 3, the configuration of the imaging system and the control system of the digital still camera 1 according to an embodiment of the present invention will be described.

  In the digital still camera 1 according to an embodiment of the present invention, the photographing of the lens optical system 32 constituting the photographing lens 3 whose focus position and aperture position are moved by driving the motor (M) 31 in the photographing mode. A CCD 33, which is an image sensor disposed behind the optical axis, is scanned and driven by a timing generator (TG) 34 and a vertical driver 35, and outputs a photoelectric conversion output corresponding to an optical image formed at fixed intervals for one screen. To do. This photoelectric conversion output is appropriately gain-adjusted for each primary color component of RGB in the state of an analog signal and then sampled and held by a sample hold circuit (S / H) 36 and converted into digital data by an A / D converter 37. After being converted, the color process circuit 38 performs color process processing including pixel interpolation processing and γ correction processing to generate digital luminance signals Y and color difference signals Cb and Cr, which are output to a DMA (Direct Memory Access) controller 39. Is done.

  The DMA controller 39 once uses the luminance signal Y and the color difference signals Cb, Cr output from the color process circuit 38 by using the composite synchronization signal, memory write enable signal, and clock signal from the color process circuit 38 once. Is transferred to the DRAM 41 and used as a buffer memory via the DRAM interface (I / F) 40. The control unit 42 is configured by a CPU, a ROM in which a computer program executed by the CPU is fixedly stored, a RAM used as a work memory, and the like, and controls the operation of the entire digital still camera 1.

  After the DMA transfer of the luminance and color difference signals to the DRAM 41 is completed, the control unit 42 reads the luminance and color difference signals from the DRAM 41 via the DRAM interface 40 and writes them into the VRAM 44 via the VRAM controller 43. The digital video encoder 45 periodically reads the luminance and color difference signals from the VRAM 44 via the VRAM controller 43, generates a video signal based on these data, and outputs the video signal to the EVF 12 and the liquid crystal display unit 19. The EVF 12 and the liquid crystal display unit 19 display based on the video signal from the digital video encoder 45, thereby displaying an image based on the image information fetched from the VRAM controller 43 at that time in real time.

  As described above, the EVF 12 and the liquid crystal display unit 19 display the image at that time in real time as a monitor image. In the so-called through image display state, when the shutter key 9 is operated at the timing at which still image shooting is desired, the trigger signal is displayed. Is generated. In response to this trigger signal, the control unit 42 cancels the DMA transfer of the luminance and color difference signals for one screen currently taken in from the CCD 33 to the DRAM 41, and again determines the aperture value and shutter speed according to the appropriate exposure conditions. Then, the CCD 33 is driven to obtain the luminance and color difference signals for one screen and transfer them to the DRAM 41. Thereafter, this path is stopped, and the recording and storage state is entered.

  In this recording and storage state, the control unit 42 reads out the luminance and color difference signals written in the DRAM 41 for each of the Y, Cb, and Cr components via the DRAM interface 40 and writes them in the image processing unit 47. The processing unit 47 compresses data by processing such as ADCT (Adaptive Discrete Cosine Transform) and Huffman coding which is an entropy coding method. Then, the obtained code data is read from the image processing unit 47, and a memory card 48 that is detachably mounted as a recording medium of the digital still camera 1 or a built-in memory that is fixedly built in the digital still camera 1 (not shown). Write on either Then, along with the compression processing of the luminance and color difference signals and the completion of writing all the compressed data to the memory card 48 or the built-in memory, the control unit 42 activates the path from the CCD 33 to the DRAM 41 again.

  A key input unit 49, a sound processing unit 50, and a strobe driving unit 51 are connected to the control unit 42. The key input unit 49 includes the power key 8, shutter key 9, shooting mode key 10, playback mode key 11, macro key 14, strobe key 15, menu key 16, ring key 17, set key 18 and the like described above. Signals accompanying these key operations are sent directly to the control unit 42. The audio processing unit 50 includes a sound source circuit such as a PCM sound source, and digitizes the audio signal input from the microphone unit 7 at the time of audio recording, and uses a predetermined data file format, for example, MP3 (MPEG-1 audio layer 3) standard. Therefore, the audio data file is created by compressing the data and sent to the memory card 48 or the built-in memory, while the audio data file sent from the memory card 48 or the built-in memory is uncompressed and converted into an analog signal when reproducing the sound. The unit (SP) 13 is driven to emit loud sounds. The strobe driving unit 51 charges a strobe capacitor (not shown) during still image shooting, and then drives the strobe light emitting unit 6 to flash based on the control from the control unit 42.

[Outline rectangle extraction processing]
The digital still camera 1 having such a configuration extracts a rectangular region including the contour of the subject by executing a contour rectangle extraction process described below. Hereinafter, the operation of the digital still camera 1 when executing the contour rectangle extraction process will be described with reference to the flowchart shown in FIG.

  The user operates the ring key 17 and the set key 18 to select a mode such as “photograph a business card or document” or “photograph a whiteboard” from the scene-specific shooting modes, and perform shooting. These modes perform frontal correction of the subject, and an image photographed in such a mode is taken into the image processing unit 47 and is shown in FIG. 3 at the timing when the outline rectangle extraction process can be executed. The flowchart starts, and the outline rectangle extraction process proceeds to step S1. The following operation of the digital still camera 1 is realized by the CPU in the control unit 42 loading a computer program stored in the ROM into the RAM and executing the computer program loaded in the RAM.

  In the process of step S1, the image processing unit 47 corrects the captured image distorted by the lens characteristics of the lens optical system 32 by executing a distortion aberration correction process on the input captured image. Thereby, the process of step S1 is completed and the outline rectangle extraction process proceeds to the process of step S2.

  In the process of step S2, the image processing unit 47 reduces the size (image size) of the captured image after correcting the distortion aberration to a predetermined size. Specifically, the image processing unit 47 calculates the size of the captured image after the distortion is corrected, and the size of the captured image is (vertical) × (horizontal) based on the calculated size: The vertical and horizontal lengths of the captured image are reduced so as to have a size of 320 × 240 (pixels). Thereby, the process of step S2 is completed, and the outline rectangle extraction process proceeds to the process of step S3.

  In the processing of step S3, the image processing unit 47 changes the display format of the color information of the captured image from the bitmap format to YUV (Y: luminance signal, U: difference between luminance signal and blue component, V: luminance signal and red component). Convert to difference format. Thereby, the process of step S3 is completed, and the outline rectangle extraction process proceeds to the process of step S4.

  In the process of step S4, the image processing unit 47 removes noise components from the image data of the captured image by applying the image data of the captured image to a median (median) filter. The median filter in this embodiment is a pixel value in a 3 × 3 (pixel) local region arranged in ascending order, and a pixel value located in the center is used as a pixel value of a pixel in the center of the region. Thereby, the process of step S4 is completed, and the outline rectangle extraction process proceeds to the process of step S5.

  In the process of step S5, as shown in FIGS. 4A to 4C, the image processing unit 47 performs vertical (vertical, x) direction and horizontal (horizontal, y) direction from the image data from which noise components have been removed. Each edge image is extracted. In the present embodiment, the image processing unit 47 calculates a spatial first derivative as shown in FIG. 5 and uses a Sobel filter that detects a contour to detect each of the vertical (vertical) direction and the horizontal (horizontal) direction. Edge image is extracted. Thereby, the process of step S5 is completed, and the outline rectangle extraction process proceeds to the process of step S6.

  In the process of step S6, the image processing unit 47, as shown in FIGS. 6A and 6B, performs fine line and binarization on the vertical and horizontal edge images extracted by the process of step S5. Apply processing. Specifically, the image processing unit 47 satisfies the condition {pixel value at the coordinate position x−1 <pixel value at the coordinate position x ≧ pixel value at the coordinate position x + 1} among the pixels constituting the vertical edge image. A pixel at the coordinate position x is detected. Similarly, the image processing unit 47 sets the coordinates satisfying the condition {the pixel value at the coordinate position y−1 <the pixel value at the coordinate position y ≧ the pixel value at the coordinate position y + 1} among the pixels constituting the horizontal edge image. The pixel at position y is detected. The image processing unit 47 sets the pixel values of the extracted coordinate positions x and y to 255 and the pixel values of other coordinate positions to 0 among the pixels constituting the edge image. Thereby, the process of step S6 is completed, and the outline rectangle extraction process proceeds to the process of step S7.

  In the processing of step S7, the image processing unit 47 performs labeling processing on each of the vertical and horizontal edge images, thereby forming the contour of the subject as shown in FIGS. 7 (a) and 7 (b). Create vertical and horizontal line segment information. In this embodiment, the image processing unit 47 scans the edge image in the x direction from the coordinate position of x = 0 while scanning in the x direction while referring to the pixels adjacent to the y direction. A pixel to be configured is detected. When a pixel is detected, the image processing unit 47 determines whether or not the pixel value of the detected pixel is 255 and is connected to another pixel, and the pixel value is 255 and the other pixel. If not connected, the trace of the line segment including the detected pixel is started in the x direction. Specifically, the image processing unit 47 traces the three points (x + 1, y−1), (x + 1, y), and (x + 1, y + 1) located on the right side of the coordinates (x, y) of the trace start position. Execute.

  Then, when any one of the following three conditions is satisfied, the image processing unit 47 assigns a unique number to the line segment (labeling), ends the trace, and continues the trace. The x coordinate position where the detected pixel is located is set as the next trace start position.

  Condition 1: At least one of the three points is already labeled.

  Condition 2: Two or more of the three points are pixels constituting the edge image.

  Condition 3: A pixel constituting an edge image was not detected twice in three points during tracing.

  On the other hand, for the edge image in the vertical direction, the image processing unit 47 detects pixels constituting the edge image by scanning in the x direction from the coordinate position of y = 0, and for the edge image in the vertical direction. The same processing as that performed is performed. The image processing unit 47 then coordinates and inclination of the start point and end point of each line segment labeled by tracing (obtained from the start point and end point), and an error (vertical line) with respect to the line segment inclination of each point constituting the line segment. If so, the average of the deviation in the x direction and the y direction in the case of a horizontal line) and the coordinate position where the error is maximized and its value are calculated as line segment information. Thereby, the process of step S7 is completed, and the outline rectangle extraction process proceeds to the process of step S8.

  In the process of step S8, the image processing unit 47 refers to the line segment information created by the process of step S7, and there is a line segment including a point where the maximum error with respect to the slope of the line segment is a predetermined value or more. If there is a line segment including a point where the maximum error is equal to or greater than a predetermined value, two line segments are included at that point (point P in the example shown in FIG. 8) as shown in FIG. Divide into The dividing point is added to the line segment whose length becomes shorter after the division. The image processing unit 47 does not divide the line segment when the length of the line segment is equal to or greater than the first threshold or when the length of the segment after division is equal to or smaller than the second threshold. Then, when there is a divided line segment, the image processing unit 47 updates the line segment information. Thereby, the process of step S8 is completed, and the outline rectangle extraction process proceeds to the process of step S9.

  In the process of step S9, the image processing unit 47 refers to the line segment information updated by the process of step S8, extracts a predetermined number of line segments having a length equal to or greater than a predetermined value as connection source line segments in the longest order, As shown in FIG. 9, the connection source line segment is connected to a line segment (connection destination line segment) that satisfies the following three conditions. Then, the image processing unit 47 connects the connection destination line segment and the connection source line segment, and then connects the connection destination line segment and the connection source line segment to determine the coordinate position of the start point and the end point of the line segment that is at least two. Calculated by multiplication. Thereby, the process of step S9 is completed, and the outline rectangle extraction process proceeds to the process of step S10.

  Condition 1: The connection source line segment and the connection destination line segment are not separated by a predetermined value or more.

  Condition 2: The connection source line segment is not completely included in the connection destination line segment.

  Condition 3: The error between the portion extended when the start point or end point of the connection source line segment is extended to the connection destination line segment and the positions of the start point and end point of the connection source line segment is less than a predetermined value.

  In the process of step S10, as shown in FIG. 10, the image processing unit 47 creates a rectangular opposite-side candidate from the vertical and horizontal line segments subjected to the division process and the connection process of step S8 and step S9. (In the example shown in FIG. 10, the opposite side candidate in the horizontal direction is shown as a pair of line segment H1 and line segment H2, and the opposite side candidate in the vertical direction is shown as a pair of line segment V1 and line segment V2.) Specifically, the image processing unit 47 determines that the distance between line segments is not less than a predetermined value and the ratio of the other line segment to the length of one line segment is within a predetermined range (for example, 1/3 to 3 times). A plurality of pairs of line segments in the vertical direction and the horizontal direction are created as opposite side candidates. Thereby, the process of step S10 is completed, and the outline rectangle extraction process proceeds to the process of step S11.

  In the process of step S11, as shown in FIG. 11, the image processing unit 47 creates a combination of opposite side candidates in the vertical direction and the horizontal direction created by the process of step S10. Then, the image processing unit 47 calculates four intersections of the opposite side candidates for each combination. At this time, it is assumed that the image processing unit 47 uses only the slope information of the line segment and only needs to have an intersection on the extension of the line segment. That is, the case where the line segment does not actually cross over the detected intersection point is also included. Then, the image processing unit 47 creates a plurality of rectangular candidates S as shown in FIG. 12 having the calculated four intersections as vertices. Thereby, the process of step S11 is completed, and the outline rectangle extraction process proceeds to the process of step S12.

In the process of step S12, the image processing unit 47 calculates the length of the outer periphery of the rectangle candidate S created by the process of step S11. The length of the outer circumference can be calculated by adding the distances between the four vertices constituting the rectangle candidate S. Further, as shown in FIG. 13, the image processing unit 47 calculates the lengths of the vertical and horizontal line segments L located on the outer periphery of the rectangular candidate S. Then, the image processing unit 47 calculates the ratio of the length of the line segment L positioned on the outer periphery of the rectangle candidate S to the length of the outer periphery of the rectangle candidate S using the following Equation 1 as the score of each rectangle candidate S ( It is calculated as the probability of the rectangular area (scoring process). In Equation 1, the P coefficient is a line segment exceeding four vertices constituting the rectangle candidate S (for example, a line segment extending beyond the outer periphery of the regions R1, R2 and the rectangle candidate S shown in FIG. 13). This means a penalty coefficient for deducting the number of points of the rectangle candidate S in some cases, for example, 1.0 if the location where the line segment exceeds four vertices is 0, 0.8 if the location is 2, If it is a location, it is set to 0.64 or the like. The penalty coefficient is not limited to the above example. For example, when the subject is in a fixed form and the ratio between the length and the width is known in advance, the penalty coefficient becomes heavier as the ratio deviates from the ratio ( If the length of the outer periphery of the subject is known, the penalty coefficient value becomes heavier (less than 1) as the error from the outer periphery increases. Application is possible. Thereby, the process of step S12 is completed and the outline rectangle extraction process proceeds to the process of step S13.

  In the process of step S13, as shown in FIGS. 14A to 14D, the image processing unit 47 performs rectangular candidates S1 to S4 in descending order of the points calculated by the process of step S12, that is, in descending order of probability. Is superimposed on the captured image and displayed on the liquid crystal display unit 19. Specifically, as shown in FIG. 15, the image processing unit 47 cyclically displays the rectangular candidates S <b> 1 to S <b> 4 on the liquid crystal display unit 19 in descending order of probability as the user operates the ring key 17. To do. In the example shown in FIG. 15, the rectangle candidates are displayed on the liquid crystal display unit 19 in descending order of probability. However, the rectangle candidates may be color-coded according to the number of points and simultaneously displayed on the liquid crystal display unit 19.

  When there are a plurality of subjects in the captured image, it is preferable that the user can select whether or not to select a plurality of rectangle candidates. When the multiple correction mode is OFF, the image processing unit 47 cyclically displays the rectangular candidates on the liquid crystal display unit 19 in descending order of probability as the user operates the ring key 17, as shown in FIG. After performing the image processing such as coordinate conversion based on the rectangle candidate displayed and selected by the user, the contour rectangle extraction processing is completed. When the multiple correction mode is ON, the image processing unit 47 cyclically displays the rectangular candidates on the liquid crystal display unit 19 in descending order of probability as the user operates the ring key 17 as shown in FIG. After displaying and performing image processing such as coordinate conversion based on the rectangle candidate selected by the user, the rectangle candidate that has not been selected by the user can be selected so that further image processing can be performed. As a result, the user can correct the image by sequentially selecting the correct rectangular light for a plurality of subjects. Thus, the process of step S13 is completed, and the series of outline rectangle extraction processes ends.

  As is apparent from the above description, according to the outline rectangle extraction process according to the embodiment of the present invention, the image processing unit 47 detects the vertical and horizontal line segment information from the captured image, and the detected vertical The candidates for opposite sides in the vertical direction and the horizontal direction constituting the rectangular area are created from the line segment information in the direction and the horizontal direction. Further, the image processing unit 47 creates a plurality of combinations of the vertical opposite-side candidates and the horizontal opposite-side candidates, and for each combination, a rectangular area whose vertex is the intersection of the vertical opposite-side candidate and the horizontal opposite-side candidate is a rectangular candidate. Create as S. Then, the image processing unit 47 calculates the ratio of the length of the line segment L located on the outer periphery of the rectangle candidate S to the length of the outer periphery of the rectangle candidate S as the score of each rectangle candidate S, and along with the captured image according to the calculation result. A rectangle candidate S is displayed. Therefore, according to such a contour rectangle extraction process, the probability of the extracted rectangle candidate S can be presented to the user.

  As mentioned above, although the form mounted in the camera was demonstrated as useful embodiment of this invention, it is also possible to apply this invention to the image processing apparatus which does not have an imaging part. That is, an image captured by an external imaging device may be captured by a memory card, a USB cable, or the like, and the series of outline rectangle extraction processes described in the above embodiment may be performed. Further, in the above embodiment, the rectangle candidates are presented in descending order of scoring, and correction is performed by coordinate conversion after letting the user select. However, the rectangle candidates are corrected by coordinate conversion from those having high scoring, and the correction result May be presented sequentially to select the most preferable one for the user.

  As mentioned above, although the embodiment to which the invention made by the present inventor is applied has been described, the present invention is not limited by the description and the drawings that form part of the disclosure of the present invention according to this embodiment. That is, it should be added that other embodiments, examples, operation techniques, and the like made by those skilled in the art based on the above embodiments are all included in the scope of the present invention.

  The present invention can be applied to an imaging apparatus that extracts a rectangular region including the contour of a subject.

1 is a perspective view illustrating a configuration of a digital still camera according to an embodiment of the present invention, in which (a) is a front configuration mainly, and (b) is a perspective view mainly illustrating a rear configuration. It is a block diagram which shows the structure of the control system of the digital still camera shown in FIG. It is a flowchart figure which shows the flow of the outline rectangle extraction process used as embodiment of this invention. It is a figure which shows an example of the edge image extracted by the process of step S5 of FIG. It is a figure for demonstrating the structure of the Sobel filter used in the edge image extraction process of step S5 of FIG. It is a figure which shows an example of the edge image after the thin line and binarization process of step S6 of FIG. It is a figure which shows an example of the line segment information obtained by the labeling process of step S7 of FIG. It is a conceptual diagram for demonstrating the line segment division | segmentation process of step S8 of FIG. It is a conceptual diagram for demonstrating the line segment connection process of FIG.3 S9. It is a figure which shows an example of the opposite side candidate of the horizontal direction and vertical direction obtained by the pairing process of FIG.3 S10. It is a figure which shows an example of the combination of the opposite side candidate of the horizontal direction shown to FIG. It is a figure which shows an example of the rectangle candidate obtained from the combination of the opposite direction candidate of the horizontal direction shown in FIG. It is a figure for demonstrating the scoring process of step S12 shown in FIG. It is a figure which shows an example of the rectangle candidate displayed in order of the score calculated by the process of step S12 shown in FIG. It is a figure which shows a mode that the rectangle candidate displayed according to a user's operation changes in order of a score. It is a figure which shows the example of a display of a rectangle candidate in case a some subject is contained in the picked-up image. It is a figure which shows the flow of a process at the time of performing an image process with respect to a to-be-photographed image based on the displayed rectangle candidate.

Explanation of symbols

1: Digital still camera 2: Main body 42: Control unit 47: Image processing unit

Claims (7)

Line segment detection means for detecting vertical and horizontal line segments from the image;
Opposite side candidate creation means for creating vertical and horizontal opposite side candidates constituting a rectangular region from the vertical and horizontal line segments detected by the line segment detection unit;
A plurality of combinations of the vertical opposite side candidate and the horizontal opposite side candidate created by the opposite side candidate creating means are created, and a rectangular region having a vertex at the intersection of the vertical opposite side candidate and the horizontal opposite side candidate for each combination Rectangle candidate creating means for creating a rectangle candidate;
Scoring means for calculating the probability of each rectangular candidate based on the ratio of the length of the opposite side candidate located on the outer periphery of the rectangular candidate to the outer peripheral length of the rectangular candidate created by the rectangular candidate creating means;
An image processing apparatus comprising: The image processing apparatus according to claim 1 .
The scoring means deducts a predetermined number of calculated values when a line segment constituting the opposite side candidate extends beyond the outer periphery of the rectangular candidate. The image processing apparatus according to claim 1 or 2 ,
An image processing apparatus, further comprising display control means for displaying a rectangular candidate together with the photographed image according to the calculation result of the scoring means. The image processing apparatus according to claim 3 .
The image processing apparatus, wherein the display control means displays the rectangle candidates in descending order of the probability calculated by the scoring means. The image processing apparatus according to claim 3 .
The image processing apparatus, wherein the display control means displays a plurality of rectangle candidates in different colors based on the probability calculated by the scoring means. The image processing apparatus according to any one of claims 1 to 5 ,
Further comprising an imaging means,
The image processing apparatus according to claim 1, wherein the image is an image captured by the imaging unit. Line segment detection processing for detecting vertical and horizontal line segments from the image;
Opposite side candidate creation processing for creating vertical and horizontal opposite side candidates constituting a rectangular region from the vertical and horizontal line segments detected by the line segment detection process;
A plurality of combinations of vertical opposite side candidates and horizontal opposite side candidates created by the opposite side candidate creation process are created, and for each combination, a rectangular area whose vertex is the intersection of the vertical opposite side candidate and the horizontal opposite side candidate Rectangle candidate creation processing to create as a rectangle candidate;
A scoring process for calculating the probability of each rectangular candidate based on the ratio of the length of the opposite side candidate located on the outer periphery of the rectangular candidate to the outer peripheral length of the rectangular candidate created by the rectangular candidate creation process. A computer program that is executed.