JP4380268B2 - Image region selection method and apparatus - Google Patents

Description

  The present invention relates to an image region selection method and apparatus suitable for use in selecting a partial region from one image for image processing such as enlargement and trimming.

  With the widespread use of digital cameras, personal computers (hereinafter referred to as PCs), image data taken with digital cameras can be easily edited on PCs. Furthermore, there is a request for simple editing within a device such as a digital camera or a mobile phone, and a request for easily checking a captured image. However, since portable devices such as digital cameras and mobile phones cannot use an interface such as a mouse, when you want to enlarge a part of an image, for example, specify the upper left and lower right of the desired area by pressing the cross key repeatedly. It was forced to perform complicated operations such as necessity.

  In the process of specifying and enlarging a certain point in the image, the simplest method is to determine the area centered on the image center / predetermined point at a predetermined magnification as described in Patent Document 1. . However, in this case, since there is no optimal magnification, for example, the user is forced to perform an operation of enlarging by 1.1 times.

  In addition, as described in Patent Document 2, there is also proposed a method of detecting an effective image area from a copy without specifying a magnification and enlarging it according to a sheet to be output. However, the effective image area is a non-blank area having no image or text, and is different from the technique for detecting the effective area in the image as the subject of the present invention.

Next, there is a method of extracting a region by calculating a certain feature from one point designated by the user. For example, in Patent Document 3, contour extraction is performed to extract a region from an image. However, with this method, only one area can be specified, and it is not possible to perform extraction over a plurality of areas, such as when it is desired to select a background or the like at the same time.
JP 2000-75416 A JP 7-327129 A JP 2000-125119 A

  By the way, in a photographic image, a plurality of types of subjects such as people, things, and scenery are often included in one photograph. In such a case, for example, there is a demand for increasing the range representing a person as much as possible and appropriately including a background image. In such a case, a partial area suitable for the user's intention can be specified by directly specifying the range to be enlarged or non-trimmed. However, the conventional technique has a problem that when a range is specified, a delicate operation with a mouse or the like is required, or a plurality of operations are required.

  On the other hand, in the conventional techniques described in Patent Documents 1 to 3 and the like, for example, a process such as enlargement is performed around a specified target. With such a technique, there are cases where sufficient consideration is not given to subjects other than the designated subject. That is, there is a possibility that the partial area cannot be designated as intended by the operator.

  The present invention has been made in view of the above circumstances, and an image region selection method and an image that can enable selection of a partial region in a photographic image with a simple operation and in a form that suits the operator's intention as much as possible. An object is to provide an area selection device.

  The present invention is a method for selecting a partial area from one image, a designated area extracting process for extracting a surrounding area including a designated point on an image designated by an operator as a designated area, and a designated area extracting process Extracted in the specified region extraction process based on the compositional analysis process to determine the positional relationship of the specified region extracted in the entire image, the positional relationship determined in the compositional analysis process, and the value related to the size of the partial region And a partial area selecting step of selecting a partial area including the designated area. According to this, the partial area is selected based on the value related to the size of the partial area to be selected on the basis of the positional relationship that the specified area occupies in the entire original image. That is, it is possible to select a partial area including the designated area in consideration of the configuration (composition) of the entire picture screen. It is thought that there are many cases where the intention of a person is included in the composition in the photograph. Therefore, according to the present invention, it is considered that there is a high possibility that the partial region can be selected in a form that matches the operator's intention as much as possible. Further, since the area around the designated point designated by the operator is automatically extracted as the designated area, the operation is simple.

  The present invention is also characterized in that, in the designated area extraction process, the image is divided into a plurality of divided areas, and then a divided area including a designated point on the image designated by the operator is extracted as the designated area. To do. According to this, the responsiveness after designating the designated point can be improved, or the designated point can be designated more easily by displaying the divided area when receiving the designation of the operator, for example.

  The present invention is also characterized in that, in the designated region extraction process, an image region having characteristics similar to the designated point around the designated point on the image designated by the operator is extracted as the designated region. According to this, it is possible to shorten the processing time before designating the designated point, and it is possible to designate a region that is finer than when a divided region is prepared.

  The present invention is also characterized in that, in the composition analysis process, the positional relationship is obtained based on respective distances from the outer periphery of the designated area to the outer periphery of the image. According to this, it is possible to select a partial region in consideration of the composition by a relatively simple calculation process.

  The present invention is also characterized in that, in the partial region selection process, the partial region is selected so as to maintain the aspect ratio of the image. According to this, since the shape of the selected partial region is constant, the operator can easily predict the processing result, and for example, it becomes easy to grasp the specification point of the specified point.

  The present invention is also characterized in that, in the partial area selection process, the partial area is selected so as to match the aspect ratio of the output medium. This makes it easier to obtain a desired result when outputting to an output medium such as a display, printer, display panel, or television screen.

  The present invention is further characterized in that in the partial region selection step, a predetermined template image is synthesized with the selected partial region. According to this, it is possible to perform image processing with a high degree of freedom by a simple operation.

  The present invention is also characterized in that the value relating to the size is either an enlargement ratio with respect to the partial region or an area ratio before and after enlargement. By selecting the size of the partial area based on values such as an enlargement ratio and an area ratio, the user can easily grasp the set value as compared with the case where the size is specified by, for example, the number of pixels or the length and width.

  The present invention is also characterized in that the partial area can be changed in accordance with an instruction from an operator in the partial area selection process. According to this, when it is desired to change the selection of the partial area in consideration of the automatic extraction of the designated area and the composition, the means is provided to the user.

  The present invention is also an apparatus for selecting a partial area from one image, a designated area extracting means for extracting a surrounding area including a designated point on an image designated by an operator as a designated area, a designated area The designated area extracting means extracts the composition based on the composition relation calculating means for obtaining the positional relationship in the entire image of the designated area extracted by the extracting means, the positional relation obtained by the composition analyzing means, and the value related to the size of the partial area. And a partial area selecting means for selecting a partial area including the designated area. The present invention is also a program for selecting a partial region from one image, and a designated region extraction process for extracting a surrounding region including a designated point on an image designated by an operator as a designated region; Based on the composition analysis process for determining the positional relationship in the entire image of the specified area extracted in the specified area extraction process, the positional relationship determined in the composition analysis process, and the value related to the size of the partial area It includes a description for executing by a computer a partial area selection process for selecting a partial area including a designated area extracted in the extraction process.

  The best mode for carrying out the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram for explaining the configuration of the image region selection apparatus according to the present embodiment. The image area selection device of the present invention is realized as a software program executed using hardware resources of a personal computer, or using a central processing unit of a portable information device such as a mobile phone or a digital camera. It can be realized as a software program to be executed. The example shown in FIG. 1 shows each function constituting a software program as a block when implemented in such a device. However, the image storage unit 10 is configured by a storage device such as a card-type nonvolatile storage medium. Here, an apparatus for executing the program constituting the present embodiment is provided with at least an operator for inputting a user operation and a display apparatus for displaying image information. And

  The image selection unit 1 selects one piece of image data that the user intends to select a partial area from a plurality of electrophotographic images stored in the image storage unit 10 in accordance with a user operation. The designated area setting unit 2 sets one or a plurality of designated points on one image selected by the image selection unit 1 according to a user operation. The designated area setting unit 2 further extracts the area around the designated point including the designated point as the designated area. The composition analysis / output region calculation unit 3 obtains the positional relationship of the designated region extracted by the designated region setting unit 2 in the entire image, and also calculates a value relating to the obtained positional relationship and the size of the partial region serving as the output region. A partial area including the designated area is selected based on the above. The composition analysis / output area calculation unit 3 stores newly created image data in the image storage unit 10 based on the selected partial area, or enlarges and displays only the selected partial area on a predetermined display device. Or print out information indicating a partial area to a printer or output it to another processing function for outputting to a predetermined display device.

  FIG. 2 is a flowchart for explaining the processing flow of the image region selection apparatus shown in FIG. (A) and (b) of FIG. 2 show processing by each unit of the image selection unit 1, the designated region setting unit 2, and the composition analysis / output region calculation unit 3 of FIG. However, the method of setting the designated area by the designated area setting unit 2 is different between FIGS.

  In the process shown in FIG. 2A, an image is first selected by the image selection unit 1 (step S11), and then the image selected by the designated region setting unit 2 is divided into a plurality of divided regions according to the image characteristics and the like. Divided (step S12). Thereafter, the divided area including the designated point on the image designated by the user is designated as the designated area (step S13). Then, composition analysis processing by the composition analysis / output region calculation unit 3 and processing for obtaining and selecting a partial region by calculation and outputting it as an output region are performed (step S14).

  The dividing process into the divided areas in step S12 by the designated area setting unit 2 can be performed as follows, for example. Here, it is assumed that color information of a given image is expressed in RGB (Red (Red) Green (Green) Blue (Blue)). The designated area setting unit 2 first converts the RGB color information into HSV (Hue (saturation) Value (lightness)) information. H (hue) is expressed in a ring shape, and the hue can be classified according to the angle. For example, when the color is classified into 24 colors, the color is classified by 15 degrees. Thereby, a given image can be classified into 24 approximate color regions. A divided region is formed for each successive approximate region classified into the same color.

  Here, the HSV method is used, but a method having hue information such as Lab (love color; a color model for digitizing color information) may be used. Further, instead of a fixed classification method such as 24 colors, a classification method such as a k-means method may be used.

  On the other hand, in the process shown in FIG. 2B, an image is first selected by the image selection unit 1 (step S21). Next, the user designates one (or a plurality of) designated points on the image (step S22), and an image region having characteristics similar to the designated points is extracted as the designated region around the designated point. (Step S23). Then, composition analysis processing by the composition analysis / output region calculation unit 3 and processing for obtaining and selecting a partial region by calculation and outputting it as an output region are performed (step S24).

  The process of extracting an image area having characteristics similar to the designated point around the designated point in step S23 by the designated area setting unit 2 can be performed as follows, for example. Various methods have been proposed as the region extraction method, and any method may be used. Here, the simplest method is shown as an example. First, it is assumed that color information of a given image is expressed in RGB. When the designated point is (R, G, B) = (222, 151, 135), the approximate color region is determined based on a preset threshold value. For example, when the threshold value is 30, pixels having color information in the range of R from 192 to 252, G from 121 to 181 and B from 105 to 165 can be determined as the approximate color region.

  Further, only pixels adjacent to the designated point are extracted from the approximate color region. If four points (up, down, left, and right) adjacent to the designated point are approximate color regions, the points are extracted. If it is not an approximate color area, it is not included in the extraction area. Similarly, by extracting adjacent points that are approximate color areas at the extracted adjacent points, it is possible to extract an approximate color area that is adjacent to the designated point. Here, the processing is performed in the RGB method, but color information such as HSV or Lab may be converted.

  Next, details of processing by the composition analysis / output region calculation unit 3 in FIG. 1 will be described with reference to FIGS. FIG. 3 shows the original image 20 selected by the image selection unit 1. In the original image 20, an area 30 representing a person, an area 40 representing the sky, an area 50 representing a mountain, and an area 60 representing a sign are projected as subjects. In addition, it is assumed that the region 30 representing a person is formed of a region 31 corresponding to the head and a region 32 corresponding to the torso. In step S12 of FIG. 2, it is assumed that at least each area is divided as a divided area.

  By calculating one of these divided areas (step S13 in FIG. 2), or from one area determined by one point indicated (step S22 in FIG. 2), the top, bottom, left and right edges are calculated. A rectangle that encloses the area is obtained. In this case, it is assumed that one point in the region 31 is designated by the user in step S13 of FIG. 2 or step S22 of FIG. In this case, an area including a divided area designated by one designated point in the area 31 or an area calculated based on one designated point by the designated area setting unit 2 in FIG. It is assumed that the designated area 21 shown in FIG.

  The composition analysis / output area calculation unit 3 firstly has a distance D_left from the left end of the designated area 21 to the left end of the image 20 shown in FIG. 4, a distance D_right from the right end of the designated area 21 to the right end of the image 20, A distance D_up from the upper end to the upper end of the image 20 and a distance D_down from the lower end of the designated area 21 to the lower end of the image 20 are calculated. Next, the composition analysis / output area calculation unit 3 obtains a partial area 22 (see FIG. 5) having a size corresponding to the enlargement ratio of the original image 20 as an output area, for example, based on an enlargement ratio preset by the user.

  FIG. 5 is a diagram for explaining the relationship among the original image 20, the designated area 21, and the partial area 22. The original image 20 is an image having a horizontal pixel number X1 and a vertical pixel number Y1. The designated area 21 is an area having a horizontal pixel number X2 and a vertical pixel number Y2. The partial area 22 is an area having a horizontal pixel number Xout and a vertical pixel number Yout. The variables d_left, d_right, d_up, and d_down are respectively the distance from the left end of the designated area 21 to the left end of the partial area 22, the distance from the right end of the designated area 21 to the right end of the partial area 22, and the portion from the upper end of the designated area 21 The distance to the upper end of the area 22 and the distance from the lower end of the designated area 21 to the lower end of the partial area 22 are shown.

  The selection of the partial area 22 by the composition analysis / output area calculation unit 3 is based on the positional relationship of the designated area 21 in the entire image 20 and the area ratio and enlargement ratio, which are values related to the size of the partial area 22 serving as the output area. Based on. Here, the contents of processing when the ratio of the vertical and horizontal distances between the designated area 21 and the image 20 is used as the value relating to the size, when the area ratio is used, and when the enlargement ratio is used will be described.

(1) Example using ratios of distances: In this example, when the ratio of distances is an enlargement ratio R, the distances d_left, d_right, d_up, and d_down from the designated area 21 to the partial area 22 shown in FIG. Is calculated as a value obtained by multiplying each distance D_left, D_right, D_up, and D_down from the designated area 21 to the image 20 by the enlargement ratio R. That is, the distances from the designated area 21 to the partial area 22 are obtained as d_left = D_left * R, d_right = D_right * R, d_up = D_up * R, and d_down = D_down * R. Here, the enlargement ratio R is set to 0-1.

となり、指定領域２１から拡張すべき増分値は、

である。 (2) Example using area ratio: In this example, for example, the area of the original image 20 is set to 1, and the area of the partial region 22 is set based on a value a of less than 1 and 0 or more. From FIG. 5, the area S1 of the original image 20 is S1 = X1 * Y1 (X1: original image horizontal pixel number, Y1: vertical pixel number). On the other hand, the area S2 of the designated region 21 is S2 = X2 * Y2 (X2: the number of horizontal pixels in the designated region, Y2: the number of vertical pixels). If the user designates an area ratio a (1> a ≧ 0), the horizontal pixel number Xout and the vertical pixel number Yout of the partial region 22 are

The increment value to be expanded from the designated area 21 is

It is.

と求められる。ただし、面積比ａによっては、部分領域２２が指定領域２１を下回る場合がある。この場合は、

として、出力領域となる部分領域２２を計算する。 Increment values from the upper, lower, left, and right areas use values related to the positional relationship between the original image 20 and the designated area 21 (upper, lower, left, and right distances D_left, D_right, D_up, and d_down = D_down),

Is required. However, depending on the area ratio a, the partial region 22 may fall below the designated region 21. in this case,

As described above, the partial area 22 to be the output area is calculated.

で求められる。この場合、指定領域２１から拡張すべき増分値は、

である。 (3) Example using an enlargement ratio: In this example, assuming that the partial area 22 matches the original image 20 when the enlargement ratio is 1, the enlargement ratio of the number of pixels (length) in the vertical and horizontal directions of the partial area 22 is Set with a value of 1 or greater. If the user specifies the enlargement ratio b (≧ 1), the horizontal pixel number Xout and the vertical pixel number Yout of the partial area 22 are

Is required. In this case, the increment value to be expanded from the designated area 21 is

It is.

となる。ただし、拡大率ｂによっては、部分領域２２が指定領域２１を下回る場合がある。この場合は、

として、出力領域となる部分領域２２を計算する。 In addition, the increment value from each of the upper, lower, left, and right areas uses values (positions D_left, D_right, D_up, and d_down = D_down) for the positional relationship between the original image 20 and the designated area 21,

It becomes. However, depending on the enlargement factor b, the partial area 22 may be lower than the designated area 21. in this case,

As described above, the partial area 22 to be the output area is calculated.

  Next, a case where the aspect ratio of the partial area 22 is adjusted by the composition analysis / output area calculation unit 3 will be described. The composition analysis / output area calculation unit 3 selects the partial area 22 so as to maintain the aspect ratio of the image 20, or selects the partial area 22 so as to match the aspect ratio of the output medium such as a printer. Can do. Here, an example of maintaining the aspect ratio of the image 20 will be described. In the selection process of the partial area 22 described above, when the ratio of each distance of (1) is used, the aspect ratio of the partial area 22 is usually different from the aspect ratio of the image 20. When the area ratio and the enlargement ratio of (2) and (3) are used, the aspect ratio of the partial region 22 is obtained in accordance with the aspect ratio of the image 20.

  Here, a case will be described in which the composition analysis / output region calculation unit 3 changes the aspect ratio of the partial region 22 selected by the enlargement ratio R to a: b = X1: Y1 which is the same as the aspect ratio of the original image 20. Here, as shown in FIG. 6, the number of horizontal pixels and the number of vertical pixels of the partial area 22 before change are set to x and y, and the number of horizontal pixels and the number of vertical pixels of the partial area 22a after change are set to X and Y. .

これによって増分値ｄｘが次式で決まる。

この増分値ｄｘを次式によって左右に振り分ける。このとき、構図を考慮して、

とする。 When the aspect ratio y / x of the partial area 22 and the aspect ratio b / a of the original image 20 have a relationship of b / a <y / x, the horizontal pixel count X and vertical pixel count Y of the partial area 22a after the change are Is obtained by the following equation.

As a result, the increment value dx is determined by the following equation.

This increment value dx is distributed to the left and right by the following equation. At this time, considering the composition,

And

で、縦方向の増分値ｄｙが求められる。そして、これらの計算結果に従ってアスペクト比の調整が行われる。 Similarly, when b / a ≧ y / x,

Thus, the vertical increment value dy is obtained. Then, the aspect ratio is adjusted according to these calculation results.

  As shown in FIG. 7, the composition analysis / output region calculation unit 3 may further have a function of synthesizing a predetermined template image 23 with the selected partial region 22. The composition analysis / output region calculation unit 3 combines the selected partial region 22 with the template image 23, changes the aspect ratio, etc., or enlarges the region, synthesizes it with the template image 23, and outputs it.

  Alternatively, as shown in FIG. 8, a function may be provided that allows the partial area 22 selected in accordance with a user instruction to be changed. The user can change the partial area 22b as the output area by providing an interface that allows the user to operate the periphery of the partial area 22 with a pointer 23 or the like in order to make a slight adjustment to the partial area 22. Can do.

  According to the present embodiment, for example, a feature region is extracted at one point (one operation / or several designation operations) in an image, and at the same time, composition analysis is performed to obtain the intention (composition) of the original image. The fetched area can be output. In particular, in a device (such as a digital camera or a mobile phone) that does not have an interface with excellent operability such as a mouse, a desired area can be easily acquired.

  Also, assuming that a photograph is originally taken in consideration of a certain amount of composition, it is effective to perform trimming in consideration of the composition even when one object in the image is selected.

  Further, the embodiment of the present invention can be applied as, for example, enlargement image processing / IF (interface) for portable devices (devices without a mouse, pen-type interface, etc.). Further, the present invention can be applied to a system / editing system / printing system to be combined with a template. For example, if you want to synthesize a photograph of a person centered on the background with a template image, if you specify a person, the person is extracted, the area that includes the background is cropped from the composition analysis, and the original intent of the photograph is taken A composite photograph that has been enlarged without deviating from the above is automatically obtained.

The block diagram which shows the structural example of the image area selection apparatus of this invention. The flowchart for demonstrating the flow of a process of the structure of FIG. Explanatory drawing for demonstrating the image of the process target by the structure of FIG. FIG. 6 is another explanatory diagram for explaining an image to be processed with the configuration of FIG. 1. FIG. 6 is another explanatory diagram for explaining an image to be processed with the configuration of FIG. 1. FIG. 6 is another explanatory diagram for explaining an image to be processed with the configuration of FIG. 1. Explanatory drawing for demonstrating the image obtained as a processing result by the structure of FIG. FIG. 6 is another explanatory diagram for explaining an image to be processed with the configuration of FIG. 1.

Explanation of symbols

1 image selection unit, 2 designated area selection unit, 3 composition analysis / output area calculation unit

Claims (8)

A method in which a portable information device, which is a digital camera or a mobile phone, selects a partial area from one image,
Hue calculated by converting RGB color information of the image expressed in RGB into HSV information
Based on the image, the image is divided into a plurality of approximate color areas and classified, and continuous images classified into the same color are classified.
A designated area extracting step of forming a divided area for each approximate area and extracting the divided area including the designated point on the image designated by the operator as the designated area;
A composition analysis process for obtaining a positional relationship in the entire image of the designated area extracted in the designated area extraction process;
A partial area selection process for selecting a partial area including the designated area extracted in the designated area extraction process based on the positional relationship obtained in the composition analysis process and a value related to the size of the partial area. An image region selection method as a feature. In the composition analysis process, a distance D_left from the left end of the designated area to the left end of the image
, The distance D_right from the right edge of the designated area to the right edge of the image, and the top edge of the designated area
The distance D_up from the lower end of the designated area to the lower end of the image D_up
down as the positional relationship,
In the partial area selection process, the horizontal pixel number Xout and the vertical pixel number Yout of the partial area are calculated.
The horizontal pixel count X1 of the image, the vertical pixel count Y1 of the image, the horizontal pixel count X1 and the vertical pixel count Y1
And an area ratio a (1> a ≧ 0) designated by the operator.
) And the following formula

The horizontal pixel count X1 of the image, the vertical pixel count Y1 of the image, and the enlargement designated by the operator.
The following formula based on large factor b (≧ 1)

The horizontal pixel count X2 of the designated area and the vertical image of the designated area
Based on the prime number Y2, the horizontal increment value dx and the vertical increment value dy to be expanded from the specified area are expressed as follows:
formula

The distance d_left from the left end of the specified area to the left end of the partial area
, A distance d_right from the right end of the specified area to the right end of the partial area, and the specified area
A distance d_up from the upper end of the area to the upper end of the partial area, and the lower end of the designated area to the part
The distance d_down to the lower end of the minute area is expressed as follows:

The image region selection method according to claim 1, wherein the partial region is selected by calculation according to the method. 3. The image region selection method according to claim 1, wherein, in the partial region selection process, the partial region is selected so as to maintain an aspect ratio of the image. The image region selection method according to any one of claims 1 to 3, wherein, in the partial region selection step, the partial region is selected so as to match an aspect ratio of an output medium. 5. The image region selection method according to claim 1, further comprising: combining a predetermined template image with the selected partial region in the partial region selection step. The image region selection method according to claim 1, wherein in the partial region selection process, the partial region can be changed in accordance with an instruction from an operator. A portable information device that is a digital camera or mobile phone that selects a partial area from one image
Where
Hue calculated by converting RGB color information of the image expressed in RGB into HSV information
Based on the image, the image is divided into a plurality of approximate color areas and classified, and continuous images classified into the same color are classified.
A designated area extracting means for forming a divided area for each approximate area and extracting the divided area including the designated point on the image designated by the operator as the designated area;
Composition analysis means for obtaining a positional relationship in the entire image of the designated area extracted by the designated area extraction means;
A partial area selecting means for selecting a partial area including the designated area extracted by the designated area extracting means based on the positional relationship obtained by the composition analyzing means and a value relating to the size of the partial area. Portable information device . A portable information device that is a digital camera or a mobile phone is a program for selecting a partial area from one image,
Hue calculated by converting RGB color information of the image expressed in RGB into HSV information
Based on the image, the image is divided into a plurality of approximate color areas and classified, and continuous images classified into the same color are classified.
A designated area extracting step of forming a divided area for each approximate area and extracting the divided area including the designated point on the image designated by the operator as the designated area;
A composition analysis process for obtaining a positional relationship in the entire image of the designated area extracted in the designated area extraction process;
A partial area selection process for selecting a partial area including the specified area extracted in the specified area extraction process based on the positional relationship obtained in the composition analysis process and the value related to the size of the partial area is executed by a computer. An image region selection program characterized by including a description for

Images