JP2005159850A - Composition of image and character string - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technique capable of overlaying a character string on an appropriate position of an object image. <P>SOLUTION: First, a noticing area PA is detected by analyzing an object image MF. The position of a character frame area CA is determined to avoid the noticing area PA. By arranging the character string in the character frame area CA, a composite image with the character string overlaid on the object image MF is created. <P>COPYRIGHT: (C)2005,JPO&NCIPI

Description

  The present invention relates to a technique for arranging a character string on a target image.

  When printing or displaying an image generated by an image generation device such as a digital camera, there are cases where it is desired to arrange character strings in an overlapping manner. As a technique for arranging a character string in an image, for example, one described in Patent Document 1 is known.

JP 2003-67175 A

  In this prior art, by setting a plurality of character sizes in advance and selecting an appropriate size from among these when displaying a screen, the arrangement relationship between character strings and images during printing and screen display is determined. Mitigating discrepancies.

  By the way, generally, when a character string is superimposed on an image (particularly a photographic image), there is a demand to arrange the character string so that it does not overlap an important part in the image. For this reason, a technique that can realize such an arrangement has been desired.

  The present invention has been made to solve the above-described conventional problems, and an object of the present invention is to provide a technique that can superimpose a character string at an appropriate position of a target image.

In order to achieve at least a part of the above object, an apparatus of the present invention is an image processing apparatus for arranging a character string on a target image,
An attention area detection unit for detecting an attention area by analyzing a target image;
A character frame region determination unit that determines the position of the character frame region so as to avoid the attention region;
An image compositing unit that generates a composite image in which the character string is superimposed on the target image by arranging a character string in the character frame region;
Is provided.

  In this apparatus, the target image is detected by analyzing the target image, and the position of the character frame region is determined so as to avoid the target region, so that the character string can be superimposed at an appropriate position of the target image.

  The character frame region determination unit may determine the position of the character frame region so that the character frame region and the attention region do not overlap.

  According to this configuration, since the character string and the attention area do not overlap at all, it is possible to prevent an important part of the target image from being hidden by the character string.

The character frame region determination unit
A color change index indicating a better value as the color change of the target image at a position corresponding to the character frame region is smaller;
The position of the character frame region may be determined so that the character frame region and the region of interest do not overlap and the color change index takes a value within a predetermined range that is close to the best value. .

  Where the color change of the image is small, the importance as an image is often low. Therefore, according to this configuration, the character string can be arranged at a position other than the region of interest and at a place where the importance of the image is low.

  The character frame region determination unit may determine the character frame region and the size of each character so that the character frame region and the attention region do not overlap.

  According to this configuration, since the character string and the attention area do not overlap at all, it is possible to prevent an important part of the target image from being hidden by the character string.

The character frame region determination unit
When the character frame area and the attention area partially overlap, calculate an overlap evaluation index indicating a better value as the amount of overlap between the character frame area and the attention area decreases,
The position of the character frame area may be determined so that the overlap evaluation index takes a value in a predetermined range close to the best value.

  According to this configuration, the character string can be arranged at a position where the overlap between the character string and the attention area is small.

The character frame region determination unit
A color change index indicating a better value as the color change of the target image at a position corresponding to the character frame region is smaller;
The position of the character frame region is determined so that the overlap evaluation index takes a value in a predetermined range close to the best value and the color change evaluation index takes a value in a predetermined range close to the best value. It may be.

  According to this configuration, the character string can be arranged at a position where the overlap between the character string and the attention area is relatively small and the importance as an image is relatively low.

  The attention area detection unit may be capable of detecting an area including a human face as a first type of attention area.

  According to this configuration, character strings can be arranged so as to avoid the human face, which is usually regarded as most important.

  The attention area detection unit further calculates an importance index indicating a better value as the importance of the image portion of the target image is higher, and detects an area having a better importance index as a second type of attention area. It is good also as what can be done.

  According to this configuration, the character string can be arranged so as to avoid an important image portion other than the face.

The character frame area determination unit, when the first type and the second type of attention area exist,
(I) There is no overlap between the character frame area and the first type of attention area, and
(Ii) An overlap evaluation index indicating a better value as the overlap between the character frame area and the second type of attention area is smaller is calculated, and the overlap evaluation index takes a value in a predetermined range close to the most favorable value. In addition,
The position of the character frame area may be determined.

  According to this configuration, the character string can be arranged so that the character string does not overlap with the first type of attention area including the face and the overlap with the second type of attention area becomes small.

The target image is arranged in a part of the print area,
The character frame area determination unit may arrange the character frame in the print area.

  According to this configuration, the character string can be arranged not only in the region of the target image but also in the print region, so that the character string can be arranged in a more appropriate position.

The image processing apparatus further includes:
A character color determining unit that determines the color of the character string in accordance with the color of the target image in the character frame region may be provided.

  According to this configuration, it is possible to display a character string clearly.

  The character color determination unit may determine the color of the character string so that a hue of the character string is different from an average color hue of the target image in the character frame region.

  Or the said character color determination part is an image processing apparatus which determines the color of each character so that the hue of each character and the hue of the said target image in the position which touches each character may differ.

  The present invention can be realized in various modes. For example, an image processing method and apparatus for arranging character strings on a target image, an image processing system, and a target image and a character string are combined. An image output method and apparatus for outputting a recorded image, a computer program for realizing the functions of the method or apparatus, a recording medium on which the computer program is recorded, and the like can be realized.

Next, the best mode for carrying out the present invention will be described in the following order based on examples.
A. First embodiment:
B. Second embodiment:
C. Variation:

A. First embodiment:
FIG. 1 is an explanatory diagram showing an image processing system as a first embodiment of the present invention. This image processing system includes a digital camera 100, a computer 200, and a color printer 300. The computer 200 includes an image processing unit 210 that executes a process of superimposing a character string on a target image. The image processing unit 210 may be provided in the digital camera 100 or may be provided in the color printer 300.

  The image processing unit 210 includes a character input reception unit 220, an image selection unit 230, a region of interest detection unit 232, a character frame region determination unit 234, a character color determination unit 236, an image composition unit 238, and image data output. A unit 240, a decoration image storage unit 250, and a decoration image selection unit 252. The decorative image storage unit 250 is provided in a hard disk of the computer 200, for example. Functions of other components of the image processing unit 210 are realized by a computer program.

  FIG. 2 is a flowchart showing a processing procedure in the first embodiment. FIG. 3 is an explanatory diagram illustrating processing contents of the embodiment. In step S1, the image selection unit 230 (FIG. 1) reads the target image MF to be processed. The target image MF may be supplied from the digital camera 100 to the computer 200 or selected by the user from a plurality of images stored in the memory (hard disk or portable storage medium) of the computer 200. It may be a thing. In the latter case, the image selection unit 230 displays a predetermined selection screen and causes the user to select the target image MF.

  FIG. 3A shows an example of the target image MF. This target image MF is a photographic image including two persons. Note that a photographic image is preferably selected as the target image MF.

  In step S2 of FIG. 2, the image selection unit 230 resizes (enlarges or reduces) the target image MF to a desired size. For example, when the entire target image MF is printed on the entire surface of a standard size (for example, postcard size) print medium, the target image MF is enlarged or reduced so as to match the size of the print medium. Note that when performing so-called borderless printing, the entire target image MF is often set slightly larger than the size of the print medium in order to prevent the occurrence of white spots at the border. In this specification, the phrase “the entire target image MF matches the size of the print medium” has a broad meaning including such a case.

  In step S3, a decoration image is selected and a character string is input. When selecting a decoration image, the decoration image selection unit 252 displays a plurality of decoration images on the screen, and the user designates a desired decoration image from the plurality of decoration images. FIG. 3E shows a state in which the decoration image FM is added to the outer periphery of the target image MF. Note that the presence or absence of the decoration image FM is arbitrary. Below, the process which synthesize | combines a character string with respect to the object image MF to which the decoration image FM is not added is demonstrated.

  When inputting a character string, the user inputs a desired character string CIN using input means such as a keyboard, and the character input receiving unit 220 receives this input character string CIN. At this time, the font type and size of the input character string CIN are also input. Note that the image processing unit 210 may synthesize the target image and the character string using a predetermined standard character string set in advance instead of the user input character string CIN. For example, a standard character string “Happy New Year” can be used when printing a New Year's card. Further, the target image and the character string may be synthesized using both the input character string CIN and the standard character string. In this specification, “character string” means a series of one or more characters.

  In step S4, the attention area detection unit 232 detects the attention area in the target image MF by analyzing the image data of the target image MF. The “attention area” is typically an area that is easily noticeable. In the first embodiment, an area including a human face is detected as an attention area. FIG. 3B shows a state in which two attention areas PA are detected in the target image MF. The attention area PA is preferably set to a predetermined geometric shape (for example, a rectangle) including the face area.

  As a method for detecting a face region in an image, for example, Technical Report of IEICE, PRMU2001-217, pp.87-94 (2002-1), “Independent of background and face direction. The method described in “Detection of face and estimation of face direction” can be used. Although this document describes not only the detection of the face area but also the estimation of the face direction, in the present embodiment, only the detection of the face area may be performed. Various other face area detection methods can also be used.

  In step S5, the character frame area determination unit 234 determines the position of the character frame area in consideration of the attention area PA. In the first embodiment, as shown in FIG. 3C, first, a character frame arrangement possible area PCA is determined, and a character frame area CA is arranged in this area PCA. The character frame allocable area PCA is an area that does not overlap with the attention area PA including the face area. In this embodiment, the character frame arrangement possible area PCA is detected as a rectangular area. Regarding the target image MF in FIG. 3A, there can exist character frame allocable areas of various shapes and sizes in addition to the character frame allocable area PCA shown in FIG. 3C. In C), only the largest one is shown. The character frame area CA is arranged in this character frame arrangement possible area PCA. The “character frame area” is an outer frame area of a character string to be arranged. In this embodiment, the font type and size of the character string are designated in advance in step S3, and the size of the character frame area CA is determined in accordance with these.

As a method for determining the position of the character frame area CA in the character frame arrangeable area PCA, the following various methods can be employed.
(A-1) It is arranged at the center of the character frame arrangement possible area PCA.
(A-2) A position where the color change of the target image MF at the position corresponding to the inside of the character frame area CA is the smallest is determined.

  The method (A-2) simply means that the character frame area CA is arranged at a position where the target image MF has a substantially uniform color. Specifically, for example, the character frame area CA may be arranged in an area where the hue variance is small. If this method (A-2) is utilized, a character string can be arrange | positioned to a less important image part. On the other hand, if the method (A-1) is employed, it is possible to arrange character strings at well-balanced positions with simple calculations. The method (A-2) can be described more generally as the following method (A-2 ′).

(A-2 ′) A color change index CCI indicating a better value as the color change of the target image MF at a position corresponding to the inside of the character frame area CA is smaller is calculated, and the color change index CCI is the most favorable value. The position of the character frame area CA is determined so as to take a value in a predetermined range close to.

  Here, as the color change index CCI, for example, a statistical value such as color dispersion or standard deviation of the target image MF in the character frame area CA can be used. Specifically, for example, the dispersion of hue values can be used as the color change index CCI. In the above method (A-2 ′), the phrase “so that the color change index CCI takes a value in a predetermined range that is close to the most favorable value” is used in various positions that the character frame area CA can take. This means that not only the position where the change index CCI takes the best value, but also a position where the change index CCI takes a value in a predetermined range close to the best value can be adopted. In such a case, it is preferable to determine the position of the character frame area CA using another index (for example, close to the left and right centers of the print area).

Incidentally, in some cases, the character frame area CA may not be arranged inside the character frame arrangeable area PCA. In this case, it is possible to determine the arrangement and size of the character frame area CA by the following various methods.
(A-3) The character frame area CA and the size of each character are determined so that the character frame area CA and the attention area PA do not overlap.
(A-4) An overlap evaluation index whose value increases as the overlap between the character frame area CA and the attention area PA increases, and the position of the character frame area CA is determined so that the overlap evaluation index becomes the smallest. .

  As the overlap evaluation index, for example, the area of the portion where the character frame area CA and the attention area PA overlap can be used. In addition, the number of pixels at a position where a pixel having a predetermined flesh color range in the attention area PA and a pixel in the character frame area CA (or a pixel constituting the outline of the character itself) overlaps is used as an overlap evaluation index. Also good. That is, as the overlap evaluation index, an arbitrary index indicating the degree of overlap between the character frame area CA and the attention area PA can be used. If these methods (A-3) and (A-4) are used, the position and / or size of the character frame area CA is determined so that the character frame area CA does not overlap the attention area as much as possible. Can do. When using the method (A-4), the position of the character frame area CA may be determined by using the method (A-2 ′) together. The method (A-4) can be described more generally as the following method (A-4 ′).

(A-4 ′) An overlap evaluation index OEI is calculated such that the value increases as the overlap between the character frame area CA and the attention area PA increases, and the overlap index OEI takes a value in a predetermined range close to the best value. In this way, the position of the character frame area CA is determined.

  Note that when the position of the character frame area CA is determined so that the overlap evaluation index OEI takes a good value, the character frame area CA and the attention area PA usually partially overlap. In this specification, “arranging the character frame area CA so as to avoid the attention area PA” has a broad meaning including such a case.

In step S6, the character color determination unit 236 determines the color of the character arranged in the character frame area CA according to the color of the target image MF. As a method for determining the character color, for example, the following method can be adopted.
(B-1) The color of the character string is determined so that the hue of the character string is different from the hue of the average color of the target image MF in the character frame area CA.
(B-2) The color of each character is determined so that the hue of each character is different from the hue of the target image MF at a position in contact with each character.
(B-3) The hue of the outline portion of each character is set to a value different from the hue of the target image MF.

  Note that when setting the hue value of a character to a value different from the hue value of the target image MF, for example, first, the color data of the character and the color data of the target image MF are set to three attributes (for example, brightness and saturation). , Hue) values are converted as necessary. Then, the hue value of the character can be determined by adding a predetermined value to the hue value of the target image MF. In this way, the character color can be determined so that the character color and the color of the target image MF do not become similar colors. The same color can be used for the entire character string. Further, a different color may be set for each character. Or you may determine the color of each pixel in each character so that it may become the gradation from which a color changes gradually in the whole character string. Further, any one of these character color determination modes may be designated by the user. If these methods (B-1) to (B-3) or other similar methods are used, the character is melted into the target image MF that is the background of the character, and the character becomes difficult to see. Can be relaxed. However, step S6 may be omitted and the character color may not be changed.

  In step S7, the image composition unit 238 synthesizes and displays an image in which a character string is arranged on the target image MF. FIG. 3D shows an example of the composite image displayed in this way. In step S8, it is determined whether or not the finish is preferable. If not, the contents of the character string and the position of the character frame area CA are changed in step S9. Then, steps S6 to S8 are repeated until a preferable composite image is obtained. When a preferred composite image is completed, the composite image is printed by the color printer 300 in step S10. At this time, the image data output unit 240 creates print data PD from the image data of the composite image and supplies it to the printer 300. The image data output unit 240 has a function as a so-called printer driver. However, instead of outputting the print data PD, the image data output unit 240 can also output a composite image represented by other types of color image data (for example, RGB data) to the outside.

  Instead of inputting the character string in step S3, the character string may be input before synthesizing the character string in step S10.

  In the first embodiment, since the character string is arranged so as to avoid the attention area PA including the face area, a state in which a highly important face part is hidden in the character string in the photographic image including the person. Can be suppressed. As a result, it is possible to overlap the character string at an appropriate position in the target image MF.

B. Second embodiment:
FIG. 4 is an explanatory diagram showing the processing contents of the second embodiment. The apparatus configuration and processing procedure of the second embodiment are the same as those of the first embodiment. In the second embodiment, the method of detecting the attention area PA and the arrangement method of the character frame area CA are different from those of the first embodiment.

  FIG. 4A shows a target image MF used in the second embodiment, and FIG. 4B shows two types of attention areas PA1 and PA2. The first type of attention area PA1 is an area including a human face area, and is detected by the same method as the attention area PA used in the first embodiment. The second type of attention area PA2 is an area where the importance index MI indicating the importance of the image portion of the target image MF is calculated by a predetermined calculation method and is detected as an area having a high importance index MI.

  As a method for calculating the importance index MI, for example, a method described in Japanese Patent Application Laid-Open No. 2001-127070 can be employed. In the method described in this publication, “attraction degree” is adopted as the importance index MI. This degree of attraction is a parameter suitable for human subjectivity, and is based on the physical (characteristic color heterogeneity, texture heterogeneity, shape heterogeneity, area heterogeneity, etc.) at the image position of the target image MF. It is a value determined quantitatively. In this method, an image portion (or a rectangular region including the image portion) having an attractiveness degree equal to or greater than a threshold value is detected as the second type of attention area PA2.

  If the importance index MI is calculated using this method or another method, the second type attention area PA2 can be detected using the importance index MI which is a quantitative index.

  FIG. 4C shows an example of the character frame arrangement possible area PCA and the character frame area CA in the second embodiment. As in the first embodiment, the character frame arrangement possible area PCA is determined as an area where no overlap occurs between the first type and the second type of attention areas PA1 and PA2. When the character frame area CA is smaller than the character frame arrangement possible area PCA, the character frame area CA is performed according to the method (A-1), (A-2), (A-2 ′), etc. described in the first embodiment. Is determined.

On the other hand, when the character frame area CA cannot be arranged inside the character frame arrangeable area PCA, the arrangement of the character frame area CA can be determined so as to satisfy the following conditions 1 to 3, for example.
(Condition 1) The character frame area CA is set to a position that does not overlap with the first type of attention area PA1.
(Condition 2) The character frame area CA is positioned so as to avoid the second type of attention area PA2 as much as possible.
(Condition 3) When the character frame area CA overlaps the second type of attention area PA2, the overlap evaluation index OEI ′ such that the value increases as the overlap between the character frame area CA and the second type of attention area PA2 increases. And the position of the character frame area CA is determined so that the overlap evaluation index OEI ′ is minimized.

  The overlap evaluation index OEI ′ of the second embodiment is, for example, a value obtained by multiplying the importance index MI of the second type of attention area PA2 by the area R where the character frame area CA and the second type of attention area PA2 overlap. (MI × R) can be used. Alternatively, the area R itself in which the character frame area CA and the second type of attention area PA2 overlap may be used as the overlap evaluation index OEI '.

  The condition 3 can be described more generally as the following condition 3 '.

(Condition 3 ′) When the character frame area CA overlaps the second type of attention area PA2, the overlap evaluation index OEI increases as the overlap between the character frame area CA and the second type of attention area PA2 increases. 'Is calculated, and the position of the character frame area CA is determined so that the overlap index OEI' takes a value in a predetermined range close to the best value.

  In the second embodiment, the character strings do not overlap the first type of attention area PA1 including the face area, and the character strings do not overlap as much as possible on the second type of attention area PA2 other than the face area. Therefore, even in an image including an important subject other than a person, the character string can be arranged at an appropriate position.

  As the character color determination method in the second embodiment, the same methods (B-1) to (B-3) as in the first embodiment can be adopted.

C. Variation:
In addition, this invention is not restricted to the said Example and embodiment, It can implement in a various aspect in the range which does not deviate from the summary, For example, the following deformation | transformation is also possible.

C1. Modification 1:
The calculation methods of various indexes in the above embodiments are examples, and various indexes may be calculated by various methods other than these.

C2. Variation 2:
In the second embodiment, the first type of attention area PA1 and the second type of attention area PA2 are detected one by one, but a plurality of these may be detected. In addition, only one of the two types of attention areas PA1 and PA2 may be detected and the other may not be detected. Also in these cases, the position of the character string can be determined according to the method described above.

C3. Variation 3:
In the above embodiment, the target image MF is enlarged or reduced so as to cover the entire area (print area) of the print medium. Instead, the target image MF is arranged only in a part of the print area. You may do it. In this case, the character frame area determination unit 234 can set the character frame area CA so that the character frame area CA is arranged at any position in the print area.

1 is an explanatory diagram showing an image processing system as a first embodiment of the present invention. FIG. The flowchart which shows the process sequence in 1st Example. Explanatory drawing which shows the processing content of 1st Example. Explanatory drawing which shows the processing content of 2nd Example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 100 ... Digital camera 200 ... Computer 210 ... Image processing part 220 ... Character input receiving part 230 ... Image selection part 232 ... Attention area detection part 234 ... Character frame area determination part 236 ... Character color determination part 238 ... Image composition part 240 ... Image Data output section 250 ... decoration image storage section 252 ... decoration image selection section 300 ... color printer CA ... character frame area CIN ... input character string FM ... decoration image MF ... target image PA ... attention area PA1 ... first type attention area PA2 ... Type 2 attention area PCA ... Character frame layout possible area

Claims (15)

An image processing apparatus for arranging a character string on a target image,
An attention area detection unit for detecting an attention area by analyzing a target image;
A character frame region determination unit that determines the position of the character frame region so as to avoid the attention region;
An image compositing unit that generates a composite image in which the character string is superimposed on the target image by arranging a character string in the character frame region;
An image processing apparatus comprising: The image processing apparatus according to claim 1,
The image processing apparatus, wherein the character frame region determination unit determines the position of the character frame region so that the character frame region and the attention region do not overlap. The image processing apparatus according to claim 2,
The character frame region determination unit
A color change index indicating a better value as the color change of the target image at a position corresponding to the character frame region is smaller;
An image processing apparatus that determines a position of the character frame region so that the character frame region and the attention region do not overlap with each other, and the color change index takes a value in a predetermined range close to the best value. The image processing apparatus according to claim 2 or 3,
The image processing apparatus, wherein the character frame region determination unit determines the character frame region and the size of each character so that the character frame region and the attention region do not overlap. The image processing apparatus according to claim 1,
The character frame region determination unit
When the character frame area and the attention area partially overlap, calculate an overlap evaluation index indicating a better value as the amount of overlap between the character frame area and the attention area decreases,
An image processing apparatus that determines the position of the character frame region so that the overlap evaluation index takes a value in a predetermined range close to a best value. The image processing apparatus according to claim 5, wherein
The character frame region determination unit
A color change index indicating a better value as the color change of the target image at a position corresponding to the character frame region is smaller;
Determining the position of the character frame region so that the overlap evaluation index takes a value in a predetermined range close to the best value and the color change evaluation index takes a value in a predetermined range close to the best value; Image processing device. The image processing apparatus according to any one of claims 1 to 6,
The said attention area detection part is an image processing apparatus which can detect the area | region containing a human face as a 1st type attention area. The image processing apparatus according to claim 7,
The attention area detection unit further calculates an importance index indicating a better value as the importance of the image portion of the target image is higher, and detects an area having a better importance index as a second type of attention area. An image processing apparatus capable of The image processing apparatus according to claim 8, wherein
The character frame area determination unit, when the first type and the second type of attention area exist,
(I) There is no overlap between the character frame area and the first type of attention area, and
(Ii) An overlap evaluation index indicating a better value as the overlap between the character frame area and the second type of attention area is smaller is calculated, and the overlap evaluation index takes a value in a predetermined range close to the most favorable value. In addition,
An image processing apparatus for determining a position of the character frame area. An image processing apparatus according to any one of claims 1 to 9,
The target image is arranged in a part of the print area,
The character frame region determination unit is an image processing device that arranges the character frame in the print region. The image processing apparatus according to any one of claims 1 to 10, further comprising:
An image processing apparatus comprising: a character color determining unit that determines a color of the character string in accordance with a color of the target image in the character frame region. The image processing apparatus according to claim 11,
The character processing unit determines the color of the character string so that the hue of the character string is different from the average color hue of the target image in the character frame region. The image processing apparatus according to claim 11,
The said character color determination part is an image processing apparatus which determines the color of each character so that the hue of each character and the hue of the said target image in the position which touches each character may differ. A method for overlaying a character string on a target image,
(A) a computer detecting a region of interest by analyzing a target image;
(B) a step of determining a position of the character frame area so as to avoid the attention area;
(C) a step of generating a composite image in which the character string is superimposed on the target image by arranging a character string in the character frame region;
A method comprising: A computer program for arranging a character string on a target image,
A function to detect a region of interest by analyzing the target image;
A function of determining the position of the character frame area so as to avoid the attention area;
A function of generating a composite image in which the character string is superimposed on the target image by arranging a character string in the character frame region;
A computer program that causes a computer to realize

Images