JP2009266166A - Harmony determination apparatus, harmony determination method and harmony determination program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To automatically determine whether or not a combination of items that when combined form one concept as a whole is a generally preferred combination. <P>SOLUTION: A harmony determination apparatus includes an item detection part for analyzing image data copying the combination of items to be determined to detect what types or outlines of items are combined, and a harmony determination part for comparing the type of each detected item with the type of each item forming one combination represented by information stored in advance or comparing the distribution of tones measured from the image data in the region where each detected item is distributed with the distribution of tones of one combination represented by the information stored in advance to compute a degree of matching indicating how they match. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a harmony determination apparatus that evaluates a combination of a plurality of items that constitute one item as a whole.

  For example, when a person decides clothes, the overall balance of the combination of clothes is often attributed to individual sensibilities and is abstractly captured. However, there is a common sensibility and fashion that occupies the majority, and clothes that are determined by the sensibility of one individual may be seen strangely in the company or in the downtown area.

  In order to achieve a generally preferred appearance, the color tone of each outfit may be determined so as to achieve a color tone balance that matches the generally preferred sensitivity.

  For example, as a technique for coordinating a user's clothes, Patent Document 1 captures a photograph of a fitting person as image data, and creates a fitting image that creates a fitting image with a clothing image having a color and pattern suitable for the user. The creation system is described.

  Further, for example, Patent Document 2 describes an apparel product sales support method that captures an image of a customer's try-on and color-coordinates the clothes so that the color matches the customer.

JP 2002-024319 A Japanese Patent Laid-Open No. 2002-207802

  However, the methods described in Patent Literature 1 and Patent Literature 2 are operated by a person having knowledge of coordination such as a user or a clothing seller, and select a color or a pattern from a displayed menu or the like. is doing. For example, in the method selected by the user, the selection is based on individual sensibilities, so that it may not be generally preferred. In addition, in the method of causing a person with knowledge of coordination to select, it may not be possible to respond immediately, and an answer cannot be obtained immediately. In particular, when the number of users increases, there is a high possibility that a long waiting time will be required before obtaining a coordination plan.

  Accordingly, the present invention provides a harmony determination device, a harmony determination method, and a harmony that automatically determine whether or not the combination is a generally preferred combination for a combination of items that together form a single concept. The purpose is to provide a degree determination program.

  The harmony determination apparatus according to the present invention is a harmony determination apparatus that determines whether a combination is a generally preferred combination with respect to a combination of items that together form a concept as a whole. Image data showing that the item pattern information indicating at least the type of each item constituting the combination is accessible and the items to be judged are combined, stored as information indicating the preferred combination of items The item detection unit that detects what type of item is combined, the type of each item detected by the item detection unit, and each of the combinations that are indicated by the item pattern information Compare the item type and the combination of each detected item Item harmony that determines whether or not the combination of items that are subject to determination is a generally preferred combination by determining the degree of coincidence that indicates how much matches the one included in the item pattern information And a determination unit.

  In addition, the harmony determination device according to the present invention is a harmony determination device that determines whether or not the combination is a generally preferred combination for a combination of items that together form one concept as a whole, Image data showing that the items to be judged are combined and accessible to at least the color pattern information indicating the distribution of the color tone based on the combination, which is stored in advance as information indicating the combination of items generally preferred. Analyzing and detecting what kind of contour item is combined, and using the image data of the area where each item detected by the item detection unit is distributed, the color tone of each item Color tone measurement unit that measures color, detection result by item detection unit and measurement result by color tone measurement unit The color tone distribution by the combination of detected items indicated by is compared with the color tone distribution by one combination indicated by the color tone pattern information, and each detected item combination is a combination included in the color tone pattern information. A color tone harmony degree determination unit that determines whether or not the combination of items to be determined is a generally preferred combination by obtaining a degree of coincidence indicating how much they match is characterized in that .

  In addition, the harmony determination method according to the present invention is a harmony determination method for determining whether or not a combination of items that together form one concept as a whole is a generally preferred combination. As the information indicating the combination of items generally preferred in advance, at least item pattern information indicating the type of each item constituting the combination is stored, and image data showing that the items to be determined are combined , To detect what type of items are combined, compare the type of each detected item with the type of each item that constitutes one combination indicated by the item pattern information, The combination of each detected item is included in the item pattern information And by obtaining a matching degree indicating whether only match any, wherein the combination of the determination target item to determine whether a combination of choice.

  In addition, the harmony determination method according to the present invention is a harmony determination method for determining whether or not a combination of items that together form one concept as a whole is a generally preferred combination. In advance, at least color tone pattern information indicating the distribution of color tone according to the combination is stored in advance as information indicating a combination of items generally preferred, and image data showing that the items to be determined are combined is analyzed. Then, it is detected what kind of contour items are combined, and the color tone of each item is measured using the image data of the area where each detected item is distributed. Color tone distribution and tone pattern by combination of items to be judged indicated by outline and tone By comparing the color distribution of one combination indicated in the report and finding the degree of coincidence indicating how much each detected item combination matches one combination included in the color pattern information, It is characterized in that it is determined whether or not the combination of items determined is a generally preferred combination.

  Further, the harmony determination program according to the present invention is a harmony determination program for determining whether or not a combination of items that together form one concept as a whole is a generally preferred combination. The item to be determined is combined with a computer that can access at least item pattern information indicating the type of each item constituting the combination, which is stored in advance as information indicating a combination of items generally preferred. Analyzing the image data that shows what was done, detecting what type of items are combined, and configuring the combination of each detected item type and item pattern information Compare each item type By determining the degree of coincidence indicating how much the combination of items matches one combination included in the item pattern information, it is determined whether or not the combination of the items to be determined is a generally preferred combination Processing is executed.

  Further, the harmony determination program according to the present invention is a harmony determination program for determining whether or not a combination of items that together form one concept as a whole is a generally preferred combination. The item to be judged is combined with a computer that can access at least the color pattern information indicating the distribution of color tone according to the combination, which is stored in advance as information indicating the combination of items generally preferred in advance. Analyzing the image data of the image, the process of detecting what kind of contour items are combined, using the image data of the area where each item specified by the detected contour is distributed, Processing to measure the color of each item, and the combination of each detected item Compares the distribution of color tones with the distribution of color tones by one combination indicated by the tone pattern information, and shows how much each combination of detected items matches with one combination included in the tone pattern information By determining the degree, a process for determining whether or not the combination of items to be determined is a generally preferred combination is executed.

  According to the present invention, for a combination of items that, when combined, form a concept as a whole, harmony that automatically determines whether the combination is a generally preferred combination (without intervention of a person other than the user) A degree determination device, a harmony degree determination method, and a harmony degree determination program can be provided.

  Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram illustrating a configuration example of a harmony determination apparatus according to the present invention. The harmony determination apparatus shown in FIG. 1 is capable of accessing, for example, item pattern information indicating at least the type of each item constituting the combination, which is stored in advance as information indicating a generally preferred combination of items. It is a determination apparatus, Comprising: The item detection part 11 and the item harmony degree determination part 12 are provided.

  In this example, the item detection unit 11 analyzes image data showing a combination of items to be determined, and detects what type of items are combined. For example, the item detection unit 11 uses the pattern matching data prepared for each type to detect the combined items including the type discrimination.

  The item harmony degree determination unit 12 compares the type of each item detected by the item detection unit 11 with the type of each item constituting one combination indicated by the item pattern information, and the combination of each detected item is Then, by determining the degree of coincidence indicating how much it matches with one combination included in the item pattern information, it is determined whether or not the combination of items to be determined is a generally preferred combination.

  Moreover, the item harmony degree determination part 12 may detect and show the difference of the combination of the item detected by the item detection part 11 with respect to the combination of the item shown by item pattern information.

  FIG. 2 is a block diagram showing another configuration example of the harmony determination apparatus according to the present invention. The harmony determination apparatus shown in FIG. 2 is, for example, a harmony determination apparatus capable of accessing at least color pattern information indicating a distribution of color tone according to a combination that is stored in advance as information indicating a combination of items generally preferred. Thus, an item detection unit 11, a color tone measurement unit 13, and a color tone harmony determination unit 14 are provided.

  In this example, the item detection unit 11 analyzes image data showing a combination of items to be determined, and detects what contour items are combined. For example, the item detection unit 11 detects the combination of items by detecting the discontinuity of the color tone and the pattern, including the specification of the outline (specifically, the specification of the distribution range in the image data). Do.

  The color tone measuring unit 13 measures the color tone of each item using image data of a region where each item detected by the item detecting unit 11 is distributed.

  The color tone harmony determination unit 14 determines the color tone distribution by the combination of detected items indicated by the detection result by the item detection unit 11 and the measurement result by the color tone measurement unit 13, and the color tone by one combination indicated by the color tone pattern information. By comparing the distributions and finding the degree of coincidence indicating how much each detected combination of items matches the one included in the color pattern information, the combination of items that are subject to determination is generally preferred. It is determined whether or not the combination is the same.

  The color tone harmony degree determination unit 14 may detect and present a difference in color tone distribution due to the combination of items detected by the item detection unit 11 with respect to the color tone distribution indicated by the color tone pattern information.

  In addition, the hue measuring unit 13 may include a hue measuring unit 131, a saturation measuring unit 132, and a lightness measuring unit 133.

  The hue measurement unit 131 measures the hue of each item using the image data of the area in which each detected item is distributed, and the main hue that is the hue distributed in the most area in the image data. Is detected.

  The saturation measuring unit 132 measures the saturation of each item using the image data of the area where each detected item is distributed, and the main hue detected by the hue measuring unit 131 is distributed. Measure the main saturation, which is the saturation of the region.

  The lightness measurement unit 133 measures the lightness of each item using the image data of the area where each detected item is distributed, and the region where the main hue detected by the hue measurement unit 131 is distributed. Measure the main brightness which is the brightness.

  In such a case, the color tone harmony determination unit 14 determines the distribution of the color tone based on the combination of detected items indicated by the measurement results of the hue measurement unit 131, the saturation measurement unit 132, and the lightness measurement unit 133, and the color pattern information. Items detected by comparing the distribution of hues by one combination indicated by, and the distribution of hues that most closely matches the main hue specified by the measured hue, saturation, and hue combination. By determining the degree of coincidence indicating how much the combination of items matches the combination included in the color tone pattern information, it is determined whether or not the combination of items to be determined is a generally preferred combination Also good.

  Note that the configuration shown in FIG. 1 and the configuration shown in FIG. 2 may be combined. At this time, in order to improve the discrimination accuracy, the measurement result of the color tone measurement unit 13 and the detection result of the item detection unit 11 may be fed back to each other.

  Further, the combination of items to be determined may be a combination of clothing items. In such a case, for example, the item detection unit 11 analyzes image data showing a user wearing one or more clothing items, detects a specific human body part of the user, and detects it as a human body part. Detection may be performed on clothing items that will be worn on the human body part from the image data of each region.

  The item detection unit 11, the item harmony degree determination unit 12, the color tone measurement unit 13, and the color tone harmony degree judgment unit 14 are realized by a CPU that operates according to a program, for example. Moreover, the harmony determination apparatus may be mounted on a mobile phone, for example.

  Hereinafter, more specific embodiments will be described. FIG. 3 is a block diagram illustrating a more specific configuration example of the harmony determination apparatus according to the present embodiment. In FIG. 3, a mobile terminal (more specifically, a mobile phone with a camera function) 10 is illustrated as an embodiment of the harmony determination apparatus. In the present embodiment, for example, a case in which the image data captured by the mobile terminal 10 is recognized, and the item of the clothing and the color tone in the image are recognized to determine the degree of harmony of the clothing. It explains using. In addition, in determining the degree of harmony of clothing, it is determined whether the combination of each clothing item is in accordance with a common sensitivity that is generally preferred, and whether the color balance of the overall clothing is in accordance with a common sensitivity that is generally preferred This will be described with reference to an example.

  The mobile terminal 10 illustrated in FIG. 3 includes an antenna 101, a TRX unit 102, a LAN-IF 103, a MOD unit 104, a communication IF 105, a control unit 106, an image processing unit 107, an imaging unit 108, and display processing. Unit 109, display unit 110, input processing unit 111, operation input unit 112, memory 113, database 114, image recognition unit 115, and coordination processing unit 116.

  The antenna 101 transmits and receives communication data by radio signals. The TRX unit 102 performs communication data transmission / reception control for the antenna 101 and modulation / demodulation control for the transmitted / received communication data. The LAN-IF 103 is an interface for connecting to a LAN such as a modular jack, for example. The MOD unit 104 performs transmission / reception control of communication data with respect to the LAN-IF 103 and modulation / demodulation control with respect to transmitted / received communication data. The communication IF 105 selects a communication method such as wireless communication or LAN communication, and issues various control instructions to the communication control unit (TRX unit 102 or LAN-IF 103) corresponding to each communication method.

  The image processing unit 107 converts the image data input from the imaging unit 108 in accordance with the processing format of the control unit 106 or the image recognition unit 115. The imaging unit 108 is configured by a CMOS sensor or the like and realizes a camera function. The display processing unit 109 performs display control on the display unit 110 such as a display based on the display data output from the control unit 106. The display unit 110 displays data as a user IF such as character display and chart display under the control of the display processing unit 109.

  The input processing unit 111 detects and analyzes data input from the operation input unit 112 such as a numeric keypad, and notifies the control unit 106 as information indicating a user operation. The operation input unit 112 inputs data in response to a user operation on the mobile terminal 10.

  The memory 113 stores programs used by the mobile terminal 10, user information, and the like. The database 114 holds various data used for image data (image data) captured by the imaging unit 108, image recognition processing performed by the image recognition unit 115, and determination processing performed by the coordination processing unit 116.

  The image recognizing unit 115 analyzes the specified image data, and recognizes (discriminates) the item of clothes that is reflected in the image indicated by the image data. In this embodiment, each part of the human body such as the head, face, and leg in the image is recognized, and clothes items such as a hat, shirt, and pants are recognized based on the recognized parts of the human body. Furthermore, the color tone (hue, saturation, brightness, etc.) of each clothing item is recognized.

  The control unit 106 is a CPU or the like that operates according to a program, and executes processing for realizing various functions implemented by the mobile terminal 10. In the present embodiment, the control unit 106 includes at least an image recognition unit 115 and a coordination processing unit 116.

  The coordination processing unit 116 determines the degree of harmony for the combination of clothing items recognized by the image recognition unit 115. In this embodiment, the quality of the overall clothing is determined by obtaining two types of harmony, a harmony degree based on the item type and a harmony degree based on the color tone. Furthermore, it has a function for proposing balanced (highly harmonious) clothes using information held in the database 114.

  FIG. 4 is an explanatory diagram showing a more specific configuration example of the image recognition unit 115. As shown in FIG. 4, the image recognition unit 115 may include a clothing item recognition unit 1151, a pattern recognition unit 1152, a hue measurement unit 1153, a saturation measurement unit 1154, and a lightness measurement unit 1155. Good.

  The clothing item recognition unit 1151 analyzes the specified image data, and determines and recognizes predetermined components of the human body such as the head, chest, arms, abdomen, and legs included in the image indicated by the image data. The image is divided into blocks according to the constituent parts of the human body. Furthermore, a clothing item to be worn on each human body block is identified and recognized. The clothing item recognition unit 1151 corresponds to the item detection unit 11 in FIGS. 1 and 2.

  For example, clothing items such as hats and glasses are identified and recognized from the head block. In addition, for example, clothing items such as jackets and shirts are identified and recognized from the chest (abdomen) block. For example, from the leg block, clothing items such as pants and shoes are identified and recognized.

  For example, the clothing item recognition unit 1151 determines the type of the clothing item as recognition of the clothing item. Further, for example, the shape of the clothing item may be specified. Then, as information indicating the recognition result of the clothing item, an identifier for identifying the clothing item is added, and information indicating the type and shape of the clothing item, the position (area) represented in the image, the size, and the like Is stored in the database 114.

  For each clothing item recognized (discriminated) by the clothing item recognition unit 1151, the pattern recognition unit 1152 determines the presence / absence of a pattern and the pattern configuration (pattern type such as stripes, lattices, and characters, and the shape and position of the pattern). Discriminate and recognize. For example, the pattern recognition unit 1152 analyzes image data in an image area where the clothing item is located, and determines whether or not a pattern is represented by the image data.

  The hue measuring unit 1153, the saturation measuring unit 1154, and the lightness measuring unit 1155, for each clothing item recognized by the clothing item recognizing unit 1151, the hue, saturation, and lightness of the image area where the clothing item is located, Measure based on 3 attributes. For clothing items that are determined to have a pattern by the pattern recognition unit 1152, the hue, saturation, and brightness are measured for each component of the pattern. Information on the hue, saturation, and lightness measured here is stored in the database 114 as color data for each clothing item. The hue measurement unit 1153, the saturation measurement unit 1154, and the lightness measurement unit 1155 may measure skin and hair color data (hue, saturation, brightness) from the human body block recognized by the clothing item recognition unit 1151. Note that the hue measurement unit 1153, the saturation measurement unit 1154, and the brightness measurement unit 1155 correspond to the hue measurement unit 131, the saturation measurement unit 132, and the brightness measurement unit 133 in FIG.

  FIG. 5 is an explanatory diagram showing a more specific configuration example of the coordination processing unit 116. As shown in FIG. 5, the coordination processing unit 116 may include a clothing item harmony degree determination unit 1161 and a color tone harmony degree determination unit 1162.

  The clothing item harmony degree determination unit 1161 obtains the harmony degree based on the type of the clothing item (how much the degree is based on the general preference and sensitivity) for the combination of the clothing items determined by the image recognition unit 115. Determine the balance of the clothes. Further, when it is determined that the clothes are not well balanced, the clothes that are well balanced (highly harmonious) are proposed using the information stored in the database 114. The clothing item harmony degree determination unit 1161 corresponds to the item harmony degree determination unit 12 in FIG.

  The color tone harmony degree determination unit 1162 is based on the color data of skin and hair from the human body block determined by the image recognition unit 115 and the color data of each clothing item, and the color tone harmony degree (color tone distribution) for the combination of clothing items. The degree of the general preference and sensitivity is determined, and the balance of the clothes is determined. Further, when it is determined that the clothes are not well balanced, the information stored in the database 114 is used to propose a well-balanced (highly harmonious) color tone and clothes. The color tone harmony degree determination unit 1162 corresponds to the color tone harmony degree determination unit 13 in FIG.

  Next, the operation of this embodiment will be described. FIG. 6 is a flowchart illustrating an example of the harmony determination operation in the present embodiment.

  In this example, first, the user operates the operation input unit 112 of the mobile terminal 10 to change the mobile terminal 10 to the imaging mode, and the whole body of the subject for which the degree of harmony is to be determined or the part to be determined (for example, on the chest) ). The imaging unit 108 generates image data obtained by photographing the subject that appears on the lens in response to a user operation (step S101). At that time, the imaging unit 108 may display the captured image via the display unit 110 as confirmation of the imaging range. Hereinafter, a case where the whole body of the subject is photographed will be described as an example. Image data captured by the imaging unit 108 is input to the image recognition unit 115 via the image processing unit 107.

  At this time, in the image processing unit 107, for example, the image data captured by the image capturing unit 108 is subjected to a color system data format such as an L * a * b display system or a Munsell display system that is easy to perform color tone recognition processing, and clothes recognition. It may be converted into an RGB data format that can be easily processed and input to the image recognition unit 115. In addition, it may be converted into a specified format and input in response to a request from the image recognition unit 115.

  In the image recognition unit 115, first, the clothing item recognition unit 1151 performs an image recognition process on the input image data (step S102).

  The clothing item recognition unit 1151 first performs a process of detecting a human body block from image data as an image recognition process (step S1021: human body block detection). For example, the clothing item recognizing unit 1151 may detect a part that becomes the face of the subject from the image data as a face block. The detection of the face block is performed, for example, by determining whether or not there are two black spots (eyes) near the center of the skin color part and the part where the skin color is distributed in a circle in the image indicated by the image data. Also good. Alternatively, pattern matching by comparison with face image data registered in advance for pattern matching may be performed to detect a face portion (face block).

  Then, using the detected face block, the chest and abdomen parts from the region below the face are detected as a body block by pattern matching or the like. Similarly, a part extending from the lower side of the abdomen is detected as a leg (leg block) by pattern matching or the like. Similarly, an elongated part extending from the left and right regions of the chest is detected by pattern matching or the like as an arm part (arm part block). The pattern matching data is not only stored in the database 114, but may be embedded in, for example, program data.

  Next, the clothing item recognition unit 1151 detects a clothing item to be worn on each human body block from each human body block (step S1022: clothing item detection). For example, the clothing item recognition unit 1151 detects a clothing item attached to the head block such as a hat by performing pattern matching or the like from the image data of the head block (image data of the area detected as the head block). . Since there are various types of hats such as a baseball cap, a knit cap, and a hunting cap, a plurality of types of pattern matching according to the shape of the eaves are performed, and the detection including the type discrimination is performed.

  Similarly, for example, clothing items to be worn on a chest block such as a jacket, a T-shirt, and a sweater are detected from the image data of the chest block. These types may be detected by detecting the degree of exposure of the neck or arm from the skin color distribution state, and performing a plurality of types of pattern matching according to the presence or absence of the button, the shape, or the like. Similarly, for example, clothing items attached to leg blocks such as slacks, cargo pants, and skirts are detected from the image data of the leg blocks. In particular, in the case of a skirt, it is difficult to distinguish between the chest and legs of a human body block, but it is necessary to detect distribution discontinuities, contours, discontinuities in colors and patterns, including the head block and chest block. Determine with. For example, a clothing item such as a dress or a joint can be identified from the fact that the contour from the chest to the leg is continuous, the color tone and the pattern are continuous, and the like. As for the color tone, a result of color tone measurement described later may be fed back and used. Further, the pattern detection result to be described later may be fed back and used for the pattern.

  The clothing item recognition unit 1151 may also determine the material of each clothing item based on a change in similar color obtained by feeding back the results of color tone measurement and pattern detection described below. For example, if the distribution of similar colors is distributed with a difference in brightness, shiny materials such as silk, nylon, and synthetic leather, and the distribution of similar colors in stripes with differences in brightness and saturation In the case of distribution, each material may be determined by detecting features, such as a corduroy material.

  Moreover, the clothing item recognition part 1151 may discriminate | determine the kind of clothing item based on the color generally used. For example, when trousers are detected as a clothing item from the leg block, if the clothing item is not a single (shaded) indigo, it may be determined that it is denim (jeans). Further, the determination may be made based on the presence or absence of a thin orange line on the skirt or crotch, the presence or absence of a pocket edge, or the like. For example, when a jacket is detected as a clothing item from a chest block, the clothing item is dark blue, there is a local pattern (emblem) on the left chest, and gold dots (buttons) are arranged vertically on the abdomen. If so, it may be determined that it is a blazer. Further, for example, when a tie is detected as a clothing item from the chest block, if the clothing item is white, it may be determined that it is a formal tie. As described above, by further detecting the features from the detected clothing items, it is possible to determine the type of detail. The image recognition method is not limited to pattern matching, and other image recognition methods may be used.

  Next, for each clothing block detected by the clothing item recognition unit 1151, the pattern recognition unit 1152 extracts a continuous change of the same color and similar color from the image data of the clothing block and performs pattern matching to extract the pattern. Is detected (step S1023: pattern detection). The result of this pattern detection is fed back to the clothing item detection, thereby improving the clothing item discrimination accuracy.

  The results of clothing item detection and pattern detection may be stored in the user information area of the database 114 as closet data (user-owned clothing data).

  Next, the hue measurement unit 1153, the saturation measurement unit 1154, and the lightness measurement unit 1155 each perform a color tone measurement process (step S103). The hue measurement unit 1153, the saturation measurement unit 1154, and the lightness measurement unit 1155 include information on clothing items detected by the clothing item recognition unit 1151 and information on patterns by clothing item detected by the pattern recognition unit 1152. Entered. In addition, image data to be processed is input. Here, for example, the image data may be converted into a color system data format and input so as to facilitate the color tone recognition process.

  The hue measuring unit 1153 measures the hue for each clothing item (step S1031: hue measurement). Further, the hue measuring unit 1153 extracts the average value of hues for each clothing item and the main hue (hues distributed in the most regions) as information indicating the hue of the entire clothing. Further, the hue is measured for each pattern in the clothing item, and the average value and the main hue are extracted.

  The saturation measuring unit 1154 measures the saturation for each clothing item (step S1032: saturation measurement). The saturation measuring unit 1154 extracts the saturation of the region in which the main hue extracted by the hue measuring unit 1153 is distributed. Further, the saturation is measured for each pattern in the clothing item, and the average value thereof is taken, or the saturation of the area where the main hue in the pattern is distributed is extracted as the main saturation.

  The brightness measurement unit 1155 measures the brightness for each clothing item (step S1033: brightness measurement). Further, the lightness measuring unit 1155 extracts the lightness of the region where the main hue extracted by the hue measuring unit 1153 is distributed. Further, the lightness is measured for each pattern in the clothing item, the average value is taken, and the lightness of the region where the main hue in the pattern is distributed is extracted as the main lightness.

  Further, the hue measuring unit 1153, the saturation measuring unit 1154, and the lightness measuring unit 1155 may measure the hue, saturation, and lightness of the hair from the face block. Further, the hue, saturation, and lightness of the skin may be measured from the face block, chest block, and leg block.

  Next, the clothing item harmony degree determination unit 1161 of the coordination processing unit 116 determines the harmony degree (item harmony degree) based on the type of the clothing item for the combination of the clothing items indicated from the detection result of the clothing item by the image recognition unit 115. It calculates and determines whether it is good or bad (step S104: clothing item harmony determination process). The clothing item harmony degree determination unit 1161 calculates the item harmony degree for the detected combination of clothing items using, for example, the clothing pattern information stored in the database 114.

  FIG. 7 is an explanatory diagram illustrating an example of clothing pattern information. As illustrated in FIG. 7, information on clothes that are commonly used for common sensibilities and preferences may be registered so as to correspond to items recognized by the image recognition unit 115. In the example shown in FIG. 7, a combination of clothes according to common sensibilities and preferences that are generally preferred is treated as one pattern, and information on clothes items constituting the clothes pattern is included together with an identifier for identifying each clothes pattern. Is registered. For example, information such as the type, detail type, color, and material of the clothing item worn on the human body block is registered for each human body block. Moreover, you may include the information of the distribution ratio about what ratio the clothing item occupies in the whole corresponding human body block.

  In addition, information about such clothing patterns may be classified and registered according to the mode according to the situation and usage of wearing the clothing patterns. For example, a clothing pattern suitable for everyday clothing may be registered as the “regular mode”. Also, for example, a suitable clothing pattern suitable for a ceremonial occasion may be registered as “formal mode”. Further, for example, as a “park mode”, a clothing pattern suitable for going to a park with children and pets may be registered. In addition, as a “shopping mode”, a clothing pattern suitable for shopping in a department store or in a town may be registered. In addition, a clothing pattern based on the latest fashion information may be registered as the “fashion mode”. In addition, you may include the classification according to a season in a mode.

  For the clothing pattern information, for example, based on a generally preferred appearance that has been found by taking a questionnaire in advance, the operator can select items matching the appearance from the list of items recognized by the image recognition unit 115. You may enter by. Further, for example, image data showing the dressed subject may be input, and the result of performing image recognition processing and color tone measurement processing on the image data may be registered as clothing pattern information.

  The clothing item harmony determination unit 1161 compares, for example, the types of clothing items constituting each clothing pattern registered as such clothing pattern information for each mode with the types of detected clothing items. The proportion of items that match all the clothing items may be obtained as the degree of coincidence and calculated as the item harmony. For example, the item harmony degree may be set to 100 (%) when all the clothing items are completely matched. For example, when one item among five items is different, the item harmony degree may be 80 (%). In addition, after obtaining the degree of coincidence (100 if matched) considering the weight (contribution rate) assigned to each item by item type and material, it depends on the number of items and the distribution ratio of each item in the human body block It is good also considering the weight and calculating | requiring the whole matching degree, and making it an item harmony degree. When a plurality of clothing patterns are included in the clothing pattern information of a certain mode, the determination result may be shown for the clothing pattern having the highest item harmony among them.

  The clothing item harmony degree determination unit 1161 determines whether or not the calculated item harmony degree satisfies a predetermined criterion (step S105), and if it satisfies, the determination result of the item harmony degree is OK. It is sufficient to proceed to the harmony determination process based on the color tone in step S107. On the other hand, if the condition is not satisfied, it is determined that the item harmony degree is NG, and an alternative is proposed (step S106). When calculating the item harmony degree, the clothing item harmony degree determination unit 1161 stores which clothing item is different from the pattern of the clothing pattern information, and uses the different clothing item as the clothing item of the clothing pattern information. You may generate and display the change clothing item list which shows changing.

For example, the following changed clothing item list is generated and proposed.
・ "Regular mode" = Item harmony 70%, Alternative: Jeans → Slacks "Formal mode" = Item harmony 30%, Alternative: Jeans → Formal suit, tie → Formal tie ・ "Park mode" = Item harmony 30%, alternative: Y-shirt → trainer, tie → unnecessary ・ "Shopping mode" = 70% item harmony, alternative: Jeans → slacks ・ "Fashion mode" = 100% item harmony, alternative: none

  Note that the user may be allowed to select a mode to be compared before determining the degree of harmony. It is also possible to search for and present alternative clothing that matches the clothing item of the clothing pattern information from the closet data stored in the database 114.

  In step S107, the color harmony degree determination unit 1162 calculates the harmony based on the color tone (color tone harmony degree) for the combination of the clothing items indicated from the detection result of the clothing item by the image recognition unit 115, and determines whether it is good or bad. (Step S107: Color tone harmony determination process). For example, the color harmony degree determination unit 1162 calculates the color harmony degree for the detected combination of clothing items using the color tone pattern information stored in the database 114.

  FIG. 8 is an explanatory diagram illustrating an example of color tone pattern information. As shown in FIG. 8, information regarding color combinations that match commonly-used sensibilities and preferences may be registered so as to correspond to items recognized by the image recognition unit 115. In the example shown in FIG. 8, a combination of color tones that are generally preferred is handled as a single pattern, and an identifier for identifying each color pattern, as well as information on the main hue in the color pattern and the color pattern. Is registered. For example, information such as a tone value (a value obtained by combining hue, saturation, and brightness), a distribution ratio, the presence / absence of a pattern, and a pointer to a tone pattern when the pattern is a single tone pattern is registered. Further, information on such a color pattern may be registered for each mode classified according to the characteristics of the color pattern.

  For example, the color tone harmony determination unit 1162 compares each color pattern registered as such color tone pattern information for each mode with the color tone pattern indicated by the detected color tone data, and the color tone patterns match. The ratio may be calculated as the color harmony degree. For example, it searches for the tone pattern that matches the main color tone most closely represented by the main hue / saturation / lightness combination, and uses the combination content and distribution ratio of the color tone (hue / saturation / lightness combination) in the color tone pattern. What is necessary is just to obtain | require in percentage how much it matches with the distribution of the color tone shown. For example, for each type of tone value included in the combination, a matching degree (100 if matching) is obtained according to the difference between hue, saturation, and lightness, and a weight corresponding to the distribution ratio is given to the matching degree. It is good also as a color harmony degree what calculated | required the ratio as a whole after multiplying. When a plurality of tone patterns are included in the tone pattern information of a certain mode, the determination result may be shown for the tone pattern having the highest tone harmony degree among them.

  The color harmony degree determination unit 1162 determines whether or not the obtained color harmony degree satisfies a predetermined standard (step S108), and if it satisfies, the determination result of the color harmony degree is OK. What is necessary is just to advance to the result display of step S110. On the other hand, if the condition is not satisfied, an alternative is proposed assuming that the item harmony determination result is NG (step S109). When calculating the color harmony degree, the color tone harmony degree determination unit 1162 obtains how the color tone harmony degree becomes 100 (%) by changing the color tone, and is subject to change of the color tone. You may produce | generate and display the change clothing item list which shows that it changes to the clothing item which has the color after a clothing item changed.

For example, if the detection result is “gray hat, black jacket, white blouse, shocking pink skirt, black shoes”, a clothing change item list as shown below is generated and proposed.
・ "Classic mode" = 80% color harmony, alternative: shocking pink skirt → black skirt ・ "Ethnic mode" = 0% color harmony, alternative:-
・ "Tropical mode" = 0% color harmony, alternative:-
・ “Elegant mode” = 60% color harmony, alternative: gray hat → white or black hat, shocking pink skirt → black skirt ・ “Sportive mode” = 0% color harmony, alternative:-

  In step S110, a determination result indicating that the item harmony degree and the color tone harmony degree satisfy predetermined criteria for the combination of the clothing items of the preview object may be displayed. In addition, when an alternative that can achieve a higher degree of harmony is detected, the alternative may be presented together.

  As described above, according to the present embodiment, for a combination of clothes items that form a single concept as a whole when combined, it is determined whether the combination is a generally preferred combination without any human intervention other than the user. Therefore, the user can check the clothes according to the displayed determination result and change the clothes to the harmonious clothes, so that the user can wear clothes that are generally preferred. As a result, it is possible to avoid being exposed to strange eyes in clothes that do not match the surroundings. In addition, since the mobile terminal 10 automatically makes a determination and proposes an alternative, it is possible to reduce the time for the user to lose clothes and to minimize the time for selecting clothes.

  The clothing pattern information and the color pattern information are received from the mobile phone network or the Internet network using the communication IF 105 on the assumption that the service pattern is stored in the server by the provider providing this service, for example. Store in database 114. In addition, the operator agrees with the appearance from the list of items recognized by the image recognition unit 115 to the operator based on the generally preferred appearance and the knowledge of color coordination that have been clarified by taking a questionnaire beforehand. May be generated by selecting. Further, for example, it can be generated by inputting image data showing a photographed subject and registering the result of image recognition processing and color tone measurement processing equivalent to the present invention for the image data.

  Further, as a transmission / reception method of the wireless section used by the antenna 101 and the TRX unit 102, it is possible to use not only a wireless telephone network but also a wireless LAN or a PHS network. In addition, as a transmission / reception method of a wired section used in the TRX unit 102 and the MOD unit 104, not only a LAN but also a general telephone network or a CATV network can be used. The imaging unit 108 may be realized by a CCD camera instead of a CMOS camera. In addition, as a user input method, an operation input unit 112 such as a keyboard, a touch panel, a mouse, a touch pad, or a pen tablet can be used.

  In the above description, an example is shown in which the degree of harmony is determined by a color system that is mainly used when a designer or the like examines a color arrangement, but it is also possible to use a representation format of digital image data such as YCbCr. It is.

  In addition, in the item harmony determination and the color tone harmony determination, information on the pattern detected by the pattern detection process may be added to the determination material. Further, as the determination result display, the result of the item harmony determination and the result of the color tone harmony determination may be displayed together. Moreover, you may feed back the determination result (alternative plan) of a mutual harmony.

For example, the result of determining the degree of harmony between the “black jacket” and the “black shoes” when considering changing the “shocking pink skirt” to the “black polka dot skirt” in the tone harmony judgment The following alternatives can also be presented.
・ "70's mode" = 80% harmony (overall), alternative: gray hat → red ribbon

  In addition, personal color types (spring type, summer type, etc.) are discriminated from the color tone of hair and skin in color tone measurement, and pattern information for the corresponding personal color is used as a comparison target. It is also possible to propose clothes that match the color.

  Further, as an alternative presentation method, not only text display but also image data registered in a database on an operator's network such as animation and photographs may be used. At that time, a collage image may be generated and presented.

  Moreover, in this Embodiment, although the example which implement | achieves a harmony determination apparatus by the portable terminal with a camera function was shown, the image data made into the harmony determination object of this invention was image | photographed by the said harmony determination apparatus. It is not limited to image data. For example, in a network configuration as shown in FIG. 9, by capturing image data captured using a portable terminal 504 equipped with a camera or a personal computer (PC) 505 via a memory card or LAN, for example, It can also be realized by the user terminal 506. In addition, the database prepared in the terminal in the example shown in FIG. 3 can be provided on a server on the mobile terminal operator's network 500 or the Internet 501 like the database A (502) or the database B (503). It is.

  For example, the user transmits image data showing a person who looks like a degree of harmony to the user terminal 506 that implements the degree of harmony judgment apparatus according to the present invention via the network 500 or the Internet 501. Note that the user terminal 506 may be, for example, a mobile terminal that is not equipped with a camera owned by the user.

  For example, the user terminal 506 receives the image data from the Internet 501 via the LAN-IF 103 and stores the image data in the memory 113. Note that the image data may be received from the network 500 via the antenna 101 and the TRX unit 102 and stored in the memory 113.

  Further, by storing image data captured using a general digital camera in a portable memory (such as an SD card) and connecting (inserting) the portable memory to the memory card IF 117, the memory card IF11 The image data may be read from and stored in the memory 113.

  In the image recognition process, image data to be determined is read from the memory 113 and the same process may be performed. When data for pattern matching exists in the database A or database B on the network, the communication IF 105 is controlled whenever necessary, and the database is transmitted via the LAN-IF 103, the antenna 101, and the TRX unit 102. Just go to

  In addition, when the image data is read, the current date is read using a built-in clock of a mobile terminal or user terminal that implements the harmony determination device, or NTP (Network Time Protocol) on the Internet. By determining fall / winter, it is also possible to make judgments and make proposals according to the season. For example, if the date when the image data is read is April 5, it is determined that the current season is spring, and if the detected clothing item includes a down jacket, What is necessary is just to produce | generate clothing pattern information so that a spring coat etc. may be proposed as an alternative. In addition to the clothing pattern information, information on the type of clothing item that matches the season may be stored, and a clothing item to be a substitute candidate may be determined based on the information.

  In addition, for example, it is possible to propose clothes suitable for the weather by storing the weather information in the database of the business server and reading the current weather information when the image data is read. For example, the information of each clothing item included in the clothing pattern information may include information indicating whether or not it is usable depending on the weather, and the degree of harmony may be determined in consideration of the information. In addition to the clothing pattern information, information on clothing items that cannot be worn according to the weather may be stored, and clothing items that are alternative candidates may be determined based on the information.

  In the above embodiment, the description is focused on the user's clothing as the target for determining the degree of harmony, but a plurality of products are combined such as the exterior of the building, the exterior of the house, and the indoor interior. It is also possible to target a single object or a space as a whole. In this case, for example, if it is a color tone harmony determination, even if the item constituting the object or space is not known, by dividing the image data based on the tone distribution as one item, It can be applied to the overall harmony determination.

  The present invention is not limited to the user's clothes, but if the user composes a single object or space as a whole by combining a plurality of products and can store it in image data, for example, Whether the exterior of the building, the exterior such as around the house, the interior of the room, and the like can be determined as to whether the combination is good or bad.

It is a block diagram which shows the structural example of the harmony determination apparatus by this invention. It is a block diagram which shows the other structural example of the harmony determination apparatus by this invention. It is a block diagram which shows the more specific structural example of a harmony determination apparatus. It is explanatory drawing which shows the more specific structural example of the image recognition part. It is explanatory drawing which shows the more specific structural example of the coordination process part. It is a flowchart which shows an example of the harmony determination operation | movement in this embodiment. It is explanatory drawing which shows the example of clothing pattern information. It is explanatory drawing which shows the example of color tone pattern information. It is explanatory drawing which shows the example of the network structure containing the harmony determination apparatus by this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Harmony determination apparatus 11 Item detection part 12 Item harmony determination part 13 Color tone measurement part 14 Color tone harmony determination part 101 Antenna 102 TRX part 103 LAN-IF
104 MOD section 105 Communication IF
106 Control Unit 107 Image Processing Unit 108 Imaging Unit 109 Display Processing Unit 110 Display Unit 111 Input Processing Unit 112 Operation Input Unit 113 Memory 114 Database 115 Image Recognition Unit 1151 Clothing Item Recognition Unit 1152 Pattern Recognition Unit 1153 Hue Measurement Unit 1154 Saturation Measurement Unit 1155 lightness measurement unit 116 coordination processing unit 1161 clothing item harmony determination unit 1162 color tone harmony determination unit

Claims (13)

It is a harmony determination device that determines whether or not the combination is a generally preferred combination for a combination of items that together form a single concept when combined,
It is possible to access item pattern information indicating at least the type of each item constituting the combination stored as information indicating a combination of items generally preferred in advance,
An item detection unit that analyzes image data showing that items to be determined are combined and detects what type of items are combined;
The type of each item detected by the item detection unit is compared with the type of each item constituting one combination indicated by the item pattern information, and the combination of the detected items is included in the item pattern information. An item harmony degree determination unit that determines whether or not the combination of the items to be determined is a combination that is generally preferred by obtaining a degree of coincidence indicating how much it matches one included combination A harmony determination device characterized by that. It is a harmony determination device that determines whether or not the combination is a generally preferred combination for a combination of items that together form a single concept when combined,
It is possible to access color pattern information indicating at least the distribution of color tone according to the combination stored in advance as information indicating a combination of items generally preferred in advance,
An item detection unit that analyzes image data showing that items to be judged are combined and detects what kind of contour items are combined;
A color tone measurement unit that measures the color tone of each item using image data of an area where each item detected by the item detection unit is distributed;
Comparing the color tone distribution by the combination of detected items indicated by the detection result by the item detection unit and the measurement result by the color tone measurement unit, and the color tone distribution by one combination indicated by the color tone pattern information, The combination of the items to be determined is generally preferred by obtaining the degree of coincidence indicating how much the combination of each detected item matches one combination included in the color pattern information. A harmony degree determination apparatus comprising: a color tone harmony degree judgment unit for judging whether or not. It is a harmony determination device that determines whether or not the combination is a generally preferred combination for a combination of items that together form a single concept when combined,
It is possible to access at least item pattern information indicating the type of each item constituting the combination, and at least color tone pattern information indicating a color tone distribution by the combination, which is stored in advance as information indicating a combination of items generally preferred. so,
An item detection unit that analyzes the image data showing that the items to be determined are combined and detects what type and contour items are combined;
A color tone measurement unit that measures the color tone of each item using image data of an area where each item detected by the item detection unit is distributed;
The type of each item detected by the item detection unit is compared with the type of each item constituting one combination indicated by the item pattern information, and the combination of the detected items is included in the item pattern information. An item harmony degree determination unit that determines whether or not the combination of items that are the determination target is a generally preferred combination by obtaining a degree of coincidence that indicates how much it matches one included combination;
Comparing the color tone distribution by the combination of detected items indicated by the detection result by the item detection unit and the measurement result by the color tone measurement unit, and the color tone distribution by one combination indicated by the color tone pattern information, The combination of the items to be determined is generally preferred by obtaining the degree of coincidence indicating how much the combination of each detected item matches one combination included in the color pattern information. The harmony degree determination apparatus according to claim 1, further comprising: a color tone harmony degree determination unit that determines whether or not the color tone harmony degree is determined.   The harmony determination apparatus according to any one of claims 1 to 3, which is mounted on a mobile phone.   The harmony determination apparatus according to any one of claims 1 to 4, wherein an item that forms one concept as a whole when combined is a clothing item. A harmony determination device for determining whether a combination of clothes items is a generally preferred combination,
The item detection unit analyzes image data showing a user wearing one or more clothing items, detects a specific human body part of the user, and from the image data of each region detected as a human body part, The harmony determination apparatus according to any one of claims 1 to 5, wherein detection is performed for a clothing item that will be worn on the human body part. The color tone measurement unit
A hue measurement unit that measures the hue of each item using the image data of the area in which each detected item is distributed, and detects the main hue that is the hue distributed in the largest area in the image data When,
Using the image data of the area where each detected item is distributed, the saturation of each item is measured, and the main saturation which is the area where the main hue detected by the hue measuring unit is distributed A saturation measurement unit for measuring saturation;
Using the image data of the area in which each detected item is distributed, the brightness of each item is measured, and the main brightness that is the brightness of the area in which the main hue detected by the hue measuring unit is distributed is determined. A brightness measurement unit to measure,
The color tone harmony determination unit includes a color tone distribution by a combination of detected items indicated by measurement results by the hue measurement unit, a saturation measurement unit, and a lightness measurement unit, and a color tone distribution by one combination indicated by the color pattern information. Of the measured main hue, saturation, and color tone, and the distribution of the color tone according to the combination that most closely matches the main color tone specified by the color tone combination. The determination as to whether or not the combination of the items to be determined is a generally preferred combination by determining a degree of coincidence indicating how much the combination is included in the information. 6. The harmony determination apparatus according to claim 1. The item harmony degree determination unit detects and presents the difference in the combination of items detected by the item detection unit with respect to the combination of items indicated by the item pattern information. The harmony determination apparatus according to the item. The color tone harmony degree determination unit detects and presents a difference in color tone distribution due to a combination of items detected by the item detection unit with respect to a color tone distribution indicated by the color tone pattern information. The harmony determination apparatus according to any one of the above. It is a harmony determination method for determining whether or not the combination is a generally preferred combination for a combination of items that together form a single concept when combined,
As information indicating a combination of items generally preferred in advance, at least item pattern information indicating the type of each item constituting the combination is stored,
Analyze the image data that shows the combination of items to be judged, detect what type of items are combined,
The type of each detected item is compared with the type of each item constituting one combination indicated by the item pattern information, and the combination of the detected items is a combination included in the item pattern information. A harmony determination method characterized by determining whether or not the combination of items to be determined is a combination that is generally preferred by obtaining a matching degree indicating how much they match. It is a harmony determination method for determining whether or not the combination is a generally preferred combination for a combination of items that together form a single concept when combined,
As information indicating a combination of items generally preferred in advance, at least color pattern information indicating a distribution of color by the combination is stored,
Analyze the image data that shows the combination of items to be judged, detect what contour items are combined,
Using the image data of the area where each detected item is distributed, measure the color tone of each item,
The color tone distribution by the combination of the items to be determined indicated by the outline and color tone of each detected item is compared with the color tone distribution by one combination indicated by the color tone pattern information, and each detected By determining the degree of coincidence indicating how much the combination of items matches one combination included in the color tone pattern information, it is determined whether or not the combination of the items to be determined is a generally preferred combination. A harmony determination method characterized by determining. A program for determining the degree of harmony for determining whether a combination is a generally preferred combination for a combination of items that together form a single concept when combined,
A computer capable of accessing item pattern information indicating at least the type of each item constituting the combination stored as information indicating a combination of items generally preferred in advance,
Analyzing image data showing the combination of items to be judged, detecting what type of items are combined, the type of each detected item, and the item pattern The type of each item constituting one combination indicated by the information is compared, and the degree of coincidence indicating how much the combination of each detected item matches one combination included in the item pattern information is obtained. Accordingly, a program for determining the degree of harmony for executing a process of determining whether or not the combination of items to be determined is a combination that is generally preferred. A program for determining the degree of harmony for determining whether a combination is a generally preferred combination for a combination of items that together form a single concept when combined,
A computer capable of accessing at least color pattern information indicating a distribution of color tone based on the combination stored in advance as information indicating a combination of items generally preferred,
Analyzing the image data that shows the combination of items to be judged and detecting what kind of contour items are combined,
Processing for measuring the color tone of each item using image data of an area in which each item specified by the detected contour is distributed, color tone distribution by a combination of each detected item, and the color tone pattern information Is compared with the distribution of the color tone by one combination indicated by the above, and by determining the degree of coincidence indicating how much the combination of each detected item matches one combination included in the color tone pattern information, Processing for determining whether or not the combination of items to be determined is a generally preferred combination.

Images