JP2016032556A - Photographing device, skin diagnosis method, program, and skin diagnosis system - Google Patents

Abstract

To provide a skin diagnosis system and the like that can easily diagnose a skin condition from a high-resolution captured image taken in the same environment. In the skin diagnosis system, a terminal and a photography apparatus that captures and outputs a photograph are connected by communication, and the terminal stores means related to a captured image received from the photography apparatus, and storage Means for transmitting information relating to the photographed image to the photography apparatus, the photography apparatus receiving from the terminal information relating to the photographed image previously taken, and a photographed image obtained by photographing the subject. Based on the generated photographing image, information on the generated captured image, and information on the received captured image, and transmits information on the generated captured image to the terminal. And a transmission means. [Selection] Figure 14

Description

  The present invention relates to a technique for diagnosing a skin condition from a photographed image obtained by photographing a subject using a photography apparatus.

  In recent years, it has become possible to take high-resolution captured images by improving the performance of cameras built in digital cameras and portable terminals. For this reason, when a face photograph or the like is taken, it is possible to form a clear image up to the detailed skin condition.

  Also, with recent improvements in image processing technology, it is possible to extract or compare characteristic portions from high-resolution captured images. By using this technique for a photographed image obtained by photographing a face photograph or the like, it is possible to extract skin spots or wrinkles and diagnose the skin condition.

  In this regard, Patent Document 1 discloses that a data collection system installed in a cosmetic store or the like photographs a customer's face and transmits a high-definition image, and a data analysis system installed in a management center displays a high-definition image. An invention is disclosed that receives, analyzes an image, and compares it with reference data to diagnose a customer's skin condition.

  Further, Non-Patent Document 1 discloses an invention of an application for a mobile terminal that takes a face photograph with a camera of a mobile terminal, performs image analysis, displays the position of spots and wrinkles, and manages the history. ing.

Japanese Patent No. 4761924

Beautiful Skin Log Plus, [online], [Search June 24, 2014], Internet <http://app-liv.jp/648748506/>

  Although the invention of Patent Document 1 enables detailed diagnosis from a photographed image, there is a problem that comparison with a photographed image photographed last time is not possible. In addition, when installed in a cosmetic store or the like, there is an atmosphere in which cosmetics and the like must be purchased, and there is a problem that it is reluctant to use, and there is a problem that men are difficult to use.

  In the invention of Non-Patent Document 1, since shooting is performed with a camera of a portable terminal, there is a problem that it is difficult to compare with a previously captured image unless the image is captured in the same illumination environment. In addition, since shooting is performed with a camera built in the front surface of the mobile terminal, there is a problem that depending on the shooting environment, the resolution of the shot image is not stabilized and a high-resolution shot image cannot be obtained. In addition, there is a problem that image processing takes time when the photographing environment is bad. Furthermore, since a captured image may be transmitted to a server or the like via a network, there is a problem that it is reluctant to use.

  The present invention has been made to solve such a problem, and an object of the present invention is to provide a technique for easily diagnosing a skin condition from a high-resolution captured image captured in the same environment.

In order to solve the above-described problem, a first invention is a photography apparatus that captures and outputs a photograph that is connected to a terminal by communication, and receives information related to a photographed image that was previously photographed from the terminal. Based on receiving means for receiving, photographing means for photographing a subject to generate a photographed image, information on the generated photographed image, and information on the received photographed image, the temporal change of the photographed image is analyzed. A photography apparatus comprising: an analysis unit; and a transmission unit that transmits information about the generated captured image to the terminal.
According to the first aspect of the present invention, the photographic device can analyze a change in the photographic image with time based on information on the current photographic image and information on the previous photographic image.

It is preferable that the image processing apparatus further includes a calculation unit that calculates specific information about the skin region from the captured image, and the information about the captured image includes the specific information.
Thereby, the photographic device can analyze the temporal change of the photographed image from the unique information regarding the skin region of the current photographed image and the unique information regarding the skin region of the previous photographed image.

It is desirable that the analysis unit analyzes a change with time of the captured image from a difference between the unique information.
Thereby, the photography apparatus can analyze the change with time of the photographed image from the difference in the unique information regarding the skin region.

It is desirable that the specific information regarding the skin region includes information regarding spots.
Thereby, the photography apparatus can analyze the change with time of information about the stain.

It is desirable that the information regarding the captured image includes a captured image.
Thereby, the photography apparatus can browse the information regarding the photographed image together with the photographed image on the user terminal.

It is desirable to further include illumination means for irradiating ultraviolet rays so as to make the subject's skin features stand out when photographing the subject.
As a result, it is possible to facilitate analysis of changes over time in the captured image.

A second invention is a skin diagnosis method executed by a photography apparatus that captures and outputs a photograph connected by communication with a terminal, and receives information related to a photographed image that was previously photographed from the terminal. A receiving step, a shooting step of shooting a subject to generate a shot image, an analysis step of analyzing a time-dependent change of the shot image based on the generated information on the shot image and the received information on the shot image And a transmitting step of transmitting information relating to the generated captured image to the terminal.
According to the second aspect of the present invention, the photographic device can analyze the change over time of the photographic image based on the information regarding the current photographic image and the information regarding the previous photographic image.

A third invention is a program for causing a computer to function as a photo photographic device that is connected to a terminal by communication and shoots and outputs a photo, and the computer shoots from the terminal before. Based on the receiving means for receiving information about the image, the shooting means for shooting the subject to generate a shot image, the information about the generated shot image, and the information about the received shot image, the temporal change of the shot image is determined. It is a program for causing an analysis unit to analyze and a transmission unit to transmit information about the generated captured image to the terminal.
According to the third aspect of the present invention, the photographic device can analyze the change over time of the photographic image based on the information regarding the current photographic image and the information regarding the previous photographic image.

A fourth aspect of the invention is a skin diagnostic system in which a terminal and a photographic device that captures and outputs a photograph are connected by communication, and the terminal receives information about a captured image received from the photographic device. Means for storing, and means for transmitting the stored information relating to the photographed image to the photography device, wherein the photography device receives from the terminal information relating to the photographed image that was previously taken. Means, photographing means for photographing a subject to generate a photographed image, analysis means for analyzing a change over time of the photographed image based on the generated information on the photographed image and the received information on the photographed image; A skin diagnosis system comprising: a transmission unit configured to transmit information regarding the generated captured image to the terminal.
According to the fourth invention, the photographic device can analyze the temporal change of the photographic image based on the information on the current photographic image and the information on the previous photographic image, and the user terminal can analyze the previous photographic image. Can be stored and provided to the photography apparatus as necessary.

  According to the present invention, it is possible to provide a technique for easily diagnosing a skin condition from a high-resolution captured image captured in the same environment.

The figure which shows the outline of the skin diagnosis system The figure which shows the structural example of a photography apparatus Block diagram showing a hardware configuration example of a photography apparatus Block diagram showing a hardware configuration example of a user terminal The figure which shows the example of the information regarding the photographed image Figure showing an example of a captured image (for information about the captured image) The figure which shows the example of the specific information (information regarding a picked-up image) The flowchart which shows the flow of the skin diagnostic process of a photography device and a user terminal Flowchart showing the flow of skin diagnosis processing of the photographic device and user terminal (continued) (A) An example of a menu selection screen, (b) An example of a photographed photo existence confirmation screen Flowchart showing the flow of processing for extracting unique information of the photographing apparatus Flow chart showing the flow of analysis processing of change over time of the photography apparatus The figure which shows the example of the analysis processing result (analysis result) of change with time Example of analysis result display screen Example of display screen for shot images and unique information

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.
[Outline of skin diagnosis system]
FIG. 1 is a diagram showing an outline of a skin diagnosis system 1.

The skin diagnosis system 1 is a system in which a photography device 2 and a user terminal 3 are directly connected by communication in a wireless manner. The wireless communication is, for example, Wi-Fi, but is not limited thereto, Bluetooth (registered trademark), infrared communication, NFC (Near Field
Communication). Further, although it is not direct communication, communication via a cellular phone network may be used.

  The photographic device 2 is a photographic device capable of taking a high-resolution ID photo installed in a city such as around a station. Further, as will be described later, the photography apparatus 2 has a computer function such as a control unit and can communicate with the user terminal 3 in a wireless manner.

  The user terminal 3 is a smartphone, a tablet terminal, a mobile phone, or the like, and can store a captured image generated by the photography apparatus 2 and can communicate with the photography apparatus 2 in a wireless manner.

The outline of the skin diagnosis system 1 is as follows.
First, the user of the user terminal 3 (also a user who takes a picture with the photography apparatus 2 and is simply referred to as a user or a subject below) goes to a place where the photography apparatus 2 is installed, and the photography apparatus In step 2, an identification photograph is taken to generate a photographed image. Then, the photographic device 2 performs image processing on the skin portion of the photographed image, and extracts unique information such as spots and wrinkles. Then, the photographic device 2 associates the photographed image with the unique information, and transmits it to the user terminal 3 for storage.

Next, the user of the user terminal 3 goes to a place where the photography apparatus 2 is installed on another occasion, and first, from the user terminal 3 to the photography apparatus 2, the photographed image and the unique information previously taken. Send. Next, the user of the user terminal 3 shoots the ID photo with the photo shooting device 2 to generate a shot image.
Here, the user is not limited to using the same photography apparatus 2 that was used last time, and has a computer function such as a control unit and communicates with the user terminal 3 in a wireless manner as will be described later. It is possible to select a photographic apparatus 2 that can be used and use the photographic apparatus 2 installed at another location.

  Then, the photographic device 2 performs similar image processing on the captured image captured this time, and extracts unique information. Then, the photographic device 2 analyzes the change of the captured image over time from the current unique information and the previous unique information, and displays the analysis result on the screen. Thereby, the user can confirm whether or not the state of the skin has changed in the photographing apparatus 2 as compared with the previous photographing. Further, the photographic device 2 associates the photographed image with the unique information and transmits it to the user terminal 3 for storage for the next analysis.

[Configuration of Photographic Equipment]
FIG. 2 is a diagram illustrating a configuration example of the photography apparatus 2.
FIG. 2A is a perspective view showing the appearance of the photographic device 2.
The photography apparatus 2 takes a box-like form including, for example, a front surface 24, a rear surface 23, a side surface 21a, a side surface 21b, a ceiling surface 22, and a floor surface 25. An entrance / exit part is provided on the side surface 21a side.

An inner wall 20 is provided in the box, and a photographing device for taking a picture, an operation screen, a port for charging a fee, and the like are provided on the inner wall 20.
A chair 28 on which a subject sits is provided near the rear surface 23 in the box. Further, a light-shielding curtain 27 is attached to the entrance / exit part of the box on the side surface 21a side in order to block light from the outside during photographing.

FIG. 2B is an overhead view of the photography apparatus 2.
A space between the inner wall 20 and the front surface 24 provided in the box is a storage box in which a photographing device, a printer, a charging device, a lighting device, a control device, and the like are stored.

FIG. 2C is a plan view showing a configuration example of the inner wall 20.
On the inner wall 20, there are an illumination (illumination unit) 208, a photographing device 204, a cash billing unit 201, an IC card reader / writer 203, an entrance sensor 207, a display / operation unit 205, a change return unit 202, a photo takeout port 206, and the like. Installed.

  The photographing device 204 is configured by a high-resolution digital camera or the like that photographs a subject sitting on the chair 28. The surface of the inner wall 20 is a shooting window, and a digital camera is installed in a storage box at the rear of the shooting window.

An illumination 208 is illumination in the box and a flash at the time of shooting.
Ultraviolet light may be used for the illumination 208 at the time of shooting by an operation by the user (subject) mode selection (ultraviolet mode). In the ultraviolet mode, an ultraviolet filter close to visible light can be imaged through the digital camera by setting an ultraviolet filter that blocks visible light on the digital camera and photographing.

  The cash billing unit 201 includes a bill insertion slot 201a, a curing slot 201b, a return lever 201c, and the like. . The coin mech is connected to a change return port 202 provided below the cash billing unit 201 and can return change or the like.

The IC card reader / writer 203 is an IC card reader for enabling payment by electronic money.
The entrance sensor 207 is an infrared sensor, for example, and detects entry of a subject into the box of the photographic device 2.

The display / operation unit 205 includes a display device having a touch panel function. The display device is a liquid crystal display, for example.
The display / operation unit 205 displays functions and descriptions of the photography apparatus 2, and performs function selection using a touch panel.
In addition, operation buttons and the like may be provided separately from the display.

The photo take-out port 206 is a take-out port for taking out the photographed ID photo and the like, and is provided in connection with a printer provided at the rear portion of the inner wall 20.
In addition to this, the photo taking-out port 206 may be provided on the side surface 21a so that the photo can be received after exiting from the entrance / exit of the photo taking device 2 after taking a picture.

FIG. 3 is a block diagram illustrating an example of a hardware configuration of the photography apparatus 2.
The photographic apparatus 2 is configured by connecting an IC card reader / writer 203, a coin mech 201, a lighting unit 208, an imaging unit 204, a display / operation unit 205, an entrance sensor 207, a printer 209, and the like to a main body 29. The

  The main unit 29 includes a control unit 291, a storage unit 292, an I / O controller 293, an I / F 295, a communication control unit 294, and the like.

The control unit 291 is a CPU (Central
A processing unit (ROM), a read only memory (ROM), a random access memory (RAM), and the like.
The CPU calls and executes a program stored in the ROM, the storage unit 292, and the like in a work memory area on the RAM, and drives and controls each connected device.
The ROM is a non-volatile memory and permanently holds a computer boot program, a program such as BIOS, data, and the like.
The RAM is a volatile memory, and temporarily stores programs, data, and the like loaded from the storage unit 292, ROM, storage medium, and the like, and includes a work area used by the control unit 291 to perform various processes.

The storage unit 292 is an HDD (Hard
Disk drive), the program executed by the control unit 291, data necessary for program execution, OS (Operating
System) and the like are stored.

The I / O controller 293 communicates between the main body 29, the illumination unit 208, the imaging device 204, the display / operation unit 205, the entrance sensor 207, the coin mech (cash billing machine) 201, and the IC card reader / writer 203. It is an intermediary interface.
The I / F 295 is an interface that mediates the exchange of information between the main body unit 29 and the printer 209, and transmits image data of a print output image and a printer control signal from the main body unit 29 to the printer 209.

  The communication control unit 294 is a communication unit including a communication interface such as a network interface card and a modem, and is connected to the user terminal 3 by a method such as Wi-Fi, Bluetooth (registered trademark), infrared communication, or NFC. Perform wireless communication.

[User terminal configuration]
FIG. 4 is a block diagram illustrating a hardware configuration example of the user terminal 3.
The user terminal 3 is configured by connecting a control unit 31, a storage unit 32, a communication control unit 33, an input unit 34, a display unit 35, an I / F unit 36, and the like via a bus 37.

The control unit 31 includes a CPU, a ROM, a RAM, and the like.
The CPU calls and executes a program stored in the ROM, the storage unit 32, and the like in a work memory area on the RAM, drives and controls each device connected via the bus 37, and realizes processing performed by the computer.
The ROM is a non-volatile memory and permanently holds a computer boot program, a program such as BIOS, data, and the like.
The RAM is a volatile memory, and temporarily stores a program, data, and the like loaded from the storage unit 32, ROM, storage medium, and the like, and includes a work area used by the control unit 31 for performing various processes.

The storage unit 32 is a flash memory or HDD, and stores a program executed by the control unit 31, data necessary for program execution, an OS, and the like. As for programs, a control program corresponding to the OS, application programs, files, and the like are stored.
These program codes are read by the control unit 31 as necessary, transferred to the RAM, and executed as various means by the CPU.
The storage unit 32 stores a program and various data according to the present embodiment of the present invention, and is read out and executed by the control unit 31 as necessary.

  The communication control unit 33 is a communication interface having a communication control device, a communication port, and the like, and can be used for the photography apparatus 2 and other devices by 3G, LTE, Wi-Fi, Bluetooth (registered trademark), infrared communication, NFC, and the like. Wireless communication with the computer.

The input unit 34 inputs data and includes an input device such as a small keyboard or a numeric keypad. An operation instruction, an operation instruction, data input, and the like can be performed on the computer via the input unit 34.
The display unit 35 includes a display device such as a liquid crystal panel and a logic circuit (a video adapter or the like) for realizing a video function in cooperation with the display device.
The input unit 34 and the display unit 35 are preferably configured integrally as a touch panel display.

The I / F unit 36 is a port or the like for connecting peripheral devices and the like.
The bus 37 is a path that mediates transmission / reception of control signals and data signals between the devices.

[Information on the storage unit of the user terminal]
Then, the information memorize | stored in the memory | storage part 32 of the user terminal 3 is demonstrated using FIGS.
FIG. 5 shows information 301 relating to a captured image.
The information 301 related to the captured image is information generated for each captured image, and is configured by associating at least an ID 3011, a captured date 3012, a captured image 3013, and unique information 3014.

The ID 3011 is an identifier for uniquely identifying the information 301 related to the captured image.
The shooting date 3012 is the date when the shot image was shot.
The captured image 3013 is information on the captured image. The captured image 3013 is the file name of the captured image (and the location of the folder, etc.), but may be the captured image data itself.
The unique information 3014 is unique information regarding the captured image, which will be described later with reference to FIG. The unique information 3014 is also a file name of unique information (and the location of the folder, etc.), but may be unique information data itself.

FIG. 6 shows a photographed image 302 that corresponds to face01.jpg of ID1 in FIG. The storage unit 32 of the user terminal 3 stores a captured image 302 for each piece of information 301 related to the captured image in FIG.
The photographed image 302 is photographed by the photograph photographing apparatus 2 and stains, wrinkles and the like are photographed.

FIG. 7 illustrates the unique information 303 corresponding to chara01.tbl of ID1 in FIG. The storage unit of the user terminal 3 stores unique information 303 for each piece of information 301 related to the captured image in FIG.
The unique information 303 includes at least an image size 3031 of the captured image, a subject area 3032, and a characteristic area 3033.

The image size 3031 of the captured image is the resolution of the captured image, and is represented by the number of pixels in the horizontal and vertical directions. For example, it is assumed that each pixel of the captured image is given a coordinate value based on the number of pixels, with the upper left of the captured image being the origin, the right direction being the X-axis positive, and the lower direction being the Y-axis positive.
The subject area 3032 indicates the position of the subject photographed in the photographed image, and is represented by, for example, the upper left coordinates and the lower right coordinates of the smallest rectangle including the subject. This can be used for alignment between captured images. The subject area 3032 may be a face area of the subject.

The characteristic area 3033 is information on a characteristic area extracted from the skin area of the subject in the photographed image, and is information on areas such as “wrinkles” and “stains”, for example.
The characteristic area 3033 is configured by linking at least an ID 3034, an attribute 3035, and a parameter 3036.

The ID 3034 is an identifier for uniquely identifying the characteristic area 3033 (attribute 3035 and parameter 3036).
The attribute 3035 is information indicating the attribute of the characteristic area 3033 and is, for example, “stain” or “wrinkle”.

  The parameter 3036 is a parameter related to the characteristic area 3033.

  For example, when the attribute is “stain”, the parameter 3036 is represented by the coordinates of all points included in the area. Further, in the case of “stain”, it may be expressed by coordinates of a plurality of points on the boundary of the area, may be expressed by coordinates of a vertex of a polygon surrounding the area, and includes the area The minimum rectangle may be represented by the upper left coordinates and the lower right coordinates, or may be represented by the coordinates and radius of the center of the smallest circle including the region. Moreover, the pixel value (color) information included in the region may be included.

  For example, when the attribute is “wrinkle”, the parameter 3036 is represented by the coordinates of all points included in the area. In the case of “wrinkle”, it may be expressed by coordinates of a plurality of points on the boundary of the area, may be expressed by coordinates of a vertex of a polygon surrounding the area, or a plurality of the areas may be expressed. The line segment may be approximated by the coordinates of the end points of a plurality of line segments, or thickness information may be added thereto. Further, the pixel value (color) information included in the region may be included.

  Note that the attribute 3035 is not limited to “stain” or “wrinkle” as long as it is a characteristic region extracted from the skin area of the subject in the captured image, and any attribute 3035 can be used. May be set accordingly.

  Note that the information 301 regarding the captured image in FIG. 5, the captured image 302 in FIG. 6, and the unique information 303 in FIG. 7 are also stored in the RAM or the like when the photograph apparatus 2 executes the skin diagnosis process.

[Skin diagnosis processing]
Next, the skin diagnosis process will be described with reference to FIGS.
FIG. 8 and FIG. 9 are flowcharts showing the flow of the skin diagnosis process of the photography apparatus 2 and the user terminal 3.

  The user terminal 3 stores and manages information 301 (photographed image 302, unique information 303) related to the photographed image shown in FIGS. 5 to 7, and an application for transmitting and receiving such information to and from the photograph photographing apparatus 2 is installed in advance. It is assumed that Further, it is assumed that the photography device 2 and the user terminal 3 are in a communicable state.

First, the control unit 291 of the photography apparatus 2 detects whether or not the user (subject) of the user terminal 3 has entered the photography apparatus 2 by using the room entry sensor 207 (step S11).
When the entrance sensor 207 detects entry into the photography apparatus 2 (Yes in step S11), the control unit 291 of the photography apparatus 2 displays the menu selection screen 211 on the display / operation unit 205 (step S12).

FIG. 10A is a diagram illustrating an example of the menu selection screen 211.
In addition to the “skin diagnostic mode” 2113, the menu selection screen 211 includes, for example, a “photographing mode” 2111 and a “skin-beautifying mode” 2112.
The “photographing mode” 2111 is a mode for taking a normal ID photo.
“Beautiful skin photographing mode” 2112 is a mode in which image processing is performed on the photographed ID photo so that the skin looks beautiful.
The “skin diagnosis mode” 2113 is a mode of the present embodiment and will be described below.

Returning to FIG. 8, the control unit 291 of the photography apparatus 2 accepts a menu selection by the user from the display / operation unit 205 (step S <b> 13).
When the selected menu is other than “skin diagnostic mode” (“other menu” in step S13), the control unit 291 of the photographic device 2 performs processing according to the selected menu (step S14). Then, the process ends.

  When the selected menu is “skin diagnosis mode” (“skin diagnosis” in step S13), the control unit 291 of the photographic device 2 proceeds to step S15.

  Next, the control unit 291 of the photographic apparatus 2 detects whether or not cash corresponding to the “skin diagnosis mode” has been inserted by the cash billing unit 201 (step S15).

When it is detected that cash has been inserted (Yes in step S15), the control unit 291 of the photographic device 2 controls the illumination unit 208, the display / operation unit 205, the photographic device 204, and the like, so that the user (subject) An ID photo is taken to generate a shot image 302 (step S16).
The captured image 302 generated here is an image of the upper body including the face of the subject, for example, as shown in FIG. A known technique, for example, a technique disclosed in Japanese Patent Application Laid-Open No. 2005-301037 can be used for taking the ID photo.

  Next, the control unit 291 of the photography apparatus 2 extracts the unique information 303 from the photographed image 302 by image processing (step S17).

  Here, with reference to FIG. 11, the unique information extraction process of the photography apparatus 2 will be described. The unique information 303 is information shown in FIG.

First, the control unit 291 of the photographic device 2 extracts a subject area 3032 from the captured image 302 (step S41).
Specifically, since the background color other than the subject is determined in the captured image 302, the control unit 291 of the photography apparatus 2 can extract the subject region 3032 by removing the pixel value of the background color. it can. Then, the control unit 291 of the photographic device 2 acquires the upper left coordinate and the lower right coordinate of the smallest rectangle including the subject in the captured image 302 as the subject region 3032.
For example, the face area of the subject may be set as the subject area 3032 using a known face recognition technique.

Next, the control unit 291 of the photographic device 2 extracts a skin region from the subject region 3032 (step S42). For example, a technique described in Japanese Patent Laid-Open No. 2005-276182 can be used to extract the skin region.
Specifically, the control unit 291 of the photographic device 2 has a rectangular area (an area that does not cover anything other than the skin such as eyes, nostrils, hair, etc.) that straddles both sides of the nose around the center of the nose of the subject area 3032. ) As a reference area. Next, the control unit 291 of the photographic device 2 obtains average pixel values μ _R1 , μ _G1 , and μ _B1 for each of RGB for the pixels included in the reference region.

Next, as shown in Expression (1), the control unit 291 of the photographic device 2 sets pixel values (R _i , G _i , B _i ) for all of RGB for each pixel in the subject region 3032. Pixels in which the difference from the average pixel value (μ _R1 , μ _G1 , μ _B1 ) falls within a predetermined threshold (δ _R1 , δ _G1 , δ _B1 ) are defined as pixels constituting the skin region.

Alternatively, as shown in Expression (2), the control unit 291 of the photographic device 2 calculates the average pixel value (μ _R1 , μ _G1 , μ _B1 ) for each pixel in the subject region 3032 in the three-dimensional color space. A pixel whose distance falls within the predetermined distance r ₁ may be a pixel constituting the skin region.

Next, the control unit 291 of the photographic device 2 extracts a dark region from the skin region (step S43). The dark area is an area representing a stain, a wrinkle or the like, and is an area where the color is close to black in the skin area.
Specifically, the control unit 291 of the photographic device 2 obtains average pixel values μ _R2 , μ _G2 , and μ _B2 for each RGB for the pixels included in the skin region (or the reference region described above).

Next, as shown in Expression (3), the control unit 291 of the photographic device 2 has pixel values (R _i , G _i , B _i ) for all pixels of the skin region for all of RGB. Pixels that are equal to or less than a value obtained by subtracting a predetermined threshold value (δ _R2 , δ _G2 , δ _B2 ) from the average pixel value (μ _R2 , μ _G2 , μ _B2 ) are defined as pixels constituting a dark region. _{However, δ R2 <δ R1, δ} G2 <δ G1, a δ _B2 <δ _B1.

Alternatively, the control unit 291 of the photographic device 2 can calculate the average pixel values (μ _R2 , μ _G2 , μ _B2 ) is a pixel whose distance is equal to or greater than a predetermined distance r ₂ (r ₂ <r ₁ ), for example, a pixel whose distance from the origin is closer than the distance between the origin and the average pixel value is a dark region. It is good also as a pixel to constitute.

  Note that the control unit 291 of the photographic device 2 examines the neighboring pixels for each pixel included in the dark region, or the area when the adjacent pixels are combined into a region. When (the number of pixels) is small, it may be deleted.

Next, the control unit 291 of the photographic device 2 determines the attribute 3035 and the parameter 3036 of the characteristic region 3033 included in the dark region (step S44).
Specifically, the control unit 291 of the photographic device 2 is characterized in that, for each pixel included in a dark region, a region obtained by combining adjacent pixels (hereinafter referred to as a connected region) satisfies a predetermined condition. The area 3033 is assigned with an ID 3034, an attribute 3035 is determined, and a parameter 3036 is extracted.

For example, the control unit 291 of the photographic apparatus 2 sets the attribute to “stain” when the connected area is substantially circular and the area (number of pixels) is included in a predetermined range, The parameter described in the parameter 3036 when the attribute 3035 of No. 7 is “stain” is extracted.
Further, for example, the control unit 291 of the photography apparatus 2 sets the attribute to “wrinkle” when the connected area is substantially linear (straight line, broken line, curved line) and the length is included in a predetermined range, With respect to the connected area, the parameter described in the parameter 3036 when the attribute 3035 in FIG. 7 is “wrinkle” is extracted.

  This is the end of the process for extracting the unique information of the photographic device 2. The photography apparatus 2 stores unique information 303 shown in FIG. 7 in a RAM or the like.

  Returning to FIG. Next, the control unit 291 of the photographic device 2 displays a photographed photo existence confirmation screen 212 on the display / operation unit 205 in order to confirm whether or not there is a previous photographed image. A selection of No is accepted (step S18).

  FIG. 10B is a diagram showing an example of a photographed photo existence confirmation screen 212. The photographed photograph existence confirmation screen 212 is provided with “Yes” and “No” buttons, which can be selected by the user.

  If there is no previous captured image (No in step S18), the control unit 291 of the photographic apparatus 2 proceeds to step S24 (FIG. 9). In addition, in the case of the user who performs skin diagnosis for the first time in the photography apparatus 2, this corresponds to the case where there is no previous photographed image.

  On the other hand, when there is a previous photographed image (Yes in step S18), the control unit 291 of the photography apparatus 2 requests the user terminal 3 for information related to the previous photographed image (step S19).

Then, the control part 31 of the user terminal 3 receives a request | requirement, and transmits the information regarding a previous picked-up image to the photography apparatus 2 (step S20).
Here, the information regarding the captured image that the control unit 31 of the user terminal 3 transmits to the photographing apparatus 2 includes information 301 regarding the captured image illustrated in FIG. 5, the captured image 302 illustrated in FIG. 6 associated with the captured image 3013, and The unique information 303 shown in FIG. 7 associated with the unique information 3014.
Here, when there are a plurality of pieces of information 301 relating to the photographed image as shown in FIG. 5, the control unit 31 of the user terminal 3 displays a selection screen (not shown) on the display unit 35, and the one selected by the user. Information relating to the photographed image is transmitted to the photography apparatus 2.

  Then, the control unit 291 of the photographic device 2 receives information regarding the previous captured image (step S21).

  Turning to FIG. Next, the control unit 291 of the photographic device 2 analyzes the temporal change of the photographic image based on the extracted information on the photographic image and the information on the previous photographic image (step S22).

  Here, with reference to FIG. 12, the analysis process of the temporal change of the photography apparatus 2 will be described. FIG. 13 is a diagram illustrating an example of a result of analysis processing of change with time (analysis result 2010).

In the following, I ₁ is information related to the previous captured image, E _{1 is} specific information, R _{1 is} the subject area, S ₁ is the characteristic area, and P ₁ is the parameter. I ₂ , unique information E ₂ , subject area R ₂ , characteristic area S ₂ , and parameter P ₂ .

  First, the control unit 291 of the photographic apparatus 2 adjusts the parameter scale of the characteristic area based on the subject area of the unique information (steps S51 to S54).

Specifically, the control unit 291 of the photographic device 2 uses the upper left coordinates of the subject area R _{1 in} the unique information E ₁ and coordinates of the subject area R ₁ and the parameter P of the characteristic area S ₁ . Each coordinate of ₁ is moved to be a coordinate with respect to the origin (0, 0) (step S51).

Similarly, the control unit 291 of the photographing apparatus 2 using the upper left coordinates of the specific information E ₂ of the subject region R _2, the subject region R ₂ coordinates, and, characteristic area S ₂ of the parameter P ₂ Each coordinate is moved to be a coordinate with respect to the origin (0, 0) (step S52).

Next, the control unit 291 of the photographing apparatus 2 obtains the ratio of the X-axis direction and the Y-axis direction of the subject region _{R 2} with respect to the subject region _{R 1} (step S53).
Next, the control unit 291 of the photographing apparatus 2, to each coordinate parameters P ₁ of the characteristic region S _1, by multiplying the ratio obtained, the parameter P ₁ of the characteristic region S ₁ and the characteristic fit a measure of a parameter _{P 2} in the region _{S 2} (step S54).

Next, the control unit 291 of the photographing apparatus 2, a characteristic area S ₁ of the information I ₁ on previous captured image, calculates a difference between the characteristic region S ₂ of the information I ₂ relating to current captured image The change with time is analyzed (steps S55 to S60).

Specifically, the control unit 291 of the photographing apparatus 2 selects one of the characteristic region S _2, among the characteristic region S _1, attribute are the same, indicated by the parameter P ₂ the difference between the area and the area indicated by the parameter P ₁ that is searching for a characteristic region S ₁ sufficiently small (step S55). The difference may be a difference in the area (number of pixels) of the region, or may be a difference in consideration of the pixel value included in the region when the pixel value is also held.

Next, the control unit 291 of the photography apparatus 2 determines whether the search has been successful (step S56). If the search is successful, a case where the characteristic region S ₁ corresponding to the characteristic region S ₂ found.

If the search is successful (Yes in step S56), the control unit 291 of the photographic device 2 determines that “no change” if the difference is almost 0, “increase” if the difference is positive, and “increase” if the difference is negative. determining a reduced ", and stores in association with the ID of the characteristic region _{S 2} ID and characteristic area _{S 1} (step S57). Examples are shown in 2011, 2012, and 2013 of the analysis result 2010 shown in FIG.

On the other hand, if the search has failed (No in step S56), the control unit 291 of the photographing apparatus 2, a characteristic region _{S 2} is determined to be "generated", is stored together with the ID (step S58). An example is shown in 2014 of the analysis result 2010 shown in FIG.

Next, the control unit 291 of the photographing apparatus 2 determines whether there is a characteristic region S ₂ of the untreated (step S59). If there is a distinctive area _{S 2} of the untreated (Yes in step S59), the control unit 291 of the photographing apparatus 2 returns to step S55.

If there is no distinctive region S ₂ of the untreated (No in step S59), the judgment control unit 291 of the photographing apparatus 2, a characteristic region S ₁ which ID is not tied with the "disappearance" And it memorize | stores with the ID (step S60). An example is shown in 2015 of the analysis result 2010 shown in FIG.

  This is the end of the analysis process for changes with time of the photographic apparatus 2. The photographic device 2 stores the result of the analysis process of change with time shown in FIG. 13 (analysis result 2010) in a RAM or the like.

  Returning to FIG. Next, the control unit 291 of the photographic apparatus 2 displays the analysis result 2010 on the display / operation unit 205 (step S23).

FIG. 14 is a diagram illustrating an example of the analysis result display screen 213.
For example, the captured image included in the information related to the previous captured image is displayed on the left as “previous captured photograph”, and the current captured image is displayed on the right as “current captured photograph”. And based on each specific information 303 (FIG. 7), the characteristic area | region 3033 is enclosed with a dotted line.

Based on the analysis result 2010 shown in FIG.
A pair of characteristic areas of “no change” 2011 is given the same reference numerals in the corresponding characteristic areas of “previous photograph” and “current photograph”, and “current photograph”. "No change" is displayed.
A pair of characteristic areas of “Increase” 2012 is similarly assigned the same reference numeral, and “Increase” is displayed in “Currently taken photograph”.
A pair of characteristic areas of “reduction” 2013 is similarly assigned the same reference numeral, and “reduction” is displayed in “currently taken photograph”.
In the characteristic area of “occurrence” 2014, a sign is added to “currently taken photograph” and “occurrence” is displayed.
In the characteristic area of “disappearance” 2015, a sign is attached to “previous photograph” and “disappearance” is displayed.

  Returning to FIG. On the other hand, when there is no information related to the previous captured image (No in step S18 in FIG. 8), the control unit 291 of the photographic apparatus 2 displays the captured image and unique information display screen 214 on the display / operation unit 205. Display (step S24).

FIG. 15 is a diagram showing an example of a captured image and unique information display screen 214.
For example, the current photographed image is displayed in the center as a “photographed photograph”. Then, based on the unique information (FIG. 7), the characteristic area is surrounded by a dotted line, and the attribute is displayed.

  Returning to FIG. Next, the control unit 291 of the photographic device 2 transmits information 301 regarding the captured image (the captured image 302 and the unique information 303) and transmits it to the user terminal 3 (step S25).

  Next, the control unit 291 of the photographic device 2 displays a print confirmation screen (not shown) on the display / operation unit 205 in order to confirm whether or not to print, and the user selects Yes / No. Accept (step S26).

When printing (Yes in step S26), the control unit 291 of the photography apparatus 2 transmits the photographed photographed image to the printer 209, prints it as an ID photo, and outputs it to the photograph extraction port 206 (step S27). ).
At this time, the control unit 291 of the photography apparatus 2 also prints and outputs the analysis result display screen shown in FIG. 14 or the photographed image and unique information display screen shown in FIG. May be.
Then, the user of the user terminal 3 acquires an ID photo or the like from the photo takeout port 206.

When not printing (No in step S26), the control unit 291 of the photographic device 2 does not perform the process of step S27.
And the photography apparatus 2 complete | finishes a skin diagnostic process. After the skin diagnosis process, the control unit 291 of the photographic device 2 deletes the information 301 (the captured image 302 and the unique information 303) regarding the captured image from the RAM or the like.

  On the other hand, the control unit 31 of the user terminal 3 receives the information 301 related to the captured image (step S28), and stores the information 301 related to the captured image in the storage unit 32 (step S29).

After that (may be after the communication connection with the photography device 2 is terminated), the control unit 31 of the user terminal 3 may accept selection of whether or not to display the photographed image 302 (and the unique information 303) by the user. Good (step S30).
When displaying (Yes in step S30), the control unit 31 of the user terminal 3 displays a captured image 302 as illustrated in FIG. 6 stored in the storage unit 32. Further, when the unique information 303 is also displayed, a screen as shown in FIG. 15 may be displayed.
When not displaying (No in step S30), the control unit 31 of the user terminal 3 does not process step S31.

  This is the end of the skin diagnosis process for the photography device 2 and the user terminal 3.

In addition, this embodiment is not restricted to the above-mentioned thing, It can change in the range which does not deviate from the meaning.
For example, in the menu selection screen 211 (FIG. 10) in step S12 or the ID photo shooting in step S16, an ultraviolet mode in which the subject is irradiated with ultraviolet rays can be selected in order to make the subject's skin characteristics stand out. It may be. Thereby, characteristic areas such as spots and wrinkles can be made conspicuous.

  Further, for example, in step S25, the photographing apparatus 2 transmits the analysis result 2010 to the user terminal 3 and displays the analysis result display screen 213 as shown in FIG. Also good. Thereby, the user of the user terminal 3 can browse the analysis result 2010 even when the user is away from the photography apparatus 2.

  Further, for example, in step S25, the photography device 2 transmits to the user terminal 3 two pieces of a photographed image for browsing on the user terminal 3 and a photographed image for analyzing temporal changes in the photograph photographing device 2. It may be stored. As a result, the user terminal 3 can browse a properly adjusted photographed image, and the photographer 2 can take a photographed image with no (or little) degradation in image quality when analyzing the temporal change of the photographed image next time. Can be used.

  The unique information extraction process of the photography apparatus 2 shown in FIG. 11 is an example, and is not limited to this method. For example, correct data of a characteristic area may be prepared and a characteristic area may be extracted from a captured image using a machine learning technique.

Similarly, the temporal change analysis processing of the photography apparatus 2 shown in FIG. 12 is also an example, and is not limited to this method. For example, in order to adjust the scale of a parameter (captured image), a feature point may be extracted from the captured image using a SIFT (Scale-Invariant Feature Transform) technique, and a transformation matrix may be calculated therefrom.
Further, for example, in step S57, the difference including the pixel value may be calculated, and detailed determination such as “darkened” or “lightened” may be performed.

  Moreover, the display screen of the analysis result of FIG. 14 is an example, and is not limited thereto. Instead of showing changes to individual characteristic areas as shown in Fig. 14, calculate the sum of the differences of characteristic areas, etc. Such information may be displayed.

[Effect of the embodiment]
1. In the photographic device, skin diagnosis such as skin beautification can be performed, and analysis of changes in the skin over time (hereinafter referred to as skin diagnosis) can be performed using a previously captured image.
2. Since the photographic device is used, skin diagnosis can be performed using a high-resolution captured image in the same shooting environment every time.
3. Since the photographed image is stored in the user terminal and the photographed image is not stored in the photography device or the server, the skin diagnosis of the photography device can be used without hesitation.
4). Since the photographing apparatus is installed in various places, the user can easily use the skin diagnosis.
5. The user can manage the history of skin diagnosis on the user terminal.
6). The user can also obtain a photograph such as an identification photograph and an analysis result by using the skin diagnosis of the photography apparatus.
7). The user can use the skin diagnosis of the photography apparatus in a short time and at a low price.

  The preferred embodiments of the skin diagnosis system and the like according to the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to such examples. It will be apparent to those skilled in the art that various changes or modifications can be conceived within the scope of the technical idea disclosed in the present application, and these naturally belong to the technical scope of the present invention. Understood.

1 ……………………… Skin diagnostic system 2 ……………………… Photographing device 3 ……………………… User terminal 211 ………………… Menu selection screen 212 ………………… Photograph existence confirmation screen 213 ………………… Analysis result display screen 214 …………………… Photographed image and unique information display screens 291 and 31 ………… Control Section 301 ………………… Information about the shot image 302 ………………… The shot image 303 ………………… Unique information 2010 ……………… Analysis result

Claims (9)

A photography device that captures and outputs a photograph connected by communication with a terminal,
Receiving means for receiving information related to a previously captured image from the terminal;
Photographing means for photographing a subject and generating a photographed image;
Based on the generated information on the captured image and the received information on the captured image, analysis means for analyzing the change over time of the captured image,
Transmitting means for transmitting information about the generated captured image to the terminal;
A photographic apparatus comprising: A calculation means for calculating specific information on the skin area from the captured image;
The photographic device according to claim 1, wherein the information regarding the captured image includes the unique information. The photography apparatus according to claim 2, wherein the analysis unit analyzes a change with time of the photographed image from a difference between the unique information. The photography apparatus according to claim 2 or 3, wherein the specific information relating to the skin region includes information relating to a stain. The photographic device according to any one of claims 1 to 4, wherein the information about the photographic image includes a photographic image. The photography according to any one of claims 1 to 5, further comprising illumination means for irradiating ultraviolet rays so as to conspicuously characterize the skin of the subject when photographing the subject. apparatus. A skin diagnosis method executed by a photography apparatus connected to a terminal by communication and taking and outputting a photograph,
Receiving from the terminal information related to a previously captured image;
A shooting step of shooting a subject and generating a shot image;
An analysis step of analyzing a time-dependent change of the captured image based on the generated information regarding the captured image and the received information regarding the captured image;
Transmitting the generated information about the captured image to the terminal; and
A skin diagnostic method comprising the steps of: A program for causing a computer to function as a photography device that is connected to a terminal by communication and captures and outputs a photograph,
The computer,
Receiving means for receiving information related to a previously captured image from the terminal,
Photographing means for photographing a subject and generating a photographed image;
Analysis means for analyzing a change over time of the captured image based on the generated information regarding the captured image and the received information regarding the captured image;
Transmitting means for transmitting information relating to the generated captured image to the terminal;
Program to function as. A skin diagnosis system in which a terminal and a photography apparatus that captures and outputs a photograph are connected by communication,
The terminal
Means for storing information relating to a photographed image received from the photography apparatus;
Means for transmitting information relating to the stored photographed image to the photography apparatus;
With
The photography apparatus includes:
Receiving means for receiving information about the captured image previously captured from the terminal;
Photographing means for photographing a subject and generating a photographed image;
Analysis means for analyzing a change over time of the captured image based on the generated information regarding the captured image and the received information regarding the captured image;
Transmitting means for transmitting information about the generated captured image to the terminal;
A skin diagnosis system comprising:

Images