JP2016168303A - Organ photographing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To determine whether the photographing state of an organ is good or no-good for acquisition of an image with good photographing state, and to reliably acquire information required for diagnosis from the image with good photographing state.SOLUTION: An organ photographing apparatus 1 comprises an imaging unit 4, an information extraction unit 14, a determination unit 15, and an imaging control unit 11. The imaging unit 4 photographs an organ of a living body to acquire an image. The information extraction unit 14 extracts, from the image, diagnostic information required for diagnosis of the organ. The determination unit 15 determines whether the photographing state of the organ is good or no-good. The imaging control unit 11 controls the imaging unit 4 according to the determination result of the determination unit 15. The determination unit 15 extracts, from the image acquired by the imaging unit 4, determination information for good/no-good determination of the photographing state, and determines whether the photographing state is good or no-good, on the basis of the determination information. The determination information contains the same information as the diagnostic information.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an organ image photographing apparatus for photographing an organ of a living body and extracting information necessary for health diagnosis.

  Diagnosis using biological images is widely used in the medical field. In the above diagnosis, not only the skin on the surface of the body such as the face and limbs, but also the diagnosis of membranous tissues such as the lips and heels, and X-rays and ultrasound on digestive and internal organs such as the stomach and intestines It is the basis of diagnosis.

  In oriental medicine, diagnostic methods (diagnosis, tongue examination) for diagnosing a health condition or medical condition by observing the condition of the face and tongue are known. Information necessary for tongue examination includes the color of the subject (tongue and moss), sharpness, surface irregularities, and gloss (humidity). A device that supports the diagnosis of health by photographing the tongue requires a function to reproduce accurate colors by photographing the tongue and a function to read detailed information (image resolution). A function that can accurately reproduce the unevenness and glossiness of the surface is required. In addition, in order to realize these functions, it is necessary to appropriately set shooting conditions (illumination color, brightness, angle, camera resolution, magnification, shooting distance, etc.) when shooting the tongue.

  In a camera studio or the like, an experienced photographer sets appropriate shooting conditions using a dedicated device, so that an appropriate shot image can be acquired. However, in the medical field, if a doctor or nurse does not have sufficient shooting technology, an appropriate shot image cannot be acquired because shooting is not performed under appropriate shooting conditions. In addition, in a health diagnosis application in which a subject to be photographed (for example, a tongue) is automatically photographed with a toilet or a wash basin or the like, the subject does not know how to put out an appropriate tongue, and thus an appropriate photographed image cannot be acquired.

  Here, when the way of putting out the tongue changes, the information related to the diagnosis item of Oriental medicine (Chinese medicine) changes. For example, because the tongue is a mucosal organ and the surface is wet, if the angle when the tongue is put out changes, the way the illumination light strikes the tongue changes, and the glossiness obtained based on the reflection component of the illumination light changes . Also, since there is a muscular tissue inside the tongue, the tongue becomes thicker or thinner (the shape of the tongue changes) depending on how the tongue is applied. In addition to how to put out the tongue, if the relative positional relationship (distance or angle) between the tongue and the device changes, the image is taken out of the appropriate imaging conditions, so information related to the diagnostic item Changes. Therefore, even if the tongue is photographed in such a state and an image is acquired, an appropriate diagnosis cannot be performed based on the image.

  In this regard, for example, in Patent Documents 1 to 3, the position of the face is fixed by supporting the chin and forehead with a support at the time of photographing. In particular, in Patent Document 1, a camera is arranged in the center of a mirror so that the tongue can be adjusted while viewing the mirror image. Moreover, in patent document 4, the instrument which contacts the front-end | tip of a tongue is prepared, and the tongue is stably made stationary by taking out a tongue and making it contact with an instrument.

Japanese Unexamined Patent Publication No. 2009-28058 (see paragraphs [0031], [0038], FIG. 4, etc.) JP 2001-314376 A (see paragraph [0024], FIG. 4 etc.) JP 7-398 A (see paragraph [0014], FIG. 11 etc.) JP 2010-259487 A (refer to claim 4, paragraphs [0023], [0025], FIG. 2, etc.)

  However, in Patent Documents 1 to 4, it is not known whether the tongue is good or bad just by supporting the jaw, forehead, or tongue (tip) with a support or instrument at the time of photographing. For this reason, depending on how the tongue is put out, there is a possibility that an image in a good photographing state cannot be acquired. In diagnosis of the tongue, information necessary for diagnosis (for example, glossiness information if lingual surface moisture is diagnosed and tongue contour information if dent marks are diagnosed) is taken. It needs to be reliably acquired from the image.

  The present invention has been made to solve the above-described problems, and its purpose is to determine whether the imaging state of the organ is good or not, to acquire an image with a good imaging state, and to acquire with a good imaging state. Another object of the present invention is to provide an organ imaging apparatus capable of reliably acquiring information necessary for diagnosis from the obtained image.

  An organ image capturing apparatus according to an aspect of the present invention includes an imaging unit that captures an image of a living organ and acquires an image; an information extraction unit that extracts diagnostic information necessary for diagnosis of the organ from the image; A determination unit configured to determine whether the imaging state of the organ is acceptable; and an imaging control unit configured to control the imaging unit according to a determination result of the determination unit, wherein the determination unit is acquired by the imaging unit. Determination information for determining the quality of the shooting state is extracted from the image, and the quality of the shooting state is determined based on the determination information. The determination information includes the same information as the diagnostic information.

  The determination unit extracts determination information from the image acquired by the imaging unit, and determines the quality of the shooting state based on the determination information. Since the determination information includes the same information as the diagnostic information extracted by the information extraction unit, the determination unit can determine the quality of the imaging state using the information (diagnosis information). Thereby, when it is determined that the imaging state is good, it is possible to reliably acquire diagnostic information used for determining the quality of the imaging state from an image with a good imaging state. As a result, appropriate diagnosis based on the diagnostic information can be performed by the present apparatus or an external apparatus. The above-mentioned “image in good imaging state” may be an image itself that is determined to be in good imaging state, or may be an image acquired by imaging an organ again in a good imaging state. Good.

  In addition, since the imaging control unit controls the imaging unit according to the determination result in the determination unit, if the determination unit determines that the shooting state is bad, for example, until the shooting state is determined to be good It is possible to prevent the information extraction unit from performing imaging (main imaging) for extracting diagnostic information by performing imaging (preliminary imaging) for determining whether the imaging state is good or bad. Thereby, the user can be prompted to adjust the imaging state of the organ, and the imaging state can be improved. Then, it is possible to acquire an image in a good photographing state and acquire diagnostic information from the image in the same manner as described above.

  In this way, the determination unit determines the quality of the imaging state of the organ, and the imaging control unit controls the imaging unit according to the determination result, finally obtaining an image with a favorable imaging state, The diagnostic information can be reliably acquired from the image.

  The organ may be a tongue. In this case, whether the imaging state of the tongue is good or bad can be determined, and an image having a good imaging state can be acquired, and the diagnostic information for the tongue can be reliably acquired from the tongue image acquired in the favorable imaging state. it can.

  The determination information may include information on at least one of glossiness of the tongue and a curved shape in a cross section perpendicular to the left-right direction of the tongue as the same information as the diagnostic information. In this case, the determination unit can determine the angle of the tongue and the angle of the imaging optical axis of the imaging unit from any of the above information, and can determine the quality of the imaging state based on the calculated angle. Become.

  The determination unit obtains at least one of the angle of the tongue with respect to the vertical direction and the angle of the photographing optical axis of the imaging unit with respect to the horizontal direction based on the determination information, and determines whether the photographing state is good based on the obtained angle. You may judge. Based on the angle, the quality of the shooting state can be reliably determined.

  The determination information may include tongue contour information as the same information as the diagnostic information. In this case, the determination unit can determine the length of the tongue from the contour line of the tongue, and can determine the quality of the photographing state based on the determined length of the tongue.

  The determination unit may determine the aspect ratio of the tongue based on information on a contour line of the tongue, and determine whether the photographing state is good or not based on the length of the tongue obtained from the determined aspect ratio. Based on the length of the tongue, it is possible to reliably determine the quality of the shooting state.

  The determination information may include, as the same information as the diagnostic information, information on at least one of the thickness of the tongue, the degree of curvature of the tongue along the left-right direction, and the glossiness of the tongue. In this case, the determination unit can determine how the tongue is applied from any of the above information, and can determine the quality of the shooting state based on the determination result.

  The determination unit may determine how the tongue force is applied based on the determination information, and determine whether the photographing state is good or not based on the determination result. The quality of the photographing state can be reliably determined based on how the tongue is applied.

  The determination information may include information on at least one of a tongue outline and a tongue color as the same information as the diagnosis information. In this case, the determination unit can determine the distance between the tongue and the device by obtaining the size or brightness of the tongue from any of the above information, and determine the quality of the shooting state based on the determination result. It becomes possible to judge.

  The determination unit determines the size or brightness of the tongue based on the determination information, determines the distance between the tongue and the device from the determined size or brightness of the tongue, and determines the distance based on the determination result. The quality of the shooting state may be determined. Based on the distance between the tongue and the device, it is possible to reliably determine the quality of the shooting state.

  You may further provide the output part which outputs the determination result of the said determination part. By outputting a determination result indicating that the imaging state is good or bad at the output unit, the user can keep the organ still when the imaging state is good, It is possible to prepare for photographing for extraction (main photographing), and when the photographing state is poor, the position of the organ can be adjusted and the photographing for preliminary determination of the photographing state (preliminary photographing) can be started again. .

  The output unit may further output a message or a sound prompting a user to adjust the imaging position of the organ when the imaging unit determines that the imaging state is defective. In this case, the user can grasp that it is necessary to adjust the imaging position of the organ based on the output message or voice, and can correct the imaging position and start another imaging (preliminary imaging).

  The organ image capturing apparatus further includes an operation unit that receives an instruction of main imaging from a user, and the imaging control unit is configured to determine whether the organ imaging state is determined to be good by the determination unit. The imaging unit is caused to perform preliminary imaging to acquire an image of the organ, and after the determination unit determines that the imaging state of the image is good, the operation unit instructs the main imaging Is input, the imaging unit is caused to perform main imaging in the imaging state determined to be good, and an image of the organ is acquired, and the information extraction unit is configured to acquire the image acquired in the main imaging. The diagnostic information may be extracted from the image.

  When the determination unit determines that the shooting state of the image acquired by the preliminary shooting is good, the imaging unit receives the instruction of the main shooting from the user by the operation unit and executes the main shooting. As a result, the main photographing based on the user's intention can be executed. In addition, the information extraction unit extracts diagnostic information from the image captured and acquired by the imaging unit in the imaging state determined to be good. Diagnosis based on it becomes possible.

  The imaging control unit causes the imaging unit to perform preliminary imaging until the determination unit determines that the imaging state of the organ is good, and acquires an image of the organ. After the imaging state of the image is determined to be good, the imaging unit is instructed to start main imaging, and the imaging unit is made to perform main imaging in the imaging state determined to be good, and the image of the organ The information extraction unit may extract the diagnostic information from the image acquired in the main photographing.

  When the determination unit determines that the shooting state of the image acquired by the preliminary shooting is good, the imaging unit immediately executes main shooting in response to an instruction from the imaging control unit. Can be performed quickly. In addition, the information extraction unit extracts diagnostic information from the image captured and acquired by the imaging unit in the imaging state determined to be good. Diagnosis based on it becomes possible.

  The imaging control unit causes the imaging unit to perform preliminary imaging until the determination unit determines that the imaging state of the organ is good, and obtains an image of the organ. It is also possible to select an image that has been determined to be in a good shooting state by the determination unit from among images acquired by preliminary shooting, and extract the diagnostic information from the selected image.

  Since the diagnostic information is extracted from the image that is determined to be good in the preliminary shooting, it is not necessary to perform the main shooting for extracting the diagnostic information after the preliminary shooting, and the processing after the pass / fail determination is quickly performed. Can be done. In addition, the information extraction unit extracts diagnostic information from the image, so that diagnosis based on the diagnostic information can be performed by the apparatus or an external apparatus.

  The organ imaging apparatus may further include a display unit that displays an image acquired by the imaging unit together with a frame line for defining the imaging position of the organ. By displaying the frame line on the display unit, the organ can be guided to an appropriate imaging position by the frame line during imaging by the imaging unit. As a result, the imaging state of the organ can be approximated roughly favorably.

  According to the above configuration, it is possible to determine whether or not the imaging state of the organ is good, and to acquire an image with a favorable imaging state, and to reliably acquire information necessary for diagnosis from the image acquired with the favorable imaging state. be able to.

It is a perspective view which shows the external appearance of the organ imaging device which concerns on one Embodiment of this invention. It is a block diagram which shows the schematic structure of the said organ image imaging device. It is explanatory drawing which shows the positional relationship of the illumination part with respect to imaging | photography object, and an imaging part. It is explanatory drawing which shows an example of the picked-up image of a tongue, an edge extraction filter, and the outline of the tongue extracted using the said filter. It is explanatory drawing which shows the typical diagnostic item regarding a tongue and moss. It is explanatory drawing which shows the picked-up image of a tongue, and each diagnostic area | region collectively. It is a graph which shows the outline of a tongue, and an approximate curve. It is the graph which plotted the difference of the y coordinate in the said outline and the said approximated curve. It is explanatory drawing which shows the picked-up image of a tongue, and the cross-sectional shape of a tongue collectively. It is explanatory drawing which shows the outline of a tongue, and the detection area | region of moisture. It is a graph which shows the spectral distribution of a tongue. It is a graph which shows typically the frequency distribution of the image data of B extracted from the said detection area. It is explanatory drawing which shows the picked-up image of the tongue which has a crack in the surface. It is a graph which shows typically the frequency distribution of the image data of B in each of the case where there is no crack on the tongue surface, and the case where there is a crack. It is a perspective view which shows the schematic structure of the light projection part of the said organ imaging device. It is explanatory drawing which shows the state by which the linear light was projected in the horizontal direction with respect to the tongue surface from the said light projection part. It is explanatory drawing which shows the picked-up image when the linear light projected on the tongue surface by the said light projection part is imaged, and the cross-sectional shape of a tongue, when a tongue is thin and when it is thick. It is explanatory drawing which shows the relationship between the thickness of a tongue, and the movement by the muscle of a tongue, and the height distribution of the horizontal direction of a tongue surface. It is explanatory drawing which shows the one part area | region approximated with a polynomial with the said height distribution. It is explanatory drawing which shows typically the other setting method of the said area | region approximated with a polynomial in the said height distribution. It is explanatory drawing which shows distribution of the RGB image data of the picked-up image in the horizontal direction which passes along the approximate center of the up-down direction of the surface of a tongue, when a tongue is thin and is thick. It is explanatory drawing which shows typically the relationship between the planar shape and cross-sectional shape of a tongue. It is explanatory drawing which shows an example of the picked-up image of a tongue. It is explanatory drawing which shows the other example of the picked-up image of a tongue. It is a side view which shows the angle with respect to the vertical direction of a tongue. It is explanatory drawing which shows typically the area | region which can be glossed when the tongue is illuminated. It is explanatory drawing which shows the relationship between the angle of the tongue in a picked-up image, and a glossy area. It is explanatory drawing which shows the state which is projecting the linear light to the tongue surface by the light projection part to the vertical direction. It is explanatory drawing which shows the curved shape of the tongue surface obtained by the said light projection of the vertical direction. It is explanatory drawing which shows the relationship between the outline of a tongue, the width | variety of the left-right direction, and the length of an up-down direction. It is sectional drawing which shows the shape in the cross section along the horizontal direction of a tongue. It is explanatory drawing which shows the curvature degree when force enters into a tongue. It is explanatory drawing which shows the positional relationship of a tongue and the said organ image imaging device. It is a flowchart which shows an example of operation | movement in the said organ imaging device. It is explanatory drawing which shows an example of the frame line displayed on a display part. It is explanatory drawing which shows the other example of the said frame line. It is a flowchart which shows the other example of operation | movement in the said organ imaging device. It is a flowchart which shows the further another example of operation | movement in the said organ imaging device. It is explanatory drawing which shows together the picked-up image and frame line of a human lower eyelid.

  An embodiment of the present invention will be described below with reference to the drawings. In the present specification, when the numerical range is expressed as A to B, the numerical value range includes the values of the lower limit A and the upper limit B.

[1. Schematic configuration of organ imaging apparatus]
FIG. 1 is a perspective view showing an external appearance of an organ image photographing apparatus 1 of the present embodiment, and FIG. 2 is a block diagram showing a schematic configuration of the organ image photographing apparatus 1. The organ image photographing apparatus 1 includes an illumination unit 2, a light projecting unit 3, an imaging unit 4, a display unit 5, an operation unit 6, a communication unit 7, and an audio output unit 8. The illumination unit 2 and the light projecting unit 3 are provided in the housing 21, and the imaging unit 4, the display unit 5, the operation unit 6, the communication unit 7, and the audio output unit 8 are provided in the housing 22. The casing 21 and the casing 22 are configured separately, but may be integrated.

(Lighting part)
The illuminating unit 2 illuminates a living organ (here, tongue) that is a subject to be imaged, and includes an illuminator that illuminates the subject to be photographed from above. As the light source of the illuminating unit 2, in order to improve color reproducibility, a light source that emits a daylight color such as a xenon lamp is used. The brightness of the light source is controlled by the illumination control unit 9 in accordance with the sensitivity of the imaging unit 4 and the distance to the shooting target. For example, the brightness of the light source is controlled so that the illuminance of the photographing target is 1000 to 10000 lx. The illumination unit 2 includes a lighting circuit and a dimming circuit in addition to the above light source, and lighting / extinguishing and dimming are controlled by a command from the illumination control unit 9.

(Lighting part)
The light projecting unit 3 includes a light projector that projects (irradiates) linear light onto the surface of the tongue, which is an organ to be imaged. The light projecting unit 3 includes a light projecting unit 3a that projects linear light in the horizontal direction with respect to the tongue, and a light projecting unit 3b that projects linear light in the vertical direction (vertical direction). doing. The light projection by the light projecting unit 3 is controlled by the light projecting control unit 10. Details of the light projecting unit 3 will be described later.

(Imaging part)
The imaging unit 4 captures a living organ (for example, a tongue) and acquires an image under illumination by the illumination unit 2. The imaging unit 4 includes an imaging lens 41 and an area sensor 42 (see FIG. 3). The aperture (brightness of the lens), shutter speed, and focal length of the imaging lens 41 are set so that the entire range to be photographed is in focus. As an example, F number: 16, shutter speed: 1/120 seconds, focal length: 20 mm.

  The area sensor 42 is composed of an image sensor such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS), and has sensitivity and resolution so that the color and shape of the subject can be sufficiently detected. Is set. As an example, sensitivity: 60 db, resolution: 10 million pixels.

  Imaging by the imaging unit 4 is controlled by the imaging control unit 11. In addition to the imaging lens 41 and the area sensor 42, the imaging unit 4 includes a focus mechanism (not shown), a diaphragm mechanism, a drive circuit, an A / D conversion circuit, and the like, and commands from the imaging control unit 11. As a result, focus, aperture control, A / D conversion, and the like are controlled. In the imaging unit 4, for example, data of 0 to 255 in 8 bits is acquired for each of red (R), green (G), and blue (B) as captured image data. That is, the imaging unit 4 acquires images obtained by shooting the shooting target for each of the different colors (RGB), and thereby acquires image data of each color. The magnification of the imaging lens 41 of the imaging unit 4 is controlled by the imaging control unit 11 so that a necessary image resolution can be obtained.

  FIG. 3 is an explanatory diagram showing a positional relationship between the illumination unit 2 and the imaging unit 4 with respect to a photographing target (tongue or face). As shown in the figure, the image pickup unit 4 including at least the image pickup lens 41 and the area sensor 42 is arranged to face the shooting target. The illumination unit 2 is arranged so as to illuminate the imaging target at an illumination angle a of, for example, 0 ° to 45 ° with respect to the imaging optical axis X of the imaging unit 4 passing through the imaging target. Note that the imaging optical axis X refers to the optical axis of the imaging lens 41 included in the imaging unit 4.

  When the illumination angle a at the time of illumination is large, the measurement accuracy of the surface unevenness is improved, but the range in which the tongue can be photographed becomes small due to the shadow of the upper lip. On the contrary, when the illumination angle a is small, the photographing range is enlarged, but the measurement accuracy is lowered, and the color jump due to the regular reflection of the illumination light is increased. Considering the above, the preferable range of the illumination angle a is 15 ° to 30 °. In particular, when the diagnosis site is an organ such as a tongue of a living body, a desirable illumination angle a is about 30 °.

(Display section)
The display unit 5 includes a liquid crystal panel (not shown), a backlight, a lighting circuit, and a control circuit, and displays an image acquired by photographing with the imaging unit 4 and a frame line P (see FIG. 35). . As said image displayed on the display part 5, the image (preview image) acquired by the preliminary | backup photography before main imaging | photography other than the image acquired by main imaging | photography of the imaging part 4 is included. The frame P is an image frame for defining the imaging position of the organ (here, the tongue) at the time of imaging by the imaging unit 4. By displaying the frame line P on the display unit 5, the photographing position of the tongue can be guided by the frame line P when photographing the tongue.

  Further, the display unit 5 displays a result of pass / fail judgment of the shooting state, which will be described later, and a message (warning) prompting the correction of the shooting state, or information acquired from the outside via the communication unit 7 (for example, to an external medical institution). It is also possible to display the result of diagnosis by sending information. Display of various types of information on the display unit 5 is controlled by the display control unit 12.

  The image to be captured acquired by the imaging unit 4 is displayed on the display unit 5 after image processing such as thinning processing and color correction processing is performed by the image processing unit (not shown). Also good. In the following description, when it is simply expressed as “a (photographed) image to be photographed”, unless otherwise specified, it is acquired by photographing with the imaging unit 4 and the above-described image processing necessary for display is performed. It shall refer to the previous image.

(Operation section)
The operation unit 6 is an input unit for instructing imaging by the imaging unit 4, and includes an OK button (imaging execution button) 6a and a CANCEL button 6b. In the present embodiment, the display unit 5 and the operation unit 6 are configured by a common touch panel display device 23 (see FIG. 1). The display of the operation unit 6 on the touch panel display device 23 is controlled by the operation control unit 13. The operation unit 6 may be configured by an input unit other than the touch panel display device 23 (the operation unit 6 may be provided at a position outside the display area of the touch panel display device 23).

(Communication Department)
The communication unit 7 transmits information obtained by the information extraction unit 14 to be described later to an external device, and conversely receives various types of information transmitted from the external device (for example, diagnosis results in an external organization). It is an interface to do. Transmission / reception of information in the communication unit 7 is controlled by the communication control unit 17.

(Audio output part)
The audio output unit 8 outputs various kinds of information as audio, and is constituted by, for example, a speaker. The voice output unit 8 can output a voice of a result of determination of pass / fail of the shooting state described later, a message for prompting correction of the shooting state, a diagnosis result based on the information extracted by the information extraction unit 14, and the like. The diagnosis result may be obtained by the device, or may be diagnosed by an external engine and transmitted to the device. The audio output in the audio output unit 8 is controlled by the audio output control unit 18.

  The display unit 5 and the audio output unit 8 described above constitute an output unit that outputs (displays and outputs audio) a determination result of the determination unit 15 described later.

  In addition to the above, the organ imaging apparatus 1 of the present embodiment includes an information extraction unit 14, a determination unit 15, and a storage unit 16.

(Information extraction unit)
The information extraction unit 14 is an image processing unit that performs processing for extracting a contour line of the tongue from the image acquired by the imaging unit 4. In the present embodiment, the information extraction unit 14 extracts the contour line of the oral cavity region based on the luminance edge of the captured image (the portion in which the brightness is rapidly changed in the image).

  Luminance edge extraction can be performed using an edge extraction filter as shown in FIG. 4, for example. The edge extraction filter is a filter that weights pixels in the vicinity of the target pixel when performing first-order differentiation (when obtaining a difference in image data between adjacent pixels). Using such an edge extraction filter, for example, with respect to R image data of each pixel of the captured image, a difference between the image data is calculated between the target pixel and a neighboring pixel, and a pixel whose difference value exceeds a predetermined threshold is extracted. Thus, a pixel that becomes a luminance edge can be extracted. Since there is a luminance difference due to the shadow and teeth of the lips in the oral cavity area inside the lips, the contour line Q of the oral cavity area can be extracted by extracting pixels that become luminance edges as described above. .

  Note that the information extraction unit 14 may extract the contour line using the distance information of the image instead of the luminance edge. Since the lips are closest to the imaging unit 4, the boundary with the oral cavity can be detected by using the distance information.

  Further, the information extraction unit 14 extracts information (diagnosis information) necessary for organ diagnosis from the organ image acquired by imaging by the imaging unit 4. Here, FIG. 5 shows typical diagnostic items related to the tongue and moss in Oriental medicine. In general, as for the tongue, five items of the color of the tongue, the thickness of the tongue, the wetness of the tongue surface (glossiness), the shape of the tongue (tooth marks), and the cleft (crack of the tongue surface) are listed as diagnostic items. . As for the moss, diagnostic items include five items: moss color, moss smoothness (degree of nipple tissue separation), moss thickness, moss presence / absence, and moss distribution. In “change of state” in FIG. 5, a circle indicates a normal (healthy) state, and an abnormal (disease) state occurs when the user leaves the state.

  These diagnostic items are information for diagnosing the tongue. Further, the degree of tongue curvature is necessary to determine the thickness of the tongue, as will be described later, and the outline of the tongue is necessary to determine the presence or absence of tooth marks. Therefore, the tongue bending degree and the contour line can also be information for tongue diagnosis. A specific method for extracting diagnostic information other than the outline will be described later.

(Judgment part)
The determination unit 15 extracts determination information for determining the quality of the shooting state from the image acquired by shooting with the imaging unit 4, and determines the quality of the shooting state based on the determination information. The determination information includes the same information as the diagnosis information described above. Thereby, the determination part 15 can determine the quality of the imaging state based on the diagnostic information. In addition, the specific method of quality determination is mentioned later. After the quality determination of the shooting state, the imaging unit 4 is controlled by the imaging control unit 11 according to the result of the quality determination. In addition, said determination part 15 may be comprised integrally with the information extraction part 14 mentioned above (the determination part 15 may serve as the information extraction part 14).

(Memory part)
The storage unit 16 stores image data acquired by the imaging unit 4, information acquired by the information extraction unit 14 and the determination unit 15, information transmitted from an external device, and the like. It is a memory for storing a program for operating. Such a memory can include a RAM, a ROM, a nonvolatile memory, and the like.

(Control unit, etc.)
The organ imaging apparatus 1 includes an illumination control unit 9, a light projection control unit 10, an imaging control unit 11, a display control unit 12, an operation control unit 13, an information extraction unit 14, a determination unit 15, a storage unit 16, and a communication control unit 17. The audio output control unit 18 and an overall control unit 20 for controlling these units are further provided. As described above, the illumination control unit 9, the light projection control unit 10, the imaging control unit 11, the display control unit 12, the operation control unit 13, the communication control unit 17, and the audio output control unit 18 are the illumination unit 2, the light projection unit. 3, the imaging part 4, the display part 5, the operation part 6, the communication part 7, and the audio | voice output part 8 are each controlled. The overall control unit 20 is configured by a CPU (Central Processing Unit), for example, and executes an operation program stored in the storage unit 16. The illumination control unit 9, the light projection control unit 10, the imaging control unit 11, the display control unit 12, the operation control unit 13, the communication control unit 17, the audio output control unit 18, and the overall control unit 20 are integrated. It may be configured (for example, with one CPU).

[2. (Method for extracting diagnostic information)
Next, a method for extracting diagnostic information by the information extracting unit 14 will be described.

(Extraction of tongue color, moss color, moss thickness)
FIG. 6 shows a photographed image of the tongue. In the following, in the photographed image of the tongue, when the region formed in the vertical direction at the center in the left-right direction of the tongue is divided into three regions arranged in the vertical direction, each region is divided into the upper region R1, the central region. These are referred to as R2 and lower region R3. These regions are defined by the size shown in FIG. 6 based on the width W in the horizontal direction and the length H in the vertical direction of the region surrounded by the contour line Q of the tongue detected by the information extraction unit 14. However, the illustrated size is an example, and the present invention is not limited to this.

  Since the color of the tongue reflects the color of blood, the R component or the B component mainly increases or decreases in the RGB image data. Therefore, by obtaining the ratio of R (R / (R + G + B)) or the ratio of B (B / (R + G + B)) in the lower tongue region R3, the color of the tongue can be quantified and extracted. As the RGB data, an average value of R image data, an average value of G image data, and an average value of B image data among a plurality of pixels constituting the lower region R3 can be used. .

  The reason why the image data of the lower tongue region R3 is used in extracting the tongue color is as follows. That is, in tongue examination used in Kampo medicine, the color of the tongue is generally diagnosed at the left and right ends of the tongue without moss or at the lower center of the tongue, but the left and right ends of the tongue are This is because the way in which the illumination light strikes changes due to the unevenness, so that shading tends to occur, and the image data tends to fluctuate from a value indicating the original tongue color.

  The moss color changes from white to brown depending on the amount of keratinized tissue, and therefore the G component or (G + R) component mainly increases or decreases in the RGB image data. Therefore, it is possible to quantify and extract the color of moss by determining the ratio of G (G / (R + G + B)) or the ratio of (G + R) ((G + R) / (R + G + B)) in the upper region R1 of the tongue. it can. As the RGB data, an average value of R image data, an average value of G image data, and an average value of B image data among a plurality of pixels constituting the upper region R1 can be used. .

  When extracting the color of the moss, the image data of the upper region R1 of the tongue is used because the moss is a keratinized papillary tissue of the lingual mucosa and exists from the upper part of the tongue to the center. This is because there are many in the upper part.

  As the moss becomes thicker, the red color of the tongue changes to the white color of the moss, so that the R component or G component mainly increases or decreases in the RGB image data. For this reason, the thickness of moss can be quantified and extracted by determining the ratio of R (R / (R + G + B)) or the ratio of G (G / (R + G + B)) in the central region R2 of the tongue. As the RGB data, an average value of R image data, an average value of G image data, and an average value of B image data among a plurality of pixels constituting the central region R2 can be used. .

  It should be noted that the image data of the tongue central region R2 is used when extracting the thickness of the moss because the moss exists at the upper part of the tongue as described above, and the color of the central region R2 is the upper region R1. This is because the thickness of the moss can be determined by whether or not the color of the moss is close. By calculating the ratio of the color difference (for example, chromaticity difference) between the upper region R1 and the central region R2 of the tongue and the color difference (for example, chromaticity difference) between the central region R2 and the lower region R3, The thickness of moss can also be quantified.

(Extraction of tooth marks)
The information extraction unit 14 approximates the contour line of the tongue with an approximate curve, and detects unevenness (smoothness) of the contour line based on the degree of correlation between the contour line and the approximate curve, thereby detecting the tooth trace on the tongue. To do. Since the approximate curve is a smooth curve without fine irregularities, it can be said that the closer the contour line is to the approximate curve, the smoother and less the tooth trace. That is, the higher the degree of correlation between the contour line and the approximate curve, the fewer the tooth traces, and the lower the degree of correlation, the more tooth traces.

  Therefore, the state of the tooth trace can be detected based on the degree of correlation. For example, when the degree of correlation is high, the state of the tooth mark is mild (level 1), when the degree of correlation is low, the state of the tooth mark is severe (level 3), and the degree of correlation is intermediate It can be determined that the state of the tooth trace is intermediate (level 2) between mild and severe.

  Here, the following first to third indices can be used as indices representing the degree of correlation between the two parties.

A first index representing the degree of correlation is a determination coefficient R ² represented by the following equation.
R ² = 1 − {(Σ (yi−fi) ² ) / (Σ (yi−Y) ² )}
However,
i: On the xy plane, let j be the x coordinate of one end of the contour line or approximate curve,
Any value from j to k when the x-coordinate of the other end is set to k. Yi: y-coordinate value at the x-coordinate i of the point on the contour line fi: approximation on the xy-plane Value of y coordinate at x coordinate i of point on curve Y: Average value of yi for all points on contour line. Note that i, j, and k are all integers, j <k, and j ≦ i ≦ k. Σ (yi−fi) ² indicates the sum of (yi−fi) ² when i is changed from j to k, and Σ (yi−Y) ² changes i from j to k. The sum of (yi-Y) ^{2 at} the time.

Figure 7 shows the tongue contour lines (see the solid line) and the approximate curve (see the broken line), and the polynomial representing the approximate curve, and a coefficient of determination R ^2. The approximate curve is obtained by the least square method and is expressed by the following polynomial. The determination coefficient R ^{2 at} this time is 0.9942. In addition, in the coefficient of the approximate curve in the figure, the notation “E−n” indicates × 10 ⁻ⁿ .
y = 5 × 10 ⁻⁷ · x ⁴ + 6 × 10 ⁻⁶ · x ³ + 2 × 10 ⁻³ · x ² + 6.29 × 10 ⁻² · x + 21.213

The difference between the contour line and the approximate curve is determined by the determination coefficient R even in the portion A where the tooth trace is severe (the contour contour is large) and in the portion B where the tooth trace is mild (the contour contour is small). Reflected in ² . Therefore, if a method of detecting a tooth trace based on the determination coefficient R ² is used, even if the tooth trace is only a slight tooth trace, the light tooth trace is detected based on the determination coefficient R ^2. This makes it possible to improve the detection accuracy of tooth marks.

The second index representing the degree of correlation is a value obtained based on the difference between the coordinate values (y coordinates) between the contour line and the approximate curve, and is the maximum value of | yi−fi | It is also called “the maximum value of”. However,
i: On the xy plane, let j be the x coordinate of one end of the contour line or approximate curve,
Any value from j to k when the x-coordinate of the other end is set to k. Yi: y-coordinate value at the x-coordinate i of the point on the contour line fi: approximation on the xy-plane This is the value of the y coordinate at the x coordinate i of the point on the curve. Note that i, j, and k are all integers, j <k, and j ≦ i ≦ k.

  FIG. 8 is a graph in which the difference (| yi−fi |) of the y coordinate between the contour line of the tongue shown in FIG. 7 and its approximate curve (xy polynomial) is plotted. The value of | yi-fi | is larger in the part A where the tooth mark is severe and in the part B where the tooth mark is mild than in the part without the tooth mark. Therefore, even if the tooth trace is only a slight tooth trace, by detecting the maximum value of | yi-fi |, it becomes possible to detect a slight tooth trace based on that value. Detection accuracy can be improved.

The third index representing the degree of correlation is a value obtained based on the difference between the coordinate values (y coordinates) between the contour line and the approximate curve, and is a coefficient A represented by the following equation. That is,
A = Σ | yi-fi |
However,
i: On the xy plane, let j be the x coordinate of one end of the contour line or approximate curve,
Any value from j to k when the x-coordinate of the other end is set to k. Yi: y-coordinate value at the x-coordinate i of the point on the contour line fi: approximation on the xy-plane This is the value of the y coordinate at the x coordinate i of the point on the curve. Note that i, j, and k are all integers, j <k, and j ≦ i ≦ k. Σ | yi−fi | indicates the sum of | yi−fi | when i is changed from j to k (hereinafter, also referred to as “sum of coordinate value differences”).

  By using the sum of coordinate value differences instead of the maximum coordinate value difference, it is possible to detect tooth marks that accurately reflect the influence of fine tooth marks, and to further improve the detection accuracy of tooth marks Can do.

  Note that the reciprocal of the sum of the coordinate value differences, that is, the value of 1 / (Σ | yi−fi |) may be used as the degree of correlation.

(Extraction of dampness (glossiness))
FIG. 9 shows a photographed image of the tongue and a sectional shape of the tongue. When photographing the tongue, the tongue is projected forward from the oral cavity. In order for the imaging unit 4 to photograph the surface of the protruding upper lip side of the tongue, the tongue is curved so that the upper lip side surface is convex toward the imaging unit 4 side (see the CC ′ section). If necessary, the tongue may be defined in a specification or manual so that the tongue is guided to an appropriate photographing position.

  When the tongue is photographed with the arrangement of the illumination unit 2 and the imaging unit 4 shown in FIG. 3, a regular reflection region is generated in the upper half of the tongue (because the illumination unit 2 is above the photographing optical axis X). On the other hand, with respect to the left-right direction of the tongue, both the center and left and right ends of the tongue are recessed and curved in an M shape (see the section D-D '). Such a cross-sectional shape is substantially the same from the upper part to the lower part of the tongue. Furthermore, the central part E of the tongue may have a crack pattern. Therefore, here, the angle a at the time of illumination is set to 15 degrees, and the remaining area on the upper half of the tongue and avoiding the center and both ends in the left-right direction is a detection area suitable for detecting the glossiness. Set as.

  More specifically, the information extraction unit 14 detects the upper and lower ends and the left and right ends of the tongue from the contour line Q of the tongue, detects the vertical length H and the horizontal width W of the tongue, and FIG. Glossiness detection regions R4 and R5 are set with reference to the contour line Q of the tongue so that the positional relationship shown in FIG.

  FIG. 11 is a graph showing the spectral distribution of the tongue. Since the tongue has a mucosal structure and no epidermis, the color of the blood appears as the color of the tongue. Blood has a large R component (wavelength 600 nm to 700 nm) and a small amount of B component (wavelength 500 nm or less). Further, when the color of the tongue is light, the ratio of the R component decreases, and when it is dark, the ratio of the R component increases.

  On the other hand, moss is formed of keratinized papillary cells and exhibits a white to yellow color. And when the moss is thin, the color of the tongue as a base appears, so the ratio of the R component becomes high as shown in the figure, and when the moss is white and dark, the G component (wavelength 500 nm to 600 nm) The ratio goes up.

  Depending on the physical condition of the living body and individual differences, the color of the tongue and moss changes as described above, but the change in the B component is small. Therefore, in the present embodiment, the glossiness of the tongue surface is detected as follows based on the B image data obtained from the photographed image of the tongue.

  First, the information extraction unit 14 extracts B image data from each pixel in the detection regions R4 and R5 of the captured image, and creates a frequency distribution thereof. FIG. 12 schematically shows the frequency distribution of the extracted B image data. In FIG. 12, the horizontal axis indicates the B pixel value (image data), and the vertical axis indicates the frequency (number of pixels). However, in order to simplify the description here, the pixel value is a value from 1 to 100, and the larger the pixel value, the brighter the pixel value.

  Next, the information extraction unit 14 obtains a pixel value Dp corresponding to the maximum frequency Np from the above frequency distribution (Dp = 70 in the example of FIG. 12), and sets a value obtained by multiplying the pixel value Dp by 1.2 as a threshold value. M (M = 84 in the example of FIG. 12), the sum of the frequencies in the section from the threshold value M to the maximum value of the image data (maximum pixel value Dm = 100) is integrated as the number of upper pixels. In determining the pixel value Dp, a function that continuously indicates the frequency change may be obtained and smoothed to remove the noise, and then the pixel value Dp corresponding to the maximum frequency Np may be obtained. Alternatively, the number of upper pixels may be obtained by integrating the smoothed function over a predetermined interval.

  Here, the frequency distribution of the B image data is only a distribution close to the normal distribution (first group G1) when there is no regular reflection on the surface of the tongue at the time of photographing, but when there is a regular reflection, the first distribution is first. A distribution (second group G2) having a large frequency on the high pixel value side is added to the group G1. In addition, since the image data of B hardly changes due to the physical condition of the living body or individual differences as described above, the width of the first group G1 (the width from the minimum pixel value to the maximum pixel value of the first group G1) is This is narrower than the frequency distribution (normal distribution) of other R and G image data. As a result, the boundary between the first group G1 and the second group G2 (the portion where the frequency is minimized so that the frequency starts to increase from the decrease) is the value of the image data (pixel value Dp) and the image where the frequency is maximum. It clearly appears between the maximum value of data (pixel value Dm), and the first group G1 and the second group G2 can be easily identified. In order to detect the gloss level, it is desirable to detect the gloss level based on the second group G2 representing the gloss component, not the first group G1 not including the gloss component (regular reflection component).

  Therefore, the information extraction unit 14 sets a threshold value M larger than the pixel value Dp, and obtains the sum of the frequencies between the threshold value M and the pixel value Dm as the number of upper pixels, whereby the frequency of the second group G2 A value close to the sum of is obtained.

  Particularly, in the frequency distribution of the B image data, it is experimentally shown that the boundary between the first group G1 and the second group G2 appears within a range of 1.1 to 1.3 times the pixel value Dp. know. For this reason, in this embodiment, the information extraction unit 14 sets the threshold value M within a range of 1.1 to 1.3 times the pixel value Dp (1.2Dp = 84 in the example of FIG. 12). The sum of the frequencies between the threshold value M and the pixel value Dm is obtained as the upper pixel number.

  Cases with dry tongue (low gloss) are known to have a low number of top pixels, and cases with wet tongue (high gloss) have a high number of top pixels . Therefore, the information extraction unit 14 can detect and detect the glossiness from, for example, level 1 (high glossiness) to level 3 (low glossiness) based on the number of upper pixels.

(Extraction of cracks)
FIG. 13 shows a photographed image of a tongue having a crack on the surface. When photographing the tongue, the tongue is projected forward from the oral cavity. The imaging unit 4 captures the surface of the protruding tongue on the upper lip side. In general, since many cracks on the tongue surface occur near the center of the tongue, in this embodiment, the upper and lower and right and left central portions (regions including the center in the vertical and horizontal directions) of the photographed image are suitable for crack detection. Is set as a detection area.

  More specifically, the information extraction unit 14 detects the upper and lower ends and left and right ends of the tongue from the obtained tongue outline Q, and the vertical length (vertical dimension) H and the horizontal width (horizontal) of the tongue. Dimension) W is detected, and the center area of the tongue of length H / 4 and width W / 4 determined by the dimensional relationship shown in FIG. 13 is set as a crack detection area F.

  When there is a crack on the surface of the tongue, the base of the tongue appears more than when there is no crack, so the range of values that can be taken by the image data of the pixels constituting the base is expanded for both RGB. For this reason, when the frequency distribution of the image data of the captured image is created, the width of the frequency distribution is widened. In particular, since the background strongly expresses the color of blood, the width of the frequency distribution is remarkably widened for R and B contained in a large amount of blood color compared to the case where there is no crack. It is known that such a tendency appears regardless of the thickness of the moss on the tongue surface and the length of the crack.

Therefore, in the present embodiment, the information extraction unit 14 creates a frequency distribution of, for example, B image data from the photographed image of the tongue (particularly the image of the detection region F described above) and an index indicating the spread of the frequency distribution. As described above, the standard deviation σ indicating the variation in the image data is obtained by calculation. The standard deviation σ is a positive square root of the variance σ ^{2 represented} by the following equation when the value of the image data takes _N values x ₁ , x ₂ ,.

  FIG. 14 schematically shows the frequency distribution of the B image data created by the information extraction unit 14. The upper frequency distribution shows a case where there is no crack on the tongue surface, and the lower frequency distribution shows the tongue surface. Shows a case where there is a crack. Note that the horizontal axis of these frequency distributions indicates the B pixel value (image data), and the vertical axis indicates the frequency (number of pixels). The pixel value is a value from 0 to 255, and the larger the pixel value, the brighter the pixel value.

  When the standard deviation σ1 in the upper frequency distribution was obtained, σ1 = 13.18. On the other hand, when the standard deviation σ2 in the lower frequency distribution is obtained, σ2 = 26.78. From this, it can be seen that when there is a crack on the tongue surface, the standard deviation σ becomes larger than when there is no crack, and the width of the frequency distribution is widened. Incidentally, the average value m1 of the pixel values in the upper frequency distribution is m1 = 177.71, and the average value m2 of the pixel values in the lower frequency distribution is m2 = 112.75.

  It is known that the standard deviation of the frequency distribution is large in the case with many cracks, and the standard deviation of the frequency distribution is small in the case with few cracks. Therefore, the information extraction unit 14 can detect and detect cracks numerically, for example, from level 1 (many cracks) to level 3 (small cracks) based on the standard deviation of the frequency distribution. In the above example, the crack is detected using the frequency distribution of the B image data, but the crack can be detected using the frequency distribution of other R and G image data.

(Extraction of moss smoothness)
The smoothness of the moss varies with the amount of deposits on the teat. That is, when there is little deposit | attachment, since the moss is thin and a teat looks independent, it becomes a rough feeling. On the other hand, when there are many deposits, the moss is thick and the nipple covers the surface of the tongue. Therefore, the information extraction unit 14 can detect the smoothness of the moss based on the ratio of the number of pixels related to the nipple and the background.

  Specifically, the information extraction unit 14 determines a nipple pixel and a ground pixel by a color (for example, R) threshold in a region near the center of the tongue to which moss adheres (for example, the central region R2 in FIG. 6). It discriminate | determines and calculates the ratio of the number of pixels of a teat and a foundation | substrate. When the ratio of the groundwork is high, it can be determined that the moss is thin and rough, and when the ratio of the teat is high, it can be determined that the moss is thick and sticky. If the range of the ratio of the number of pixels is associated with three levels of moss smoothness, the degree of moss smoothness can be determined based on the ratio of the number of pixels actually obtained by calculation. .

(Extraction of tongue thickness)
FIG. 15 is a perspective view illustrating a schematic configuration of the light projecting unit 3. As shown in the figure, the light projecting unit 3 includes a point light source 31 and a columnar lens 32, and a drive circuit (not shown) that drives the point light source 31. The point light source 31 is configured by an LD (Laser Diode) or an LED (Light Emitting Diode). The columnar lens 32 is a horizontal columnar condensing lens that condenses light emitted from the point light source 31 only in one direction (for example, the vertical direction). As the light emitted from the point light source 31 is condensed in one direction by the columnar lens 32, linear light that is long in the horizontal direction is projected onto the surface of the tongue as shown in FIG. The light projecting units 3a and 3b have a point light source 31 and a columnar lens 32, except that the condensing direction of the columnar lens 32 is different (one condenses in the vertical direction and the other condenses in the horizontal direction). In common.

  FIG. 17 shows a captured image when the imaging unit 4 captures linear light (indicated by a thick solid line) projected on the tongue surface by the light projecting unit 3 when the tongue is thin and thick, and the tongue The cross-sectional shape is shown. As shown in the figure, when the tongue is thin and its surface is almost flat, the projected linear light is slightly curved upward at the tongue portion. On the other hand, when the tongue is thick and the surface thereof is convex, the projected linear light is greatly curved upward at the tongue portion. Therefore, the thickness of the tongue (whether the tongue is thick or thin) can be detected by detecting the curved shape of the projected linear light. Note that the method of detecting the shape of the subject by irradiating the subject with linear light is also called a light cutting method.

  When the organ to be imaged is a tongue, if a point light source that emits G or B light, which is a complementary color of the tongue color (R), is used as the point light source 31, the color of the projected light and the color of the tongue Therefore, it is easy to detect the curved shape of the projected light by photographing with the imaging unit 4. Even when the point light source 31 that emits R light is used, the curved shape of the projected light can be detected by photographing with the imaging unit 4 by adjusting the amount of emitted light.

  In addition, in this embodiment, although the light projection part 3 is comprised using the point light source 31 and the columnar lens 32, the structure of the light projection part 3 is not necessarily limited to this. For example, instead of the columnar lens 32, a slit having a long opening in the horizontal direction may be disposed, and linear light may be projected onto the tongue surface in the horizontal direction through the opening. Alternatively, a polygon mirror may be disposed in place of the columnar lens 32, and light may be projected in the horizontal direction by scanning the light from the light source at high speed in the horizontal direction with the polygon mirror.

  By the way, there are two types of tongue muscles, an external tongue muscle and an internal tongue muscle. The former controls a large movement of the whole tongue, and the latter controls a fine movement of the tongue. Movements affecting the surface irregularities of the tongue are controlled by the internal tongue muscle. The internal tongue muscle includes an upper longitudinal tongue muscle that winds the side of the tongue upward, a lateral lower muscle that stretches the tongue thinly, and a vertical tongue muscle that flattens the tongue. When photographing the tongue, it is instructed to spread the tongue sideways, but depending on the person, there may be unconscious power.

  FIG. 18 shows the relationship between the thickness of the tongue and the movement of the tongue muscle. When the tongue is spread laterally by the action of the vertical tongue muscle (normal photographing state), the surface of the tongue becomes flat when the tongue is thin, and becomes convex when the tongue is thick. When the side of the tongue is raised by the action of the upper longitudinal tongue muscle, the surface of the tongue is concave when the tongue is thin, and is gently convex when the tongue is thick. When the tongue is stretched thinly by the action of the lateral tongue muscle, the surface of the tongue becomes convex when the center is thick, and when the tongue is thick, the surface of the tongue is raised as a whole.

  When a linear light is projected in the horizontal direction by the light projecting unit 3 on the surface of such a tongue and the curved shape is photographed by the imaging unit 4, the information extracting unit 14 displays the curved shape from the curved shape. A data distribution as shown in FIG. 18 is acquired. This data distribution indicates the degree of unevenness on the tongue surface, and is a height distribution in which the height changes according to the unevenness on the tongue surface. Note that (a1) to (a3) indicate the horizontal height distribution when the tongue is thin, and (b1) to (b3) indicate the horizontal height distribution when the tongue is thick.

  In the present embodiment, the linear light projected from the light projecting unit 3 is projected so as to pass through the substantially vertical center of the tongue surface as shown in FIG. Therefore, the height distribution shown in FIG. 18 is obtained as a horizontal height distribution that passes through the substantially vertical center of the tongue surface. Here, the vertical direction on the surface of the tongue refers to the direction connecting the tip of the tongue (the tongue tip) and the root (the tongue base) (the same applies hereinafter).

  Next, a method for extracting and determining the tongue thickness from the above height distribution will be described. First, as illustrated in FIG. 19, the information extraction unit 14 sets a region A closer to the end than the center in the horizontal direction in the height distribution of the tongue surface. Here, the area A is an area determined by the dimensional relationship shown in FIG. 19 (with a width W / 4 at a distance W / 8 from the end of the tongue, where W is the width of the tongue determined from the contour line of the tongue. Area). In addition, the said edge part may be an edge part on either side with respect to the center part of a tongue. Further, the height distributions of (a1) to (a3) and (b1) to (b3) in FIG. 19 correspond to those shown in FIG.

  Next, the information extraction unit 14 approximates the shape of the region A in the height distribution with a polynomial, detects irregularities on the surface of the tongue (the curvature of the shape) based on the coefficients of the approximated polynomial, and the irregular shape To detect the thickness of the tongue. The order of the polynomial is not particularly limited, but a quadratic polynomial is used here as an example.

  In the thin tongues (a1) to (a3), the shape of the region A is horizontal or has changed from a horizontal shape to a concave shape. Therefore, the second-order coefficient of the approximate polynomial is 0 or a positive value. On the other hand, when the tongue is thick (b1) to (b3), the shape of the region A is a convex shape, and therefore the quadratic coefficient of the approximate polynomial is a negative value. Note that the thicker the tongue, the greater the negative value of the coefficient.

  In this way, by looking at the coefficients of the approximate polynomial, the degree of unevenness of the shape (height distribution) of the region A can be obtained to determine whether the tongue is thin or thick. For example, the information extraction unit 14 can determine that the tongue is thin when the second-order coefficient of the approximate polynomial is 0.003 or more, and can determine that the tongue is thin when the second-order coefficient is 0 or less. If the second-order coefficient is greater than 0 and less than 0.003, the tongue thickness can be determined to be medium.

  In the above, the thickness of the tongue is obtained by grasping the cross-sectional shape (curvature) along the horizontal direction of the tongue using the light projecting unit 3a that projects linear light in the left-right direction (horizontal direction, lateral direction). Although an example of detecting the height has been described, the cross-sectional shape of the tongue (based on the same principle as described above) by using the light projecting unit 3b that projects linear light in the vertical direction (vertical direction, vertical direction). In this case, since the shape and curvature of the cross section perpendicular to the horizontal direction can be grasped, it is possible to detect the thickness of the tongue based on this cross sectional shape.

<< Other setting methods for area A >>
FIG. 20 is an explanatory diagram schematically showing another setting method of the region A in the horizontal data distribution (height distribution). As shown in the figure, the region A (a part of the height distribution) in which the information extraction unit 14 approximates the shape of the height distribution may include the data distribution of the end portion of the tongue.

  In this case, when the circle is used as the approximation polynomial and the information extraction unit 14 approximates the shape of the height distribution (curvature) in the region A with a circle and detects the radius, it is found that the following relationship is obtained. It was. That is, when the radius of the approximate circle when the tongue is thin is R1 (mm) and the radius of the approximate circle when the tongue is thick is R2 (mm), the radius R1 is significantly smaller than the radius R2. It was. Therefore, it is easy to determine whether the tongue is thick or thin based on the radius of the circle approximating the region A, and the detection accuracy of the tongue thickness can be further improved.

<< Other examples of data distribution >>
FIG. 21 is a distribution of image data obtained when the surface of the tongue is imaged by the imaging unit 4 under illumination by the illuminating unit 2, and the captured image in the horizontal direction passing through the substantially vertical center of the tongue surface. The distribution of RGB image data is shown. However, the upper distribution is for the case where the tongue is thin, and the lower distribution is for the case where the tongue is thick. The solid line indicates the distribution of R image data, the alternate long and short dash line indicates the distribution of G image data, and the broken line indicates the distribution of B image data.

  When the tongue is thick, the tongue includes a portion that protrudes upward from its end to the center (see (b1) to (b3) of FIG. 18). Since such a convex portion on the tongue surface is brightly illuminated as it approaches the illumination unit 2, the value of the image data increases in the portion corresponding to the convex portion in the photographed image of the tongue. Conversely, when the tongue is thin, the surface of the tongue includes a portion that is substantially flat from the end to the center or concave downward (see (a1) to (a3) in FIG. 18). Since the flat part and recessed part of the tongue surface move away from the illumination part 2 compared with said convex part, even if illuminated, it is darker than a convex part. For this reason, in the photographed image of the tongue, the value of the image data decreases in the portion corresponding to the flat portion or the concave portion on the surface compared to the portion corresponding to the convex portion. This tendency is the same for all RGB image data. Note that the distribution shape of the image data shown in FIG. 21 corresponds to the shapes of (a2) and (b2) in FIG.

  Therefore, whether or not the tongue is thick or thin based on the unevenness of the distribution, even from the distribution of the image data of one of RGB colors (monochromatic distribution) in the photographed image of the tongue obtained under illumination of the illumination unit 2 Can be detected. In other words, it is possible to detect the tongue thickness with high accuracy (the tongue is detected by using the horizontal distribution of image data of one of RGB colors included in the photographed image of the tongue as a data distribution indicating the degree of unevenness of the tongue surface. Regardless of the outer shape). Also, with this method, the above-described light projecting unit 3 is not necessary, and thus it is possible to detect and diagnose the thickness of the tongue with a small and inexpensive configuration.

  As a result of various experiments, it was found that when the component ratio of R, that is, the distribution of R / (R + G + B) is used as the data distribution, a distribution that substantially matches the above-described height distribution can be obtained. Therefore, the tongue thickness can be accurately detected by using the data distribution indicating the R component ratio in the photographed image of the tongue obtained under illumination of the illumination unit 2 as the data distribution indicating the degree of unevenness of the tongue surface. I can say that. Note that the tongue thickness can be accurately determined in the same manner as described above using the distribution of data indicating the G component ratio (G / (R + G + B)) and the B component ratio (B / (R + G + B)) in the photographed image of the tongue. Can be detected.

<Other tongue thickness extraction methods>
FIG. 22 is an explanatory view schematically showing the relationship between the planar shape and the cross-sectional shape of the tongue. The surface area S (cm ² ) of the tongue in the standard state of the tongue muscle (the state where the tongue is not expanded or narrowed) is obtained by integrating the portion surrounded by the contour line Q (see FIG. 5) extracted from the photographed image of the tongue. Sought by.

The surface area S is approximated as follows using an index W _S (cm) corresponding to the lateral width of the tongue in the standard state of the tongue muscle and an index W _L (cm) corresponding to the vertical width of the tongue. be able to.
S ≒ W _S × W _L

Here, the aspect ratio of the tongue in the standard state of the tongue muscle is length / width≈1.0 to 1.3, but if length / width = 1.2,
W _L = 1.2W _S
Can be considered. Therefore, the surface area S is
_{S = W S × (1.2W S} )
= 1.2W _S ²
It becomes. From this formula:
W _S = √ (S / 1.2)
Is obtained.

On the other hand, the cross-sectional area of the tongue in the standard state of the tongue muscle is simplified by twice the area of the height distribution (area surrounded by the horizontal axis and the distribution curve) shown in FIG. Can be requested. That is, when the sectional area of the tongue is Sc (cm ² ),
Sc = 2 × ∫h × dw
It is represented by Here, h is the height from the horizontal axis of the height distribution curve, and dw is the minute width of the curve.

When the thickness of the tongue in the standard state of the tongue muscle is Hs (cm), the thickness Hs is
Hs = Sc / W _S
= Sc / (√ (S / 1.2))
It is represented by

  When the tongue muscle is moved, the length and width of the tongue change, but the surface area S and the cross-sectional area Sc hardly change. Therefore, by determining the surface area S and the cross-sectional area Sc of the tongue, the tongue thickness Hs in the standard state of the tongue muscle can be easily obtained from the above equation regardless of the state of the tongue muscle at the time of photographing. And it becomes possible to diagnose a health degree based on the calculated | required tongue thickness.

[3. Relationship between diagnostic items and imaging conditions)
Next, the relationship between each diagnostic item (diagnostic information) and the imaging conditions will be described.

(Relationship between dampness and tooth marks and imaging conditions)
The condition of the illumination angle by the illuminating unit 2 is important for detecting wetness and tooth marks on the tongue. As described above, the wetness level of the tongue is determined by measuring the reflection component of the illumination light on the tongue surface, detecting the glossiness of the tongue, and determining the level of the glossiness. When the humidity is high, the glossiness of the tongue surface increases and the reflection component of the illumination light increases. On the other hand, when the humidity is low, the glossiness of the tongue surface decreases and the reflection component of the illumination light decreases. By setting the illumination angle to about 30 °, it is possible to appropriately detect the reflected component of the illumination light from the curved upper region of the tongue, thereby appropriately detecting the degree of wetness of the tongue. .

  The degree of the tooth trace is determined by detecting the contour line of the tongue and the degree of unevenness. Since the face, lips, and tongue are the same living tissue, it is difficult to detect the contour line by the difference in color. Therefore, the contour line can be easily extracted by creating a shadow (shadow, dark part) around the tongue with the illumination light and detecting the shadow. Again, by setting the illumination angle to about 30 °, it is possible to form an appropriate shadow around the tongue, thereby properly detecting the contour line of the tongue.

(Relationship between tongue crevice and moss smoothness and shooting conditions)
The condition of imaging resolution is important for detection of tongue cracks and moss smoothness. Both the cleft of the tongue and the smoothness of the moss are determined by detecting the state of the papillary tissue present on the surface of the tongue. Tongue clefts are caused by partial loss of papillary tissue or growth failure. Moreover, when old tissues are stacked due to metabolic failure of the nipple, the granularity on the surface of the moss disappears and becomes smooth (poor state).

  The size of the nipple is about 0.5 mm in diameter. The average tongue size is about 50 mm both vertically and horizontally. In order to separate the nipple and appropriately detect tongue crests and moss smoothness, a resolution of at least 200 × 200 = 40000 pixels, preferably 400 × 400 = 16000 pixels is required.

(Relationship between tongue color and tongue thickness and shooting conditions)
To detect the color and thickness of the tongue, the illuminance condition when the tongue is illuminated by the illumination unit 2 is required. When the illuminance is low, the S / N ratio of the photographic signal (image data of the photographic image) decreases, and accurate detection and diagnosis become difficult. When illumination light in a shooting environment such as a fluorescent lamp is superimposed, it is necessary to consider the influence of its chromaticity, flicker, angle, and the like. Generally, the illuminance of the indoor environment is about 100 to 500 lx, and in order to eliminate this influence, the illuminance several times to several tens of times (for example, about 1000 to 10000 lx) is required.

[4. Example of shot image)
FIG. 23 and FIG. 24 show examples of images acquired by photographing by the imaging unit 4 and displayed on the display unit 5 before the diagnostic information is extracted by the information extracting unit 14. FIG. 23 shows an example in which the photographing state of the tongue is good. In any of the images in FIG. 23, the tongue has no force, the tongue is sufficiently protruded, and the tongue is extended substantially straight downward so that the surface of the tongue faces the imaging unit 4. Also, the positional relationship (distance, angle) between the tongue and the device is good. Furthermore, a shadow is formed at the lower part of the tongue, and the contour can be extracted appropriately. The resolution of the image is sufficient, and the nipple can be clearly identified. Diagnosis information can be extracted stably and accurately from such a captured image, and appropriate diagnosis can be performed based on the extracted information.

  On the other hand, FIG. 24 shows an example in which the photographing state of the tongue is bad. The tongue protrudes forward (the tongue angle is inappropriate), and the tongue does not protrude much (the tongue length is inadequate). (Appropriate), the tongue is too strong, the tongue is too far away from the device, the tongue is too close to the device, or the tongue is photographed from above. Information necessary for diagnosis cannot be stably and accurately extracted from such a photographed image, and the accuracy of diagnosis based on the extracted information may be reduced.

  Therefore, in the present embodiment, the determination unit 15 uses the same information as the diagnosis information described above as determination information for determining the quality of the imaging state of the tongue (how to put out the tongue and the positional relationship with the device). Based on this determination information, the quality of the shooting state is determined.

[5. (Shooting pass / fail judgment)
Hereinafter, a method for determining the shooting state using the determination information will be described. In addition, the method for extracting the determination information described below can employ the method described above, that is, the same method as the method for extracting diagnostic information and the contour line by the information extraction unit 14.

(About the angle of the tongue)
FIG. 25 shows a side view of the tongue. As shown in the figure, an angle θ in the XY plane with respect to a vertical axis Y passing through the intersection point O of the tangent T at the intersection point O between the photographing optical axis X and the tongue of the imaging unit 4 is defined as the tongue angle. . The quality of the tongue angle θ can be determined based on the glossiness of the tongue.

  FIG. 26 shows a region where the tongue surface can be glossed when the tongue is illuminated (hereinafter referred to as gloss region J). When the angle θ of the tongue is small, the light from the illumination unit 2 is reflected by the left and right regions of the upper part of the tongue (on the base of the tongue) and is incident on the imaging unit 4. Region J appears. On the other hand, when the angle θ of the tongue is increased, the reflection position of the illumination light when the light from the illumination unit 2 is reflected by the tongue surface and enters the imaging unit 4 approaches the lower part of the tongue (the tongue tip side). That is, the gloss region J due to the reflection of illumination light approaches the tongue tip side.

  The relationship between the angle of the tongue and the position of gloss is examined in the captured image as follows. FIG. 27 schematically shows the relationship between the tongue angle θ and the glossy region J ′ in the photographed image. Note that the gloss region J ′ in the captured image corresponds to the gloss region J of the tongue surface described above. When the angle θ of the tongue is 0 °, a glossy area J ′ appears in each of the left and right areas on the upper part of the tongue.

  As the angle θ of the tongue increases, the glossy region J approaches the tongue apex side as described above, and thus the glossy region J ′ approaches the tongue apex side also in the captured image. Accordingly, the determination unit 15 shifts the gloss detection areas R4 and R5 from the base of the tongue to the tip of the tongue as needed, detects the glossiness at the position of each detection area R4 and R5, and determines the glossiness area J ′ having the highest glossiness. By detecting the position, the angle θ of the tongue corresponding to the glossy region J ′ can be detected. Then, the determination unit 15 determines that the shooting state is good if the detected tongue angle θ is within a target range (for example, a range of 0 to 10 °), and determines that the shooting state is poor if it is outside the target range. .

  Also, the quality of the tongue angle θ can be determined based on the curved shape (longitudinal distance information, curvature) in a cross section perpendicular to the left-right direction of the tongue. FIG. 28 schematically shows a state in which linear light is projected onto the tongue surface in the vertical direction by the light projecting unit 3 (particularly, the light projecting unit 3b). FIG. 29 shows a curved shape in the cross section of the tongue surface obtained by the above light projection. From the curved shape of the tongue surface, the distance K of each position on the tongue surface in the direction (optical axis direction) along the photographing optical axis X from the reference position (for example, the position of the tongue tip in FIG. 29) can be known. As the tongue angle θ increases, the distance K in the optical axis direction of the tongue base relative to the tongue apex increases. Therefore, the angle θ of the tongue can be obtained based on the distance K, and the angle θ is within an appropriate range ( For example, it is possible to determine whether or not the shooting state is good.

(About tongue length)
FIG. 30 shows the relationship between the contour line Q of the tongue, the width W in the left-right direction, and the length H in the up-down direction. Whether or not the length of the tongue is appropriate can be determined based on the information on the contour line Q of the tongue. More specifically, the determination unit 15 determines the aspect ratio W / H of the tongue from the information on the contour line Q of the tongue, and determines whether the length of the tongue is appropriate based on the determined aspect ratio W / H. be able to. For example, if 0.85 ≦ W / H ≦ 1.15, it can be determined that the tongue length is good as a shooting state, and if W / H is out of the above range, the tongue sticking out is short. If it is too long or too long, it can be determined that the shooting state is poor.

(About the strength of force)
FIG. 31 shows a cross-sectional shape along the horizontal direction of the tongue. In the normal case where no force is applied to the tongue, the thickness of the tongue is as small as T1, and when the force is applied to the tongue, the muscle of the tongue expands, so that the thickness of the tongue becomes T2, which is larger than T1. Therefore, the determination unit 15 can determine how the tongue is applied based on the thickness of the tongue, and can determine the quality of the photographing state based on the determination result. For example, if the thickness T of the tongue is within a range of 10 to 20% of the width W of the tongue (see FIG. 30), it is possible to determine that there is no force on the tongue and the photographing state is good, and the above range is satisfied. When it comes off, the tongue is strong, and it can be determined that the shooting state is poor.

  Further, when the thickness of the tongue changes, the curved shape in the cross section along the left-right direction of the tongue also changes (see FIG. 18 and the like). Therefore, the determination unit 15 can also determine how the tongue is applied based on the curved shape of the tongue, and can determine the quality of the shooting state based on the determination result.

  Further, when force is applied to the tongue, as shown in FIG. 32, the tongue may be curved in a cross section perpendicular to the horizontal direction (left-right direction). Therefore, it is possible to determine the degree of force applied to the tongue based on the degree of curvature. For example, when the radius of curvature r when the curved shape of the tongue in the above-mentioned cross section is approximated by a circle is 100 mm or more, it can be determined that the tongue is hardly curved and the tongue has no force. It can be determined that the state is good. On the other hand, when the radius of curvature r is less than 100 mm, it can be determined that the tongue is greatly curved and the tongue has a force, so that it is determined that the photographing state is poor.

  Further, when a force is applied to the tongue and the tongue is bent as shown in FIG. 32, the position of the glossy region formed by the reflection of the illumination light on the tongue surface changes as in the case of the change in the angle θ of the tongue. Therefore, as in the determination of the tongue angle θ, the determination unit 15 determines the degree of tongue curvature (how the tongue is applied) by detecting the position with the highest glossiness in the captured image of the tongue. Thus, it can be determined whether or not the photographing state of the tongue is good.

(About distance between tongue and device)
FIG. 33 schematically shows the positional relationship between the tongue and the device (organ image capturing device 1). When the image is taken with the tongue approaching the apparatus, the tongue becomes larger or the tongue becomes brighter in the captured image. On the other hand, if the image is taken with the tongue away from the device, the tongue becomes smaller or the tongue becomes darker in the photographed image. Therefore, for example, if the correspondence relationship between the size of the tongue or the brightness of the tongue in the captured image and the distance between the tongue and the device is stored in the storage unit 16 as a table, the determination unit 15 refers to the table. The distance between the tongue and the device can be determined based on the size of the tongue or the brightness of the tongue in the photographed image, and based on the determination result, the quality of the shooting state (positional relationship between the tongue and the device) can be determined. Whether or not is appropriate). For example, if the distance L between the tongue and the device is within a range of ± 10% of the design value, it can be determined that the shooting state is good, and if it is outside the above range, the shooting state can be determined as poor.

  Here, the size of the tongue can be obtained from the contour line Q of the tongue. For example, by grasping the area of the portion surrounded by the contour line Q of the tongue shown in FIG. 30, the width W in the horizontal direction of the contour line Q, and the length H in the vertical direction of the contour line Q, the size of the tongue is obtained. (Large or small) can be judged. On the other hand, the brightness of the tongue can be determined based on the image data of the tongue color. For example, it can be determined that the larger the value of the image data, the brighter the tongue, and the smaller the value, the darker the tongue. Since the tongue is reddish, which is the color of blood, the brightness of the tongue can be accurately determined by using the R image data as the image data, but other B and G image data It is also possible to determine the brightness of the tongue using.

(About the angle of the device)
As shown in FIG. 33, the angle of the apparatus, that is, the angle of the imaging optical axis X of the imaging unit 4 with respect to the horizontal direction is γ (clockwise in FIG. 33 is the positive direction). When the device is tilted with respect to the tongue (when the angle γ is changed), the tongue is tilted relative to the device, so that the same phenomenon as when the tongue angle θ is changed occurs. That is, when the angle γ increases (increases to the positive side), the glossy region of the tongue surface approaches the lingual apex side and the angle γ decreases (increases to the negative side), as in the case where the angle θ increases. The gloss area on the tongue surface approaches the tongue base side.

  Therefore, as in the determination of the tongue angle θ, the determination unit 15 determines the position with the highest glossiness so that the angle γ of the device is in a favorable range (for example, a range of design value ± 10 °). It can be determined whether or not there is, and thereby it can be determined whether or not the photographing state of the tongue is good.

  Further, as shown in FIG. 29, from the curved shape of the tongue surface, the distance K of each position on the tongue surface in the direction along the photographing optical axis X from the reference position is known, and the larger the angle γ (the angle θ As the distance increases, the distance K in the optical axis direction of the tongue base with respect to the tongue tip increases. Therefore, the angle γ of the apparatus can be obtained based on the distance K, and it is determined whether or not the angle γ is within an appropriate range (for example, within the range of the design value ± 10 °). Can be determined.

  Note that the target range described above (for example, 0 to 10 ° for the tongue angle θ) is an example, and is not limited to the numerical values described above. The target range (determination threshold) may be obtained from an average value of a plurality of subjects or may be obtained from a photographed image of the person.

[6. Control flow)
FIG. 34 is a flowchart showing an example of the operation in the organ image capturing apparatus 1 of the present embodiment. Hereinafter, the operation of the organ image capturing apparatus 1 will be described.

  When receiving a photographing instruction by pressing the OK button 6a of the operation unit 6, the illumination control unit 9 turns on the illumination unit 2 (S1), and the display control unit 12 defines the photographing position of the tongue as shown in FIG. A frame line P is displayed on the display unit 5 (S2). Note that the frame line P is configured by a straight line (broken line) whose upper part extends in the left-right direction, imitating the ideal shape of a tongue, and the lower part of the straight line is a curved line (half of an ellipse). However, the present invention is not limited to this shape, and may be a quadrangular shape as shown in FIG.

  Subsequently, under the control of the imaging control unit 11, the imaging unit 4 starts preliminary imaging (S3), and the display unit 5 displays a captured image (preview image) of the user's tongue together with the frame P in a moving image. indicate. This prompts the user to confirm the composition, such as the position and angle of the tongue and how to put out the tongue. By adjusting the position of the tongue so that it falls within the frame line P, the user can approximate the photographing state of the tongue roughly well.

  When the composition is determined, the determination unit 15 extracts determination information for determining whether the photographing state of the tongue is good or bad from the photographed image obtained by the preliminary photographing (S4). The determination information is the same as at least one of the information (diagnosis information) of each diagnosis item extracted in S13 of the subsequent stage. That is, the determination information includes information such as glossiness of the tongue, curved shape of the tongue (curvature), tongue contour, tongue thickness, tongue color, and the like.

  Subsequently, the determination unit 15 determines the quality of the tongue photographing state based on the extracted determination information (S5). For example, the determination unit 15 determines the angle θ of the tongue based on the glossiness of the tongue by the above-described method, determines whether the calculated angle θ is within the target range, and determines whether the shooting state is good or bad. In S5, for example, when the angle θ is out of the target range and the shooting state is determined to be bad, a determination result that the shooting state is bad is output (displayed on the display unit 5 or voiced from the voice output unit 8). Output). At this time, a message prompting the user to adjust the photographing position of the tongue may be displayed on the display unit 5, or a sound to that effect may be output from the audio output unit 8.

  After S6, the process returns to the preliminary shooting of S3, and the operations after S3 are performed. That is, the imaging control unit 11 causes the imaging unit 4 to continuously perform preliminary shooting until the determination unit 15 determines that the shooting state of the tongue is good.

  In S5, for example, when it is determined that the angle θ of the tongue is within the target range and the shooting state of the tongue is good, the imaging control unit 11 instructs the main shooting from the user through the operation unit 6. Until the image is input, the imaging unit 4 is put on standby, and when an instruction for main shooting is input by the operation unit 6 (S7), the imaging unit 4 is caused to execute main shooting (S8). Since it is determined that the photographing state of the tongue is good, in this photographing, the tongue is photographed again in the good photographing state. At this time, in the actual photographing, the tongue may be photographed under a higher level photographing condition such as photographing the tongue at a higher resolution than in the preliminary photographing.

  If it is determined in S5 that the shooting state of the tongue is good, a message indicating that the shooting state is good may be displayed on the display unit 5 in order to prompt the user to input an instruction for main shooting. Then, it may be output from the audio output unit 8 by voice.

  Subsequently, the information extraction unit 14 extracts the contour line of the tongue from the photographed image obtained by the main photographing (S9), and based on the contour line, the diagnostic region (upper region) for extracting diagnostic information R1, central region R2, lower region R3, detection regions R4 and R5, etc.) are set (S10).

  Thereafter, the information extraction unit 14 supplies the diagnostic information (tongue color, tongue thickness, tongue wetness, tooth marks, fissures, moss smoothness, etc.) necessary for diagnosis from the image acquired in the actual photographing. (S11), and the extracted information is classified (numerized) into a plurality of levels (S12). For example, when information about the color of the tongue is extracted at S11, the information is classified into four levels (level 1 to level 4) according to the example of FIG. 5, and information about the thickness of the tongue is obtained. Is extracted, the information is classified into three levels (level 1 to level 3) according to the example of FIG.

  Next, the information extraction unit 14 determines the health level of the user based on the digitized information (S13). For example, if the color of the tongue is “level 3”, the degree of health is determined to be normal (light red, good blood flow, and healthy). For example, if the thickness of the tongue is “level 1”, it is determined that the health level is abnormal (the tongue is thin, blood flow is insufficient, or water is insufficient).

  The information digitized by the information extraction unit 14 and the determination result of the health level are displayed on the display unit 5 and stored in the storage unit 16 (S14). The display on the display unit 5 allows the user to grasp the digitized information or the health level for each diagnosis item. It should be noted that the processing in S13 is omitted (the health level is not judged by the device), and the information extracted by the information extraction unit 14 is transferred to the outside in S14, and the health level of the user is externally transmitted. And the result may be received by the apparatus and displayed on the display unit 5.

  As described above, the determination unit 15 determines the quality of the shooting state based on the determination information extracted from the image acquired by the imaging unit 3. Since the determination information includes the same information as diagnostic information necessary for organ diagnosis, the determination unit 15 can determine the quality of the imaging state using the information (diagnosis information). As a result, when the imaging state is determined to be good, diagnostic information is reliably acquired from an image obtained by imaging the tongue again in the imaging state determined to be good (an image obtained by actual imaging). can do. As a result, an appropriate diagnosis with high accuracy based on the diagnostic information can be performed by this apparatus or an external apparatus.

  In addition, since the imaging control unit 11 controls the imaging unit 4 according to the determination result in the determination unit 15, if the imaging state is determined to be poor, the imaging unit 4 is determined until the imaging state is determined to be good. It is possible to repeat (continue) the preliminary shooting in order to prevent the main shooting from being executed if the shooting state remains defective. Accordingly, the user can be prompted to adjust the photographing state of the tongue, and the photographing state can be improved. Then, it is possible to acquire an image in a good photographing state and acquire diagnostic information from the image in the same manner as described above.

  That is, the determination unit 15 determines the quality of the photographing state of the tongue based on the determination information (diagnosis information), and the imaging control unit 11 controls the imaging unit 4 according to the determination result, so that finally, It is possible to acquire an image in a good shooting state and reliably acquire diagnostic information from the image.

  In addition, since the determination information includes at least one of the glossiness of the tongue and the curved shape in a cross section perpendicular to the left-right direction of the tongue, the determination unit 15 determines the angle of the tongue, The angle of the apparatus (the angle of the photographic optical axis X of the imaging unit 4) can be obtained. Thereby, it is possible to determine the quality of the photographing state (the quality of the tongue or the angle of the device) based on the obtained angle.

  Further, since the determination information includes information on the outline of the tongue, the determination unit 15 obtains the aspect ratio of the tongue based on the information on the outline of the tongue, and the length of the tongue obtained from the obtained aspect ratio. The quality of the shooting state can be determined based on the above.

  Further, since the determination information includes at least one of the thickness of the tongue, the degree of curvature in the cross section along the left-right direction of the tongue, and the glossiness, the determination unit 15 selects any one of these pieces of information. Based on the above, it is possible to determine how the tongue is applied, and based on the determination result, it is possible to determine the quality of the photographing state.

  In addition, since the determination information includes at least one information of the outline of the tongue and the color of the tongue, the determination unit 15 obtains the size or brightness of the tongue based on any one of these pieces of information. The distance between the tongue and the device can be determined from the size or brightness of the tongue, and the quality of the photographing state can be determined based on the determination result.

  In addition, since the output unit (display unit 5, audio output unit) outputs the determination result of the determination unit 15, when the user recognizes that the shooting state is good based on the above output, When it is possible to input an instruction for main shooting and recognize that the shooting state is poor, the position of the tongue can be adjusted to start another shooting (preliminary shooting).

  The output unit outputs a message or a voice prompting the user to adjust the shooting position of the tongue when the determination unit 15 determines that the shooting state is poor. Alternatively, it is possible to grasp the necessity of adjusting the photographing position of the tongue by voice, and it is possible to correct the photographing position and start another photographing (preliminary photographing).

  In addition, the imaging control unit 11 causes the imaging unit 4 to perform preliminary photographing of the tongue until the determination unit 15 determines that the photographing state of the tongue is good, and the determination unit 15 determines that the photographing state is good. Thereafter, when an instruction for main shooting is input from the user through the operation unit 6, the imaging unit 4 is caused to perform main shooting. Then, the information extraction unit 14 extracts the diagnostic information again from the image acquired by the main imaging of the imaging unit 4. In the main shooting, the tongue is shot in the same shooting state as that determined to be good in the preliminary shooting. Therefore, by extracting diagnostic information again from the image acquired in the main shooting, the tongue or the external device can be extracted. Thus, the tongue can be properly diagnosed based on the extracted diagnostic information.

[7. About other controls)
FIG. 37 is a flowchart showing another example of the operation in the organ image photographing apparatus 1. In the same figure, the flow omits the step S7 of FIG. That is, the imaging control unit 11 causes the imaging unit 4 to perform preliminary shooting until the determination unit 15 determines that the shooting state of the tongue is good (S3 to S5), and the determination unit 15 determines that the shooting state is good. After the determination, it is possible to directly instruct the imaging unit 4 to start the main shooting without waiting for the shooting instruction from the user's operation unit 6 and to execute the main shooting in the shooting state determined to be good. Good (S8). And the information extraction part 14 may extract again the diagnostic information from the image acquired by the main imaging | photography of the imaging part 4 (S11).

  When the determination unit 15 determines that the shooting state is good, the imaging unit 4 automatically performs the main shooting without receiving an instruction input from the user. Can be done.

  FIG. 38 is a flowchart showing still another example of the operation in the organ image photographing apparatus 1. In FIG. 37, the actual photographing process in S8 of FIG. 37 is changed to an image selection process in S8 '. That is, the imaging control unit 11 causes the imaging unit 4 to perform preliminary shooting until the determination unit 15 determines that the shooting state of the tongue is good (S3 to S5). From the images acquired by the preliminary shooting, an image that is determined to be in a good shooting state by the determination unit 15 may be selected (S8 ′), and diagnostic information may be extracted again from the selected image (S11). ).

  Since the image selected in S8 ′ is an image that is determined to have a good shooting state, selecting this image and extracting the diagnostic information again by the information extraction unit 14 can also be based on the extracted information. The diagnosis can be appropriately performed by the device or an external device. In addition, since the main shooting is not required after the preliminary shooting, it is possible to quickly perform the processing after the quality determination of the shooting state.

[8. About the program)
The organ imaging apparatus 1 described above can be configured by, for example, a multifunctional portable terminal (computer) such as a smartphone in which a predetermined program (application software) is installed. By reading and executing the above program by a computer (for example, the overall control unit 20 as a CPU), the above-described processes in the organ imaging apparatus 1 can be realized. Such a program is acquired, for example, by downloading via a network, and is stored in the storage unit 16.

[9. Others]
The case where the living body is a human has been described above, but an animal other than a human may be used. For example, even for the tongue of an animal such as a pet or a domestic animal, the method of this embodiment is applied to determine the quality of the shooting state and extract diagnostic information from the image shot in a good shooting state. It is possible to diagnose the health level. In this case, the user can quickly and accurately determine the poor physical condition of the animal that cannot communicate the intention. Since the average tongue size changes for each animal to be photographed, a target range (determination threshold value) for determining the quality of the photographing state may be set for each animal.

  In the above, the case where the organ of the living body to be imaged is the tongue has been described, but the imaged object is not limited to the tongue. FIG. 39 shows a photographed image obtained by photographing the lower eyelid S located under the human eye (for example, the right eye). The quality of blood flow and water metabolism can be diagnosed from the color and shape of the lower eyelid S. As shown in the figure, a frame P that guides the eye and the area below it is displayed on the display unit 5, the position of the lower eyelid S is guided, the lower eyelid S is photographed at that position, You may make it perform quality determination. Even in this case, it is possible to extract diagnostic information from an image with a good shooting state and perform an appropriate diagnosis.

  As described above, according to the organ image capturing apparatus 1 of the present embodiment, a medical worker such as an inexperienced doctor, nurse, caregiver, or pharmacist, or a patient himself / herself becomes a user to capture an image of an organ. However, since an image in a good shooting state can always be acquired, an appropriate diagnosis can always be performed based on the image. In addition, the diagnosis based on the photographed image can be performed stably without the photographing date and the photographing condition being changed by the subject. Furthermore, by combining imaging by the patient and computer diagnosis, it can be used for daily physical condition management and early detection of diseases.

  INDUSTRIAL APPLICABILITY The present invention can be used for an apparatus that captures an organ of a living body and extracts information necessary for health diagnosis.

DESCRIPTION OF SYMBOLS 1 Organ imaging device 2 Illumination part 4 Imaging part 5 Display part (output part)
6 Operation part 8 Audio output part (output part)
DESCRIPTION OF SYMBOLS 11 Imaging control part 14 Information extraction part 15 Judgment part P Borderline

Claims (16)

An imaging unit that captures an image of a living organ and acquires an image;
An information extraction unit for extracting diagnostic information necessary for diagnosis of the organ from the image;
A determination unit for determining the quality of the imaging state of the organ;
An imaging control unit that controls the imaging unit according to a determination result in the determination unit;
The determination unit extracts determination information for determining the quality of the shooting state from the image acquired by the imaging unit, determines the quality of the shooting state based on the determination information,
The organ image capturing apparatus, wherein the determination information includes the same information as the diagnostic information.   The organ image capturing apparatus according to claim 1, wherein the organ is a tongue.   The organ according to claim 2, wherein the determination information includes at least one information of glossiness of the tongue and a curved shape in a cross section perpendicular to the left-right direction of the tongue as the same information as the diagnostic information. Image shooting device.   The determination unit obtains at least one of the angle of the tongue with respect to the vertical direction and the angle of the photographing optical axis of the imaging unit with respect to the horizontal direction based on the determination information, and determines whether the photographing state is good based on the obtained angle. The organ imaging apparatus according to claim 3, wherein the determination is performed.   The organ image capturing apparatus according to claim 2, wherein the determination information includes information on a contour line of a tongue as the same information as the diagnostic information.   The determination unit determines an aspect ratio of the tongue based on information on a contour line of the tongue, and determines whether the photographing state is good based on a length of the tongue obtained from the determined aspect ratio. Item 6. The organ image photographing apparatus according to Item 5.   The determination information includes, as the same information as the diagnostic information, information on at least one of the thickness of the tongue, the degree of curvature of the tongue along the left-right direction, and the glossiness of the tongue. The organ image photographing device according to any one of claims 2 to 6.   The organ image capturing apparatus according to claim 7, wherein the determination unit determines the degree of tongue force based on the determination information, and determines whether the imaging state is acceptable based on the determination result. .   9. The organ image photographing apparatus according to claim 2, wherein the determination information includes at least one of a tongue outline and a tongue color as the same information as the diagnostic information.   The determination unit determines the size or brightness of the tongue based on the determination information, determines the distance between the tongue and the device from the determined size or brightness of the tongue, and determines the distance based on the determination result. The organ image photographing apparatus according to claim 9, wherein the quality of the photographing state is determined.   The organ imaging apparatus according to claim 1, further comprising an output unit that outputs a determination result of the determination unit.   12. The output unit further outputs a message or a sound prompting a user to adjust the imaging position of the organ when the imaging unit determines that the imaging state is defective. An organ imaging apparatus according to claim 1. It further includes an operation unit that accepts an instruction for actual shooting from the user,
The imaging control unit causes the imaging unit to perform preliminary imaging until the determination unit determines that the imaging state of the organ is good, and acquires an image of the organ. After the image shooting state is determined to be good, when an instruction for main shooting is input from the user through the operation unit, the imaging unit is caused to execute main shooting in the shooting state determined to be good. To acquire an image of the organ,
The organ image capturing apparatus according to claim 1, wherein the information extraction unit extracts the diagnostic information from the image acquired in the main imaging. The imaging control unit causes the imaging unit to perform preliminary imaging until the determination unit determines that the imaging state of the organ is good, and acquires an image of the organ. After the imaging state of the image is determined to be good, the imaging unit is instructed to start main imaging, and the imaging unit is made to perform main imaging in the imaging state determined to be good, and the image of the organ And get
The organ image capturing apparatus according to claim 1, wherein the information extraction unit extracts the diagnostic information from the image acquired in the main imaging. The imaging control unit causes the imaging unit to perform preliminary imaging until the determination unit determines that the imaging state of the organ is good, and acquires an image of the organ,
The information extraction unit selects an image that is determined to be in a good shooting state by the determination unit from images acquired in the preliminary shooting, and extracts the diagnostic information from the selected image. The organ image capturing apparatus according to claim 1, wherein the organ image capturing apparatus is characterized.   The organ image according to claim 1, further comprising a display unit that displays an image acquired by the imaging unit together with a frame line for defining an imaging position of the organ. Shooting device.

Images