JP2008301877A - Apparatus and program for processing image - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To estimate the amount of movement of an imaging device in a lumen by a simple structure. <P>SOLUTION: An arithmetic section 13 acquires image data of an image captured by the imager 2 moving through the lumen of the body of a subject 1 from a portable recording medium 4 mounted on an external interface 10. A movement estimation section 13d estimates the amount of movement of the imaging device 2 in the lumen by processing the image of the interior of the lumen. A position estimation section 13e estimates the position of the imaging device 2 in the lumen at the time when the image of the interior of the lumen is captured on the basis of the amount of movement of the imaging device 2 estimated by the movement estimation section 13d. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and an image that can estimate the position of the image pickup device in the tube based on the time-series tube image taken by the image pickup device that moves inside the tube in the body. It relates to a processing program.

  In recent years, as represented by a capsule endoscope, a medical imaging machine that sequentially captures time-series images in a tube while moving in a tube such as a digestive tract has been developed. After being swallowed from the patient's mouth, the imager sequentially captures images while moving in the tube by a peristaltic motion and the like, transmits them to an external receiver, and is finally discharged out of the body. The doctor confirms the time-series in-pipe images received by the external receiving device on a diagnostic workstation, etc., and if he finds the affected area, insert the medical treatment tool into the patient's body again if necessary. Or incising the patient's body to perform medical procedures such as tissue sampling, hemostasis, and excision of the affected area.

  In order to perform such medical treatment efficiently, information on where the target affected part is in the tube is necessary. However, when the image pickup device picks up the affected part, the position of the image pickup unit at the time of imaging is the affected part. It corresponds to the vicinity of. Therefore, if the position of the imaging device at the time of imaging is grasped, the position of the affected part can be estimated.

  Information on the position of the imager at the time of imaging is taken at the position along the tube from the entrance and exit of the tube, or the start position or end position of a specific organ, rather than the spatial coordinates in the body. Information on whether or not it was done is useful. For example, in an organ whose shape changes in the body, such as the small intestine, even if the position of the imaging device at the time of imaging is grasped by coordinates, if the organ is deformed, the specified position matches the actual position of the affected part. Disappear. If the position of the imaging device at the time of imaging is grasped by the distance from the base point such as the entrance of the tube, the position of the affected part can be known even when the organ is deformed. Here, the amount of movement of the capsule endoscope from a predetermined position in the tube is calculated by using a plurality of magnetic fields generated by a single-core coil for generating a magnetic field in the imaging device arranged outside the body. What is calculated | required by detecting with a coil is known (refer patent document 1).

JP 2005-198789 A

  However, in Patent Document 1, in order to obtain the amount of movement of the capsule endoscope from a predetermined position, it is necessary to provide a single core coil in the capsule endoscope, which increases the size of the capsule endoscope. There is a problem. In addition, a coil for detecting the amount of movement must be disposed outside the patient's body, which complicates the system configuration and increases the cost.

  The present invention has been made in view of the above, and an object of the present invention is to provide an image processing apparatus and an image processing program capable of estimating the amount of movement of the image pickup device in the tube with a simple configuration. .

  In order to solve the above-described problems and achieve the object, an image processing apparatus according to the present invention processes a time-series tube image captured by an imager that moves inside a tube in the body, An image pickup device movement amount estimation means for estimating the movement amount of the image pickup device is provided.

  In addition, an image processing program according to the present invention is a computer that processes a time-series image in a tube taken by a camera that moves inside a tube in the body and estimates the amount of movement of the imager. By accumulating a plurality of movement amount values estimated by processing a plurality of the tube interior images in the movement amount estimation step and the imaging device movement amount estimation step, each of the tube interior images is captured. A position estimating step of estimating a position of the imaging device in the tube.

  In addition, an image processing program according to the present invention allows a computer that processes time-series tube images captured by an imaging device moving in a tube in the body to store the tube images in the tube images at different times. A predetermined part extracting step for extracting a predetermined part and a movement amount estimating step for estimating a movement amount of the imaging device based on the position of the predetermined part are executed.

  According to the image processing apparatus of the present invention, it is possible to estimate the amount of movement of the image pickup device in the tube by processing the images taken in time series by the image pickup device. The position can be estimated with a simple configuration.

  Hereinafter, an image processing apparatus which is the best mode for carrying out the present invention will be described with reference to the drawings.

(Embodiment)
FIG. 1 is a schematic diagram showing the overall configuration of an image processing acquisition system including an image processing apparatus according to an embodiment of the present invention. Note that in this embodiment, a capsule endoscope is used as an example of an imaging device that captures an image of the inside of the body (hereinafter referred to as an intraluminal image). This capsule endoscope has an imaging function, a wireless function, an illumination function for illuminating an imaging part, and is swallowed from the mouth of a subject such as a person or an animal for examination. It is introduced into the subject. And until it is spontaneously discharged, it sequentially captures and obtains intraluminal images inside the esophagus, stomach, small intestine, large intestine, etc., and wirelessly transmits it outside the body.

  As shown in FIG. 1, the image processing system receives an intravascular image data wirelessly transmitted from an imaging device (capsule endoscope) 2 that captures an intraluminal image inside the subject 1 and the imaging device 2. A receiving device 3 that includes the image processing device 5 that processes the intraluminal image captured by the imaging device 2 based on the intraluminal image received by the receiving device 3. For example, a portable recording medium (portable recording medium) 4 is used for the transfer of the intraluminal image data between the receiving device 3 and the image processing device 5.

  The receiving device 3 receives a wireless signal transmitted from the imaging device 2 via a plurality of receiving antenna groups A1 to An attached to the external surface of the subject 1, and the wireless unit 3a receives the wireless signal. And a receiving body unit 3b for processing a wireless signal. The wireless unit 3a and the receiving body unit 3b are detachably connected via a connector or the like. The receiving device 3 is configured so that the portable recording medium 4 can be freely attached and detached. The receiving device 3 receives the image data of the intraluminal image inside the subject 1 imaged by the imaging device 2 and performs portable recording in time series. Accumulate in medium 4. In the present embodiment, an image of the tube interior image captured on the portable recording medium 4 from the time t (0) at the entrance of the tube to the time t (T) at the exit of the tube. It is assumed that data is accumulated in chronological order. Here, the time t (0) at the entrance of the tube is equivalent to, for example, the time when the imaging device 2 is introduced into the subject, and the time t (T) at the exit of the tube is outside the body. It corresponds to the time when it was discharged.

  The image processing apparatus 5 is detachably mounted with a portable recording medium 4, and an external interface 10 that acquires image data of an intraluminal image stored in the portable recording medium 4 and the overall operation of the image processing apparatus 5. A control unit 11 that controls the image, a storage unit 12, a calculation unit 13 that performs various calculation processes for estimating the position of the image pickup device 2 at the time of imaging based on the image in the tube, and various instruction information Is input, and a display unit 15 that displays and outputs the calculation result of the calculation unit 13 and the like.

  The storage unit 12 is realized by various IC memories such as ROM and RAM such as flash memory that can be updated and stored, a hard disk connected by a built-in or data communication terminal, an information storage medium such as a CD-ROM, and a reading device thereof. A program relating to the operation of the image processing apparatus 5, a program for realizing various functions of the image processing apparatus 5, data relating to execution of these programs, and the like are stored. In addition, an image processing program 12a is stored for the calculation unit 13 to process the image in the tube and estimate the position of the imaging device 2 at the time of imaging.

  The calculation unit 13 includes a structure region extraction unit 13a, a corresponding region extraction unit 13b, a duct depth extraction unit 13c, a movement amount estimation unit 13d, and a position estimation unit 13e. The structure region extraction unit 13a extracts a structure region from one tube interior image captured by the image pickup device 2 moving in the tube. The corresponding region extraction unit 13b extracts the structure extracted from the one intraluminal image by the structural region extraction unit 13a from the other intraluminal image captured at a time different from that of the one intraductal image. A corresponding area corresponding to the area is extracted. The tubular deep part extracting unit 13c extracts a tubular part (hereinafter referred to as “pipe deep part”) on the far side in the traveling direction of the image pickup device 2 from one tubular image. Based on the positional relationship between the structural region in one tube image, the corresponding region in the other tube image, and the deep tube portion, the movement amount estimation unit 13d The amount of movement of the image pickup device 2 that has moved from the time of imaging to the time of imaging of another image in the tube is estimated. The position estimation unit 13e estimates the position of the image pickup device 2 in the tube air when each tube image is picked up based on the movement amount of the image pickup device 2 estimated by the movement amount estimation unit 13d. .

  FIG. 2 is an overall flowchart showing a calculation processing procedure performed by the calculation unit 13 of the image processing apparatus 5. Note that the processing described here is realized by the arithmetic unit 13 executing the image processing program 12a stored in the storage unit 12.

  As shown in FIG. 2, the calculation unit 13 first initializes a code i indicating the time-series order of the target intraluminal images to “0” (step S <b> 101). Then, the calculation unit 13 continues through the external interface 10 and the control unit 11 in time-series with the tube image at time t (i), which is the tube image to be processed, and the tube image. A tube interior image at time t (i + 1), which is a tube interior image, is acquired (step S102). In addition, although it was decided here to acquire the tube images that are continuous in time series, if the common part in the tube images is reflected in each tube image, it is not always continuous in time series. There is no need to acquire a tube image.

  FIG. 3 is a diagram illustrating an example of an intraluminal image captured by the imaging device 2. In the tube interior image, in addition to the mucosa (hereinafter referred to as “tubule mucosa”) 21 in the tube lumen, the contents 22 floating in the tube lumen, the bubbles 23 and the like, the back side in the traveling direction of the image pickup device 2 This shows the deep tube section 24. The intraluminal image captured by the imaging device 2 is a color image having pixel levels (pixel values) for the respective color components of R (red), G (green), and B (blue) at each pixel position.

  Then, as shown in FIG. 2, the structural region extraction unit 13a executes a structural region extraction process for extracting the structural region from the tube image at time t (i) (step S103), and the corresponding region extraction unit 13b. Executes a corresponding region extraction process for extracting a corresponding region from the image in the tube at time t (i + 1) (step S104), and the tube deep portion extraction unit 13c performs each of the times t (i) and t (i + 1). Tube depth extraction processing is performed to extract the tube depth from the tube inner image (step S105), and the amount of movement of the image pickup device 2 from the time t (i) to t (i + 1) is determined by the movement amount estimation unit 13d. The movement amount estimation process for estimating the value is executed (step S106).

  Then, after the movement amount estimation process in step S106, the calculation unit 13 increments the code indicating the time series order to i = i + 1 (step S107), and the tube at time t (i) to be processed next. The presence / absence of the inner image is determined by t (i) ≦ t (T). Here, t (T) is the final time relating to the intraluminal image to be processed. If t (i) ≦ t (T) (step S108: Yes), the processing from step S101 to step S107 is executed again. In addition, when processing tube-in-tube images continuous in time series, the tube-in-pipe portion extracted from the tube-in-tube image at time t (i + 1) by the tube-in-tube deep portion extraction process in step S105 is stored in the storage unit 12. By using this, the tube image at time t (i + 1) is used as the tube image at time t (i), and this is used when the processing from step S102 to step S107 is executed again. This can reduce the calculation load. On the other hand, when t (i)> t (T) (step S108: No), the process proceeds to step S109, and the position estimation unit 13e determines the position of the image pickup device 2 when each tube aerial image is captured. A position estimation process for estimation is executed (step S109). Then, based on the result of the position estimation process, the calculation unit 13 outputs position information of the image pickup device 2 in the tube when each tube image is captured (step S110), and the image processing device 5 finishes the calculation process of the calculation unit 13. For example, the calculation unit 13 causes the display unit 15 to display and output position information via the control unit 11.

  Next, processing executed by each unit of the calculation unit 13 will be described in detail. First, the structure region extraction process in step S103 of FIG. 2 will be described. In this structure area extraction process, the structure area extraction unit 13a extracts a plurality of structure areas from the in-tube image at time t (i). The structure region is a region between tube images taken at different times (in this process, the tube images at time (i) acquired in step S102 in FIG. 2 and the tube images and time series Between the images at the time (i + 1) that are continuous with each other) and can be associated with each other, for example, a structure that characterizes a local part on the tubular mucous membrane (hereinafter referred to as “characteristic structure”). ). Generally, there is a wrinkle in the tubular mucous membrane, and it is common that blood vessels are partially seen through the surface. These wrinkles of the luminal mucosa and blood vessels that can be seen through the surface are different in shape from each other and can be distinguished from others, and thus can be said to be characteristic structures. The structure region extraction unit 13a extracts a region where these feature structures are clearly reflected.

  FIG. 4 is a flowchart showing a detailed processing procedure of the structure area extraction processing. For example, the structural region extraction unit 13a sets a rectangular region of a predetermined size with the upper left position of the tube inner image at time t (i) as an initial position, and shifts the position of the rectangular region by a predetermined amount while moving the tube inner image. By scanning to the lower right position of the image, the structural region in the image in the tube at this time t (i) is searched.

  First, the structure area extraction unit 13a sets a rectangular area of a predetermined size at an initial position on the image in the tube at time t (i) (step S301). Then, the structure area extraction unit 13a obtains a feature amount related to the color in the set rectangular area (step S302). Here, the color-related feature amount is, for example, an RGB average value or RGB histogram in a set rectangular area, or a color ratio, color difference, hue, or the like that is secondarily calculated from the RGB values of each pixel in the rectangular area. An average value such as saturation and a histogram.

  Next, the structural region extraction unit 13a calculates the similarity between the feature amount relating to the color in the rectangular region obtained in step S302 and the preset feature amount relating to the color of the tubular mucosa (step S303). For example, the structural region extraction unit 13a calculates, as the similarity, a distance between vector tips, a cosine of an angle formed by the vectors, and the like when each feature amount is considered as one feature vector.

  Next, the structure area extraction unit 13a compares the similarity calculated in step S303 with a reference similarity set in advance as a threshold, and determines whether the set rectangular area is an area on the tubular mucous membrane. For example, if the calculated similarity is greater than or equal to the reference similarity, the structural region extraction unit 13a determines that the set rectangular region is a region that reflects the tubular mucosa (step S304: Yes), and proceeds to step S305. On the other hand, if the calculated similarity is less than the reference similarity, it is determined that the set rectangular area is, for example, an area showing contents or bubbles, and is not an area showing the tubular mucous membrane (step S304). : No), the process proceeds to step S306.

  In step S305, the structure area extraction unit 13a calculates a distribution range of pixel values in the rectangular area. The calculated distribution range is stored in the storage unit 12 in association with information on the position of the rectangular area at the current time. As the distribution range of pixel values, the distribution range of pixel values for each RGB color component in the rectangular area may be calculated, and the widest distribution range may be selected from these, or the distribution of specific color components A range may be calculated. When the obtained distribution range is narrow, the pixel values in the rectangular area are in a uniform state, and it is unlikely that the feature structure is reflected. On the other hand, when the obtained distribution range is wide, there is a high possibility that the feature structure is reflected in the rectangular area. The pixel value distribution range may not be calculated using all the pixels in the rectangular area. For example, the distribution range may be calculated using pixel values of pixel columns in a predetermined direction within the rectangular area, or the distribution range may be calculated using pixel values of two orthogonal pixel columns. As compared with the case where the calculation is performed using all the pixels in the rectangular area, the calculation load can be reduced.

  Then, the structural region extraction unit 13a changes the position of the rectangular region (step S307) until the scanning of the setting region on the tube inner image at time t (i) ends (step S306: No). The processes from S302 to S306 are repeatedly executed. Then, when the scanning of the set area on the image in the tube at time t (i) is completed (step S306: Yes), the process proceeds to step S308. In step S308, the structural region extraction unit 13a selects a predetermined number from the rectangular region whose distribution range has been calculated in step S305 in order from the wide distribution range, and extracts it as a structural region in which the feature structure is reflected. At this time, a rectangular area is selected so that the structure areas do not overlap. Data relating to the selected structure area is held in the storage unit 12. Then, the structure area extraction unit 13a returns to step S103 in FIG.

  FIG. 5 is a schematic diagram showing the result of the structure area extraction process. The structure extracted as a result of the structure area extraction process using the tube interior image illustrated in FIG. 3 as the tube interior image at time t (i). An example of the area is shown. In the example of FIG. 5, four structural regions T1, T4, T3, and T4 are extracted as the structural regions, and by this structural region extraction processing, the contents 22 and bubbles 23 that float in the tube are reflected. , And a structural region in which the characteristic structure of the region 21 of the hollow mucosa including wrinkles and blood vessels is reflected is extracted.

Next, the corresponding area extraction process in step S104 of FIG. 2 will be described. In the corresponding region extraction process, the corresponding region extraction unit 13b is extracted from the tube image at time t (i) as a result of the structure region extraction process from the tube image at time t (i + 1). An area determined to be the same as each structural area is extracted as a corresponding area. Specifically, the corresponding region extraction unit 13b sets each structural region extracted by the structural region extraction process as a template, performs a known template matching process, and from the tube inner image at time t (i + 1), A region similar to each template is detected. As a template matching method, for example, the method disclosed in “CG-ARTS Association, Digital Image Processing, 202p, Template Matching” can be used. The matching search range is centered on the center coordinates (x _j , y _j ) (j = 1, 2,..., N (N: number of templates)) of each template in time series of the tube interior image. It may be set in consideration of the amount of change. Further, in order to increase the speed, a sparse / dense search method, a residual successive examination method, or the like may be used. For example, the technique disclosed in “CG-ARTS Association, Digital Image Processing, 206p, Fast Search Method” can be used. As a result, for each template corresponding to each structural region extracted from the tube interior image at time t (i), the coordinates of the most similar region from the tube interior image at time t (i + 1) ( x _j ', y _j ') and their similarity. A template having a low similarity at the time of matching is not determined as the same portion and is not extracted as a corresponding region. In addition, the structure area in which the corresponding area is not extracted as a result of matching is not used in the subsequent processing.

  FIG. 6 is a schematic diagram showing the result of the corresponding region extraction process. Each structural region is based on the tube interior image of t (i + 1) time-sequentially continuous with the tube interior image illustrated in FIG. An example of corresponding areas extracted as a result of matching using T1 to T4 as templates is shown. In the example of FIG. 6, the corresponding region T1 ′ corresponding to the structural region T1 in FIG. 5, the corresponding region T2 ′ corresponding to the structural region T2, the corresponding region T3 ′ corresponding to the structural region T3, and the corresponding region corresponding to the structural region T4. T4 ′ is extracted. In FIG. 6, the search ranges ST1 'to ST4' for each template are indicated by broken lines. For example, if attention is paid to the structure region T1 in the tube interior image at t (i) shown in FIG. 5, matching using the structure region T1 as a template is performed in the tube interior image at t (i + 1) shown in FIG. A region having a high degree of similarity is extracted as the corresponding region T1 ′.

  Next, the tube-pipe deep part extraction process by step S105 of FIG. 2 is demonstrated. In the tube deep portion extraction process, the tube deep portion extractor 13c extracts the tube deep portion from the tube interior images at times t (i) and t (i + 1), and calculates the position of the center of gravity. Since the distance from the image pickup device 2 is far away from the image pickup device 2, it is difficult for the illumination from the image pickup device 2 to reach the tube deep portion and is obtained as a dark region. The area where the dark pixels are gathered is extracted as the deep tube portion, and the position of the center of gravity is obtained. In the tube interior images shown in FIGS. 3, 5, and 6, the center of the deep tube portion is schematically shown bright so that the tube space can be easily understood, but the center portion is actually dark. The deep tube portion extraction unit 13c extracts a region (dark portion) where dark pixels gather from the tube internal images at times t (i) and t (i + 1) as the deep tube portion, and obtains the position of the center of gravity.

  FIG. 7 is a flowchart showing a detailed processing procedure of the deep tube extraction processing. The deep tube portion extraction unit 13c executes the processing of loop A with the tube internal image at time t (i) and the tube internal image at t (i + 1) as processing targets (steps S501 to S506). Here, in the description of the processing in the loop A, the tube interior image to be processed is referred to as “target tube interior image”.

  In the loop A, first, the G value of each pixel constituting the target tube aerial image is compared with a predetermined threshold value, and the G value is determined from among the pixels of the target tube aerial image. Pixels below the threshold are extracted as dark pixels (step S502). The G value is used here because it is close to the wavelength of the absorption band of hemoglobin in the blood and has high sensitivity and resolution, and therefore expresses the light / dark information of the target tube aerial image well. Note that the dark pixel may be extracted using the value of the color component other than the G value. Or it is good also as extracting a dark part pixel using the value which shows the brightness information calculated using the well-known conversion technique. For example, the brightness calculated by YCbCr conversion and the brightness calculated by HSI conversion can be used. Also, a predetermined threshold value set in advance for the G value is set. However, the G value distribution of each pixel constituting the target tube aerial image is calculated and set using a known discriminant analysis method or the like. It is good to do. As the method of discriminant analysis, for example, the method disclosed in “CG-ARTS Association, Digital Image Processing, 175p, Discriminant Analysis” can be used.

  Subsequently, the deep tube extraction unit 13c performs a known labeling process on the dark pixel extracted in step S502, and attaches a unique value (label) to the adjacent dark pixel group (step S503). Thereby, the dark part area | region in a target tube aerial image can be recognized. As a labeling processing method, for example, the method disclosed in “CG-ARTS Association, Digital Image Processing, 181p, Labeling” can be used.

  Subsequently, the deep tube portion extraction unit 13c calculates the area of each dark region in the target tube image recognized in step S503, and extracts the dark region having the largest area as the deep tube portion (step S504). . In the intraluminal image, there are dark areas other than the deep part of the tube, such as wrinkled shadows of the tubular mucous membrane, but these areas are usually smaller in area than the deep part of the tube. Identification is possible. Then, the deep tube portion extraction unit 13c calculates the position of the center of gravity of the dark region extracted as the deep tube portion (step S505). The extracted region of the deep tube portion and the data relating to the gravity center position of the deep tube portion are stored in the storage unit 12 in association with the identification information of the target tube image. Then, when the process of loop A is executed for each of the tube image at time t (i) and the tube image at time t (i + 1), the tube depth extraction unit 13c returns to step S105 in FIG. .

  FIG. 8 is a schematic diagram illustrating a result of the tube hollow depth extraction process for the tube inner image of t (i + 1) illustrated in FIG. In the example of FIG. 8, the dark space region indicated by the hatched region 31 is extracted as the deep tube portion as a result of the deep tube portion extraction process, and the barycentric position 33 is calculated.

  Next, the movement amount estimation processing in step S106 in FIG. 2 will be described. In this movement amount estimation process, the movement amount estimation unit 13d uses the structure region extraction unit 13a to extract the position of the structure region extracted from the tube image at time t (i), and the corresponding region extraction unit 13b uses the time t (i). The position of the corresponding region extracted from the tube interior image of i + 1) and the tube extracted from the tube interior image at time t (i) and the tube interior image of t (i + 1) by the tube space depth extraction unit 13c. The amount of movement of the imaging device 2 from time t (i) to t (i + 1) is estimated based on the sky depth and the center of gravity position. The estimated movement amount is stored / accumulated in the storage unit 12.

  FIG. 9 is a model diagram of the inside of the tube and the image pickup device 2 for explaining the movement amount estimation processing, and an image pickup device that picks up an image inside the tube including the feature structure 51 inside the tube at time t (i). 2 shows the imaging situation model of the imaging device 2 that has captured the image in the tube including the feature structure 51 at time t (i + 1). In the imaging situation model shown in the lower part of FIG. 9, changes in the imaging position (position of the imaging device 2) and changes in the imaging direction are seen with respect to the imaging situation model shown in the upper part. Here, D represents the feature structure distance obtained by projecting the distance from the imaging device 2 to the feature structure 51 of the tubular mucous membrane at the time t (i) onto the inner wall surface of the tube, and D ′ represents the time t (i + 1). Represents a feature structure distance obtained by projecting the distance from the image pickup device 2 to the feature structure 51 of the tubular mucous membrane on the inner wall surface of the tube. O is an optical center corresponding to a principal point of an optical system such as a lens included in the image pickup device 2. R is the hollow radius. For example, an average tube radius is used as the tube radius R.

  Further, in the upper model diagram of FIG. 9, image coordinates 53 a of an intraluminal image obtained by being projected onto the image pickup device of the image pickup device 2 by this image pickup state model are shown. The image coordinate 53a is a coordinate system with the position intersecting with the optical axis 52 of the image pickup device 2 as the origin, and f is the distance from the optical center O of the image pickup device 2 to the image pickup device. Here, the coordinates of the center of the structure region in which the feature structure 51 in the tube interior image obtained by this imaging situation model is shown are the structure region center coordinates T (xT, yT), and the tube space depth portion in this tube interior image The center-of-gravity position coordinates of the tube are defined as the deep-pipe portion center-of-gravity coordinates C (xC, yC). In addition, θ is an angle formed by a vector OC from the optical center O toward the deep tube portion and a vector OT from the optical center O to the feature structure 51 at time t (i).

  Similarly, in the model diagram in the lower part of FIG. 9, the image coordinates 53b of the intraluminal image obtained by this imaging state model are shown. The image coordinate 53b is a coordinate system with the position intersecting with the optical axis 52 of the image pickup device 2 as the origin, and f is the distance from the optical center O of the image pickup device 2 to the image pickup device. Here, the coordinates of the center of the corresponding region in which the feature structure 51 in the image inside the tube obtained by this imaging situation model is shown is the corresponding region center coordinate T ′ (xT ′, yT ′), and The coordinate of the gravity center position of the deep tube portion is defined as the deep tube portion gravity center coordinate C ′ (xC ′, yC ′). Further, θ ′ is an angle formed by a vector OC ′ from the optical center O to the center of gravity 54 of the tube air depth portion and a vector OT ′ from the optical center O to the feature structure 51 at time t (i + 1).

Here, the following equation (1) is obtained from the characteristic structure distance D, the structure region center coordinate T, the tube depth deep barycenter coordinate C, the distance f, and the tube radius radii R of the imaging state model in the upper part of FIG. Note that δ represents the pitch of the image pickup device of the image pickup device 2. The values of the camera parameters such as the distance f and the imaging element pitch δ are acquired in advance.

Similarly, the following equation (2) is obtained from the characteristic structure distance D ′, the structure region center coordinate T ′, the tube depth deep center coordinate C ′, the distance f, and the tube radius R of the imaging state model in the lower part of the figure. It is done.

Then, the following equation (3) is obtained from the equations (1) and (2).

When Expression (3) is transformed, the following Expression (4) is obtained.

  DD ′ represented by the equation (4) is a projection of the distance from the imaging device 2 to the characteristic structure 51 of the tubular mucosa at each time of time t (i) and t (i + 1) on the inner wall surface of the tube. This is the difference between the feature structure distances and corresponds to the movement amount d of the image pickup device 2 from the time t (i) to t (i + 1) shown in the lower part of FIG. By obtaining D-D ′ in this way, it is possible to estimate the amount of movement of the imaging device 2 from time t (i) to t (i + 1). Specifically, the movement amount estimation unit 13d obtains D-D ′ for each feature structure corresponding to each structure region extracted by the structure region extraction process. Then, the movement amount estimation unit 13d calculates an average value of the obtained plurality of movement amount values, and estimates the movement amount from time t (i) to t (i + 1) of the imaging device 2.

  Next, the position estimation process in step S109 in FIG. 2 will be described. In this position estimation process, the position estimation unit 13e calculates the accumulated value of the movement amount of the imaging device 2 from time t (i) to t (i + 1) estimated as a result of the movement amount estimation process and accumulated in the storage unit 12. Then, the position of the image pickup device 2 at the time of capturing each tube image is estimated. This accumulated value corresponds to the moving distance of the image pickup device 2 from the time t (0) at the entrance of the tube to the time t (T) at the exit of the tube. The obtained cumulative value is divided by the total amount of movement of the image pickup device 2 from the tube inlet to the outlet, that is, the total length of the tube empty, so that the relative value of the image pickup device 2 in the tube air at each time is calculated. The position may be determined.

  FIG. 10 is a schematic diagram showing the result of the position estimation process, where the horizontal axis indicates time t and the vertical axis indicates the relative position of the image sensor 2 from the tube empty inlet to the tube empty outlet. The time-series change of the relative position of is shown. According to this, based on the imaging time of the tube interior image showing the affected part, relative to the tube entrance or tube exit of the image pickup device 2 in the tube atmosphere when the tube interior image is captured. It is possible to know the actual position and to estimate the position of the affected part.

  As described above, in the present embodiment, by processing the intraluminal images captured in time series by the image pickup device 2, the image pickup device 2 can be configured with a simple configuration without providing a separate device inside the image pickup device or outside the patient's body. The amount of movement can be estimated, and the position of the image pickup device 2 in the tube space when each tube image is captured can be estimated. As a result, there is no need to increase the size of the image pickup device or complicate the system configuration as if the image pickup device had a separate device such as a single-core coil for measuring the amount of movement. Can be produced in a compact and low-cost manner.

  Note that the position of the image pickup device 2 when the image pickup device 2 images the affected area is in the vicinity of the affected area and is not completely the same as the position of the affected area. However, using the intraluminal image of the affected area, it is a case where it is determined whether the insertion route when inserting the medical treatment tool into the patient's intraluminal should be taken from the oral cavity or the transcavity, When it is determined whether the position where the patient's body should be incised should be closer to the upper part of the specific organ or closer to the lower part, the position of the affected part may not be strictly grasped. In this case, if information on the approximate position of the affected area can be obtained by this embodiment, the information on this position can be used in subsequent medical procedures such as tissue collection, hemostasis, and excision of the affected area.

  In the present embodiment, the description has been made from the tube entrance to the exit, but the position of the imaging device 2 is estimated based on the start position or end position of a specific organ such as the small intestine or large intestine. May be. According to this, it is possible to obtain information on how much the affected part is located along the lumen from the start position or end position of a specific organ.

  Further, in the present embodiment, the structural region is extracted before the deep tube portion, but the deep tube portion may be extracted first, and the structural region may be extracted based on this position. You may extract a structure area | region so that it may surround.

  In the present embodiment, a method of extracting a plurality of appropriate regions as a structure region while scanning a rectangular region of a predetermined size has been described. However, the shape of the structure region does not have to be a rectangle and may be an arbitrary shape. . For example, it is possible to determine whether each pixel in the rectangular area is a pixel of the tubular mucous membrane, and to extract a region having an arbitrary shape excluding pixels determined not to be the tubular mucous membrane as a structural region.

  In addition, the image in the tube is divided in advance into a grid of a predetermined size, and steps S302 to S305 shown in FIG. 4 are performed in each divided region. Finally, a predetermined number of divided regions are sequentially arranged from the divided region having the largest distribution range. The structure area may be extracted, and as a result, the calculation load for extracting the structure area can be reduced. Here, the calculation load can be further reduced by extracting only one structural region without extracting a plurality of structural regions.

1 is a schematic diagram illustrating an overall configuration of an image processing acquisition system including an image processing apparatus. It is a whole flowchart which shows the arithmetic processing procedure which the calculating part of an image processing apparatus performs. It is a figure which shows an example of the tube interior image imaged with the imaging device. It is a flowchart which shows the detailed process sequence of a structure area | region extraction process. It is a schematic diagram which shows the result of a structure area | region extraction process. It is a schematic diagram which shows the result of a corresponding area extraction process. It is a flowchart which shows the detailed process sequence of a deep-pipe part extraction process. It is a schematic diagram which shows the result of a tube air deep part extraction process. It is a model figure of the inside of a tube and an image pick-up machine for explaining movement amount presumption processing. It is a schematic diagram which shows the result of a position estimation process.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Subject 2 Imaging device 3 Receiving device A1-An Receiving antenna group 3a Wireless unit 3b Reception main body unit 4 Portable recording medium 5 Image processing device 10 External interface 11 Control unit 12 Storage unit 12a Image processing program 13 Calculation unit 13a Structure Region extraction unit 13b Corresponding region extraction unit 13c Tube air depth extraction unit 13d Movement amount estimation unit 13e Position estimation unit 14 Input unit 15 Display unit

Claims (12)

  An image processing apparatus comprising: an imager movement amount estimation means for processing a time-series tube image captured by an imager moving within a body tube and estimating a movement amount of the imager . The imaging device movement amount estimation means includes
Predetermined part extracting means for extracting a predetermined part in the tube in the tube image at different times;
A movement amount estimating means for estimating a movement amount of the imaging device based on the position of the predetermined part;
The image processing apparatus according to claim 1, further comprising: The predetermined part extracting means includes
A structure region extracting means for extracting a structure region from the tube interior image;
Corresponding region extracting means for extracting a corresponding region corresponding to the structural region from the tubular image at a different time from the tubular image from which the structural region has been extracted;
A tube deep portion extracting means for extracting a tube deep portion that is a portion in which the depth in the tube is reflected from the tube hollow image;
And extracting the structural region, the corresponding region and the tube air depth as the predetermined portion,
The image processing apparatus according to claim 2, wherein the movement amount estimation unit estimates a movement amount of the image pickup device based on the positions of the structure area, the corresponding area, and the deep tube portion.   The image processing apparatus according to claim 3, wherein the structural region extracting unit extracts a region in which the mucous membrane in the duct is reflected as the structural region.   5. The structure area extracting unit extracts an area in which at least one of a wrinkle of a mucous membrane in the duct and a blood vessel on a mucosal surface in the duct is reflected as the structure area. The image processing apparatus described.   The image processing apparatus according to claim 3, wherein the structural region extracting unit extracts a region having a pixel value distribution range larger than a predetermined threshold as the structural region.   The image processing apparatus according to claim 3, wherein the structural region extraction unit extracts a plurality of regions in the intraluminal image as the structural region.   The corresponding region extraction means performs matching processing using the structural region as a template between in-vivo images in the body at different times, and extracts a region whose similarity at the time of matching is higher than a predetermined threshold as the corresponding region The image processing apparatus according to claim 3, wherein the image processing apparatus is an image processing apparatus.   The image processing apparatus according to any one of claims 3 to 8, wherein the tube-pipe deep portion extraction unit extracts a region where dark pixels gather in the tube-pipe image as the tube-pipe deep portion.   By accumulating a plurality of movement amount values estimated by processing the plurality of tube images in the image pickup device movement amount estimation means, the tube images at the time of capturing each tube image are captured. The image processing apparatus according to claim 1, further comprising a position estimation unit that estimates a position of the imaging device. On the computer,
An imager movement amount estimation step for processing a time-series tube aerial image captured by an imager moving within a body tube and estimating an amount of movement of the imager;
By accumulating a plurality of movement amount values estimated by processing a plurality of the images in the tube in the image pickup device movement amount estimation step, in the tube atmosphere when each tube image is captured. A position estimating step for estimating the position of the imager;
An image processing program for executing A computer that processes time-series tube images taken by an imager that moves inside the tube.
A predetermined part extraction step for extracting a predetermined part in the tube in the tube image at different times;
A movement amount estimation step for estimating a movement amount of the imaging device based on the position of the predetermined part;
An image processing program for executing

Images