JP2012228346A - Image display device - Google Patents

Abstract

An object of the present invention is to easily grasp an imaging location in a tubular organ and prevent oversight of a non-imaging location that has not been imaged.
An image display device (4) includes a plurality of images captured by a capsule endoscope (2) which is an example of an intra-organ imaging device that moves in a tubular organ and captures images in time series, and an intra-organ organ for each image. The acquisition unit 4c that acquires the imaging position and the imaging posture of the imaging device 2, and the degree of abnormality that represents the possibility of a medical abnormality for each image are obtained, and a plurality of imaging positions and orientations for each image are used. A processing unit 4d that classifies the image into a plurality of segmented regions corresponding to the extending direction and the circumferential direction of the organ, and generates a developed view in which the abnormal degree of the image is assigned to each divided region, and a display unit 4e that displays the developed view With.
[Selection] Figure 1

Description

  Embodiments described herein relate generally to an image display apparatus.

  In recent years, capsule endoscopes targeting the digestive tract, which is one of the tubular organs, have been put into practical use, and about 55,000 images are captured in an examination using the capsule endoscope. For this reason, it is practically impossible to check these images one by one, and it is necessary to devise an image display. The dedicated workstation provides an efficient image display using an image analysis technique. For example, in the time bar display, the imaging time is plotted on the horizontal axis and the degree of image abnormality is indicated by color.

JP 2004-337596 A JP 2010-240000 A

  However, since the capsule endoscope moves by the peristaltic movement of the digestive tract, the moving speed is not constant, and the position of the capsule endoscope on the digestive tract cannot be accurately determined from the imaging time. In addition, since the direction of the capsule endoscope is constantly changing, different portions may be photographed or may not be photographed at all even in images captured at the same position. For these reasons, in the above-described time bar display, it is difficult for an operator such as a doctor to grasp the imaging location in the digestive tract, and the non-imaging location that is not imaged may be overlooked.

  The problem to be solved by the present invention is to provide an image display device that can easily grasp an imaging location in a tubular organ and can prevent oversight of a non-imaging location that is not imaged. It is.

  An image display device according to an embodiment includes a plurality of images captured by an intra-organ imaging device that moves in a tubular organ and images in time series, and an imaging position and an imaging posture of the intra-organ imaging device for each image. Using an acquisition unit to acquire, an abnormality detection means for obtaining an abnormality degree representing the possibility of a medical abnormality for each image, and an imaging position and an imaging posture for each image, a plurality of images are arranged in the extending direction and circumference of the organ. Classification means for classifying into a plurality of divided areas corresponding to directions, generation means for generating a developed view in which the degree of image abnormality is assigned to each divided area, and a display unit for displaying the developed view.

It is a block diagram showing a schematic structure of a capsule endoscopy system concerning an embodiment. It is a flowchart which shows the flow of the process which the process part with which the image display apparatus of the capsule endoscopy system shown in FIG. 1 is provided. It is explanatory drawing for demonstrating the estimation process of the centerline of a digestive tract. It is explanatory drawing for demonstrating the division of the digestive tract which concerns on the division | segmentation process of a digestive tract. It is explanatory drawing for demonstrating the area of the digestive tract which concerns on the division | segmentation process of a digestive tract. It is explanatory drawing for demonstrating the relationship between the division area | region of a digestive tract, and the imaging visual field of a capsule endoscope. It is a figure which shows the expansion | deployment image which shows the abnormal degree of an image in the expansion | deployment figure of a digestive tract. It is a figure which shows the capsule endoscopy screen containing the expansion | deployment image shown in FIG.

  An embodiment will be described with reference to the drawings.

  As shown in FIG. 1, a capsule endoscopy system 1 according to the present embodiment includes a capsule endoscope 2 that is an example of an intra-organ imaging device that images a tubular organ, and the capsule endoscope 2. A receiving device 3 that receives data and an image display device 4 that displays an image in the organ based on the data of the receiving device 3 are provided.

  The capsule endoscope 2 includes an illumination unit 2a that illuminates the inside of a tubular organ (for example, the digestive tract), an imaging unit 2b that images the inside of the tubular organ, and a position detection unit that detects an imaging position of the capsule endoscope 2 2c, a posture detection unit 2d that detects the imaging posture of the capsule endoscope 2 at the imaging position, and a transmission unit 2e that transmits information such as the captured image, the imaging position, and the imaging posture to the reception device 3 outside the body. The power supply unit 2f that supplies power to each unit and the control unit 2g that controls each unit are provided. Each of these parts is enclosed in a capsule housing.

  The illuminating unit 2a is an illuminating device that irradiates light into a tubular organ that can be imaged by the imaging unit 2b. As this illumination part 2a, LED (light emitting diode) etc. are used, for example. The capsule housing is formed with a transparent tip portion, for example, so that the internal wall of the organ can be irradiated with light by the built-in illumination unit 2a.

  The imaging unit 2b is an imaging device that performs imaging in a time series with a constant time interval (a constant cycle). For example, a CCD (charge coupled device) camera or the like is used as the imaging unit 2b. In addition to the light irradiation of the illuminating unit 2a described above, the capsule housing is formed so that the inner wall of the organ can be imaged by the built-in imaging unit 2b.

  The position detection unit 2c is a detection device that detects the imaging position of the capsule endoscope 2 in synchronization with the imaging of the imaging unit 2b. This imaging position is a spatial position of the capsule endoscope 2 and is determined by the X axis, the Y axis, and the Z axis. For this spatial position detection, various known position detection means such as a position detection means using the strength of electromagnetic waves can be used.

  The posture detection unit 2d is a detection device that detects the imaging posture (that is, the imaging direction) of the capsule endoscope 2 at the imaging position together with the above-described imaging position in synchronization with the imaging of the imaging unit 2b. As the posture detection unit 2d, various known posture detection units such as a posture detection unit using an acceleration sensor, a gyro sensor, or the like can be used.

  The transmission unit 2e includes the image captured by the imaging unit 2b, the imaging position of the capsule endoscope 2 detected by the position detection unit 2c, and the imaging posture of the capsule endoscope 2 detected by the posture detection unit 2d. This is a transmission device that wirelessly transmits the included information to the reception device 3. For example, a transmission antenna is used as the transmission unit 2e.

  The power supply unit 2f is a power supply device that supplies power to each unit. For example, a button-type battery is used as the power supply unit 2f. A plurality of batteries are provided according to the capacity required for the inspection time.

  The control unit 2g performs control of imaging the inner wall of the organ at a constant time interval (a constant cycle) by the imaging unit 2b while irradiating the inside of the organ by the illumination unit 2a. Further, the control unit 2g sequentially transmits information including an image captured by the image capturing unit 2b, an image capturing position detected by the position detecting unit 2c, and an image capturing posture (image capturing direction) detected by the posture detecting unit 2d by the transmitting unit 2e. Control to transmit to the receiver 3 is performed.

  When the capsule endoscope 2 is swallowed by a patient, the capsule endoscope 2 performs imaging at a constant time interval (a constant cycle) while moving in a tubular organ by a peristaltic motion. In this imaging, irradiation light is irradiated from the illumination unit 2a to the inner wall of the organ by the power from the power supply unit 2f, and the reflected light reflected by the inner wall is photoelectrically converted by the imaging unit 2b to generate image data. At this time, the position detection unit 2c detects the imaging position of the capsule endoscope 2 in synchronization with the generation of the image data. Similarly, the posture detection unit 2d also detects the capsule endoscope in synchronization with the generation of the image data. 2 imaging postures are detected. The image data, the imaging position and the imaging posture of the capsule endoscope 2 are transmitted to the receiving device 3 outside the patient's body by wireless communication by the transmission unit 2e. The generation of image data, the detection of the imaging position by the position detection unit 2c, the detection of the imaging posture by the posture detection unit 2d, and the transmission of data are controlled at a constant time interval (for example, several frames / second) under the control of the control unit 2g. Done in

  The receiving device 3 includes a receiving unit 3a such as a receiving antenna that wirelessly receives information transmitted from the capsule endoscope 2, and a storage medium 3b such as a flash memory that stores the received information. The receiving device 3 receives information (image data, information including the imaging position and imaging posture of the capsule endoscope 2) transmitted from the capsule endoscope 2, and sequentially stores the information in the storage medium 3b. . The storage medium 3b is detachable. The storage medium 3b is attached to the image display device 4, and all information in the storage medium 3b is input to the image display device 4.

  The image display device 4 includes a control unit 4a that centrally controls each unit, a storage unit 4b that stores various programs and various data, an acquisition unit 4c that acquires various data from the storage unit 4b, and various processes. A processing unit 4d to perform, a display unit 4e for displaying various images, and an operation unit 4f for receiving an input operation from an operator such as a doctor are provided. This image display device 4 functions as a workstation.

  The control unit 4a controls each unit based on various programs and various data stored in the storage unit 4b. Furthermore, the control unit 4a executes a series of data processing, display processing for displaying images, and the like based on various programs and various data stored in the storage unit 4b.

  The storage unit 4b includes a memory for storing various programs and various data in addition to a memory functioning as a work area for the control unit 4a. For example, a ROM, a RAM, a magnetic disk device (HDD), a flash memory (semiconductor disk device), or the like is used as the storage unit 4b. The storage unit 4b stores all information in the storage medium 3b of the receiving device 3 (information including image data of all images, the imaging position and the imaging posture of the capsule endoscope 2).

  The acquisition unit 4c acquires information such as image data and the imaging position and imaging posture of the capsule endoscope 2 for each image from the storage unit 4b, and transmits the information to the processing unit 4d. The information is acquired in response to, for example, an operator such as a doctor performing an input operation on the operation unit 4f and instructing the processing unit 4d to execute the process.

  The processing unit 4d receives the image data, the imaging position and the imaging posture of the capsule endoscope 2 for each image from the acquisition unit 4c, obtains the degree of abnormality for each image, and displays the degree of abnormality for each obtained image. A diagram is generated, and the generated development diagram is transmitted to the display unit 4e. This processing unit 4d is an abnormality detecting means for obtaining the degree of abnormality representing the possibility of a medical abnormality for each image, and a classification means for classifying a plurality of images into a plurality of divided regions corresponding to the extending direction and circumferential direction of the organ. Also, it functions as a generation means for generating a development view in which the degree of image abnormality is assigned to each divided area (details will be described later).

  The display unit 4e is a display device that displays various images such as a development view and other images transmitted from the processing unit 4d in color. For example, a liquid crystal display or a CRT (Cathode Ray Tube) display is used as the display unit 4e.

  The operation unit 4f is an input unit that is input by a worker such as a doctor and receives various input operations such as image display, image switching, and setting change. As the operation unit 4f, for example, an input device such as a mouse or a keyboard is used.

  Next, processing performed by the processing unit 4d of the image display device 4 described above will be described in detail. When an operator such as a doctor performs an input operation on the operation unit 4f to instruct execution of processing by the processing unit 4d, the following processing is executed. Here, processing for the digestive tract as an example of a tubular organ will be described.

  As shown in FIG. 2, the processing unit 4d first estimates the center line L1 (see FIG. 3) of the digestive tract from the position information of the capsule endoscope 2 (step S1). In this step S1, as shown in FIG. 3, the center line L1 of the digestive tract is estimated based on the position information (capture position for each image) of the capsule endoscope 2 stored in the storage unit 4b. The position information of the capsule endoscope 2 is time-series position data corresponding to the number of captured images. As described above, since the capsule endoscope 2 itself moves due to the peristaltic movement of the digestive tract, the capsule endoscope 2 advances in the longitudinal direction of the digestive tract while being finely shaken. By smoothing by taking a moving average, the shape of the digestive tract can be estimated, and the center line L1 can be estimated in a thin digestive tract such as the small intestine.

  Returning to FIG. 2, the processing unit 4d calculates the degree of abnormality of each image (step S2). In step S2, the degree of abnormality is calculated for each image by analyzing each image. For example, the processing unit 4d determines whether or not there is red in the image. If the processing unit 4d determines that there is no red, the processing unit 4d determines that there is no bleeding and the degree of abnormality of the image is 0 (zero). On the other hand, when it is determined that there is red, it is determined that there is bleeding, and then it is determined whether the density of red is equal to or higher than a predetermined value. When it is determined that the density of red is equal to or higher than the predetermined value, it is determined that the bleeding is coagulated blood and the degree of abnormality of the image is 1. On the other hand, when it is determined that the density of red is smaller than the predetermined value, it is determined that the bleeding is fresh blood and the degree of abnormality of the image is 2. In this way, the degree of abnormality of the image is determined. The abnormality level may be determined based on the bleeding area of the bleeding, for example, when it is determined whether the red area is equal to or greater than a predetermined value, and the red area is determined to be equal to or greater than the predetermined value. When the degree of abnormality of the image is determined to be 2, while it is determined that the red area is smaller than the predetermined value, it is determined that the degree of abnormality of the image is 1. However, the means for calculating the degree of abnormality is not limited to the above, and various known analysis methods can be used.

  Next, the processing unit 4d classifies each image into each divided region (section and section) of the digestive tract (step S3). As shown in FIG. 4, each segmented region has a section A (n) “n = natural number” along the longitudinal direction of the digestive tract, and a circumferential direction within those sections as shown in FIG. 5. The gastrointestinal tract is set so as to be divided into a section B (m) along which “m = natural number” (in FIG. 5, m = 1 to 4 as an example). Therefore, each segment area is specified by R (A (n), B (m)). In step S3, each image is classified into each divided area and distributed (details will be described later). At this time, the classification of each image is not based on the imaging position of the capsule endoscope 2 when the image is captured, but to which segmented region (section and section) the image is the captured image in the digestive tract. Dependent. In addition, the number and shape of a division area are not limited.

  Here, the above-described classification method will be described in detail. First, as shown in FIG. 6, the processing unit 4d starts from the imaging position V0 of the capsule endoscope 2 (a position determined by the X axis, the Y axis, and the Z axis), and the center line L1 of the digestive tract obtained in step S1 described above A line parallel to is extended to be a digestive tract centerline vector V1. Further, an imaging direction from the imaging position V0 is obtained based on the imaging posture of the capsule endoscope 2, and a line is extended from the imaging position V0 to the imaging direction to obtain an imaging direction vector V2. Here, since the capsule endoscope 2 advances in a state where the imaging surface side faces forward when traveling in the digestive tract, if the imaging posture of the capsule endoscope 2 is known, the imaging direction is determined from the imaging posture. Will be revealed.

  In addition, the processing unit 4d estimates a virtual digestive tract (tubular virtual organ) along the center line L1 of the digestive tract, divides the virtual digestive tract by a certain length along the extending direction, Each section is defined as a section A (n). In addition, each section is equally divided along the circumferential direction, each section is defined as a section B (m), and each divided region R (A (n), B (m)) Set. Here, information necessary for estimation of the digestive tract, such as the thickness of the digestive tract (average thickness according to age and physique, or patient-specific thickness obtained from other tests) It is stored in advance in the storage unit 4b. In addition, division of a section and an area is not limited to equal division.

  Next, the processing unit 4d obtains the imaging field F1 of the capsule endoscope 2 based on the imaging direction vector V2. The viewing angle of the capsule endoscope 2 is stored in advance in the storage unit 4b together with the information such as the thickness of the digestive tract described above, and the processing unit 4d uses the information such as the thickness of the digestive tract from the imaging position V0. The distance (subject distance) along the imaging direction to the digestive tract is calculated, and the imaging visual field F1, that is, the imaging area, is obtained based on the subject distance and the aforementioned viewing angle. The imaging field F1 itself may be stored in the storage unit 4b in advance, and in this case, the processing unit 4d reads the imaging field F1 from the storage unit 4b and uses it.

  As described above, since the viewing angle and subject distance of the capsule endoscope 2, that is, the imaging field of view F1, can be known in advance, any segmented region in the digestive tract when imaged at a predetermined viewing angle from the imaging position V0. It can be calculated whether (section and area) is imaged. However, when the imaging direction vector V2 of the capsule endoscope 2 is inclined with respect to the center line L1 of the digestive tract and the imaging field of view F1 extends over a plurality of divided regions, one image is divided into regions and classified. However, for the sake of simplicity, the image is classified into a segment area having the largest occupation area. When the imaging direction vector V2 of the capsule endoscope 2 is substantially parallel to the center line L1 of the digestive tract and the capsule endoscope 2 goes straight through the digestive tract, the imaging visual field F1 has a plurality of areas (in FIG. 6). In this case, the image data is almost equally spread over four regions), and therefore, one image is divided into regions (divided into four regions in FIG. 6) and classified into respective divided regions.

  Here, in the example of FIG. 6, the image captured from the imaging position V0 is classified into segmented regions R (A (a), B (1)) determined by the section A (a) and the section B (1). . This classification is performed on all images. As a result, the entire screen is assigned to each divided area, but there are also divided areas to which a plurality of images are assigned.

  Returning to FIG. 2, the processing unit 4d calculates the degree of abnormality in the image group of the same segmented area (section and section) (step S4). In step S4, an integrated abnormality degree is calculated for the image groups classified into the same segmented area. Since the degree of abnormality of each image is calculated in the above-described step S2, for example, the average value or the maximum value of the degree of abnormality of the image group classified into the same segmented area is calculated as an integrated abnormality degree.

  Next, the processing unit 4d maps the degree of abnormality of each image to a development view showing each divided region of the digestive tract (step S5). In step S5, the degree of abnormality of each segmented area calculated in step S2 or step S4 described above is mapped on the development view. In this development view, as shown in FIG. 7, the cells in the horizontal direction represent sections, and the cells in the vertical direction represent sections. For example, when it is determined that there is an abnormality in an image of a certain divided area (section and section), a color (shown by hatching in FIG. 7) is added to a cell corresponding to the divided area. At this time, the color is changed depending on the degree of abnormality (in FIG. 7, it is indicated by a difference in hatching). If the degree of abnormality is zero (no abnormality), a specific color (shown in white in FIG. 7) is assigned to the cell. In this way, the cell corresponding to the segmented area where the image is present is assigned a color indicating the degree of abnormality, and the degree of abnormality is assigned. In addition, as a result of the above-described step S3, for a segmented region (section and segment) in which no image to be classified exists, a cell corresponding to the segmented region, that is, a cell to which no abnormality is assigned is specified. A color (shown in black in FIG. 7) is added. As described above, a color indicating that there is no image is given to the cell corresponding to the divided area where there is no image.

  Thereafter, a development view showing the degree of abnormality and the presence or absence of an image, that is, a development view image is displayed on the display unit 4e. The developed view is a pseudo-developed view as if the digestive tract was cut open. In particular, the vertical and horizontal directions are divided into multiple cells. It is colored. In addition, when there is no image corresponding to the cell, a colored display for understanding the image is performed. Thereby, the abnormal part of the digestive tract and its extent can be observed in a list. Furthermore, since the division of cells corresponds to distance rather than time, the spread of abnormal locations can be confirmed, and in addition, the imaging location in the tubular organ and the non-imaging location that was not imaged are accurately listed. Can be confirmed. In particular, the developed view becomes a sketch in the subsequent endoscopy, so that the imaging location in the tubular organ can be easily grasped, and oversight can be prevented. As a result, the accuracy and safety of the procedure can be improved.

  As shown in FIG. 8, in the capsule endoscopy image G1, the development image G2 and the main image G3 are displayed in cooperation. The developed image G2 is a color map colored according to the degree of abnormality of the image, not the image (medical image) itself showing the inner wall of the digestive tract. For this reason, in diagnosis, it is necessary to refer not only to the developed image G2 but also to the main image G3 that is an actually captured image (medical image). Therefore, when an operator such as a doctor selects a cell corresponding to the segmented region R (A (a), B (1)) in the developed view using the operation unit 4f such as a mouse, the segmented region R (A (a), A main image G3 corresponding to B (1)) is displayed. As the main image G3, for example, thumbnails of image groups classified into the segmented region R (A (a), B (1)) and the degree of abnormality of the segmented region R (A (a), B (1)). A representative image or the like prioritized in determining the image is displayed. A plurality of cells may be selected simultaneously. On the other hand, by switching the display of the main image G3, cells corresponding to the divided areas (sections and sections) in which the image is classified may be highlighted.

  As described above, according to the present embodiment, the processing unit 4d of the image display device 4 obtains the degree of abnormality representing the possibility of a medical abnormality for each image, and the capsule endoscope 2 for each image. Using the imaging position and orientation, multiple images are classified into multiple segment areas corresponding to the stretching direction and circumferential direction of a tubular organ such as the digestive tract, and an image abnormality level is assigned to each segment area. A diagram is generated, and the developed view is displayed on the display unit 4e. As a result, unlike the normal time bar display, the developed view looks like an image that opens up the digestive tract, and shows the abnormal points and extent of the digestive tract in a list to show the abnormal points and presence of the images for each segmented area. It becomes possible to do. In particular, since the division of the divided area corresponds to the distance instead of the time, it is possible to check the spread of the abnormal part. In addition, it is possible to accurately confirm the imaging location in the tubular organ and the non-imaging location that was not imaged in a list. Therefore, the imaging location in the tubular organ can be easily grasped, and further, oversight of a non-imaging location that is not imaged can be prevented.

  In addition, the imaging field F1 of the capsule endoscope 2 with respect to the imaging position is obtained using the imaging position and the imaging posture of the capsule endoscope 2 for each image, and the imaging field of the capsule endoscope 2 for each obtained image. By classifying a plurality of images into each segmented area according to F1, it is possible to classify each image into each segmented area accurately, and to grasp an imaging location and a non-imaging location more accurately. In addition to using the imaging field of view F1 of the capsule endoscope 2, as this classification method, an imaging direction based on the imaging position is obtained based on the imaging posture of the capsule endoscope 2, and from the imaging position. You may make it use the intersection of the straight line (imaging direction vector) extended in an imaging direction, and the inner wall of a tubular organ. However, since the tubular organ has a large number of curved portions, the classification accuracy is higher when the above-described imaging field of view F1 is used.

  Further, the center line L1 of the organ extending in the extending direction of the tubular organ is obtained based on the imaging position for each image, the tubular virtual organ whose axis is the obtained center line L1 is obtained, and the obtained virtual organ is extended. By dividing into directions and circumferential directions and dividing into a plurality of divided regions, it becomes possible to divide according to the actual organ path, so it is possible to classify images into more accurate divided regions, As a result, it is possible to obtain a developed view showing the abnormal points and presence / absence of the image more accurately.

  In addition, when multiple images are classified into the same segmented area, the average value or maximum value of the degree of abnormality of the image group in the segmented area is obtained, and the degree of image abnormality (total abnormality of the image group) is calculated in the segmented area. As a result, even if multiple images are classified into the same segmented area, the degree of abnormality for each segmented area can be accurately shown in the development view, so that the abnormal point of the image can be more accurately identified. The developed view shown can be obtained, and as a result, more accurate observation can be performed.

  Although one embodiment of the present invention has been described above, this embodiment is presented as an example and is not intended to limit the scope of the invention. These novel embodiments can be implemented in various other forms, and various omissions, replacements, and changes can be made without departing from the scope of the invention. These embodiments and modifications thereof are included in the scope and gist of the invention, and are included in the invention described in the claims and the equivalents thereof.

2 Capsule endoscope (Intra-organ imaging device)
4 image display device 4c acquisition unit 4d processing unit 4e display unit

Claims (4)

An acquisition unit that acquires a plurality of images captured by an intra-organ imaging device that moves in a tubular organ and images in time series, and an imaging position and an imaging posture of the intra-organ imaging device for each image;
Abnormality detection means for obtaining an abnormality degree representing the possibility of a medical abnormality for each image,
Classifying means for classifying the plurality of images into a plurality of segment regions corresponding to the extending direction and the circumferential direction of the organ using the imaging position and the imaging posture for each image;
Generating means for generating a development view in which the degree of abnormality of the image is assigned to each of the divided areas;
A display unit for displaying the developed view;
An image display device comprising:   The classification means obtains an imaging field of view of the intra-organ imaging device with respect to the imaging position using the imaging position and the imaging orientation for each image, and images the intra-organ imaging device for each obtained image. The image display device according to claim 1, wherein the plurality of images are classified into the plurality of divided regions according to a field of view.   The classification means obtains a center line of the organ extending in the extending direction of the organ based on an imaging position for each image, obtains a tubular virtual organ having the obtained center line as an axis, and obtains the obtained virtual 3. The image display device according to claim 1, wherein the organ is divided into the plurality of divided regions by dividing the organ into the extending direction and the circumferential direction.   When the plurality of images are classified into the same segmented area, the classifying unit obtains an average value or a maximum value of the degree of abnormality of the image group of the segmented area, and assigns it to the segmented area as the degree of abnormality of the image. The image display device according to claim 1, 2, or 3.

Images