JP2025100867A - COMPUTER PROGRAM, INFORMATION PROCESSING METHOD, AND INFORMATION PROCESSING APPARATUS - Google Patents

Abstract

To provide a computer program capable of presenting information on anatomical feature points of a hollow organ so as to be grasped more, an information processing method, and an information processing device.SOLUTION: A computer program makes a computer for acquiring a plurality of tomographic images of a hollow organ generated on the basis of a signal detected by an imaging device provided in a catheter inserted into a first tube of the hollow organ with a bifurcation execute detection processing including determination of the presence/absence of a second tube bifurcated from the first tube and angle calculation of the second tube to the plurality of tomographic images, and executes processing for noise removal processing on the basis of distribution information including a position in a longitudinal axis direction of the hollow organ of a tomographic image determined to be an image showing the second tube by the detection processing and an angle of the second tube in the tomographic image calculated by the detection processing.SELECTED DRAWING: Figure 14

Description

The present disclosure relates to a computer program, an information processing method, and an information processing device for processing medical images.

In medical examinations, images of the subject are either taken directly or obtained by imaging the results of measurements using electromagnetic waves, and these images are used for diagnosis. In particular, in examinations of hollow organs, various techniques are used that use images obtained by moving an imaging element inside the organ.

Diagnostic imaging of vascular organs, particularly blood vessels, is essential for the safe and reliable performance of procedures such as percutaneous coronary intervention (PCI). For this reason, in addition to angiography, which uses contrast media to take images from outside the body, intravascular imaging techniques such as IVUS (Intra Vascular Ultra Sound) and OCT (Optical Coherence Tomography)/OFDI (Optical Frequency Domain Imaging) using catheters are becoming widespread.

In the image diagnosis described above, it is not easy to obtain accurate diagnostic information from the captured medical images. To assist in the interpretation of medical images, various techniques have been proposed that use image analysis or machine learning to correct images or add information (Patent Document 1, etc.).

JP 2012-075702 A

Interpreting medical images requires accurate identification of anatomical landmarks from the images. Therefore, it is necessary to capture medical images accurately and to present information about anatomical landmarks in a way that is easier to understand and more visible.

The objective of the present disclosure is to provide a computer program, an information processing method, and an information processing device that can present information about anatomical landmarks of hollow organs in a manner that is easier to understand.

The computer program of the present disclosure causes a computer to acquire multiple tomographic images of a tubular organ having a branch, generated based on a signal detected by an imaging device provided in a catheter inserted into a first tube of the tubular organ, to execute a detection process for the multiple tomographic images, including determining whether or not a second tube branches off from the first tube and calculating the angle of the second tube, and to execute a process of performing a noise removal process based on distribution information including the position in the longitudinal direction of the tubular organ of a tomographic image determined by the detection process to be an image showing the second tube, and the angle of the second tube in the tomographic image calculated by the detection process.

In the information processing method disclosed herein, a computer acquires multiple tomographic images of a tubular organ having a branch, generated based on a signal detected by an imaging device provided in a catheter inserted into a first tube of the tubular organ, performs a detection process on the multiple tomographic images, including determining whether or not a second tube branches off from the first tube and calculating the angle of the second tube, and performs a noise removal process based on distribution information including the position in the longitudinal direction of the tubular organ of a tomographic image determined by the detection process to be an image showing the second tube, and the angle of the second tube in the tomographic image calculated by the detection process.

The information processing device according to the present disclosure is an information processing device that acquires multiple tomographic images of a tubular organ having a branch, generated based on a signal detected by an imaging device provided in a catheter inserted into a first tube of the tubular organ, and includes a processing unit that executes image processing on the multiple tomographic images, and the processing unit executes a detection process on the multiple tomographic images, including determining whether or not a second tube branches off from the first tube and calculating the angle of the second tube, and performs noise removal processing based on distribution information including the position in the longitudinal direction of the tubular organ of a tomographic image determined by the detection process to be an image showing the second tube, and the angle of the second tube in the tomographic image calculated by the detection process.

The present disclosure makes it possible to display information about anatomical landmarks of hollow organs in a way that is easier to understand.

FIG. 1 is a diagram illustrating an example of the configuration of an imaging diagnostic apparatus. FIG. 13 is an explanatory diagram showing the operation of the catheter. FIG. 1 is a block diagram showing a configuration of an image processing device. FIG. 1 is an overview of a trained model. FIG. 13 is a diagram showing detected boundaries (contours). 11 is a flowchart illustrating an example of an information processing procedure performed by the image processing device. 11 is a flowchart illustrating an example of an information processing procedure performed by the image processing device. 13 is a flowchart showing an example of a detailed processing procedure of a side branch detection process. FIG. 13 is a schematic diagram of a side branch detection process. FIG. 13 is a schematic diagram of a side branch detection process. FIG. 13 is a schematic diagram of a side branch detection process. FIG. 13 is a schematic diagram of a side branch detection process. FIG. 13 is a schematic diagram of a side branch detection process. 13 is a flowchart illustrating an example of a processing procedure for removing noise data in side canal detection. FIG. 13 is a schematic diagram of noise data removal processing for side canal detection. 1 shows an example screen containing information presented on a display device. FIG. 11 is a diagram showing another example of a screen displayed on the display device. FIG. 11 is a diagram showing another example of a screen displayed on the display device. FIG. 11 is a diagram showing another example of a screen displayed on the display device. FIG. 11 is a diagram showing another example of a screen displayed on the display device. FIG. 11 is a diagram showing another example of a screen displayed on the display device.

Specific examples of a computer program, an information processing method, and an information processing device according to an embodiment of the present invention will be described below with reference to the drawings.

Figure 1 shows an example of the configuration of an image diagnostic device 100. The image diagnostic device 100 is a device for generating medical images including ultrasonic tomographic images of blood vessels (hollow organs) using the IVUS method and for performing ultrasonic examinations and diagnoses within blood vessels.

The imaging diagnostic device 100 includes a catheter 1, an MDU (Motor Drive Unit) 2, an image processing device (information processing device) 3, a display device 4, and an input device 5.

The catheter 1 is a flexible tube for medical use. The catheter 1 is particularly known as an imaging catheter, which has an imaging device 11 at its tip and rotates in a circumferential direction when driven from its base end. In the case of the IVUS method, the imaging device 11 is an ultrasound probe including an ultrasound transducer and an ultrasound sensor. In the case of OCT, the imaging device 11 is an OCT device including a near-infrared laser and a near-infrared sensor. The imaging device 11 may also be another device that uses electromagnetic waves of other wavelengths, such as visible light.

The MDU 2 is a drive unit attached to the base end of the catheter 1, and controls the operation of the catheter 1 by driving the internal motor in response to the operation of a medical professional.

The image processing device 3 generates multiple medical images, such as cross-sectional images of blood vessels, based on the signal output from the imaging device 11 of the catheter 1. The configuration of the image processing device 3 will be described in detail later.

The display device 4 uses a liquid crystal display panel, an organic EL display panel, or the like. The display device 4 displays the medical images generated by the image processing device 3 and information related to the medical images.

The input device 5 is an input interface that accepts operations for the image processing device 3. The input device 5 may be a keyboard, a mouse, etc., or may be a touch panel, soft keys, hard keys, etc. built into the display device 4.

Figure 2 is an explanatory diagram showing the operation of the catheter 1. In Figure 2, the catheter 1 is inserted into a tubular blood vessel L by a medical professional along a guide wire W that has been inserted into the coronary artery shown in the figure. In the enlarged view of the blood vessel L in Figure 2, the right side corresponds to the distal side from the insertion point of the catheter 1 and the guide wire W, and the left side corresponds to the proximal side.

Driven by the MDU 2, the catheter 1 moves from the distal end to the proximal end of the blood vessel L as indicated by the arrow in the figure, and while rotating in the circumferential direction, the imaging device 11 scans the blood vessel in a spiral manner.

In the image diagnostic device 100 of this embodiment, the image processing device 3 acquires a signal for each scan output from the imaging device 11 of the catheter 1. In one scan, the imaging device 11 emits a detection wave in the radial direction and detects the reflected wave. The imaging device 11 performs this scan tens to thousands of times while rotating 360 degrees, scanning in a spiral shape. The image processing device 3 generates a tomographic image (transverse cross-sectional image) obtained by polar coordinate conversion (inverse conversion) of the signal for each scan for every 360 degrees (I1 in FIG. 2). The tomographic image I1 is also called a frame image. The reference point (center) of the tomographic image I1 corresponds to the range of the catheter 1 (not imaged). The image processing device 3 further generates a long-axis image (longitudinal cross-sectional image) in which the pixel values on a straight line passing through the reference point of the tomographic image I1 are arranged along the length direction (long axis direction) of the blood vessel by the catheter 1 (I2 in FIG. 2). The image processing device 3 analyzes and processes the branching structure of the blood vessels based on the obtained tomographic image I1 and long-axis image I2, and outputs a two-dimensional or three-dimensional image showing the structure of the blood vessels so that it can be viewed by a medical professional. In the image diagnostic device 100 of the present disclosure, information about the side branches, such as the side branch position and outline of the first tube (e.g., main trunk) into which the guidewire W and catheter 1 are inserted, and the second tube (e.g., side branch) branching from the first tube, is superimposed on both the long-axis image and the tomographic image displayed on the display device 4. The process by which the image processing device 3 creates information about the side branches and superimposes it on the display device 4 will be described in detail below.

Figure 3 is a block diagram showing the configuration of the image processing device 3. The image processing device 3 is a computer, and includes a processing unit 30, a storage unit 31, and an input/output I/F 32.

The processing unit 30 includes one or more CPUs (Central Processing Units), MPUs (Micro-Processing Units), GPUs (Graphics Processing Units), GPGPUs (General-purpose computing on graphics processing units), TPUs (Tensor Processing Units), etc. The processing unit 30 incorporates a non-temporary storage medium such as a RAM (Random Access Memory), and executes calculations based on a computer program 3P stored in the storage unit 31 while storing data generated during processing in the non-temporary storage medium.

The storage unit 31 is a non-volatile storage medium such as a hard disk or a flash memory. The storage unit 31 stores the computer program 3P, setting data, etc., that are read by the processing unit 30. The storage unit 31 also stores the trained model 3M.

The computer program 3P and the trained model 3M may be copies of the computer program 9P and the trained model 9M stored in a non-temporary storage medium 9 outside the device, read out via the input/output I/F 32. The computer program 3P and the trained model 3M may be distributed by a remote server device, acquired by the image processing device 3 via a communication unit (not shown), and stored in the storage unit 31.

The input/output I/F 32 is an interface to which the catheter 1, the display device 4, and the input device 5 are connected. The processing unit 30 acquires a signal (digital data) output from the imaging device 11 via the input/output I/F 32. The processing unit 30 outputs screen data of a screen including the generated tomographic image I1 and/or long axis image I2 to the display device 4 via the input/output I/F 32. The processing unit 30 accepts operation information input to the input device 5 via the input/output I/F 32.

Figure 4 is an overview diagram of the trained model 3M. In the present disclosure, the trained model 3M is a model trained to output an image showing the area of one or more objects appearing in an image when an image is input. The trained model 3M is, for example, a model that performs semantic segmentation. The trained model 3M is designed to output an image tagged with data indicating which object each pixel in the input image is within, and a probability.

For example, as shown in FIG. 4, the trained model 3M uses a so-called U-net in which a convolution layer, a pooling layer, an upsampling layer, and a softmax layer are symmetrically arranged. When a tomographic image I1 created by a signal from the catheter 1 is input, the trained model 3M outputs a tag image IS and accuracy. The tag image IS is obtained by tagging the range of the blood vessel lumen, the range of the membrane corresponding to the area between the lumen boundary of the blood vessel including the tunica media and the blood vessel boundary, the range in which the guidewire W and its reflection are captured, and the range corresponding to the catheter 1 with different pixel values (shown in FIG. 4 with different types of hatching and solid color) for the pixels at those positions.

As described above, semantic segmentation and U-net have been exemplified as trained model 3M, but it goes without saying that this is not limited to these. In addition, trained model 3M may be a model that realizes individual recognition processing using instance segmentation, etc. Trained model 3M is not limited to being U-net-based, and may also be a model based on SegNet, R-CNN, or an integrated model with other edge extraction processing, etc.

The processing unit 30 can detect the edges of the lumen boundary and vascular boundary of the blood vessel shown in the tomographic image I1 by using pixel values in the tag image IS obtained by inputting the image I1 to the trained model 3M and the coordinates in the image. Strictly speaking, the vascular boundary is the external elastic membrane (EEM) between the tunica media and tunica adventitia of the blood vessel, and is shown relatively clearly with low brightness in the image I1 by the IVUS method. Figure 5 is a diagram showing the detected boundary (contour). Figure 5 shows a state in which a curve B1 indicating the lumen boundary obtained based on the output from the trained model 3M and a curve B2 indicating the vascular boundary are superimposed on the tomographic image I1 shown in Figure 4.

The image processing device 3 of the present disclosure further derives and displays anatomical feature points using the tomographic image I1 obtained from the signal from the catheter 1 and the information on the lumen boundary and blood vessel boundary obtained when the tomographic image I1 is input to the trained model 3M. The detailed processing procedure is described below.

Figures 6 and 7 are flowcharts showing an example of an information processing procedure by the image processing device 3. When a signal is output from the imaging device 11 of the catheter 1, the processing unit 30 of the image processing device 3 starts the following processing.

Each time the processing unit 30 acquires a predetermined amount (e.g., 360 degrees) of signals (data) from the imaging device 11 of the catheter 1 (step S301), it performs polar coordinate conversion (inverse conversion) on the rectangularly arranged signals to generate a tomographic image I1 (step S302). The processing unit 30 outputs the generated tomographic image I1 so that it can be displayed in real time on the screen displayed on the display device 4 (step S303). The processing unit 30 stores the signal data acquired in step S301 and the tomographic image I1 in the memory unit 31 in association with the position (position on the long axis, angle) of the imaging device 11 (step S304).

The processing unit 30 inputs the tomographic image I1 to the trained model 3M (step S305). Based on the tag image IS obtained from the trained model 3M, the processing unit 30 calculates data on the lumen boundary and vascular boundary in the tomographic image I1 (step S306). In step S306, the processing unit 30 calculates the lumen range output from the trained model 3M, the inner membrane range including the tunica media of the blood vessel, the contour (edge) of the lumen range as the lumen boundary, and the outer contour of the membrane range as the vascular boundary. In step S306, the processing unit 30 may implement high-speed processing, such as reducing the size of the tomographic image I1 before inputting it to the trained model 3M.

Based on the data (coordinate data) of the lumen boundary and vascular boundary obtained in step S306, the processing unit 30 calculates parameters for determining whether or not a side branch is shown (step S307). If the tomographic image I1 shows the branching portion of the main trunk and the side branch, the shape of the vascular boundary derived from the tomographic image I1 deviates from a circle or an ellipse. In step S307, the processing unit 30 calculates parameters corresponding to the degree of deviation from a circle or an ellipse.

In step S307, the processing unit 30 may basically calculate parameters only for the vascular boundary. However, if the vascular boundary intersects with the outside of the tomographic image I1, i.e., the membrane range extends outside the image of the tomographic image I1, the processing unit 30 may calculate parameters for the inner region of the lumen boundary instead of the vascular boundary.

In step S307, in the first example, the processing unit 30 calculates the eccentricity by dividing the difference between the maximum and minimum diameters passing through the center of gravity of the inner region of the blood vessel boundary by the maximum diameter.

In step S307, in the second example, the processing unit 30 may use a learning model for judgment (not shown) that has been trained to output a degree of accuracy corresponding to the possibility that a side branch is captured when data on the lumen boundary and vascular boundary are input. Here, the learning model for judgment may be trained to output a degree of accuracy corresponding to the possibility that a side branch is captured when data on the lumen range and membrane range in the tag image output from the trained model 3M is input. In step S307, the processing unit 30 calculates the output from the learning model for judgment as a parameter.

In step S307, in the third example, the processing unit 30 may calculate a parameter obtained by comparing the diameter (maximum diameter and minimum diameter) of the target blood vessel boundary with the diameter of the blood vessel boundary for the already scanned tomographic image I1. If the diameter changes suddenly by more than a predetermined percentage or by more than a predetermined length, it can be determined that there is a high possibility that a side branch is captured.

As a fourth example, the processing unit 30 may calculate circularity instead of eccentricity. Circularity is the ratio of the area of the inner region of the vascular boundary to the circumference of the vascular boundary. The closer the circularity is to the ratio of the area to the circumference of a circle, the higher the circularity is, and it can be determined that the possibility of a side branch being captured is low.

The processing unit 30 stores the parameters in association with the position of the imaging device 11 (step S308). Based on the calculated parameters, the processing unit 30 determines whether the target tomographic image I1 is a candidate for an image showing not only the main trunk into which the catheter 1 is inserted, but also a side branch (step S309). In step S309, the processing unit 30 may make a determination based on, for example, whether the eccentricity is higher than a predetermined value. If it is determined in step S309 that it is not a candidate (S309: NO), the processing unit 30 proceeds to step S313.

If it is determined to be a candidate (S309: YES), the processing unit 30 executes a side branch detection process (step S310), which includes determining the presence or absence of a side branch based on the vascular boundary calculated from the target tomographic image I1, identifying the boundary of the main trunk in the tomographic image, and calculating the angle of the side branch.

The processing unit 30 superimposes the image showing the area boundary of the main trunk and the data on the angle of the side branch obtained as a result of the side branch detection process in step S310 on the tomographic image I1 being displayed in step S303 (step S311). Step S311 may be skipped if it is determined that the side branch is not present in the tomographic image I1. The processing unit 30 stores the result of the side branch detection process in association with the tomographic image I1 stored in step S304 (step S312). If it is determined that a side branch is present, data such as a flag indicating a side branch image is associated with the tomographic image I1. The position corresponding to the tomographic image I1 associated with the data of the flag indicating a side branch image is the position of the branch of the main trunk and the side branch on the long axis.

The processing unit 30 may also perform the processing of steps S307-S310 on the lumen boundary calculated in step S306.

The processing unit 30 determines whether scanning by the imaging device 11 of the catheter 1 has been completed (step S313). If it is determined that scanning has not been completed (S313: NO), the processing unit 30 returns the process to step S301 and generates the next tomographic image I1.

If it is determined that the scan is complete (S313: YES), the processing unit 30 executes a process of removing noise data based on the presence or absence of side branches and the distribution of the angles of the detected side branches relative to the position of the imaging device 11 (step 314). In step S314, the processing unit 30 re-determines that there are no side branches for the tomographic image I1 that is determined to have low accuracy even if the tomographic image I1 is determined to have a side branch by the process of step S310, and stores the determined result. Details will be described later.

The processing unit 30 causes the display device 4 to display the long axis image I2 with a mark indicating the presence of a side branch at a location corresponding to the position of the tomographic image I1 where it has been determined by the processing in step S314 that the side branch is shown (step S315), and ends the processing.

Figure 8 is a flowchart showing an example of a detailed processing procedure for side branch detection processing. The flowchart in Figure 8 corresponds to the details of step S310 shown in the flowcharts in Figures 6 and 7.

The processing unit 30 extracts a circle along the boundary of the target (blood vessel boundary or lumen boundary) by, for example, a Hough transform (step S101). When, for example, a Hough transform is used in step S101, the processing unit 30 extracts multiple circles of various sizes.

The processing unit 30 determines a plausible circle corresponding to the main trunk from the circles extracted in step S101 (step S102). In step S102, the processing unit 30 selects a circle whose diameter is equal to or greater than a predetermined length corresponding to the size of the blood vessel diameter and whose center is closest to the center of gravity of the area within the boundary of the target as the plausible circle corresponding to the main trunk. In step S102, the processing unit 30 may select a circle whose center is closest to the center of the image as the plausible circle, or may select a circle whose center is closest to the center of gravity of the previous or next frame image as the plausible circle. The processing unit 30 may select a combination of some or all of these selection methods as appropriate.

The processing unit 30 determines the center of the circle determined to correspond to the main trunk as the center of the blood vessel (step S103), and calculates the distance from the center of the blood vessel to points on the boundary of the target along the entire circumference (360 degrees) of the boundary (step S104). The processing unit 30 creates a distribution of the distance from the center of the blood vessel along the entire circumference (step S105), and determines whether there are a predetermined number or more consecutive points on the boundary whose distance from the center of the blood vessel is longer than a predetermined reference value (step S106).

If it is determined that there is a predetermined number or more (S106: YES), the processing unit 30 determines that a side branch is present (is captured) in the tomographic image I1 (step S107). The processing unit 30 identifies arcs on the main trunk circle determined in step S102 that correspond to points on the boundary whose distance from the center of the blood vessel is continuously longer than a predetermined reference value (step S108).

The processing unit 30 calculates the side branch angle as the angle of a straight line connecting the center of the tomographic image I1 to the center point of the arc identified in step S108 for the tomographic image I1 from, for example, the 12 o'clock direction (upward) in the tomographic image I1 (step S109).

The processing unit 30 identifies an interpolation trajectory that connects two points on the boundary that are closest to the end points of the arc identified in step S108 with a new arc (step S110). The processing unit 30 stores the interpolation trajectory identified in step S110 as an image showing the area boundary of the main trunk (step S111), and returns the process to step S311 in Figs. 6 and 7.

If it is determined in step S106 that the number is less than the predetermined number (S106: NO), the processing unit 30 determines that no side branches are present in the tomographic image I1 (step S112) and returns the process to step S311 in Figures 6 and 7.

The processing procedure shown in the flowchart of FIG. 8 will be explained using a specific example. FIGS. 9 to 13 are schematic diagrams of side branch detection processing. FIG. 9 is a diagram showing an example of circles extracted for a vascular boundary. FIG. 9 shows a cross-sectional image I1 superimposed with a curve B2 of the vascular boundary calculated for the cross-sectional image I1, and shows in bold a number of circles extracted after the circle extraction process for the curve B2. As shown in FIG. 9, from a cross-sectional image I1 that is a candidate for showing a side branch, circles are extracted for a portion corresponding to the main trunk in the lower left part of the cross-sectional image I1 and a portion corresponding to the side branch. However, in FIG. 9, the circle with the larger diameter in the lower left part is determined to be closer to the center of gravity.

Figure 10 shows the distribution obtained by the processing of step S104 in the flowchart of Figure 8. The upper part of Figure 10 shows the calculated vascular boundary curve B2 and the determined circle superimposed on the tomographic image I1. The lower part of Figure 10 shows the distribution of the distance from the center of the blood vessel to the coordinate points on the curve B2. The arrows in Figure 10 indicate the order in which the distances are calculated. As shown in Figure 10, in the tomographic image I1 in which a side branch is shown, there is a clear peak in the distance, as shown in the distribution at the bottom.

Figure 11 is a diagram showing the determination of the presence or absence of side branches based on the distribution in Figure 10 (S106). In Figure 11, a reference value for distance is shown by a thick line on the distribution shown in Figure 10. In the diagram in Figure 11, it is determined that there are a predetermined number of pixels whose distance is equal to or greater than the reference value. In the lower part of Figure 11, points on the boundary (B2) whose distance from the center of the blood vessel is continuously longer than the predetermined reference value are shown in black, and the corresponding arc (black end point circle) is shown (S108). Specifically, the arc of the black end point circle is specified as the range between two straight lines connecting the center of the blood vessel and both ends of a part of the boundary (B2) that is continuously longer than the predetermined reference value, of the circle corresponding to the main trunk.

The determination of whether or not a side branch exists (is captured) with a high probability (S106) is not limited to the content shown in FIG. 11. For example, it is not limited to whether or not there are a predetermined number or more consecutive coordinate points at a distance equal to or greater than the radius of the circle determined for the vascular boundary. It may be determined that there are a predetermined number or more consecutive coordinate points on the lumen boundary at a distance equal to or greater than the radius of the circle determined for the lumen boundary, and that the range overlaps with the range specified for the vascular boundary.

Figure 12 shows an example of an interpolation trajectory identified based on the arc identified in Figure 11. As shown in Figure 12, the interpolation trajectory is identified as an arc that passes through two points on the curve B2 of the vascular boundary that are close to each of the end points of the arc identified in step S108. Also, in Figure 12, information on the calculated side branch angle (side branch direction) is shown as a white arrow. At this time, the image (arrow) indicating the side branch angle is displayed outside the curve B2 so that the curve B2 and the interpolation trajectory can be easily viewed.

Figure 13 shows an example of display on the display device 4. In Figure 13, a tomographic image I1, the lumen boundary and vascular boundary curves B1 and B2 superimposed on the tomographic image I1, an interpolation trajectory, and information on the side branch angle are superimposed. In the example shown in Figure 13, an interpolation trajectory determined in a similar manner is superimposed on the lumen boundary. This makes it easy for even medical personnel who are not accustomed to reading images to interpret the presence or absence of side branches and the locations of side branches where plaque is likely to accumulate.

Figure 14 is a flowchart showing an example of a processing procedure for removing noise data in side branch detection. The flowchart in Figure 14 corresponds to details of step S314 shown in the flowcharts in Figures 6 and 7.

The processing unit 30 judges whether or not there is one or more other tomographic images I1 in which a side branch is judged to exist within a predetermined range from the position of the tomographic image I1 in which a side branch is judged to exist within the scanning range of the catheter 1 (step S401). The predetermined range is, for example, a range corresponding to the diameter of the side branch (1 to 5 mm). In a location where a side branch actually exists, the side branch should be judged to exist in multiple consecutive tomographic images I1. Therefore, if a tomographic image I1 in which a side branch is judged to exist is isolated, this is presumed to be a false detection. In step S401, the processing unit 30 may judge that there are multiple side branches, for example, if the side branch is judged to exist in 80% of the multiple tomographic images I1 generated for each position within a predetermined range from the position of the tomographic image I1 in which a side branch is judged to exist.

Therefore, when it is determined in step S401 that no other tomographic image I1 exists (S401: NO), the processing unit 30 removes the target tomographic image I1 from the image in which the side branch exists (step S402). Specifically, in step S402, the processing unit 30 deletes the flag indicating the side branch image, information on the side branch, etc., that was associated with the tomographic image I1. The processing unit 30 returns the process to step S315 in Figs. 6 and 7.

If it is determined that another tomographic image I1 exists (S401: YES), it is compared with other tomographic images I1 in which a side branch is determined to exist within a predetermined range, and it is determined whether the side branch angles are similar (step S403). If the same side branch is detected, the calculated side branch angles should be similar.

Therefore, if the processing unit 30 determines that they are not similar (S403: NO), the processing unit 30 removes the target tomographic image I1 from the image in which the side branch is present (S402) and returns the process to step S315 in Figures 6 and 7.

If it is determined that they are similar (S403: YES), the processing unit 30 returns the process to step S315 in FIG. 6 and FIG. 7. In this case, the tomographic image I1 determined to be similar remains associated with data such as a flag indicating that it is a side branch image. The tomographic image I1 that was not removed by the flowchart in FIG. 14 becomes an image that has been confirmed to contain a side branch from among the candidate images that contain a side branch.

Figure 15 is an overview of the noise data removal process for side branch detection. Figure 15 is a distribution diagram of side branch angles for positions in tomographic image I1 where a side branch is determined to exist within the scanning range of catheter 1. The horizontal axis of Figure 15 is the long axis direction of the blood vessel through which catheter 1 moves, and the vertical axis indicates the intravascular angle. Each point indicates the position in the long axis direction of tomographic image I1 where a side branch is determined to exist, the angle range for that side branch, and the side branch angle, which is the center value of that range. In Figure 15, the range on the long axis where a side branch actually exists is hatched.

As shown in FIG. 15, where a side branch actually exists, consecutive points continue in a similar angle range. In contrast, for points corresponding to tomographic image I1 where a side branch is determined to exist in a location where a side branch does not actually exist, the calculated side branch angles are inconsistent and there is no continuity. Therefore, depending on whether there is continuity in the position on the long axis and the side branch angle, the information of tomographic image I1 where a side branch is erroneously determined to exist can be removed as noise.

By clustering the features of each point shown in FIG. 15 (position in the long axis direction, side branch range, and side branch angle), points that do not belong to a cluster may be removed as noise, assuming that the tomographic image I1 is a case in which a side branch is erroneously determined to exist. For example, DBSCAN (Density-Based Spatial Clustering of Applications with Noise) may be used for the clustering, or other known methods may be used.

In this way, the image diagnostic device 100 can present each tomographic image I1 together with data such as the angle of the side branch in the tomographic image I1 by narrowing down the information on the tomographic image I1 that is determined to have a high probability of having a side branch rather than noise. Specifically, the memory unit 31 of the image processing device 3 stores signal data at each time point from the imaging device 11 and the tomographic image I1 converted into polar coordinates for one examination using the catheter 1. The memory unit 31 also stores coordinate data related to the lumen boundary and vascular boundary (curve, center of gravity, center of blood vessel, blood vessel diameter (minimum diameter), etc.), the presence or absence of a side branch, the angle of the side branch, data on the interpolated trajectory, etc. obtained by processing each tomographic image I1. Based on these data, the image processing device 3 can present data on anatomical features in real time during an examination using the catheter 1 or after the fact.

Figure 16 shows an example of a screen including information presented on the display device 4. The screen 400 shown in Figure 16 includes display areas for a long-axis image I2 and a tomographic image I1 of the position being scanned or selected at the time of display. As shown in Figure 13, the display area for the tomographic image I1 displays superimposed information on the lumen boundary, the vascular boundary curves B1 and B2, the interpolation trajectory, and the side branch angle.

As shown in FIG. 16, the long axis image I2 on the screen 400 is a longitudinal cross-sectional image with the long axis direction of the blood vessel through which the catheter 1 moves as the horizontal direction. By default, it is displayed as a collection of brightness data along the vertical direction in the tomographic image I1. A cursor 403 indicating the position on the long axis corresponding to the tomographic image I1 is displayed on the long axis image I2. The screen 400 also includes a first button 401 and a second button 402 at locations on the long axis image I2 that correspond to the position in the long axis direction of the tomographic image I1 that has been determined and confirmed to contain a side branch. The buttons 401 and 402 are displayed for the number of side branches that have been confirmed to exist.

The first button 401 and the second button 402 are buttons for displaying the tomographic image I1 in which the presence of each side branch has been confirmed, and the data related to the corresponding side branch. When the first button 401 and the second button 402 are selected, the long axis image I2 becomes a longitudinal section image at the side branch angle corresponding to each button in each tomographic image I1.

17 and 18 show other examples of screens displayed on the display device 4. FIGS. 17 and 18 are screens that are displayed when the first button 401 and the second button 402 in the screen shown in FIG. 16 are pressed, respectively. FIG. 17 is a screen that is displayed when the first button 401 is selected by the input device 5, and FIG. 18 is a screen that is displayed when the second button 402 is selected by the input device 5. Comparing FIGS. 16 to 18, it can be seen that the cross-sectional direction of the long-axis image I2 is different between the default, when the first button 401 is selected, and when the second button 402 is selected. This makes it easier for medical personnel who interpret the images to observe the entire blood vessel with a focus on the side branches and understand the anatomical structure from the examination results using the catheter 1.

By accurately detecting the position of the side branch in the direct axis direction, and by detecting the interpolated trajectory of the main trunk and side branch, as well as the side branch angle, other anatomical features at the branching portion of the blood vessel can be displayed in various forms on the display device 4. FIG. 19 shows another example of a screen displayed on the display device 4. The screen 400 shown in FIG. 19 includes a schematic longitudinal section image I3 in which the blood vessel region identified for each tomographic image I1 is connected in the longitudinal direction with the center of the blood vessel as the axis. In addition to the schematic, a longitudinal section image may be displayed that takes into account dimensions based on the diameter of the blood vessel boundary and the diameter of the lumen boundary obtained from each tomographic image I1.

In the screen 400 of FIG. 19, a first button 401, a second button 402, and a cursor 403 are also displayed on the long axis image I2. When the first button 401 is selected by the input device 5, the cutting direction of the schematic longitudinal section image I3 is changed to the side branch angle of the side branch corresponding to the first button 401.

When the first button 401 or the second button 402 for selecting the location where the side branch is present is selected, as shown in FIG. 19, a tomographic image I4 centered on the center of the side branch may be further generated and displayed for the schematic longitudinal section image I3. Such an image makes it easier for medical personnel to visually check the display device 4 and understand the structure of the blood vessel.

As shown in FIG. 4, since the guidewire W and its reflection range can be recognized, a point in each tomographic image I1 corresponding to the guidewire W (e.g., a point closer to the catheter 1) can be recognized, and an image in which the vascular boundaries are connected with that point as the center can be displayed.

Figure 20 shows another example of a screen displayed on the display device 4. Figure 20 shows an example of a three-dimensional image. Figure 20 shows a screen 400 displayed on the display device 4 in the same way as Figures 16-19. Screen 400 in Figure 20 includes a three-dimensional image I5 of the main trunk and side branches. Three-dimensional image I5 is created by simplifying both the main trunk and side branches into a tubular shape, and by using data on the side branches obtained by the above-mentioned processing to visually grasp the position of the side branches in the longitudinal direction and the angle of the side branches. This allows medical personnel to easily grasp the anatomical characteristics of the blood vessels (side branches).

Since the image processing device 3 stores the side branch detection results in association with the position in the longitudinal direction (S312), it is also possible to use these stored data to display the anatomical characteristics of the blood vessels in a way that makes it easier for medical personnel to grasp them. FIG. 21 shows another example of a screen displayed on the display device 4. In FIG. 21, a curve B3 indicating the lumen boundary detected at each position and a curve B4 indicating the area boundary (vascular boundary) of the main trunk are superimposed on the longitudinal image I2. The curves B3 and B4 may be drawn superimposed on the longitudinal image I2 in a color that is easy to see. The processing unit 30 creates a curved surface in which the curve B1 of the lumen boundary and the curve B2 of the vascular boundary of each tomographic image I1 shown in FIG. 13 are superimposed in the longitudinal direction, and a longitudinal cross-sectional image is created by cutting the curved surface in the longitudinal direction. At a position where a side branch is captured and the area boundary of the main trunk is interpolated, the processing unit 30 connects the interpolation trajectory shown by the dashed line to the curve B2 of the vascular boundary. The same applies to the lumen boundary. The processing unit 30 creates a curved surface by connecting points on the curve B1 and points on the curve B2 with a spline curve in the long axis direction, and creates a longitudinal section at a specific cutting angle from the created curved surface. The boundary curve of the lumen region connecting the points on the curve B1, and the curve connecting the interpolated boundary in some places, are shown as B3, and the boundary curve connecting the points on the curve B2 of the blood vessel boundary and, in some places, the points on the trunk region boundary are shown as B4. The specific cutting angle may be selected by the input device 5. This makes it easier for medical personnel to grasp the anatomical characteristics of the blood vessels across the scanning range.

In this embodiment, the image processing device 3 connected to the catheter 1 generates a tomographic image I1 in almost real time based on a signal from the imaging device 11, judges whether the image shows a side branch, and displays data showing anatomical features on the display device 4. However, each process including the above-described judgment by the image processing device 3 of whether the image shows a side branch may be performed after the fact on the generated tomographic image I1. In other words, the image processing device 3 is not necessarily directly connected to the imaging device 11 of the catheter 1. The image processing device 3 may be a device capable of reading a storage device that stores a signal from the imaging device 11 via a network, such as a server device. In other words, the processing procedure of steps S301-S304 shown in the flowcharts of Figures 6 and 7 may be performed by an existing image diagnostic device, and the processing of steps S305-S315 and the flowchart of Figure 8 may be executed by the image processing device 3 connected to the processing device, and displayed on the display device 4.

In this embodiment, the medical images are described using images obtained by IVUS of the coronary arteries as an example. However, the subject of application is not limited to this, and may be OCT/OFDI, etc., and the luminal organ is not limited to blood vessels.

The embodiments disclosed above are illustrative in all respects and are not limiting. The scope of the present invention is defined by the claims, and includes all modifications within the meaning and scope of the claims.

1 Catheter 11 Imaging device 3 Image processing device (information processing device)
30 Processing unit 31 Memory unit 3P Computer program 3M Trained model 4 Display device 400 Screen 401 First button 402 Second button I1, I4 Tomographic image I2 Long-axis image I3 Schematic longitudinal section image I5 Three-dimensional image

Claims (5)

a computer that acquires a plurality of tomographic images of a hollow organ based on a signal detected by an imaging device provided in a catheter that is inserted into a first tube of the hollow organ having a branch,
performing a detection process for the plurality of tomographic images, the detection process including determining whether or not a second pipe branching from the first pipe exists and calculating an angle of the second pipe;
A computer program that executes a process of performing a noise removal process based on distribution information including the position in the longitudinal axis direction of the tubular organ of a tomographic image that has been determined by the detection process to be an image in which the second tube is captured, and the angle of the second tube in the tomographic image calculated by the detection process. The noise removal process includes:
determining whether or not there is one or more other tomographic images in which the second vessel is determined to be present within a predetermined range from a position in the longitudinal direction of the hollow organ of the tomographic image of the object in which the second vessel is determined to be present;
The computer program according to claim 1 , further comprising a process of removing a result of the detection process corresponding to the target tomographic image as noise when it is determined that there is no other tomographic image. The noise removal process includes:
Executing a clustering process on the distribution information;
The computer program product according to claim 1 , further comprising a process of removing, as noise, a result of the detection process that corresponds to a position or angle that does not belong to a cluster. a computer that acquires a plurality of tomographic images of a tubular organ based on a signal detected by an imaging device provided in a catheter that is inserted into a first tube of the tubular organ having a branch,
performing a detection process for the plurality of tomographic images, the detection process including determining whether or not a second pipe branching from the first pipe exists and calculating an angle of the second pipe;
An information processing method comprising: performing a noise removal process based on distribution information including the position in the longitudinal direction of the tubular organ of a tomographic image that has been determined by the detection process to be an image in which the second tube is captured; and the angle of the second tube in the tomographic image calculated by the detection process. 1. An information processing device for acquiring a plurality of tomographic images of a tubular organ having a branch, the tomographic images being generated based on signals detected by an imaging device provided in a catheter inserted into a first tube of the tubular organ, the information processing device comprising:
a processing unit that performs image processing on the plurality of tomographic images,
The processing unit includes:
performing a detection process for the plurality of tomographic images, the detection process including determining whether or not a second pipe branching from the first pipe exists and calculating an angle of the second pipe;
An information processing device that performs noise removal processing based on distribution information including the position in the longitudinal axis direction of the tubular organ of a tomographic image that has been determined by the detection processing to be an image in which the second tube is captured, and the angle of the second tube in the tomographic image calculated by the detection processing.

Images