KR102732544B1 - Method and device for providing information of m-mode ultrasound images - Google Patents

Abstract

The present invention provides a method for providing information on an M-mode ultrasound image implemented by a processor, comprising the steps of: receiving an M-mode ultrasound image of an object; segmenting a plurality of cardiac slice regions within the M-mode ultrasound image using a segmentation model learned to segment the M-mode ultrasound image into a plurality of cardiac slice regions by inputting the M-mode ultrasound image; and determining measurements for the plurality of segmented cardiac slice regions. The present invention also provides a device using the same.

Description

Method for providing information on M-mode ultrasound images and device for providing information on M-mode ultrasound images using the same {METHOD AND DEVICE FOR PROVIDING INFORMATION OF M-MODE ULTRASOUND IMAGES}

The present invention relates to a method for providing information on an M-mode ultrasound image and a device for providing information on an M-mode ultrasound image using the same.

A cardiac ultrasound examination is performed by projecting ultrasound waves on the three-dimensional structure of the heart in multiple planes to obtain images of the heart and measure hemodynamic variables.

At this time, the medical staff positions the ultrasound probe in a location where it is easy to obtain ultrasound images, so as to obtain multi-faceted images through the anatomical structures around the heart, such as between the ribs, and records the images by finding the appropriate slice through rotation and tilting.

Among these, M-mode ultrasound images are ultrasound images that set a line of interest in the part of the object to be observed and display the internal structure of the tissue included in the line over time.

M-mode ultrasound images of the heart can be primarily used to measure the thickness of tissues including the left ventricle or right ventricle of the heart. At this time, it can be common to measure LVIDd (left ventricle internal dimension diastole) and LVIDs (left ventricle internal dimension systole) of the heart at the end-diastolic and end-systolic sections, respectively. This is because the sealing volume of the left ventricle is the largest at the end-diastolic section, and the sealing volume of the left ventricle is the smallest at the end-systolic section.

Meanwhile, M-mode ultrasound images of objects that move periodically, such as the heart, have periodicity and can be used to derive clinically significant micro-measurements, such as tissue thickness or blood vessel diameter.

However, since the measurement values may vary greatly depending on the skill level of the medical staff, there is a continuous demand for the development of a new information provision system that can derive highly accurate measurement values from M-mode images.

The background technology of the invention has been written to facilitate understanding of the present invention. It should not be understood that the matters described in the background technology of the invention are recognized as prior art.

Meanwhile, in order to solve the above-mentioned problem, the inventors of the present invention attempted to develop an information providing system based on an artificial neural network trained to segment cardiac slice regions for M-mode ultrasound images.

In particular, the inventors of the present invention were able to recognize that, by applying an artificial neural network, high-precision discrimination is possible for a region of the fault that is difficult to distinguish with the naked eye (e.g., the anterior wall of the right ventricle, the posterior wall of the left atrium, etc.), regardless of the type of M-mode (e.g., LA-Ao, or LV).

As a result, the inventors of the present invention developed an information provision system based on an artificial neural network.

Meanwhile, the inventors of the present invention designed the information providing system to automatically determine measurements, such as the aorta diameter or the left atrial diameter, based on the region segmented by the artificial neural network.

At this time, the inventors of the present invention attempted to design a method to determine the measurement value in different ways depending on the presence or absence of ECG (electrocardiogram) data that provides information in determining the measurement value.

More specifically, the information providing system is constructed to automatically determine end-diastole and end-systole measurements within a segmented image when ECG data exists.

At this time, the above information provision system was built to automatically determine each measurement value by utilizing the following three methods when ECG data does not exist.

1) Determine the periodicity.

2) Determine local maxima and local minima.

3) Determine by calculating the gradient.

Accordingly, the inventors of the present invention expected that by providing the above information provision system, it would be possible to obtain M-mode-based measurements even in secondary and tertiary hospitals where it is difficult to obtain ECG data.

Furthermore, the inventors of the present invention expected that by providing a new information provision system, it would be possible to provide highly reliable analysis results for M-mode ultrasound images regardless of the skill level of medical staff.

Accordingly, the problem to be solved by the present invention is to provide a method for providing information on an M-mode ultrasound image, which is configured to classify a plurality of cardiac slice regions from a received M-mode ultrasound image using an artificial neural network-based segmentation model, and to determine measurement values therefrom, and a device using the same.

The tasks of the present invention are not limited to the tasks mentioned above, and other tasks not mentioned will be clearly understood by those skilled in the art from the description below.

In order to solve the above-mentioned problem, a method for providing information on an M-mode ultrasound image according to one embodiment of the present invention is provided. The method comprises:

A method for providing information on an M-mode ultrasound image implemented by a processor, comprising the steps of: receiving an M-mode ultrasound image of an object; segmenting a plurality of cardiac slice regions within the M-mode ultrasound image using a segmentation model learned to segment the M-mode ultrasound image into a plurality of cardiac slice regions by inputting the M-mode ultrasound image; and determining measurements for the plurality of segmented cardiac slice regions.

According to a feature of the present invention, the method may further include, after the segmenting step, the step of receiving an electrocardiogram (ECG) for the subject, and the step of determining end-diastole and end-systole based on the ECG. In this case, the step of determining the measurement value may include the step of determining the measurement value based on the end-diastole and end-systole.

According to another feature of the present invention, the method may further include, after the segmenting step, a step of determining periodicity for a plurality of cardiac slice regions. In this case, the step of determining the measurement value may include a step of determining the measurement value based on the periodicity.

According to another feature of the present invention, the step of determining the periodicity may include the step of calculating the degree of auto-correlation for a plurality of cardiac slice regions, the step of determining a peak based on the degree of auto-correlation, and the step of determining the periodicity based on the peak.

According to another feature of the present invention, the step of determining the periodicity may include the step of calculating the degree of cross-correlation for adjacent regions of a plurality of cardiac slice regions, the step of determining a peak based on the degree of cross-correlation, and the step of determining the periodicity based on the peak.

According to another feature of the present invention, the method may further include, after the segmenting step, a step of determining a plurality of local maxima and a plurality of local minima for a plurality of cardiac slice regions. In this case, the step of determining the measurement value may include a step of determining the measurement value based on the local maxima and the local minima.

According to another feature of the present invention, the method may include, after the segmenting step, the step of calculating a gradient for a plurality of cardiac slice regions, and the step of determining a measurement value based on the gradient.

According to another feature of the present invention, the method may further comprise, after the segmenting step, a step of selectively receiving an ECG for the object.

According to another feature of the present invention, the method may further include, after the receiving step, a step of determining end-diastole and end-systole based on the ECG when the ECG is received. Furthermore, the step of determining the measurement value may include a step of determining the measurement value based on end-diastole and end-systole.

According to another feature of the present invention, the method further comprises, after the segmenting step, a step of determining periodicity for a plurality of cardiac slice regions if an ECG is not received, and the step of determining the measurement may comprise a step of determining the measurement based on the periodicity.

According to another feature of the present invention, the method further comprises, after the segmenting step, a step of determining a plurality of local maxima and a plurality of local minima for a plurality of cardiac slice regions when an ECG is not received, and the step of determining the measurement value may comprise a step of determining the measurement value based on the plurality of local maxima and the plurality of local minima.

According to another feature of the present invention, the method further comprises, after the segmenting step, a step of calculating a gradient for a plurality of cardiac slice regions when an ECG is not received, and the step of determining the measurement value may comprise a step of determining the measurement value based on the gradient.

According to another feature of the present invention, the method may further include, after the step of determining the measurement value, the step of determining an entropy value, and the step of verifying the measurement value based on the entropy value.

According to another feature of the present invention, the plurality of cardiac cross-sectional regions are at least two regions selected from the group consisting of the right ventricle anterior wall, the right ventricle (RV), the anterior wall of the aorta, the aorta, the posterior wall of the aorta, the left atrium (LA), and the posterior wall of the LA, and the measurement may be the aorta diameter or the left atrium diameter.

According to another feature of the present invention, the plurality of cardiac cross-sectional regions are at least two regions selected from the group consisting of the right ventricle (RV) anterior wall, the right ventricle, the interventricular septum (IVS), the left ventricle (LV), and the LV posterior wall, and the measurement may be at least one of the interventricular septum thickness, the LV internal diameter (LVID), and the LV posterior wall thickness.

According to another feature of the present invention, the interventricular septal thickness may include the interventricular septal thickness in diastole and the interventricular septal thickness in systole, the left ventricular medial thickness may include the left ventricular medial thickness in diastole and the left ventricular medial thickness in systole, and the left ventricular posterior wall thickness may include the left ventricular posterior wall thickness in diastole and the left ventricular posterior wall thickness in systole.

According to another feature of the present invention, the method may further include, after the segmenting step, a step of removing a segmented area other than an M-mode area or filling a hole in a segmented area to obtain a plurality of post-processed cardiac tomography areas. In this case, the step of determining a measurement value may include a step of determining a measurement value based on the plurality of post-processed cardiac tomography areas.

In order to solve the problem as described above, a device for providing information on an M-mode ultrasound image according to another embodiment of the present invention is provided. The device includes a communication unit configured to receive an M-mode ultrasound image of an object, and a processor functionally connected to the communication unit. At this time, the processor is configured to segment a plurality of cardiac slice regions within the M-mode ultrasound image using a segmentation model learned to segment an M-mode ultrasound image into a plurality of cardiac slice regions as input, and to determine measurements for the plurality of segmented cardiac slice regions.

According to a feature of the present invention, the communication unit may be further configured to receive an electrocardiogram (ECG) for the object. At this time, the processor may be further configured to determine an end-diastole and an end-systole based on the ECG, and determine a measurement value based on the end-diastole and the end-systole.

According to another feature of the present invention, the processor can be further configured to determine periodicity for a plurality of cardiac slice regions and to determine measurements based on the periodicity.

According to another feature of the present invention, the processor may be further configured to calculate a degree of auto-correlation for a plurality of cardiac slice regions, determine a peak based on the degree of auto-correlation, and determine a periodicity based on the peak.

According to another feature of the present invention, the processor may be further configured to calculate a degree of cross-correlation for adjacent regions of a plurality of cardiac slice regions, determine a peak based on the degree of cross-correlation, and determine a periodicity based on the peak.

According to another feature of the present invention, the processor may be further configured to determine a plurality of local maxima and a plurality of local minima for a plurality of cardiac slice regions, and to determine a measurement value based on the plurality of local maxima and the plurality of local minima.

According to another feature of the present invention, the processor can be further configured to calculate gradients for a plurality of cardiac slice regions and determine measurements based on the gradients.

According to another feature of the present invention, the communication unit may be further configured to selectively receive an ECG for the object.

According to another feature of the present invention, the processor may be further configured to, when an ECG is received, determine end-diastole and end-systole based on the ECG, and determine a measurement value based on the end-diastole and end-systole.

According to another feature of the present invention, when an ECG is not received, the processor can be further configured to determine periodicity for a plurality of cardiac slice regions and determine measurements based on the periodicity.

According to another feature of the present invention, when an ECG is not received, the processor may be further configured to determine a plurality of local maxima and a plurality of local minima for a plurality of cardiac slice regions, and determine a measurement value based on the plurality of local maxima and the plurality of local minima.

According to another feature of the present invention, when an ECG is not received, the processor may be further configured to calculate slopes for a plurality of cardiac slice regions and determine measurements based on the slopes.

According to another feature of the present invention, the processor may be further configured to determine an entropy value and verify the measurement based on the entropy value.

According to another feature of the present invention, the processor may be further configured to remove a segmented area other than an M-mode area, or fill a hole in a segmented area, so as to obtain a plurality of post-processed cardiac slice areas, and determine a measurement value based on the plurality of post-processed cardiac slice areas.

Specific details of other embodiments are included in the detailed description and drawings.

The present invention provides an information providing system for an M-mode ultrasound image based on an artificial neural network configured to segment a cardiac tomography region using an M-mode ultrasound image, thereby providing a highly reliable cardiac ultrasound diagnosis result.

In particular, the present invention provides an information providing system configured to automatically measure measurements based on segmented cardiac tomography regions, thereby enabling faster and more accurate provision of cardiac ultrasound diagnostic results.

That is, the present invention can provide highly reliable analysis results for M-mode ultrasound images regardless of the skill level of medical staff, and can contribute to establishing more accurate decision-making and treatment plans at the image analysis stage.

Furthermore, the present invention provides an information provision system in which the measurement value determination method is different depending on whether or not an ECG is present, thereby making it possible to obtain M-mode-based measurement values even in secondary and tertiary hospitals where it is difficult to obtain ECG data.

The effects according to the present invention are not limited to those exemplified above, and further diverse effects are included in the present specification.

FIG. 1 illustrates a system for providing information on an M-mode ultrasound image using a device for providing information on an M-mode ultrasound image according to one embodiment of the present invention.
FIG. 2a is a block diagram showing the configuration of a medical device according to one embodiment of the present invention.
FIG. 2b is a block diagram showing the configuration of a server for providing information according to one embodiment of the present invention.
FIG. 3 illustrates a procedure of a method for providing information on an M-mode ultrasound image according to one embodiment of the present invention.
FIG. 4 illustrates an example of a procedure for providing information on an M-mode ultrasound image according to one embodiment of the present invention.
FIGS. 5a and 5b exemplarily illustrate a procedure of a method for providing information on an M-mode ultrasound image according to another embodiment of the present invention.
FIG. 6 illustrates an example of a procedure for providing information on an M-mode ultrasound image according to another embodiment of the present invention.
FIG. 7 illustrates an example of a post-correction procedure in a method for providing information on an M-mode ultrasound image according to various embodiments of the present invention.
Figures 8a and 8b illustrate training data of a segmentation model used in various embodiments of the present invention.
FIGS. 8c and 8d illustrate measurements determined in a method for providing information on an M-mode ultrasound image according to various embodiments of the present invention.
Figures 9a to 9f illustrate evaluation results for automatic measurement according to information provision methods according to various embodiments.

The advantages of the invention and the method for achieving them will become apparent by referring to the embodiments described in detail below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and these embodiments are provided only to make the disclosure of the present invention complete and to fully inform those skilled in the art of the scope of the invention, and the present invention is defined only by the scope of the claims.

The shapes, sizes, ratios, angles, numbers, etc. disclosed in the drawings for explaining embodiments of the present invention are exemplary, and therefore the present invention is not limited to the matters illustrated. In addition, in describing the present invention, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the present invention, the detailed description will be omitted. When the terms “includes,” “has,” “consists of,” etc. are used in this specification, other parts may be added unless “only” is used. When a component is expressed in the singular, it includes a case where the plural is included unless there is a specifically explicit description.

When interpreting components, it is interpreted as including the error range even if there is no separate explicit description.

The individual features of the various embodiments of the present invention may be partially or wholly combined or combined with each other, and as can be fully understood by those skilled in the art, various technical connections and operations are possible, and each embodiment may be implemented independently of each other or may be implemented together in a related relationship.

For clarity in the interpretation of this specification, the terms used in this specification are defined below.

The term "subject" as used herein may mean any subject from which information on M-mode ultrasound images is provided. Meanwhile, the subject disclosed in this specification may be any mammal other than a human, but is not limited thereto.

The term "M-mode ultrasound image" as used herein may mean an ultrasound image that sets a line of interest in a part of a subject to be observed and displays the internal structure of a tissue (particularly, a cardiac cross-sectional structure) included in the line over time.

Preferably, the M-mode ultrasound image may be, but is not limited to, an M-mode ultrasound image of the cardiac region of the subject.

According to a feature of the present invention, the M-mode ultrasound image may be an M-mode image for left atrium-aorta (LA-Ao) analysis or left ventricle (LV) analysis.

As used herein, the term "multiple cardiac slice regions" may mean multiple regions of cardiac structure visible on an M-mode image.

According to a feature of the present invention, the plurality of cardiac cross-sectional regions may include at least two regions selected from the group consisting of the right ventricle anterior wall, the right ventricle (RV), the anterior wall of the aorta, the aorta, the posterior wall of the aorta, the left atrium (LA), and the posterior wall of the LA.

According to another feature of the present invention, the plurality of cardiac slice regions may be at least two regions selected from the following regions: the right ventricle anterior wall (RV anterior wall), the right ventricle, the interventricular septum (IVS), the left ventricle (LV), and the LV posterior wall.

As used herein, the term "segmentation model" may be a model configured to take as input an M-mode ultrasound image and output a cardiac slice region.

More specifically, the segmentation model may be a model configured to segment multiple cardiac slice regions by taking an M-mode ultrasound image (in particular, an M-mode region) as input.

For example, the segmentation model may be a model configured to input an M-mode ultrasound image and probabilistically segment and output each of the right ventricle anterior wall, right ventricle, aortic anterior wall, aorta, aortic posterior wall, left atrium, and left atrium posterior wall.

In addition, the segmentation model may be a model configured to input an M-mode ultrasound image and probabilistically segment and output the right ventricle, right ventricle, interventricular septum, left ventricle, and left ventricular posterior wall, respectively.

Meanwhile, the segmentation model may be based on at least one algorithm selected from among DenseNet-121, U-net, VGG net, DenseNet, and FCN (Fully Convolutional Network) with encoder-decoder structure, SegNet, DeconvNet, DNN (deep neural network) such as DeepLAB V3+, Transformer such as Lawin+, SegFormer, Swin, SqueezeNet, Alexnet, ResNet18, MobileNet-v2, GoogLeNet, Resnet-v2, Resnet50, RetinaNet, Resnet101, Inception-v3, HRNet, ResNeXt, EfficientNet. Furthermore, the segmentation model may be an ensemble model based on at least two algorithm models among the above-mentioned algorithms. However, the type of the model is not limited thereto.

As used herein, the term "measurement" may mean a numerical value, such as thickness or diameter, that can be measured from multiple cardiac slice regions.

According to a feature of the present invention, the measurement value may be the aorta diameter or the left atrial diameter.

According to another feature of the present invention, the measurement may be at least one of interventricular septal thickness, left ventricular internal diameter (LVID) and left ventricular posterior wall thickness.

Here, the interventricular septal thickness may include the interventricular septal thickness in diastole and the interventricular septal thickness in systole, and the left ventricular inner thickness may include the left ventricular inner thickness in diastole and the left ventricular inner thickness in systole. Furthermore, the left ventricular posterior wall thickness may include the left ventricular posterior wall thickness in diastole and the left ventricular posterior wall thickness in systole.

According to another feature of the present invention, the measurements can be determined from the segmentation results of multiple cardiac slice regions for M-mode ultrasound images and from an electrocardiogram (ECG).

More specifically, measurements can be determined based on end-diastolic and end-systolic values for cardiac slice regions determined from the ECG.

According to another feature of the present invention, the measurements can be determined solely from the segmentation results of multiple cardiac slice regions for M-mode ultrasound images.

More specifically, measurements can be determined based on the periodicity of the segmented cardiac slice regions.

At this time, the periodicity can be determined based on the degree of auto-correlation for multiple cardiac slice regions and the peak based on the degree of auto-correlation.

However, it is not limited thereto, and periodicity may also be determined based on the degree of cross-correlation between adjacent regions of multiple cardiac slice regions.

According to another feature of the present invention, the measurement value can be determined based on a plurality of local maxima and a plurality of local minima of the segmented cardiac slice region.

According to another feature of the present invention, the measurement value can be determined based on the slope value of the segmented cardiac slice region.

Hereinafter, with reference to FIGS. 1 and 2a and 2b, an information providing system for an M-mode ultrasound image using a device for providing information for an M-mode ultrasound image according to one embodiment of the present invention and a device for providing information for an M-mode ultrasound image will be described.

FIG. 1 illustrates a system for providing information on an M-mode ultrasound image using a device for providing information on an M-mode ultrasound image according to one embodiment of the present invention. FIG. 2a illustrates an exemplary configuration of a medical device for receiving information on an M-mode ultrasound image according to one embodiment of the present invention. FIG. 2b illustrates an exemplary configuration of a device for providing information on an M-mode ultrasound image according to one embodiment of the present invention.

First, referring to FIG. 1, the information providing system (1000) may be a system configured to provide information related to an M-mode ultrasound image based on an M-mode ultrasound image of an object. At this time, the information providing system (1000) may be configured with a medical device (100) that receives information related to an M-mode ultrasound image, an ultrasound image diagnosis device (200) that provides an M-mode ultrasound image, and an information providing server (300) that generates information about an M-mode ultrasound image based on the received M-mode ultrasound image.

First, next, the medical device (100) is an electronic device that provides a user interface for displaying information associated with an M-mode ultrasound image, and may include at least one of a smart phone, a tablet PC (Personal Computer), a laptop, and/or a PC.

The medical device (100) can receive prediction results associated with M-mode ultrasound images of an object from an information providing server (300) and display the received results through a display unit to be described later.

The server (300) for providing information may include a general-purpose computer, laptop, and/or data server, etc., which performs various operations to determine information associated with an M-mode ultrasound image based on an ECG provided from an ECG measuring device (not shown) and an M-mode ultrasound image provided from an ultrasound imaging diagnostic device (200), such as an ultrasound diagnostic device. In this case, the server (300) for providing information may be a device for accessing a web server providing a web page or a mobile web server providing a mobile website, but is not limited thereto.

More specifically, the information providing server (300) receives an M-mode ultrasound image from an ultrasound imaging diagnostic device (200) and segments a heart slice region within the received M-mode ultrasound image. At this time, the information providing server (300) can segment the heart slice region from the M-mode ultrasound image using a prediction model.

Furthermore, the information providing server (300) can determine measurements such as wall thickness, tube diameter, etc. based on the segmented cardiac slice regions and/or received ECG data.

The information providing server (300) can provide determined measurements, segmentation results, etc. to the medical device (100).

Information provided from the information provision server (300) in this way may be provided as a web page through a web browser installed on a medical device (100), or may be provided in the form of an application or program. In various embodiments, such data may be provided in a form included in a platform in a client-server environment.

Next, components of the information providing server (300) of the present invention will be specifically described with reference to FIGS. 2a and 2b.

First, referring to FIG. 2A, the medical device (100) may include a memory interface (110), one or more processors (120), and a peripheral interface (130). Various components within the medical device (100) may be connected by one or more communication buses or signal lines.

The memory interface (110) is connected to the memory (150) and can transmit various data to the processor (120). Here, the memory (150) can include at least one type of storage medium among flash memory type, hard disk type, multimedia card micro type, card type memory (for example, SD or XD memory, etc.), RAM, SRAM, ROM, EEPROM, PROM, network storage, cloud, and blockchain data.

In various embodiments, the memory (150) can store at least one of an operating system (151), a communication module (152), a graphical user interface (GUI) module (153), a sensor processing module (154), a telephone module (155), and an application module (156). Specifically, the operating system (151) can include instructions for processing basic system services and instructions for performing hardware operations. The communication module (152) can communicate with at least one of other devices, computers, and servers. The graphical user interface module (GUI) (153) can process a graphical user interface. The sensor processing module (154) can process sensor-related functions (e.g., processing voice input received using one or more microphones (192)). The telephone module (155) can process telephone-related functions. The application module (156) can perform various functions of the user application, such as electronic messaging, web browsing, media processing, navigation, imaging, and other processing functions. In addition, the medical device (100) can store one or more software applications (156-1, 156-2) (e.g., information providing applications) associated with one type of service in the memory (150).

In various embodiments, the memory (150) may store a digital assistant client module (157) (hereinafter, DA client module), and thus store instructions for performing client-side functions of the digital assistant and various user data (158).

Meanwhile, the DA client module (157) can obtain the user's voice input, text input, touch input, and/or gesture input through various user interfaces (e.g., I/O subsystem (140)) provided in the medical device (100).

Additionally, the DA client module (157) can output data in audiovisual and tactile forms. For example, the DA client module (157) can output data consisting of a combination of at least two or more of voice, sound, notification, text message, menu, graphic, video, animation, and vibration. Additionally, the DA client module (157) can communicate with a digital assistant server (not shown) using a communication subsystem (180).

In various embodiments, the DA client module (157) may collect additional information about the surroundings of the medical device (100) from various sensors, subsystems, and peripheral devices to construct a context associated with the user input. For example, the DA client module (157) may provide context information along with the user input to a digital assistant server to infer the user's intent. Here, the context information that may accompany the user input may include sensor information, such as lighting, ambient noise, ambient temperature, images of the surroundings, video, etc. As another example, the context information may include the physical state of the medical device (100) (e.g., device orientation, device position, device temperature, power level, speed, acceleration, motion pattern, cellular signal strength, etc.). As another example, context information may include information related to the software state of the medical device (100) (e.g., processes running on the medical device (100), installed programs, past and current network activity, background services, error logs, resource usage, etc.).

In various embodiments, the memory (150) may include additional or deleted instructions, and may further include additional configurations other than those illustrated in FIG. 2A of the medical device (100), or may exclude some configurations.

The processor (120) can control the overall operation of the medical device (100) and execute various commands to implement an interface that provides information related to M-mode ultrasound images by driving an application or program stored in the memory (150).

The processor (120) may correspond to a computational unit such as a CPU (Central Processing Unit) or an AP (Application Processor). In addition, the processor (120) may be implemented in the form of an integrated chip (IC) such as a SoC (System on Chip) in which various computational units such as an NPU (Neural Processing Unit) are integrated.

The peripheral interface (130) can be connected to various sensors, subsystems, and peripheral devices to provide data so that the medical device (100) can perform various functions. Here, it can be understood that the function performed by the medical device (100) is performed by the processor (120).

The peripheral interface (130) can receive data from a motion sensor (160), a light sensor (light sensor) (161), and a proximity sensor (162), through which the medical device (100) can perform orientation, light, and proximity detection functions, etc. For another example, the peripheral interface (130) can receive data from other sensors (163) (positioning system-GPS receiver, temperature sensor, biometric sensor), through which the medical device (100) can perform functions related to the other sensors (163).

In various embodiments, the medical device (100) may include a camera subsystem (170) connected to a peripheral interface (130) and an optical sensor (171) connected thereto, which may enable the medical device (100) to perform various photographic functions, such as taking photographs and recording video clips.

In various embodiments, the medical device (100) may include a communication subsystem (180) coupled with a peripheral interface (130). The communication subsystem (180) may be comprised of one or more wired/wireless networks and may include various communication ports, radio frequency transceivers, and optical transceivers.

In various embodiments, the medical device (100) includes an audio subsystem (190) coupled to the peripheral interface (130), the audio subsystem (190) including one or more speakers (191) and one or more microphones (192), such that the medical device (100) can perform voice-activated functions, such as speech recognition, voice replication, digital recording, and telephony functions.

In various embodiments, the medical device (100) may include an I/O subsystem (140) coupled with a peripheral interface (130). For example, the I/O subsystem (140) may control a touch screen (143) included in the medical device (100) via a touch screen controller (141). As an example, the touch screen controller (141) may detect a user's contact and movement or cessation of contact and movement using any one of a number of touch sensing technologies, such as capacitive, resistive, infrared, surface acoustic wave technology, proximity sensor array, and the like. As another example, the I/O subsystem (140) may control other input/control devices (144) included in the medical device (100) via other input controller(s) (142). As an example, the other input controller(s) (142) may control one or more buttons, rocker switches, thumb-wheels, infrared ports, USB ports, and pointer devices such as a stylus.

Next, referring to FIG. 2b, the information providing server (300) may include a communication interface (310), a memory (320), an I/O interface (330), and a processor (340), each component of which may communicate with each other through one or more communication buses or signal lines.

The communication interface (310) can be connected to a medical device (100), an ultrasound imaging diagnosis device (200), and an ECG measurement device (not shown) via a wired/wireless communication network to exchange data. For example, the communication interface (310) can receive an M-mode ultrasound image from the ultrasound imaging diagnosis device (200), an ECG from an ECG measurement device (not shown), and transmit information related to measurements determined therefrom to the medical device (100).

Meanwhile, a communication interface (310) that enables transmission and reception of such data includes a communication port (311) and a wireless circuit (312), wherein the wired communication port (311) may include one or more wired interfaces, for example, Ethernet, Universal Serial Bus (USB), FireWire, etc. In addition, the wireless circuit (312) may transmit and receive data with an external device via an RF signal or an optical signal. In addition, the wireless communication may use at least one of a plurality of communication standards, protocols, and technologies, for example, GSM, EDGE, CDMA, TDMA, Bluetooth, Wi-Fi, VoIP, Wi-MAX, or any other suitable communication protocol.

The memory (320) can store various data used in the information providing server (300). For example, the memory (320) can store an M-mode ultrasound image or a segmentation model learned to segment a cardiac slice region within an M-mode ultrasound image.

In various embodiments, the memory (320) may include a volatile or nonvolatile storage medium capable of storing various data, commands, and information. For example, the memory (320) may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., an SD or XD memory, etc.), a RAM, an SRAM, a ROM, an EEPROM, a PROM, a network storage storage, a cloud, and blockchain data.

In various embodiments, the memory (320) may store a configuration of at least one of an operating system (321), a communication module (322), a user interface module (323), and one or more applications (324).

An operating system (321) (e.g., an embedded operating system such as LINUX, UNIX, MAC OS, WINDOWS, VxWorks, etc.) may include various software components and drivers to control and manage general system operations (e.g., memory management, storage device control, power management, etc.) and may support communication between various hardware, firmware, and software components.

The communication module (323) can support communication with other devices through the communication interface (310). The communication module (320) can include various software components for processing data received by the wired communication port (311) or wireless circuit (312) of the communication interface (310).

The user interface module (323) can receive a user's request or input from a keyboard, touch screen, microphone, etc. through an I/O interface (330) and provide a user interface on the display.

The application (324) may include a program or module configured to be executed by one or more processors (330). Here, the application for providing information associated with M-mode ultrasound images may be implemented on a server farm.

The I/O interface (330) can connect at least one of an input/output device (not shown) of the information providing server (300), such as a display, a keyboard, a touch screen, and a microphone, to the user interface module (323). The I/O interface (330) can receive user input (e.g., voice input, keyboard input, touch input, etc.) together with the user interface module (323) and process a command according to the received input.

The processor (340) is connected to a communication interface (310), a memory (320), and an I/O interface (330) to control the overall operation of the information providing server (300), and can perform various commands for providing information through an application or program stored in the memory (320).

The processor (340) may correspond to a computational device such as a CPU (Central Processing Unit) or an AP (Application Processor). In addition, the processor (340) may be implemented in the form of an integrated chip (Integrated Chip (IC)) such as a SoC (System on Chip) in which various computational devices are integrated. Alternatively, the processor (340) may include a module for calculating an artificial neural network model such as an NPU (Neural Processing Unit).

In various embodiments, the processor (340) can be configured to segment and provide cardiac slice regions within an M-mode ultrasound image using the predictive models. Optionally, the processor (340) can be configured to provide information about measurements obtainable from the M-mode ultrasound image.

Hereinafter, a method for providing information on an M-mode ultrasound image according to one embodiment of the present invention will be specifically described with reference to FIGS. 3 to 6.

FIG. 3 illustrates a procedure of a method for providing information on an M-mode ultrasound image according to one embodiment of the present invention. FIG. 4 illustrates an example of a procedure of a method for providing information on an M-mode ultrasound image according to one embodiment of the present invention. FIGS. 5a and 5b illustrate an example of a procedure of a method for providing information on an M-mode ultrasound image according to another embodiment of the present invention. FIG. 6 illustrates an example of a procedure of a method for providing information on an M-mode ultrasound image according to still another embodiment of the present invention.

First, referring to FIG. 3, the information providing procedure according to one embodiment of the present invention is as follows. First, an M-mode ultrasound image of an object is received (S310). Then, a plurality of heart slice regions are segmented by a segmentation model (S320). Then, a measurement value is determined based on the segmented heart slice regions (S330).

More specifically, in the step (S310) where an M-mode ultrasound image is received, an M-mode ultrasound image of the target area, i.e., the heart area, can be received.

According to a feature of the present invention, in the step (S310) of receiving an M-mode ultrasound image, an M-mode ultrasound image with an M-mode region cropped can be received. That is, before the step (S310) of receiving an M-mode ultrasound image, cropping of an essential region for segmentation of the M-mode ultrasound image can be performed.

Next, a step (S320) in which the heart section area is segmented is performed.

According to a feature of the present invention, in the step (S320) of segmenting a cardiac tomography region, at least two regions selected from among the right ventricle anterior wall, the right ventricle (RV), the anterior wall of the aorta, the aorta, the posterior wall of the aorta, the left atrium (LA), and the posterior wall of the left atrium (LA) are segmented in an M-mode ultrasound image by a segmentation model.

According to another feature of the present invention, in the step (S320) of segmenting a cardiac slice region, at least two regions selected from among the right ventricle anterior wall (RV anterior wall), the right ventricle, the interventricular septum (IVS), the left ventricle (LV), and the left ventricle posterior wall (LV posterior wall) are segmented by a segmentation model.

However, it is not limited to this, and more diverse areas may be divided depending on the type of M-mode.

According to another feature of the present invention, after the step (S320) of segmenting the cardiac tomography region, further post-processing can be performed.

For example, referring to (a) and (b) of FIG. 7, further post-processing may be performed, such as removing a segmented area other than the M-mode area or filling a hole within the segmented area.

Meanwhile, according to another feature of the present invention, after the step (S320) of segmenting a cardiac tomography region, an electrocardiogram (ECG) for the subject is received, and end-diastole and end-systole can be determined based on the ECG.

Next, in the step (S330) where the measurement value is determined, the measurement value is determined based on the end-diastolic and end-systolic values.

For example, referring to FIG. 4 together, in the step (S320) where a cardiac slice region is segmented, when an M-mode ultrasound image (412) is input into a segmentation model (420), a plurality of cardiac slice regions (422) are output. Then, an electrocardiogram (ECG) for an individual is received, and end-diastole (EDd) and end-systole (EDs) are determined for the plurality of cardiac slice regions (422) based on the ECG. Then, in the step (S330) where a measurement is determined, a measurement (e.g., aorta diameter, left atrial diameter (LA diameter), interventricular septal thickness, LV internal diameter (LVID), or left ventricular posterior wall thickness) is automatically determined based on the end-diastole and end-systole.

However, the determination of measurements is not limited to this and may be determined solely from segmented cardiac slice regions.

According to another feature of the present invention, after the step (S320) of segmenting the cardiac tomography regions, periodicity for a plurality of cardiac tomography regions is determined.

Here, the periodicity can be determined based on the degree of auto-correlation for multiple cardiac slice regions, the degree of auto-correlation is calculated, and the peak based on the degree of auto-correlation is determined.

However, it is not limited thereto, and the periodicity can be determined by calculating the degree of cross-correlation for adjacent regions of multiple cardiac slice regions and determining a peak based on the degree of cross-correlation.

Next, in the step (S330) where the measurement value is determined, the measurement value is determined based on the periodicity of the cardiac slice area determined.

For example, referring to FIG. 5A together, in the step (S320) where a heart slice region is segmented, when an M-mode ultrasound image (412) is input to a segmentation model (420), a plurality of heart slice regions (422) are output. Then, the autocorrelation of each of the plurality of heart slice regions (422) is calculated and then a peak is detected. As a result, the periodicity (436) can be detected. That is, the measurement value (442) can be determined based on the periodicity (436) even without determining the end-diastole and end-systole based on the ECG.

Optionally, the measurement may be determined by determining multiple local maxima and multiple local minima (437) for the segmented region, or by calculating the gradient (438) for the segmented region.

That is, the determination of the measurement value can be performed by at least one of the processes of determining periodicity, determining maximum and minimum values, and calculating the slope.

For example, referring further to Fig. 5b, after the gradient is calculated, a specific point whose gradient is close to 0 is determined, and then the determined specific point can be clustered.

At this time, in the step (S330) where the measurement value is determined, the measurement value (e.g., aorta diameter, left atrial diameter (LA diameter), interventricular septal thickness, left ventricular internal diameter (LVID), or left ventricular posterior wall thickness) is automatically determined based on at least one of the determined peak, maximum value, minimum value, and slope.

Therefore, when it is difficult to obtain ECG data, automatic determination of measurements may be possible using only M-mode ultrasound images.

In other words, the present invention provides an information provision system in which the measurement value determination method is different depending on whether or not an ECG is present, thereby making it possible to obtain M-mode-based measurement values even in secondary and tertiary hospitals where it is difficult to obtain ECG data.

Meanwhile, according to a feature of the present invention, after the step (S330) in which the measurement value is determined, an entropy value is determined, and the measurement value can be verified based on the entropy value.

For example, a measurement may be excluded if its entropy value corresponding to uncertainty is above a predetermined level.

Accordingly, medical staff can easily identify measurement values from acquired M-mode ultrasound images according to the information providing method according to various embodiments of the present invention, thereby quickly proceeding with the ultrasound analysis step and establishing more accurate decision-making and treatment plans.

Meanwhile, the information providing method according to various embodiments of the present invention is not limited to the above-described procedure.

For example, referring to FIG. 6, a step (S630) of selectively receiving ECG data is performed, and when an ECG is received, a step (S6302) of determining end-diastole and end-systole based on the ECG is performed, and a measurement value is determined based on this (S640). Meanwhile, when an ECG is not received, a step (S6304) of determining periodicity for a plurality of cardiac slice regions is performed, and a measurement value is determined based on the periodicity (S640).

That is, depending on whether ECG data is received or not, more diverse measurement decision procedures can be performed.

Hereinafter, with reference to FIGS. 8a to 8d, the segmentation model and measurements used in various embodiments of the present invention will be described.

FIGS. 8A and 8B illustrate training data of a segmentation model used in various embodiments of the present invention. FIGS. 8C and 8D illustrate measurements determined in a method for providing information on an M-mode ultrasound image according to various embodiments of the present invention.

First, referring to Fig. 8a, the segmentation model can use the M-mode ultrasound image for training, in which seven regions, namely the anterior wall of the right ventricle, the anterior wall of the right ventricle, the anterior wall of the aorta, the aorta, the posterior wall of the aorta, the left atrium, and the posterior wall of the left atrium, are each labeled.

That is, the segmentation model may be a model trained to segment the above region for left atrium-aorta (LA-Ao) analysis.

Next, referring further to Fig. 8b, the segmentation model can use the M-mode ultrasound images for training, in which five regions, namely the right ventricular anterior wall, right ventricle, midventricular wall, left ventricle, and left ventricular posterior wall, are each labeled.

That is, the segmentation model may be a model trained to segment the above region for left ventricle (LV) analysis.

Meanwhile, M-mode images for learning are not limited to this, and the labeling area may be different depending on the area to be segmented and the analysis goal.

Next, referring to Figure 8c, the measurements may be left atrial thickness corresponding to systole and aortic thickness corresponding to end diastole.

Referring further to FIG. 8d, the measurements may be interventricular septum (IVS) thickness, left ventricular internal diameter (LVID) and left ventricular posterior wall (LVPW) thickness corresponding to end systole and end diastolic, respectively.

However, the measurements are not limited to those described above and can be set more diversely depending on the analysis objectives.

Evaluation 1: Measurement Evaluation Results

Hereinafter, with reference to FIGS. 9a to 9f, evaluation results for measurements determined according to information providing methods according to various embodiments of the present invention will be described.

First, referring to FIG. 9a, in left atrial-aortic analysis, correlation coefficients of measurements manually determined by an expert and measurements automatically determined according to an information provision method according to various embodiments of the present invention are illustrated.

More specifically, referring to Figures 9b and 9c together, the correlation coefficient for the measurements of left atrial thickness is 0.91, and the correlation coefficient for the measurements of aortic thickness is 0.97, showing a high correlation with the expert's measurements.

These results may imply that the clinical reliability of measurements determined according to the information provision method according to various embodiments of the present invention is high.

Next, referring to FIG. 9d , in left ventricular analysis, correlation coefficients of measurements manually determined by an expert and measurements automatically determined according to an information provision method according to various embodiments of the present invention are illustrated.

More specifically, referring to FIGS. 9e and 9f together, the correlation coefficient for the end-diastolic interventricular septum diameter is 0.93, for the end-diastolic LV internal diameter is 0.95, and for the end-diastolic LV posterior wall diameter is 0.81.

Furthermore, the correlation coefficient for end-systolic interventricular septal thickness was 0.80, for end-systolic left ventricular medial thickness was 0.97, and for end-systolic left ventricular posterior wall thickness was 0.85.

These results may imply that the measurements determined by the information provision method according to various embodiments of the present invention have a high correlation with the measurements of experts, and furthermore, have high clinical reliability.

That is, the present invention can provide highly reliable analysis results for M-mode ultrasound images regardless of the skill level of medical staff, and can contribute to establishing more accurate decision-making and treatment plans at the image analysis stage.

Furthermore, the present invention provides an information provision system in which the measurement value determination method is different depending on whether or not an ECG is present, thereby making it possible to obtain M-mode-based measurement values even in secondary and tertiary hospitals where it is difficult to obtain ECG data.

Although the embodiments of the present invention have been described in more detail with reference to the attached drawings, the present invention is not necessarily limited to these embodiments, and various modifications may be made without departing from the technical idea of the present invention. Accordingly, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention, but to explain it, and the scope of the technical idea of the present invention is not limited by these embodiments. Therefore, it should be understood that the embodiments described above are exemplary in all aspects and not restrictive. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of the rights of the present invention.

100: Medical Devices
110: Memory Interface 120: Processor
130: Peripheral Interface 140: I/O Subsystem
141: Touch screen controller 142: Other input controllers
143: Touch screen
144: Other input control devices
150: Memory 151: Operating System
152: Communication Module 153: GUI Module
154: Sensor processing module 155: Phone module
156: Applications
156-1, 156-2: Applications
157: Digital Assistant Client Module
158: User Data
160: Motion sensor 161: Light sensor
162: Proximity sensor 163: Other sensors
170: Camera subsystem 171: Optical sensor
180: Communication Subsystem
190: Audio Subsystem
191: Speaker 192: Microphone
300: Server for information provision
310: Communication Interface
311: Wired communication port 312: Wireless circuit
320: Memory
321: Operating System 322: Communication Module
323: User Interface Module 324: Application
330: I/O interface 340: Processor

Claims (25)

A method for providing information on an M-mode ultrasound image implemented by a processor,
A step of receiving an M-mode ultrasound image of an object;
A step of segmenting a plurality of cardiac slice regions within the M-mode ultrasound image using a segmentation model learned to segment the M-mode ultrasound image into a plurality of cardiac slice regions as input;
A step of removing a segmented area other than the M-mode area or filling a hole within the segmented area to obtain multiple post-processed cardiac cross-section areas, and
A method for providing information on an M-mode ultrasound image, comprising the step of determining measurements for a plurality of post-processed cardiac slice regions. In the first paragraph,
After the above dividing step,
A step of receiving an ECG (electrocardiogram) for the above object, and
Further comprising a step of determining the end-diastole and end-systole based on the above ECG,
The steps for determining the above measurements are:
A method for providing information on an M-mode ultrasound image, comprising the step of determining the measurement based on the end-diastolic and end-systolic values. In the first paragraph,
After the above dividing step,
Further comprising a step of determining periodicity for the plurality of post-processed cardiac tomography regions;
The steps for determining the above measurements are:
A method for providing information on an M-mode ultrasound image, comprising the step of determining the measurement value based on the periodicity. In the third paragraph,
The step of determining the above periodicity is:
A step of calculating the degree of auto-correlation for the above-mentioned multiple post-processed cardiac slice regions;
A step of determining a peak based on the above degree of autocorrelation, and
A method for providing information on an M-mode ultrasound image, comprising the step of determining periodicity based on the above peak. In the third paragraph,
The step of determining the above periodicity is:
A step of calculating the degree of cross-correlation for adjacent regions of the above-mentioned multiple post-processed cardiac tomography regions;
A step of determining a peak based on the above degree of cross-correlation, and
A method for providing information on an M-mode ultrasound image, comprising the step of determining periodicity based on the above peak. In the first paragraph,
After the above dividing step,
Further comprising a step of determining multiple local maxima and multiple local minima for the multiple post-processed cardiac tomography regions,
The steps for determining the above measurements are:
A method for providing information on an M-mode ultrasound image, comprising the step of determining the measurement value based on the plurality of maximum values and the plurality of minimum values. In the first paragraph,
After the above dividing step,
Further comprising the step of determining a slope for the plurality of post-processed cardiac tomography regions,
The steps for determining the above measurements are:
A method for providing information on an M-mode ultrasound image, comprising the step of determining the measurement value based on the slope. In the first paragraph,
After the above dividing step,
A method for providing information on M-mode ultrasound images, further comprising the step of selectively receiving an ECG for said object. In Article 8,
When the above ECG is received,
After the above receiving step,
Further comprising a step of determining the end-diastole and end-systole based on the above ECG,
The steps for determining the above measurements are:
A method for providing information on an M-mode ultrasound image, comprising the step of determining the measurement based on the end-diastolic and end-systolic values. In Article 8,
If the above ECG is not received,
After the above dividing step,
Further comprising a step of determining periodicity for the plurality of post-processed cardiac tomography regions;
The steps for determining the above measurements are:
A method for providing information on an M-mode ultrasound image, comprising the step of determining the measurement value based on the periodicity. In the first paragraph,
After the step of determining the above measurement value,
Steps for determining the entropy value, and
A method for providing information for diagnosis of an M-mode ultrasound image, further comprising a step of verifying the measurement value based on the entropy value. In the first paragraph,
The above-described post-processed multiple cardiac cross-sectional regions are at least two regions selected from the right ventricle anterior wall, right ventricle (RV), anterior wall of aorta, aorta, posterior wall of aorta, left atrium (LA), and posterior wall of LA.
The above measurements are,
A method for providing information for diagnosis of M-mode ultrasound images, such as aorta diameter or left atrial diameter. In the first paragraph,
The above-mentioned post-processed multiple cardiac tomography regions are at least two regions selected from the right ventricle anterior wall (RV anterior wall), right ventricle, interventricular septum (IVS), left ventricle (LV), and LV posterior wall,
The above measurements are,
A method for providing information on an M-mode ultrasound image, at least one of interventricular septal thickness, left ventricular internal diameter (LVID), and left ventricular posterior wall thickness. In Article 13,
The above ventricular septal thickness includes the ventricular septal thickness in diastole and the ventricular septal thickness in systole.
The above left ventricular inner thickness includes the left ventricular inner thickness in diastole and the left ventricular inner thickness in systole.
A method for providing information on M-mode ultrasound images, wherein the left ventricular posterior wall thickness includes the left ventricular posterior wall thickness in diastole and the left ventricular posterior wall thickness in systole. delete A communication unit configured to receive an M-mode ultrasound image of an object, and
Including a processor functionally connected to the above communication unit,
The above processor,
Using a segmentation model learned to segment the M-mode ultrasound image into multiple cardiac slice regions as input, multiple cardiac slice regions are segmented within the M-mode ultrasound image,
To obtain multiple post-processed cardiac slice regions, remove segmented regions other than the M-mode region or fill holes within the segmented region.
A device for providing information on M-mode ultrasound images, configured to determine measurements for multiple post-processed cardiac slice regions. In Article 16,
The above communication department,
Further configured to receive an ECG (electrocardiogram) for said entity,
The above processor,
Based on the above ECG, end-diastole and end-systole are determined,
A device for providing information on M-mode ultrasound images, further configured to determine said measurements based on said end-diastolic and said end-systolic values. In Article 16,
The above processor,
Determine the periodicity for the multiple post-processed cardiac tomography regions,
A device for providing information on M-mode ultrasound images, further configured to determine said measurement based on said periodicity. In Article 18,
The above processor,
Calculate the degree of auto-correlation for the multiple post-processed cardiac slice regions,
Based on the above autocorrelation degree, the peak is determined,
A device for providing information on M-mode ultrasound images, further configured to determine periodicity based on said peak. In Article 18,
The above processor,
The degree of cross-correlation between adjacent regions of the above-mentioned post-processed multiple cardiac tomography regions is calculated,
Determine the peak based on the above degree of cross-correlation,
A device for providing information on M-mode ultrasound images, further configured to determine periodicity based on said peak. In Article 16,
The above communication department,
A device for providing information on M-mode ultrasound images, further configured to selectively receive an ECG for said entity. In Article 21,
The above processor,
When the above ECG is received,
Based on the above ECG, end-diastole and end-systole are determined,
A device for providing information on M-mode ultrasound images, further configured to determine said measurements based on said end-diastolic and said end-systolic values. In Article 21,
If the above ECG is not received,
The above processor,
Determine the periodicity for the multiple post-processed cardiac tomography regions,
A device for providing information on M-mode ultrasound images, further configured to determine said measurement based on said periodicity. In Article 16,
The above processor,
Determine the entropy value,
A device for providing information on an M-mode ultrasound image, further configured to verify the measurement based on the entropy value. delete