[go: up one dir, main page]

WO2025090750A1 - Interface utilisateur graphique ayant des indicateurs spatiaux dynamiques d'images dans une séquence - Google Patents

Interface utilisateur graphique ayant des indicateurs spatiaux dynamiques d'images dans une séquence Download PDF

Info

Publication number
WO2025090750A1
WO2025090750A1 PCT/US2024/052793 US2024052793W WO2025090750A1 WO 2025090750 A1 WO2025090750 A1 WO 2025090750A1 US 2024052793 W US2024052793 W US 2024052793W WO 2025090750 A1 WO2025090750 A1 WO 2025090750A1
Authority
WO
WIPO (PCT)
Prior art keywords
setting
artificial intelligence
images
user interface
indications
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/US2024/052793
Other languages
English (en)
Inventor
David W. Blue
Ha Hong
Ryan S. SOHLDEN
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Covidien LP
Original Assignee
Covidien LP
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Covidien LP filed Critical Covidien LP
Publication of WO2025090750A1 publication Critical patent/WO2025090750A1/fr
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A61B1/000096—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope using artificial intelligence
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/041—Capsule endoscopes for imaging
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/30—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for calculating health indices; for individual health risk assessment
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10068—Endoscopic image

Definitions

  • the present disclosure relates to endoscopy images, and more particularly, to a graphic user interface for viewing endoscopy images.
  • Endoscopy allows examining of a gastrointestinal tract (GIT) endoscopically.
  • GIT gastrointestinal tract
  • capsule endoscopy systems and methods that are aimed at examining a specific portion of the GIT, such as the small bowel (SB) or the colon.
  • SB small bowel
  • CE is a non-invasive procedure which does not require the patient to be admitted to a hospital, and the patient can continue most daily activities while the capsule is in his body.
  • the patient is referred to a procedure by a physician.
  • the patient arrives at a medical facility (e.g., a clinic or a hospital), to perform the procedure.
  • the capsule which is about the size of a multi-vitamin, is swallowed by the patient under the supervision of a healthcare professional (e.g., a nurse or a physician) at the medical facility and the patient is provided with a wearable device (e.g., a belt having a recorder in a pouch and a strap to be placed around the patient’s shoulder).
  • the wearable device typically includes a storage device. The patient may be given guidance and/or instructions and then is released to his or her daily activities.
  • the capsule captures images as it travels naturally through the GIT. Images and additional data (e.g., metadata) are then transmitted to the recorder that is worn by the patient.
  • the capsule is typically disposable and passes naturally with a bowel movement.
  • the procedure data (e.g., the captured images or a portion of them and additional metadata) is stored in the storage device of the wearable device.
  • the procedure data is uploaded from the wearable device to a computing system, which has a software engine stored thereon.
  • the procedure data is then processed by the software engine to generate a compiled study.
  • the number of images in the procedure data to be processed is of the order of tens of thousands, and the generated study typically includes thousands of images.
  • a reader (which may be the procedure supervising physician, a dedicated physician, or the referring physician) may access the study via a reader application. The reader then reviews the study, evaluates the procedure, and provides input via the reader application. Since the reader needs to review thousands of images, the reading time of a study may usually take between half an hour to an hour on average and the reading task may be tiresome. A report is then generated by the reader application based on the compiled study and the reader’s input. On average, it may take an hour to generate a report.
  • the report may include, for example, images of interest (e.g., images which are identified as including pathologies) selected by the reader; evaluation or diagnosis of the patient’s medical condition based on the procedure’s data (i.e., the study), and/or recommendations for follow up and/or treatment provided by the reader.
  • the report may then be forwarded to a referring physician.
  • the referring physician may decide on a required follow up or treatment based on the report.
  • a system for presenting in-vivo studies includes: at least one processor and at least one memory storing instructions. The instructions, when executed by the at least one processor, cause the system at least to perform: displaying, on a screen, a graphical user interface configured to present at least a portion of an in- vivo study, where the in-vivo study includes a sequence of in-vivo images of at least a portion of a gastrointestinal tract (GIT) of a person, where the graphical user interface includes: a user interface element configured to be engaged by a user to adjust a setting for an artificial intelligence detector, and a graphical spatial representation of an entirety of the sequence of in-vivo images; receiving a setting for the artificial intelligence detector based on a user engagement of the user interface element; and displaying a plurality of indications in spatial relation to the graphical spatial representation, where the spatial relation corresponds to locations of
  • the user interface element includes at least three selectable settings, and the setting received for the artificial intelligence detector is one of the at least three selectable settings.
  • the at least three selectable settings include a first selectable setting and a second selectable setting.
  • the first selectable setting sets a lower threshold than the second selectable setting for classifying an object in an image as an indicator of a pathology, resulting in a greater number of indications in the plurality of indications.
  • the second selectable setting sets a higher threshold than the first selectable setting for classifying an object in an image as an indicator of a pathology, resulting in a smaller number of indications in the plurality of indications.
  • the artificial intelligence detector classifies an object in an image as an indicator of a pathology based on at least one classification score.
  • the graphical spatial representation of the entirety of the sequence of in-vivo images is a bar
  • the plurality of indications are lines in the bar, where the lines are brighter than a brightness of the bar.
  • the instructions when executed by the at least one processor, further cause the system at least to perform: transmitting, to a server, the setting received for the artificial intelligence detector, where the server is configured to execute the artificial intelligence detector; and receiving, from the server, information for generating the plurality of indications.
  • the instructions when executed by the at least one processor, further cause the system at least to perform: in a case the user interface element is set to a highest certainty setting for the artificial intelligence detector, displaying surrounding images for the images, of the sequence of in-vivo images, which are selected based on the setting for the artificial intelligence detector.
  • a method for presenting in-vivo studies includes: displaying, on a screen, a graphical user interface configured to present at least a portion of an in-vivo study, where the in-vivo study includes a sequence of in-vivo images of at least a portion of a gastrointestinal tract (GIT) of a person, where the graphical user interface includes: a user interface element configured to be engaged by a user to adjust a setting for an artificial intelligence detector, and a graphical spatial representation of an entirety of the sequence of in-vivo images; receiving a setting for the artificial intelligence detector based on a user engagement of the user interface element; and displaying a plurality of indications in spatial relation to the graphical spatial representation, where the spatial relation corresponds to locations of images, of the sequence of in-vivo images, which are selected based on the setting for the artificial intelligence detector.
  • GIT gastrointestinal tract
  • the user interface element includes at least three selectable settings, and the setting received for the artificial intelligence detector is one of the at least three selectable settings.
  • the at least three selectable settings include a first selectable setting and a second selectable setting.
  • the first selectable setting sets a lower threshold than the second selectable setting for classifying an object in an image as an indicator of a pathology, resulting in a greater number of indications in the plurality of indications.
  • the second selectable setting sets a higher threshold than the first selectable setting for classifying an object in an image as an indicator of a pathology, resulting in a smaller number of indications in the plurality of indications.
  • the graphical spatial representation of the entirety of the sequence of in-vivo images is a bar
  • the plurality of indications are lines in the bar, where the lines are brighter than a brightness of the bar.
  • the method further includes: transmitting, to a server, the setting received for the artificial intelligence detector, where the server is configured to execute the artificial intelligence detector; and receiving, from the server, information for generating the plurality of indications.
  • the method further includes: in a case the user interface element is set to a highest certainty setting for the artificial intelligence detector, displaying surrounding images for the images, of the sequence of in-vivo images, which are selected based on the setting for the artificial intelligence detector.
  • a processor-readable medium stores instructions which, when executed by at least one processor of a system, cause the system at least to perform: displaying, on a screen, a graphical user interface configured to present at least a portion of an in-vivo study, where the in-vivo study includes a sequence of in-vivo images of at least a portion of a gastrointestinal tract (GIT) of a person, where the graphical user interface includes: a user interface element configured to be engaged by a user to adjust a setting for an artificial intelligence detector, and a graphical spatial representation of an entirety of the sequence of in-vivo images; receiving a setting for the artificial intelligence detector based on a user engagement of the user interface element; and displaying a plurality of indications in spatial relation to the graphical spatial representation, where the spatial relation corresponds to images, of the sequence of in-vivo images, which are selected based on the setting for the artificial intelligence detector.
  • GIT gastrointestinal tract
  • the user interface element includes at least three selectable settings, and the setting received for the artificial intelligence detector is one of the at least three selectable settings.
  • the at least three selectable settings include a first selectable setting and a second selectable setting.
  • the first selectable setting sets a lower threshold than the second selectable setting for classifying an object in an image as an indicator of a pathology, resulting in a greater number of indications in the plurality of indications.
  • the second selectable setting sets a higher threshold than the first selectable setting for classifying an object in an image as an indicator of a pathology, resulting in a smaller number of indications in the plurality of indications.
  • the artificial intelligence detector classifies an object in an image as an indicator of a pathology based on at least one classification score.
  • the graphical spatial representation of the entirety of the sequence of in-vivo images is a bar
  • the plurality of indications are lines in the bar, where the lines are brighter than a brightness of the bar.
  • the instructions when executed by the at least one processor, further cause the system at least to perform: transmitting, to a server, the setting received for the artificial intelligence detector, where the server is configured to execute the artificial intelligence detector; and receiving, from the server, information for generating the plurality of indications.
  • the instructions when executed by the at least one processor, further cause the system at least to perform: in a case the user interface element is set to a highest certainty setting for the artificial intelligence detector, displaying surrounding images for the images, of the sequence of in-vivo images, which are selected based on the setting for the artificial intelligence detector.
  • FIG. l is a diagram of a gastrointestinal tract (GIT), in accordance with aspects of the present disclosure
  • FIG. 2 is a block diagram of an example of a system for analyzing medical images captured in-vivo via a capsule endoscopy (CE) procedure and for providing the images to a user system or device, in accordance with aspects of the disclosure;
  • CE capsule endoscopy
  • FIG. 3 is a block diagram of an example of components of a computing system or of a user system, in accordance with aspects of the disclosure
  • FIG. 4 is a diagram of an example of a graphical user interface having one setting of a user interface element, in accordance with aspects of the disclosure
  • FIG. 6 is a diagram of the graphical user interface having yet another setting of the user interface element, in accordance with aspects of the disclosure.
  • FIG. 7 is a diagram of another example of a graphical user interface having a setting of the user interface element, in accordance with aspects of the disclosure;
  • FIG. 8 is a diagram of the graphical user interface of FIG. 8 having another setting of the user interface element, in accordance with aspects of the disclosure.
  • FIG. 9 is a flow diagram of an example of an operation, in accordance with aspects of the disclosure.
  • GUI graphic user interface
  • a healthcare profession reads an in-vivo study to look for indications of a pathology, such as bleeding, polyps, inflammation, Crohn’s disease, Celiac disease, and/or diverticulitis, among other pathologies.
  • Machine learning has come a long way in being able to detect one or more such pathologies. For example, detection of indicators of Celiac disease using machine learning is disclosed in International Publication No. WO2022107124, which is hereby incorporated by reference herein in its entirety.
  • aspects of the present disclosure relate to a GUI that allows a user to adjust one or more settings of an artificial intelligence detector and that graphically indicates which images of an in-vivo study are selected based on the settings.
  • the artificial intelligence settings may be changed to set a lower threshold or a higher threshold for detecting pathologies.
  • a lower threshold for detecting pathologies may yield a greater number of images that may contain pathologies (and have possibly higher false positives), and a higher threshold for detecting pathologies may yield a smaller number of images (and have possibly lower false positives).
  • a user may adjust the user interface element to also control the number of images that are provided for review.
  • the terms “plurality” and “a plurality” as used herein may include, for example, “multiple” or “two or more.”
  • the terms “plurality” or “a plurality” may be used throughout the specification to describe two or more components, devices, elements, units, parameters, or the like.
  • exemplary means “an example” and is not intended to mean preferred. Unless explicitly stated, the methods described herein are not constrained to a particular order or sequence. Additionally, some of the described methods or elements thereof can occur or be performed simultaneously, at the same point in time, or concurrently.
  • GIT may mean a portion of the gastrointestinal tract and/or the entirety of a gastrointestinal tract.
  • disclosures relating to a GIT may apply to a portion of the GIT and/or the entirety of a GIT.
  • image and frame may each refer to or include the other and may be used interchangeably in the present disclosure to refer to a single capture by an imaging device.
  • image may be used more frequently in the present disclosure, but it will be understood that references to an image shall apply to a frame as well.
  • in-vivo study means and includes a compilation of medical images that are captured in-vivo by an imaging device for a single in-vivo procedure. An in-vivo study may include any number of such medical images for a single in-vivo procedure, which may be a capsule endoscopy procedure or may be another type of in-vivo procedure.
  • the present disclosure may refer to in-vivo images or studies that have an “indication” of one or more GIT conditions. Such descriptions do not mean and are not intended to mean that such images or studies definitely show the GIT condition(s). Additionally, the present disclosure may refer to in-vivo images or studies that have an “indication” of no GIT conditions. Such descriptions do not mean and are not intended to mean that such images or studies definitely do not show the GIT condition(s). Rather, such descriptions mean and are intended to mean that the images or studies merely have such indications.
  • machine learning means and includes any technique which analyzes existing data to learn a model between inputs and outputs in the existing data.
  • artificial intelligence and “machine learning model” mean and include any implementation of the learned model, in software and/or hardware, that can receive new input data and that can predict/infer output data by applying the learned model to the new input data.
  • Machine learning may include supervised learning and unsupervised learning, among other things. Examples of machine learning models include, without limitation, deep learning neural networks and support vector machines, among other things.
  • artificial intelligence and “machine learning” will be used interchangeably herein.
  • the term “device” means and refers to any physical product that can provide an associated function and may include a physical product implemented in a single housing or include a physical product implemented in more than one housing.
  • the GIT 100 is an organ system within humans and animals.
  • the GIT 100 generally includes a mouth 102 for taking in sustenance, salivary glands 104 for producing saliva, an esophagus 106 through which food passes aided by contractions, a stomach 108 to secret enzymes and stomach acid to aid in digesting food, a liver 110, a gall bladder 112, a pancreas 114, a small intestine/small bowel 116 (“SB”) for the absorption of nutrients, and a colon 40 (e.g., large intestine) for storing water and waste material as feces prior to defecation.
  • a mouth 102 for taking in sustenance
  • salivary glands 104 for producing saliva
  • an esophagus 106 through which food passes aided by contractions
  • a stomach 108 to secret enzymes and stomach acid to aid in digesting food
  • a liver 110 a gall bladder 112
  • a pancreas 114 a small intestine
  • the colon 40 generally includes an appendix 42, a rectum 48, and an anus 43. Food taken in through the mouth is digested by the GIT to take in nutrients and the remaining waste is expelled as feces through the anus 43.
  • the type of procedure performed may determine which portion of the GIT 100 is the portion of interest. Examples of types of procedures performed include, without limitation, a procedure aimed to specifically exhibit or check the small bowel, a procedure aimed to specifically exhibit or check the colon, a procedure aimed to specifically exhibit or check the colon and the small bowel, or a procedure to exhibit or check the entire GIT: esophagus, stomach, SB, and colon, among other possibilities.
  • FIG. 2 shows a block diagram of a system for analyzing medical images captured in- vivo via a capsule endoscopy (“CE”) procedure and providing the images to a user system or device.
  • the system generally includes a capsule system 210 configured to capture images of the GIT and a computing system 230 (e.g., local system and/or cloud system) configured to process the captured images.
  • a computing system 230 e.g., local system and/or cloud system
  • the capsule system 210 includes a swallowable CE imaging device 212 (e.g., a capsule) configured to capture images of the GIT as the CE imaging device 212 travels through the GIT.
  • the images may be stored on the CE imaging device 212 and/or transmitted to a receiving device 214, typically via an antenna.
  • the receiving device 214 may be located on the patient who swallowed the CE imaging device 212 and may, for example, take the form of a belt worn by the patient or a patch secured to the patient.
  • the capsule system 210 may be communicatively coupled with the computing system 230 and can communicate captured images to the computing system 230.
  • the computing system 230 processes the received images using image processing technologies, machine learning technologies, and/or signal processing technologies, among other technologies.
  • the computing system 230 may include local computing devices that are local to the patient and/or local to the patient’s treatment facility, a cloud computing platform that is provided by cloud services, or a combination of local computing devices and a cloud computing platform.
  • the images captured by the capsule system 210 may be transmitted to the cloud computing platform.
  • the images can be transmitted by or via the receiving device 214 worn or carried by the patient.
  • the images can be transmitted via the patient’s smartphone or via any other device which is connected to the Internet and which may be coupled with the CE imaging device 212 or the receiving device 214.
  • the images processed by the computing system 230 and/or the processing results may be communicated to a user system or user device 250 for a healthcare professional to read.
  • the computing system 230 may host a downloadable application and may provide the downloadable application to the user system/device 250.
  • the user system/device 250 may download, install, and execute the application.
  • the application executing on the user system/device 250 may communicate with the computing system 230 to obtain study images and study information and then display such images and information locally on the user system/device 250 in a graphical user interface (GUI).
  • GUI graphical user interface
  • the computing system 230 may provide a web service, and the user system/device 250 may access the web service using a web browser.
  • the computing system 230 provides the study images and study information to the user system/device 250 in a GUI as website content.
  • Such examples are merely illustrative and do not limit how a computing system may provide study images and study information to a user system/device.
  • FIG. 3 shows a block diagram of example components of the computing system 230 of FIG. 2 and/or the user system/device 250 of FIG. 3.
  • the system 300 includes a processor 305, an operating system 315, a memory 320, a communication device 322, a storage 330, input devices 335, and output devices 340.
  • the communication device 322 of the system 300 may allow communications with other systems or devices via a wired network (e.g., Ethernet) and/or a wireless network (e.g., Wi-Fi, cellular network, etc.).
  • a wired network e.g., Ethernet
  • a wireless network e.g., Wi-Fi, cellular network, etc.
  • the processor 305 may be or may include one or more central processing units (CPU), graphics processing unit (GPU), controllers, microcontrollers, microprocessors, and/or other computational devices.
  • the operating system 315 may be or may include any code segment designed and/or configured to perform tasks involving coordination, scheduling, arbitration, supervising, controlling or otherwise managing operation of system 300, for example, scheduling execution of programs.
  • Memory 320 may be or may include, for example, a Random Access Memory (RAM), a read-only memory (ROM), a Dynamic RAM (DRAM), a Synchronous DRAM (SD-RAM), a double data rate (DDR) memory chip, a Flash memory, a volatile memory, a non- volatile memory, a cache memory, a buffer, a short term memory, a long term memory, and/or other memory devices.
  • the memory 320 stores executable code 325 that implements the data and operations of the present disclosure, which will be described later herein.
  • Executable code 325 may be any executable code, e.g., an application, a program, a process, task, or script. Executable code 325 may be executed by the processor 305 possibly under control of operating system 315.
  • Storage 330 may be or may include, for example, a hard disk drive, a solid-state drive (SSD), a digital versatile disc (DVD), a universal serial bus (USB) device, and/or other removable and/or fixed device for storing electronic data. Instruct! ons/code and data (e.g., images) may be stored in the storage 330 and may be loaded from the storage 330 into the memory 320, where it may be processed by processor 305.
  • SSD solid-state drive
  • DVD digital versatile disc
  • USB universal serial bus
  • Input devices 335 may include, for example, a mouse, a keyboard, a touch screen, and/or any other device that can receive an input.
  • Output devices 340 may include one or more monitors, screens, displays, speakers, and/or any other device that can provide an output.
  • aspects of the present disclosure relate to a GUI that allows a user to adjust one or more artificial intelligence settings and that graphically indicates which images of an in-vivo study are selected based on the settings.
  • an application may execute on a user system/device and may display study images and study information locally on the user system/device 250 in a GUI.
  • a user system/device may access a web service using a web browser, and the web service may provide the study images and study information to the user system/device in a GUI as website content.
  • a web service may provide the study images and study information to the user system/device in a GUI as website content.
  • the GUI 400 includes a region 410 for displaying images of an in-vivo study, a user interface element 420 for adjusting a setting for an artificial intelligence detector, and a graphical spatial representation 430 of the entirety of the images in the in-vivo study.
  • the arrangement, shapes, and dimensions of the GUI 400 and the various components of the GUI 400 are merely examples.
  • the region 410 for displaying images may include one or more bounding boxes 412 that indicate the location(s) of one or more pathologies detected by the artificial intelligence detector (not shown).
  • the artificial intelligence detector may execute in the computing system 230, which may be a server, such as a proprietary server, a cloud server, and/or an on-site server, among other possibilities.
  • the user interface element 420 is a slider that has multiple positions corresponding to different settings for the artificial intelligence detector.
  • the slider is merely an example.
  • the user interface element 420 may be any interface element that has multiple selections or positions (e.g., lists, buttons, spin wheels, etc.) and that enables a user to choose a setting.
  • the user interface element 420 may have any number of positions or selections, such as two positions or selections, three positions or selections, or more than three positions or selections.
  • the GUI 400 may include multiple user interface elements (not shown) corresponding to multiple settings for the artificial intelligence detector.
  • selections made in the user interface element 420 determine the extent of images that a user can review.
  • the user interface element 420 has one end labeled as “Details” and the other end labeled as “Overview.”
  • the “Details” end of the user interface element 420 is selected, the entirety of the images in the in-vivo study is available for the user to review.
  • the “Details” end may be labeled as “Full Video” instead of “Details.”
  • the “Overview” end of the user interface element 420 is selected, generally much fewer images will be available for the user to review. As explained in more detail below, the fewer images for review may result from setting the artificial intelligence detector to a high certainty threshold for selecting images of interest so that only high certainty images are available for a user to review.
  • the graphical spatial representation 430 of the entirety of the images in the in-vivo study is in the form of a bar, where the beginning of the bar (e.g., left end) corresponds to the first image of the in-vivo study and the end of the bar (e.g., right end) correspond to the last image of the in-vivo study.
  • Indications 432 e.g., lines
  • the indications 432 may be brighter than the brightness of the graphical spatial representation 430.
  • the bar may have a dark gray color, while the lines may be brightly colored.
  • the indications 432 may create a “color bar” by, for example, using a representation of average colors in the respective images as the indication 432.
  • An example of creating a color bar using average color of images is described in U.S. Patent No. 7,215,338, which is incorporated by reference herein in its entirety.
  • a bar is just one example of a graphical spatial representation 430, and lines are just one example of indications 432.
  • the graphical spatial representation 430 and the indications 432 may have other shapes.
  • the user interface element 420 has the illustrated position/selection 422, which corresponds to a setting for an artificial intelligence detector. Accordingly, the position/selection 422 may also be characterized as a setting 422 for the artificial intelligence detector.
  • the artificial intelligence detector may execute in the computing system 230.
  • the GUI may be implemented or may execute in a user system or device (e.g., 250, FIG. 2).
  • the user system or device receives the setting 422 from the user input and transmits the setting 422 to the computing system 230.
  • the computing system 230 may execute the artificial intelligence detector on the images in the in-vivo study using the setting 422 to determine which images of the in-vivo study are selected based on setting 422. In various embodiments, the computing system 230 may be able to determine which images of the in-vivo study are selected based on the setting 422 without re-executing the artificial intelligence detector on all images of the in-vivo study (an example is provided below).
  • the computing system 230 transmits, to the user system or device 250, information about which images are selected based on the setting 422, and the user system or device 250 receives the information to generate the indications 432. In the example of FIG. 4, indications 432 occupy almost the entirety of the graphical spatial representation 430.
  • FIG. 5 and FIG. 6 show different settings 424, 426 of the user interface element 420 and show different indications 434, 436 in the graphical spatial representation 430 corresponding to the settings.
  • the setting 424 corresponds to fewer indications 434
  • the setting 426 corresponds to even fewer indications 436.
  • a user reviewing the images corresponding to the spaced apart indications may see “skipping” when reviewing the images. Such visual skipping may be visually disorienting to the user.
  • a number of images before the target image and/or after the target image may be displayed, as well, so that the user’s review of the target image is smoother and the “skipping” effect is reduced.
  • a number of images surrounding a target image may be displayed when the user interface element 420 is set to the “Overview” setting.
  • a number n of images surrounding a target image may be displayed whenever two target images to be displayed are separated by more than a threshold number of images.
  • a threshold number of images Such and other variations are contemplated to be within the scope of the present disclosure.
  • the differences in the quantity of indications 432, 434, 436 in the graphic spatial representation 430, among FIGS. 4-6, are attributable to the different settings 422, 424, 426 for the artificial intelligence detector, as set by the user interface element 420.
  • the different settings 422, 424, 426 correspond to different thresholds used by the artificial intelligence detector to classify an object in an image as an indicator of a pathology.
  • an object may be, for example, blood or a polyp.
  • Machine learning models for detecting and classifying objects that are indicators of a pathology will be understood by persons skilled in the art.
  • a machine learning model for detecting polyps is disclosed in International Publication No. W02022054046A2, which is hereby incorporated by reference herein in its entirety.
  • An artificial intelligence detector may produce one or more classification scores and may classify an object in an image as an indicator of a pathology based on the classification score(s).
  • classification scores and will understand that the classification score(s) may be compared to one or more thresholds to determine how an object in an image should be classified.
  • a lower threshold for classifying an object in an image as an indicator of a pathology would result in a greater number of images detected as having an indicator of a pathology
  • a higher threshold for classifying an object in an image as an indicator of a pathology would result in a smaller number of images detected as having an indicator of a pathology.
  • the settings 422, 424, 426 may be used to adjust a number of images to be reviewed.
  • changes to the user interface element correspond to changing the threshold(s) for classifying an object in an image.
  • the classification scores produced by the artificial intelligence detector may be stored (e.g., in the user system/device and/or in the computing system) and may be compared to the changed threshold(s) when the user interface element is adjusted. In this manner, the artificial intelligence detector may not need to be re-executed when the user interface element is adjusted.
  • the new threshold(s) may be compared to the stored classification scores (e.g., by the user system/device and/or by the computing system) to determine which images (and their corresponding locations) are selected as a result of the new threshold corresponding to the adjusted user interface element.
  • FIG. 7 shows another example of a graphical user interface (GUI) that includes the user interface element 420 and the graphical spatial representation 430.
  • GUI graphical user interface
  • the indications 730 e.g., lines
  • the indications 730 are shown adjacent to and outside of the graphical spatial representation 430, but still shown in spatial relation to the graphical spatial representation 430 to indicate the locations of images in the in-vivo study which are selected based on the artificial intelligence detector setting corresponding to position/selection 712 of the user interface element 420.
  • FIG. 8 shows the GUI of FIG. 7 with a different position/selection 812 of the user interface element 420.
  • the number of indications 830 in FIG. 8 is less than the number of indications 730 in FIG. 7 due to the different position/selection of the user interface element 420.
  • the GUI of FIG. 7 and FIG. 8 includes another user interface element 720 (e.g., a checkbox, a toggle, etc.) that allows a user to turn on or turn off display of the indications 730, 830.
  • the user interface element 720 is set to turn off display of the indications 730, 830, the indications 730, 830 would not be shown.
  • the graphical spatial representation 430 does not change and may be a “color bar,” as described above herein, that represents the entirety of images in the in-vivo study.
  • the user interface element 420 and the user interface element 720 may be unavailable and non-selectable by a user.
  • the functionality provided by the user interface element 420 and the user interface element 720 may be enhanced features that are not available in every implementation.
  • the user interface element 420 and the user interface element 720 may both be shown in gray color or in another visualization to indicate they are not available to be used, and they can be disabled.
  • the user interface element 720 can be used to enable or disable the artificial intelligence detector.
  • the user interface element 420 becomes disabled and can be shown in gray color or another visualization to indicate that it is not available to be used.
  • the user interface element 720 is set to enable the artificial intelligence detector, the user interface element 420 becomes enabled and can be shown normally to indicate that it is available to be used.
  • FIG. 9 shows an example of an operation from the perspective of the user system or device.
  • the operation involves displaying, on a screen, a graphical user interface configured to present at least a portion of an in-vivo study, where the in-vivo study includes a sequence of in-vivo images of at least a portion of a gastrointestinal tract (GIT) of a person, and where the graphical user interface includes: a user interface element configured to be engaged by a user to adjust a setting for an artificial intelligence detector, and a graphical spatial representation of an entirety of the sequence of in-vivo images.
  • the GUI may have a user interface element and a graphical spatial representation as shown in FIGS. 4-6.
  • the operation involves receiving a setting for the artificial intelligence detector based on a user engagement of the user interface element.
  • the setting may be, for example, a setting for a threshold used by the artificial intelligence detector to classify an object in an image as an indicator of a pathology.
  • the operation involves displaying a plurality of indications in spatial relation to the graphical spatial representation, where the spatial relation corresponds to locations of images, of the sequence of in-vivo images, which are selected based on the setting for the artificial intelligence detector.
  • the graphical spatial representation may be a bar, and the plurality of indications may be lines, as described in connection with FIGS. 4-8. The lines are positioned in or adjacent to the bar according to locations of images of an in-vivo study which are selected based on the setting for the artificial intelligence detector.
  • the operation may involve transmitting, to a server, the setting received for the artificial intelligence detector (where the server is configured to execute the artificial intelligence detector), and receiving, from the server, information for generating the plurality of indications.
  • FIG. 9 is merely an example.
  • the operations may include other blocks not shown in FIG. 9.
  • the operations may not include every block shown in FIG. 9. Such and other embodiments are contemplated to be within the scope of the present disclosure.
  • phrases “in an embodiment,” “in embodiments,” “in various embodiments,” “in some embodiments,” or “in other embodiments” may each refer to one or more of the same or different embodiments in accordance with the present disclosure.
  • a phrase in the form “A or B” means “(A), (B), or (A and B) .”
  • a phrase in the form “at least one of A, B, or C” means “(A); (B); (C); (A and B); (A and C); (B and C); or (A, B, and C) .”
  • the systems, devices, and/or servers described herein may utilize one or more processors to receive various information and transform the received information to generate an output.
  • the processors may include any type of computing device, computational circuit, or any type of controller or processing circuit capable of executing a series of instructions that are stored in a memory.
  • the processor may include multiple processors and/or multicore central processing units (CPUs) and may include any type of device, such as a microprocessor, graphics processing unit (GPU), digital signal processor, microcontroller, programmable logic device (PLD), field programmable gate array (FPGA), or the like.
  • the processor may also include a memory to store data and/or instructions that, when executed by the one or more processors, causes the one or more processors to perform one or more methods and/or algorithms.
  • any of the herein described methods, programs, algorithms or codes may be converted to, or expressed in, a programming language or computer program.
  • programming language and “computer program,” as used herein, each include any language used to specify instructions to a computer, and include (but is not limited to) the following languages and their derivatives: Assembler, Basic, Batch files, BCPL, C, C+, C++, Delphi, Fortran, Java, JavaScript, machine code, operating system command languages, Pascal, Perl, PL1, Python, scripting languages, Visual Basic, metalanguages which themselves specify programs, and all first, second, third, fourth, fifth, or further generation computer languages. Also included are database and other data schemas, and any other meta-languages.

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Public Health (AREA)
  • General Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biomedical Technology (AREA)
  • Pathology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Surgery (AREA)
  • Radiology & Medical Imaging (AREA)
  • Primary Health Care (AREA)
  • Data Mining & Analysis (AREA)
  • Epidemiology (AREA)
  • Physics & Mathematics (AREA)
  • Databases & Information Systems (AREA)
  • Animal Behavior & Ethology (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Optics & Photonics (AREA)
  • Veterinary Medicine (AREA)
  • Heart & Thoracic Surgery (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Signal Processing (AREA)
  • Artificial Intelligence (AREA)
  • Theoretical Computer Science (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Quality & Reliability (AREA)
  • Endoscopes (AREA)

Abstract

Un système de présentation d'études in vivo comprend un ou plusieurs processeurs et une mémoire stockant des instructions. Les instructions, lorsqu'elles sont exécutées, amènent le système à effectuer : l'affichage d'une interface utilisateur graphique (IUG) configurée pour présenter au moins une partie d'une étude in vivo, l'étude in vivo comprenant une séquence d'images in vivo d'un tractus gastro-intestinal, et l'IUG comprenant : un élément d'interface utilisateur configuré pour être mis en prise par un utilisateur pour ajuster un réglage pour un détecteur d'intelligence artificielle (IA), et une représentation spatiale graphique d'une totalité de la séquence d'images in vivo ; la réception d'un réglage pour le détecteur d'IA sur la base d'un engagement de l'utilisateur de l'élément d'interface utilisateur ; et l'affichage d'une pluralité d'indications en relation spatiale avec la représentation spatiale graphique, la relation spatiale correspondant à des emplacements d'images, de la séquence d'images in vivo, qui sont sélectionnées sur la base du réglage pour le détecteur d'IA.
PCT/US2024/052793 2023-10-25 2024-10-24 Interface utilisateur graphique ayant des indicateurs spatiaux dynamiques d'images dans une séquence Pending WO2025090750A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202363593051P 2023-10-25 2023-10-25
US63/593,051 2023-10-25

Publications (1)

Publication Number Publication Date
WO2025090750A1 true WO2025090750A1 (fr) 2025-05-01

Family

ID=93455687

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2024/052793 Pending WO2025090750A1 (fr) 2023-10-25 2024-10-24 Interface utilisateur graphique ayant des indicateurs spatiaux dynamiques d'images dans une séquence

Country Status (1)

Country Link
WO (1) WO2025090750A1 (fr)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7215338B2 (en) 2003-10-02 2007-05-08 Given Imaging Ltd. System and method for presentation of data streams
US20190385302A1 (en) * 2018-06-13 2019-12-19 Cosmo Technologies Limited Systems and methods for processing real-time video from a medical image device and detecting objects in the video
WO2020079696A1 (fr) 2018-10-19 2020-04-23 Given Imaging Ltd. Systèmes et procédés de génération et d'affichage d'une étude d'un flux d'images in vivo
WO2020236683A1 (fr) 2019-05-17 2020-11-26 Given Imaging Ltd. Systèmes, dispositifs, applications et procédés pour des procédures d'endoscopie par capsule
WO2022054046A2 (fr) 2020-09-08 2022-03-17 Given Imaging Ltd. Systèmes et méthodes pour l'identification d'images de polypes
WO2022107124A1 (fr) 2020-11-18 2022-05-27 Given Imaging Ltd. Systèmes et procédés d'identification d'images contenant des indicateurs d'une maladie de type cœliaque

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7215338B2 (en) 2003-10-02 2007-05-08 Given Imaging Ltd. System and method for presentation of data streams
US20190385302A1 (en) * 2018-06-13 2019-12-19 Cosmo Technologies Limited Systems and methods for processing real-time video from a medical image device and detecting objects in the video
WO2020079696A1 (fr) 2018-10-19 2020-04-23 Given Imaging Ltd. Systèmes et procédés de génération et d'affichage d'une étude d'un flux d'images in vivo
US20210345865A1 (en) * 2018-10-19 2021-11-11 Given Imaging Ltd Systems and methods for generating and displaying a study of a stream of in-vivo images
WO2020236683A1 (fr) 2019-05-17 2020-11-26 Given Imaging Ltd. Systèmes, dispositifs, applications et procédés pour des procédures d'endoscopie par capsule
WO2022054046A2 (fr) 2020-09-08 2022-03-17 Given Imaging Ltd. Systèmes et méthodes pour l'identification d'images de polypes
WO2022107124A1 (fr) 2020-11-18 2022-05-27 Given Imaging Ltd. Systèmes et procédés d'identification d'images contenant des indicateurs d'une maladie de type cœliaque

Similar Documents

Publication Publication Date Title
JP7752529B2 (ja) 生体内画像ストリームの精査用情報を生成及び表示するためのシステム並びに方法
US8934722B2 (en) System and method for classification of image data items based on indirect user input
EP4211649B1 (fr) Systèmes et méthodes pour l'identification d'images de polypes
US11934491B1 (en) Systems and methods for image classification and stream of images segmentation
KR20230118054A (ko) 의료 영상으로부터 바이오마커 발현을 예측하는 방법및 시스템
JP7789774B2 (ja) セリアック病様疾患の指標を含む画像を識別するためのシステム及び方法
US20230397794A1 (en) Analysis of in-vivo images using connected graph components
US20230298306A1 (en) Systems and methods for comparing images of event indicators
Ahn et al. Finding small-bowel lesions: challenges in endoscopy-image-based learning systems
CN115485719B (zh) 用于分析图像流的系统和方法
US20240013375A1 (en) Systems and methods for polyp size estimation
WO2025090750A1 (fr) Interface utilisateur graphique ayant des indicateurs spatiaux dynamiques d'images dans une séquence
US12518389B2 (en) Systems and methods for selecting and deduplicating images of event indicators
US20240386692A1 (en) Medical image processing device, method of operating medical image processing device, non-transitory computer readable medium, and endoscope system
US20230351592A1 (en) Clinical decision support system having a multi-ordered hierarchy of classification modules
JP7787161B2 (ja) 手順の妥当性を推定すること
EP4360102A1 (fr) Systèmes et procédés d'évaluation de nettoyage gastro-intestinal
WO2025014628A1 (fr) Interface utilisateur graphique pour des études in vivo préciblées
US20230122280A1 (en) Systems and methods for providing visual indicators during colonoscopy
WO2025210667A1 (fr) Dispositifs de détection de lésion(s) gastro-intestinale(s) et leurs procédés de fonctionnement
KR20200135142A (ko) 의료 영상의 메타데이터 예측 장치 및 방법
JPWO2022054043A5 (fr)

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24804670

Country of ref document: EP

Kind code of ref document: A1