WO2016189313A1

WO2016189313A1 - Imaging feedback system and method

Info

Publication number: WO2016189313A1
Application number: PCT/GB2016/051524
Authority: WO
Inventors: James Alexander PATERSON; Julia Alison Noble
Original assignee: Intelligent Ultrasound Ltd
Current assignee: Intelligent Ultrasound Ltd
Priority date: 2015-05-28
Filing date: 2016-05-26
Publication date: 2016-12-01
Anticipated expiration: 2017-11-28
Also published as: GB201509164D0

Abstract

A system (10) and a method for providing feedback during an imaging procedure, and optionally for generating a summary of acquired images or of the imaging procedure. The system (10) has a protocol database (30) which includes at least one imaging protocol which defines one or more images to be acquired by an imaging machine (12), an image acquisition component (20) which is configured to determine when an image has been acquired by the imaging machine (12), an image assessment component (22) configured to compare at least one characteristic of the acquired image to the imaging protocol and to determine a feedback response and a feedback delivery component (24) configured to provide feedback to the operator of the imaging machine (12). In use, the provision of feedback to the machine operator during the imaging procedure can facilitate image collection and gathering of statistics on the imaging protocol, and help to maximise adherence to imaging protocols.

Description

Imaging feedback system and method

The present invention relates to systems and methods for providing feedback during an imaging procedure, such as a medical imaging procedure. More particularly, the present invention relates to systems and methods which provide feedback regarding the acquisition of each of a series of required images during an imaging procedure and / or to the quality of the acquired images.

Imaging is now a prevalent technique in modern medicine, biological sciences, manufacturing etc. In many domains a human operator is responsible for interacting with an imaging machine to receive information on imaging parameters, make decisions, and control the device. Some clinical professional bodies, for instance, offer "best practice" guidance on how to carry out standardised imaging processes, which are interpreted locally into instructions for carrying out imaging procedures or protocols. These procedures will detail the necessary anatomical or biological structures to be imaged, and the imaging modes to be applied during image acquisition, in order to record sufficient imaging data for the required purpose, for example a medical diagnosis.

Often, imaging procedures can be lengthy and complex, and due to the nature of the subject being imaged, for example an unborn fetus, and technological limitations, the procedure will be carried out in an opportunistic way, recording and storing images to a database as and when possible. This, in many cases, results in operators producing "incomplete" records, providing, for instance, only partial diagnostic and medical record information. There is therefore a need to provide systems and methods which enhance the "acquisition-to-storage" imaging procedure.

According to a first aspect of the invention there is provided a system for providing feedback during an imaging procedure, the system comprising: a protocol database comprising at least one imaging protocol which defines one or more images to be acquired by an imaging machine, such as a series of required images; an image acquisition component configured to determine when an image has been acquired by the imaging machine; an image assessment component configured to compare at least one

characteristic of the acquired image to the imaging protocol and to determine a feedback response; and a feedback delivery component configured to provide feedback to the operator of the imaging machine during the imaging procedure.

In a second aspect of the invention there is provided a computer-implemented method for providing feedback during an imaging procedure, for example using a system as described herein, the method comprising the steps: (i) determining that an image has been acquired by an imaging machine; (ii) comparing at least one characteristic of the acquired image to an imaging protocol and determining a feedback response, wherein the imaging protocol defines one or more images to be acquired during the procedure, such as a series of required images; (iii) providing feedback to the operator of the imaging machine during the imaging procedure.

In a third aspect of the invention there is provided an imaging system for providing feedback during an imaging procedure, the system comprising: an imaging machine; a protocol database comprising at least one imaging protocol which defines one or more images to be acquired by the imaging machine, such as a series of required images; an image acquisition component configured to determine when an image has been acquired by the imaging machine; an image assessment component configured to compare at least one characteristic of the acquired image to the imaging protocol and to determine a feedback response; a feedback delivery component configured to provide feedback to the operator of the imaging machine during the imaging procedure; means for providing the feedback.

The provision of feedback to the imaging machine operator during an imaging procedure enhances the acquisition of image collection and can provide improvements to the quality of the recorded imaging data and help to maximise adherence to an imaging protocol. It will be understood that the feedback is provided in a timely manner during the imaging procedure such that the feedback is useful to the machine operator and can influence the way the machine is operated. Preferably, the feedback is in real-time.

The system may additionally be configured to generate a summary of the one or more acquired images and / or of the imaging procedure. The method of the invention may therefore additionally comprise the step of (iv) generating a summary of one or more acquired images and / or of at least one imaging procedure. It will be understood that the summary may be generated, for example, at the end of the imaging procedure or at the end of a series of imaging procedures, for example after multiple iterations of steps (i) - (iii). Preferably, the summary comprises data relating to adherence to imaging protocols and / or to the quality of the one or more acquired images, for example the proportion or percentage of images successfully acquired or the proportion or percentage of images meeting at least one quality metric. The system and / or method may be configured to collate information from more than one imaging procedure and to generate a summary of the images acquired from the multiple procedures, for example providing summary data relating to the adherence to imaging protocols and / or to the quality of the images acquired from a series of imaging procedures. The system and / or method may additionally enable stratification of the data to generate a summary of imaging procedures, for example, divided by operator, the type or identity of imaging machine, the type of imaging procedure etc. The system may additionally be configured to store the generated summaries and / or to deliver the summaries to a remote database. This enables analysis and reporting, for example the computation of statistics across multiple scans from an organisation, or from an individual operator.

The generation of a summary of the imaging procedure provides a means for rapidly assessing the adherence to imaging protocols for a single imaging procedure, or across a number of imaging procedures.

The system incorporates a protocol database comprising at least one imaging protocol which defines one or more images to be acquired by an imaging machine, such as an ultrasound, MRI, PET, CT or x-ray machine, during an imaging procedure The definition of the one or more images to be acquired may include, for example, the identity of the object to be imaged, such an anatomical feature, a particular view of the object to be imaged, a view captured using a particular imaging mode, etc., and may also include qualitative and quantitative characteristics of the acquired image. The protocol may define, for example, a series of images that are required to be recorded during an imaging procedure. The system may then provide a feedback response to indicate which images within the series of required images have been collected and which images are still required. Preferably, the protocol defines at least one quality metric for each of the one or more images to be acquired. This metric may, for example, relate to the presence of one or more required features, the angle of capture of a required feature in the image, the presence of a measurement of a feature in the image, etc. The inclusion of quality metrics in the protocol enhances the ability of the system to provide a feedback response which can improve the overall quality of data captured during an imaging procedure.

The system comprises an image acquisition component which is configured to determine when an image has been acquired by the imaging machine. It will be understood that images acquired during an imaging procedure can be single frames or can be image sequences (videos). The image frames may be 1 D, 2D or 3D, and may be of, for example, human or animal anatomy, or physical or biological samples. The acquired images may include overlaid signal processing modes, such as 1 D signals, 2D signals, plots, colour map overlays, measurements, performance estimated tools, etc.

The image acquisition component may monitor the control interface of the imaging machine to detect when images are recorded, for example to a database. Alternatively, the image acquisition component may monitor a network communication between the imaging machine and a database to determine when an image has been recorded or the database itself may be monitored to detect when images are recorded, or the image acquisition component may be integrated into the imaging machine. In one embodiment, the image acquisition component is additionally configured to monitor the output of the imaging machine, to determine whether the current image matches the definition of one of the images in the imaging protocol, and to prompt the imaging machine or the operator of the imaging machine to acquire the image.

The system also comprises an image assessment component which is configured to compare at least one characteristic of the acquired image to the imaging protocol and determine a feedback response. This comparison may comprise the identification of at least one object contained within the image and a confirmation of the presence of the at least one object in the imaging protocol. The comparison may additionally comprise the assessment of at least one qualitative and / or quantitative feature of the image, for example the presence or absence of a measurement, the presence of one or more anatomical features, etc., and a comparison of the one or more features with information relating to the one or more features stored in the imaging protocol. The image assessment component may, for example, use object recognition techniques to match the acquired image against a library of example images. The image assessment component may utilise, for example, one or more of object or feature detection, optical character recognition or template detection methods. The image assessment component then determines a feedback response, for example the confirmation that at least one image required in the imaging protocol has been acquired, or that at least one acquired image has the features that are required in the imaging protocol, or that the quality of at least one acquired image meets the criteria of the imaging protocol.

The system comprises a feedback delivery component which is configured to provide feedback to the operator of the imaging machine during the imaging procedure. The feedback delivery component is configured to provide feedback on a means suitable for providing the feedback, for example, on a monitor screen, or any other display screen, such as a mobile electronic device which may be connected to the system by wireless means, such as a Bluetooth^® or wireless internet connection, via at least one speaker, etc. The system may also be configured to display other metrics relating to the imaging procedure, for example the length of scanning time, the number of scanning modes used, the probe types in use, etc. The feedback delivery component may be configured to provide the display of a checklist of the one or more images to be acquired. The checklist may, for example, display a list of the images that are required during an imaging procedure. The checklist may then be updated when each of the required images is acquired. Alternatively the checklist may, for example, show only those images that have not yet been acquired. This enables the machine operator to quickly and clearly identify the progress of the imaging procedure. The checklist may relate to an individual imaging procedure, or may form part of a schedule of tasks to be done by the machine operator, for example over the period of a day, week etc. The schedule of tasks may indicate the time of day at which the tasks should be performed.

The feedback delivery component may alternatively or additionally be configured to provide an audible feedback signal which may be, for example, a signal indicating an image of sufficient quality has been acquired. The feedback signal may comprise spoken word elements, such as a confirmation that a particular image has been acquired, an indication of progress through an imaging procedure, a prompt to record a particular image, a suggestion as to the priority of the next image to be acquired, or an indication of the quality of an acquired image etc. The feedback may comprise a visual and audible component, such as a checklist and an audible signal.

Preferably, the feedback comprises an indication of the quality of acquired images, for example by the display of an indicator providing feedback on the acquired image quality or the provision of an audible signal. This enables the machine operator to rapidly identify captured images which do not satisfy one or more of the quality metrics. In one

embodiment, the indication of the quality of the acquired images forms part of the checklist of the one or more images to be acquired. Preferably, the feedback comprises an indication of the priority of the remaining images to be acquired, for example based on a stratification of the required images into categories based on the importance of the contribution of each image to a successful imaging procedure outcome. The feedback may also comprise an indication of the priority of images to be re-acquired, for example based on an assessment of the relative quality of those images that have already been acquired and indicating which of the images acquired so far is of the lowest quality and therefore which should be re-captured if time allows. The indication of the priority of images to be acquired or re-acquired is updated as the imaging procedure progresses, providing the machine operator with a valuable tool to help prioritise actions, for example during a fixed time period allocated to an imaging procedure. It will be understood that the image acquisition component, the image assessment component and the feedback delivery component may operate, for example, on a computer processor. The system may additionally be configured to store the recorded images and / or associated metadata. The system may also be configured to deliver the recorded images and metadata to a remote database. This enables analysis and reporting, for example the computation of statistics across multiple scans from an organisation, or from an individual operator.

The system may additionally be configured to enable the operator to provide indications about the imaging process during the imaging procedure, for example "patient non-compliant", "machine requires re-calibration" etc.

In a further aspect of the invention there is provided an imaging device, such as a medical imaging device, configured to execute a method for providing feedback during an imaging procedure as described herein. In another aspect of the invention there is provided a processor or computer-readable medium configured to execute a method for providing feedback during an imaging procedure as described herein. In an additional aspect there is provided a computer program comprising program code means for performing all of the steps of the method as defined herein when the program is run on a computer.

The present invention will now be described, by way of example only, and with reference to the accompanying drawings, in which:

Figure 1 shows a schematic representation of an embodiment of a system for providing feedback during an imaging procedure.

Figure 2 shows a flow chart showing operations performed by an embodiment of the system for providing audible feedback relating to the quality of an acquired image.

Figure 3 shows a flow chart showing operations performed by an embodiment of the system for providing feedback in the form of a displayed checklist.

Figure 4 shows a schematic representation of an embodiment of a displayed checklist. Figure 1 shows an embodiment of a system 10 for providing feedback during an ultrasound imaging procedure. The system 10 is connected to an ultrasound imaging machine 12 which includes a signal processing unit 14, a probe 16, and a display monitor 18. The system 10 comprises an image acquisition component 20, image assessment component 22 and a feedback delivery component 24 which operate on a computer processor 26. The image acquisition component 20 monitors the control interface of the imaging machine 12 to detect when images are recorded to an external database 28, such as a hospital picture archiving and communication system (PACS). Once the image acquisition component 20 detects that an image has been recorded, the image assessment component 22 assesses the recorded image and compares at least one characteristic of the image with an imaging protocol stored on a protocol database 30 and then determines a feedback response. Depending on the determination of the feedback response, the feedback delivery component 24 generates a signal which is transmitted to the speaker 32. The feedback delivery component 24 may also deliver a signal to a monitor 34 as an alternative or as an addition to the speaker 32. The system may be configured to generate a summary of the acquired images from one or more imaging procedures and to transmit the summaries to an external database 36.

Figure 2 shows a flow chart detailing operations performed during the ultrasound imaging procedure in which the system 10 provides an audible feedback to the operator. At the start of the procedure at step 100, the ultrasound machine operator positions the probe 16 to acquire a view required by the protocol, such as an image of a fetal femur. Once the operator considers that the current view meets the image requirements, at step 102 the operator records the image to the database 28. The image acquisition component 20 monitors the control interface of the imaging machine 12 and at step 104 detects that an image has been recorded to the database 28 and acquires the image.

The image assessment component 22 then assesses, at step 106, the type and quality of the image using automatic image recognition methods; and, at step 108, determines whether the image meets the image characteristics defined in the imaging protocol which determines a feedback response. The automatic image recognition methods firstly determine which protocol view the image belongs to and secondly determines quality aspects of the image. In both cases standard object recognition techniques are used to match the acquired image against a library of "known good" example images. If the image quality is determined to be good quality then at step 1 10 the feedback delivery component 24 generates a signal which is transmitted to speaker 32. The speaker 32 provides an audible signal which indicates to the operator that a good quality image has been acquired. The operator at step 1 12 can then move onto the next image to be acquired during the imaging procedure. If, at step 108, the image quality is determined to be of poor or bad quality then at step 1 14 the feedback delivery component 24 generates a signal which is transmitted to speaker 32. The speaker 32 provides an audible signal which indicates to the operator that a poor quality image has been acquired. The operator at step 1 16 can then repeat the image acquisition.

Figure 3 shows a flow chart detailing operations performed during the ultrasound imaging procedure which utilises a system substantially as shown in Figure 1 , however with the speaker 32 replaced by a display monitor 34 (represented by a broken line in Figure 1 ). The monitor 34 displays a checklist of the required images as detailed in the image protocol for the particular imaging procedure. Figure 4 shows an example of a checklist 300 which shows seven required images for a 20 week fetal anomaly scan (Head Transventricular View (HeadTV), Head Transcerebellum View (HeadTC), Abdomen, Femur, Face, Spine and Heart Four Channel (FourCh)). Those images which have been successfully acquired with an acceptable quality (302, 304, 306) are highlighted by a change in colour of the text label of the required image in the checklist, for example from white to green.

At step 200 (Figure 3) the ultrasound machine operator reviews the checklist 300 to determine which images have been acquired and which remain to be acquired. If all the required images have been acquired with an acceptable quality then at step 202 the operator is prompted to end the imaging procedure. If at step 202 it is determined that any good quality images remain to be acquired then at step 204 the ultrasound machine operator positions the probe 16 to acquire an image corresponding to one of the remaining images required by the protocol. Once the operator considers that the current image meets the imaging protocol requirements, at step 206 the operator records the image to the database 28. The image acquisition component 20 monitors that control interface of the imaging machine 12 and at step 208 detects that an image has been recorded to the database 28, and acquires the image.

The image assessment component 22 then, at step 210, automatically assesses the quality of the image and, at step 212, automatically determines whether the image meets the image characteristics defined in the imaging protocol which determines a feedback response. For example, a number of different anatomical views of the heart may be defined in the imaging protocol and for each view a variety of different imaging modes are required, such as a four chamber view with a colour-flow Doppler (CF) map of the tricuspid valve. The image assessment component 22 identifies that the acquired image is a four chamber (4CH) view using object recognition techniques. Template matching may then be used on overlaid text in the image to determine imaging mode (e.g. CF enabled). Colour histogram matching may then be used to identify Doppler pixels in the image, and a cascaded, boosted object detector used to locate the boundaries of the tricuspid valve in the image. With this resolved information the image assessment component 22 identifies that the 4CH CF image of the tricuspid valve has been acquired, and whether or not the acquired image meets quality metrics defined in the imaging protocol, for example accuracy of Doppler placement etc.

If the system 10 determines that the acquired image quality is good then at step 214 the checklist 300 is updated by a change in colour of the name of the required image in the checklist 300, for example from white to green. If the system 10 determines that the acquired image quality is bad then at step 216 the checklist 300 is updated by a change in colour of the name of the required image in the checklist 300, for example from white to red. The checklist 300 may also indicate those images which meet minimum standards as defined in the protocol, however the quality of which could be improved, for example by the display of the image name in orange. A continuous scale of quality may also be used.

The checklist 300 may also display additional information pertinent to the imaging procedure, for example the name of the machine operator 308, the type of imaging procedure 310, the conclusions from the imaging procedure 312 (for example 'CIO' - Clinical Indication Open (e.g. live fetus has identified anomaly) or 'CIC - Clinical Indication Closed (e.g. unviable fetus)), or comments on the imaging process 314 (for example 'Difficult' - Scanning is difficult). Variations and modifications will be apparent to the skilled person, for example, the system 10 may be arranged such that the image acquisition component 20 monitors a network connection between the imaging machine 12 and the database 28, or the database 28 itself, to detect when an image is recorded. Alternatively the image acquisition component 10 may be configured to continually monitor the imaging output of the imaging machine 10, determining itself when an image of sufficient quality is available and then triggering the recording of that image to the database 28 or prompting the operator to record an image.

The image assessment component 22 may utilise a range of computer vision and image processing techniques depending on which aspects of the image are relevant for quality assessment purposes e.g. object and feature detection, optical character recognition, template detection, etc.

The system 10 may also be configured to store the recorded images, along with associated metadata, and / or deliver them to a central database (not shown in Figure 1 ) for analysis and reporting, e.g. computation of statistics across multiple imaging procedures from an organisation, or from an individual operator. The system 10 may also provide the operator with the ability to provide indications about the imaging process as it is carried out to assist process management, e.g. "patient non-compliant", "machine requires re- calibration" etc. The feedback to the operator may be provided in a number of ways, for example by a buzzer, synthesised audible words, coloured light indicators, or other display means. The feedback may be provided via a mobile electronic device, such as a smartphone or tablet, which may be connected to system 10, for example by a wireless internet or Bluetooth® connection. It will be also apparent to the skilled person that the components of the system 10 or means to deliver the feedback response may be incorporated directly into a medical imaging device. It will also be apparent that the system 10 may be connected to more than one imaging machine 12.

Other variations and modifications will be apparent to the skilled person. Such variations and modifications may involve equivalent and other features that are already known and which may be used instead of, or in addition to, features described herein. Features that are described in the context of separate embodiments may be provided in combination in a single embodiment. Conversely, features that are described in the context of a single embodiment may also be provided separately or in any suitable sub- combination.

It should be noted that the term "comprising" does not exclude other elements or steps, the term "a" or "an" does not exclude a plurality, a single feature may fulfil the functions of several features recited in the claims and reference signs in the claims shall not be construed as limiting the scope of the claims. It should also be noted that the Figures are not necessarily to scale; emphasis instead generally being placed upon illustrating the principles of the present invention.

Claims

1 . A system for providing feedback during an imaging procedure, the system comprising:

a protocol database comprising at least one imaging protocol which defines one or more images to be acquired by an imaging machine;

an image acquisition component configured to determine when an image has been acquired by the imaging machine;

an image assessment component configured to compare at least one characteristic of the acquired image to the imaging protocol and to determine a feedback response; and a feedback delivery component configured to provide feedback to the operator of the imaging machine during the imaging procedure.

2. A system according to any preceding claim additionally configured to generate a summary of the one or more acquired images and / or the imaging procedure.

3. A system according to any preceding claim in which the protocol defines at least one quality metric for each of the one or more images to be acquired.

4. A system according to any preceding claim in which the feedback delivery component is configured to provide the display of a check-list of the one or more images to be acquired.

5. A system according to any preceding claim in which the feedback delivery component is configured to provide an audible feedback signal.

6. A system according to any preceding claim in which the feedback comprises an indication of the quality of acquired images.

7. A system according to any preceding claim in which the feedback comprises an indication of the priority of the remaining images to be acquired or to be re-acquired.

8. A system according to any preceding claim in which the image acquisition component is additionally configured to monitor the output of the imaging machine, to determine whether the current image matches the definition of one of the images in the imaging protocol, and to prompt the imaging machine, or the operator of the imaging machine, to acquire the image.

9. A computer-implemented method for providing feedback during an imaging procedure using the system according to claim 1 , the method comprising the steps: (i) determining that an image has been acquired by an imaging machine;

(ii) comparing at least one characteristic of the acquired image to an imaging

protocol and determining a feedback response, wherein the imaging protocol defines one or more images to be acquired during the procedure;

(iii) providing feedback to the operator of the imaging machine during the imaging procedure.

10. A method according to claim 9 additionally comprising the step of (iv) generating a summary of the one or more acquired images and / or at least one imaging procedure.

1 1 . A method according to any one of claims 9 to 10 in which the protocol defines at least one quality metric for each of the one or more images to be acquired.

12. A method according to any one of claims 9 to 1 1 in which the feedback comprises the display of check-list of the one or more images to be acquired and / or an audible signal.

13. A method according to any one of claims 9 to 12 in which the feedback comprises an indication of the quality of acquired images.

14. A method according to any one of claims 9 to 13 in which the feedback comprises an indication of the priority of the remaining images to be acquired or to be re-acquired.

15. A method according to any one of claims 9 to 14 additionally comprising the steps of monitoring the output of the imaging machine, determining whether the current image matches the definition of one of the images in the imaging protocol, and prompting the imaging machine, or the operator of the imaging machine, to acquire the image.

16. A processor or computer-readable medium configured to execute the method of any one of claims 9 to 15, or a computer program comprising program code means for performing all of the steps of the method as defined in any one of claims 9 to 15 when the program is run on a computer, or a medical imaging device configured to execute the method of any one of claims 9 to 15.