JP2018007960A - Endoscope device - Google Patents

Abstract

A real-time image that captures the same part and a recorded moving image are displayed simultaneously without complicated calculation processing. When a marking operation for setting a frame image to be used as an index at the time of moving image reproduction is performed while a real-time moving image is recorded, information other than the image data related to the frame image displayed at the time of the Markin operation ( For example, the time from the start of recording or the frame number) is recorded as the metadata of the moving image file. Then, in a new endoscope operation, when the same subject as the subject that was used as an index during the previous recording process is projected in the moving image display, the operator performs a recorded moving image reproduction start operation. In response to this, the frame image is extracted from a series of frame images constituting the recording moving image based on the index frame image related information, and the reproduction display is started from the reproduction start frame image corresponding to the frame image. . [Selection] Figure 4

Description

  The present invention relates to an endoscope apparatus that images a body organ and the like and performs a treatment or the like as necessary, and particularly relates to simultaneous display of a real-time moving image and a recorded moving image.

  Endoscope devices display a moving image of an observation site in real time by imaging with a scope. However, in order to improve lesion diagnosis, a previously recorded moving image to be compared is displayed together with a real-time moving image at the same time. It is also possible (see Patent Document 1). In this case, a moving image recorded during past endoscopic work is displayed simultaneously with a real-time moving image. At this time, a frame image including the same part as the real-time moving image is extracted from a series of frame images constituting the recorded moving image by blood vessel pattern matching.

JP 2008-194334 A

  In pattern matching, it is necessary to perform shift vector calculation or the like on the frame image, the calculation processing amount becomes enormous, and it is difficult to quickly extract the frame image of the same part by pattern matching.

  Therefore, it is required to simultaneously display a real-time moving image and a recorded moving image that show the same part without complicated calculation processing.

  The endoscope apparatus of the present invention is based on an image processing unit that simultaneously displays on a monitor a real-time moving image captured by a scope and a recorded moving image composed of a series of frame images, and a series of index frame-related information. A frame image extraction unit for extracting the index frame image from the frame image, and determining a playback start frame image based on the extracted index frame image, and starting playback display of the recorded moving image from the playback start frame image A moving image reproduction control unit.

  Here, the “index frame image related information” means information related to a frame image other than image data, which is a frame image recorded in association with a recording moving image. For example, the index frame image-related information includes at least one of the number of a frame image serving as an index in a recording moving image or the time after recording starts. The reproduction start frame image may be the extracted index frame image itself, or a frame image that is around a predetermined number of frames or a predetermined time from there may be used as the reproduction start frame image.

  The endoscope apparatus of the present invention can be used for observation of organs such as the lungs, otolaryngology, digestive organs such as the esophagus, stomach, duodenum, small intestine, and large intestine, or organs such as the uterus. It is considered that the effect becomes more remarkable when the observation target is the large intestine. This is because the large intestine is a particularly digestive organ that is easily deformed, and it is particularly difficult to find a matching target based on image data. Therefore, it is preferable that the endoscope apparatus is an endoscope apparatus for large intestine.

  The endoscope apparatus can further include a moving image recording processing unit that records a real-time moving image in a memory, and the moving image recording processing unit performs marking for setting a frame image that serves as an index during moving image reproduction. When the operation is performed, the frame image displayed at the time of the marking operation can be used as the index frame image, and the frame image related information for the index frame image can be recorded in association with the index frame image. Further, the frame image extraction unit can extract the index frame image in response to a recording moving image reproduction display start operation by the user.

  According to the present invention, it is possible to simultaneously display a real-time moving image and a recorded moving image that show the same part without complicated calculation processing.

It is a block diagram of the endoscope apparatus which is this embodiment. It is the figure which showed the simultaneous display screen of a real-time moving image and a recording moving image. It is a flowchart of a moving image recording process. It is a flowchart of the simultaneous display process of a real-time moving image and a recorded moving image.

  Below, the endoscope apparatus which is this embodiment is demonstrated with reference to drawings.

  FIG. 1 is a block diagram of an endoscope apparatus according to this embodiment.

  The endoscope apparatus 10 includes a video scope (scope) 20 and a processor 30 to which the video scope 20 is detachably connected. A monitor 40 is connected to the processor 30. Light emitted from a light source (not shown) provided in the processor 30 passes through a light guide (not shown) provided in the video scope 20 and is irradiated toward the subject from the distal end portion of the video scope 20. The As the light source, a lamp such as a discharge lamp (for example, a xenon lamp, a halogen lamp, a mercury lamp, or a metal is a ride lamp), a light emitting element such as an LED, or a semiconductor light source such as a laser can be applied. Further, a light source device may be provided separately from the processor.

  The light reflected from the subject forms an image on an image sensor (for example, CCD, CMOS, etc.) 21 of the video scope 20, and a subject image is formed on the light receiving surface of the image sensor 21. Pixel signals for one frame are read from the image sensor 21 at predetermined time intervals and sent to the image processing circuit 32 of the processor 30.

  The image processing circuit 32 performs a series of image processing such as color conversion processing and gamma correction processing on the pixel signal to generate a color image signal. By outputting the color image signal to the monitor 40, the observation image is displayed on the monitor 40 as a real-time moving image. The system control circuit 33 outputs a control signal to each circuit such as the image processing circuit 32 and controls the operation of the processor 30.

  In the endoscope apparatus 10, it is possible to record a real-time moving image displayed on the monitor 40, and by operating a button (not shown) provided on the front panel of the video scope 20 or the processor 30, A moving image recording process is executed. Specifically, the moving image recording unit 36 performs compression processing on the image signal sent from the image processing circuit 32, and records the compressed moving image data as a moving image file in a moving image file storage unit (for example, a memory) 38. Here, the moving image is compressed and recorded in accordance with the MPEG system.

  When the recorded moving image is reproduced and displayed, the moving image reproducing unit 34 reads the target moving image file from the moving image file storage unit 38 and performs a restoration process on the compressed moving image data. The image processing circuit 32 generates an image signal based on the moving image data sent from the moving image reproduction unit 34 and outputs the image signal to the monitor 32. As a result, the recorded moving image is reproduced and displayed on the monitor 40.

  In addition to such recording and playback display of moving images, the endoscope apparatus 10 can simultaneously display a real-time moving image by the video scope 20 and a recorded moving image on the monitor 40. When the operator operates a simultaneous display mode setting button (not shown) provided on the front panel of the processor 30, the system control circuit 33 controls the moving image reproduction unit 34 and the image processing circuit 32.

  The moving image reproducing unit 34 restores the compressed moving image data of the corresponding moving image file and transmits it to the image processing circuit 32. The image processing circuit 32 executes display control processing so that the real-time moving image and the recorded moving image sent from the moving image reproducing unit 34 are simultaneously displayed on the monitor 40.

  FIG. 2 is a diagram showing a simultaneous display screen for a real-time moving image and a recorded moving image. The real-time moving image is displayed on the left half of the monitor screen 40S, while the recorded moving image is reproduced and displayed near the upper right corner.

  Such a moving image simultaneous display mode is usually used for diagnosing the progress of a lesion by comparing a recorded moving image with a real-time moving image showing the same part of the same patient. In the present embodiment, marking is performed at the timing when an object as an index is projected by the operator during moving image recording. In the subsequent endoscopic operation, an operation for starting reproduction display of the recorded moving image is performed at the timing when the same subject is displayed on the real-time moving image. At this time, the reproduction display is started from the scene marked at the time of the previous moving image recording (the subject as an index is shown). This will be described in detail below.

  FIG. 3 is a flowchart of the moving image recording process.

  When an operation for starting moving image recording is performed by the operator in synchronization with the start of the endoscopic work (procedure), the moving image recording process is started (S101). At the same time, information such as user data (patient information), recording date and time, and type of procedure is recorded as metadata (S102).

  The operator inserts the insertion portion of the video scope 20 into the body until the distal end of the scope becomes a diagnosis target or reaches the vicinity of the site of interest with the start of the endoscopic work. For example, when diagnosing the cecum in the lower digestive tract, the video scope 20 is inserted until the cecum is reached.

  When a subject to be diagnosed is displayed on the monitor 40, the operator performs an operation of marking an image of this scene. That is, it is set as a frame image that serves as an index when a moving image is subsequently played back. Specifically, a button provided on the video scope 20 or a button provided on the front panel of the processor 30 is operated.

  When such an operator performs a marking operation, information regarding the image displayed on the monitor screen 40S when the operation is performed is recorded as metadata (S103, S104). That is, information about the frame image that serves as an index when the recorded moving image is reproduced later is recorded as metadata.

  Here, the time when the marking operation is performed (that is, the time from the start of recording to the display of the frame image) or the frame number of the frame image is recorded on the metasection of the moving image file. When the operator finishes moving image recording, the recording process ends (S105). The recorded moving image data stored as a moving image file is composed of a series of frame images, and each frame image is associated with a time from the start of recording and a frame number.

  The marking operation can be performed a plurality of times during moving image recording. In this case, the type of the subject serving as an index may be recorded as metadata.

  FIG. 4 is a flowchart of the simultaneous display processing of the real-time moving image and the recorded moving image.

  The operator performs the endoscope operation while viewing the real-time moving image until the part that is used as an index when the previous moving image is recorded is displayed. When the display of the subject image (for example, the cecum image) serving as the index is confirmed, an input operation is performed to start playback of the recorded moving image. Here, a button provided on the video scope 20 or a button provided on the front panel of the processor 30 is operated. The foot switch may be operated.

  If it is determined that the recorded moving image reproduction start operation has been performed by the operator (S201), processing for searching for a corresponding moving image file is executed (S202). The system control circuit 33 determines whether or not the moving image file that matches the patient information set at that time exists in the moving image file storage unit 38 based on the metadata collation of the moving image file. If there are a plurality of moving image files, the moving image file with the same diagnosis target or the latest moving image file may be automatically selected, or the operator may select it.

  When the corresponding moving image file exists, it is determined whether or not the frame image information as an index is included in the metadata (S203, S204). When the frame image information as an index, that is, the time or frame number information is included in the metadata, the moving image reproduction process is executed (S205, S206). At this time, the moving image reproduction unit 34 is controlled to start reproduction and display from the frame image corresponding to the frame number or time.

  When the moving image reproduction display is started, the operator performs an endoscope operation while comparing the recorded moving image being reproduced and displayed with the real-time moving image. Usually, when the same part is a diagnosis target, the endoscope operation of the operator is repeated the same operation. For example, the insertion part is gradually pulled out or pushed in gradually. Therefore, when playback and display of a recorded moving image is started from a frame image that serves as an index and the endoscope operation is continued, comparing the real-time moving image and the subject on which the recorded moving image is projected, It will be copied over time.

  When the reproduction of the recorded moving image is completed or the reproduction ending operation by the operator and the endoscope operation are completed, the simultaneous display process is completed and the display returns to the display of only the real-time moving image (S207, S208).

  As described above, according to the present embodiment, when a real-time moving image is recorded, information (time or frame number) related to a frame image in which a subject serving as an index is displayed by the operator's marking operation is recorded as metadata. Then, in a new endoscopic operation, when the same subject as the subject used as an index during the previous recording process is projected in the moving image display, a recorded moving image reproduction start operation is performed. In response to this, the frame image is extracted from a series of frame images constituting the recording moving image based on the information of the frame image serving as an index, and reproduction display is started from the frame image.

  For example, when an endoscope tip is sent to a diagnosis target site over a long period of time, such as an endoscope pull-out operation, and the site is observed and diagnosed while gradually pulling the insertion portion, the site It takes time to reach, and the amount of moving image data recorded so far also becomes enormous. However, it is only necessary to compare the recorded moving image after reaching the site with the real-time moving image, and the moving image data recorded before that is not helpful.

  In this embodiment, since the recorded moving image starts to be reproduced and displayed from the scene of the frame image as an index by the operator, the frame image showing the same part as the real-time moving image is immediately displayed, and the subsequent endoscopic operation is waited. Can be displayed simultaneously. In addition, since a frame image serving as an index is extracted from the frame information recorded in association with the moving image recording without performing complicated calculation processing such as pattern matching, the moving image reproduction display can be started quickly.

  In particular, an endoscopic observation image is an image that is mostly a reddish subject and emphasizes the brightness and darkness of the brightness. Therefore, if the shooting range (the range to be observed during work) is wide, The moving image data to be matched becomes enormous and it is difficult to select frame images of the same part. In addition, a matching target may not be extracted due to a change in a digestive tract as a photographing target. However, in this embodiment, since the marking operation and the moving image reproduction display start operation are performed at the timing when the scene intended by the operator is projected, it is not necessary to perform special image processing.

  Further, in order to accurately perform diagnosis, it is preferable to obtain a high-definition image, and the resolution of the image is preferably at least 1 million pixels or more, with 2 million pixels, 8 million pixels, or more. Preferably there is. However, the higher the resolution of the image, the larger the amount of data of the frame image and the moving image, and the amount of calculation when the frame image of the same part is extracted by pattern matching and the real-time moving image and the recorded moving image are displayed simultaneously is enormous. There is a problem that becomes. On the other hand, according to the present embodiment, it is possible to suppress the load of calculation processing.

  As described above, the conventional matching method has problems such as poor matching accuracy, difficulty in finding a matching target, enormous amount of calculation, or a problem that an unintended part is reproduced and displayed due to a matching error. On the other hand, in this embodiment, it is possible to reduce the amount of calculation processing and to simultaneously display the recorded image of the part intended by the operator together with the real-time moving image with high accuracy.

  It should be noted that the reproduction display may be started from a frame image that has been shifted by a predetermined time or a predetermined number of frames with respect to the frame image serving as an index. For example, by starting from an image several frames before the index frame image, the endoscope operation start timing can be easily adjusted during simultaneous display. That is, a playback start frame image may be determined based on the extracted frame image serving as an index, and playback may be started therefrom.

  Further, when a frame image in which a subject to be marked is automatically detected is detected regardless of the marking operation, information on the frame image may be automatically recorded as metadata. Further, information other than metadata may be stored in association with each other.

10 Endoscopic device 20 Videoscope (scope)
30 processor 32 image processing circuit (image processing unit)
33 System control circuit (frame image extraction unit, video playback control unit)
34 Movie playback unit 36 Movie recording unit (moving image recording processing unit)

Claims (5)

An image processor that simultaneously displays on a monitor a real-time moving image captured by a scope and a recorded moving image composed of a series of frame images;
Based on information other than image data (hereinafter referred to as index frame image related information), which is information related to the indexed frame image recorded in association with the recorded moving image, the index frame image is extracted from a series of frame images. A frame image extraction unit to perform,
An endoscope apparatus comprising: a moving image reproduction control unit that determines a reproduction start frame image based on an extracted index frame image and starts reproduction display of a recorded moving image from the reproduction start frame image. A moving image recording processing unit for recording the real-time moving image in the memory;
When the marking operation for setting the index frame image is performed by the moving image recording processing unit, the frame image displayed at the time of the marking operation is set as the index frame image, and the frame image related information for the index frame image is The endoscope apparatus according to claim 1, wherein recording is performed in association with the index frame image.   The endoscope according to claim 2, wherein the moving image recording processing unit records a real-time moving image as a moving image file in the memory, and records index frame image related information as metadata of the moving image file. apparatus.   The endoscope apparatus according to any one of claims 1 to 3, wherein the frame image extraction unit extracts an index frame image in response to a recording moving image reproduction display start operation by a user.   5. The index frame image related information includes at least one of a number of a frame image serving as an index in a recording moving image or a time from the start of recording. The endoscope apparatus described.

Images