JP2011167301A - Surgery video image accumulating device, surgery video image accumulating method, and program - Google Patents

Abstract

A surgical video storage apparatus for storing surgical videos that can be easily reused is provided.
An operation image receiving unit 102 that receives one or more operation images that are images during an operation, an operation information acquisition unit 109 that acquires operation information including operation dynamic information that is information related to information that can be acquired during an operation, and the like. The storage unit 110 stores the surgical video and the surgical information by associating the surgical dynamic information with the portion corresponding to the surgical dynamic information of one or more surgical videos received by the surgical video receiving unit 102. It was.
[Selection] Figure 1

Description

  The present invention relates to a surgical video storage device that stores surgical videos.

  As an apparatus for accumulating conventional surgical images, for example, various kinds of ID information are embedded as watermark information in a surgical video and recorded so that a medical image with high evidence can be left. An information recording medium in which video is recorded is provided to a patient to provide an advantageous medical method for both the medical institution and the patient and to prevent medical errors (see Patent Document 1). ).

JP 2004-248895 A (1st page, FIG. 1 etc.)

  However, in a conventional device for storing surgical images, in order to obtain medically important scenes from the stored surgical images, fast-forwarding, fast-forwarding playback, etc. are performed while checking the images. It is necessary, and it has been difficult to search and acquire images of medically important scenes in a short time. For this reason, there is a problem that it takes time and labor to use the stored surgical images, and it is not easy to reuse the stored surgical images.

  The surgical video storage apparatus of the present invention is a surgical video receiving unit that receives one or more surgical videos that are videos during surgery, and surgical information that acquires surgical information including surgical dynamic information that is information related to information that can be acquired during surgery. An acquisition unit and a storage unit that stores surgical video and surgical information by associating surgical dynamic information with a portion corresponding to surgical dynamic information of one or more surgical videos received by the surgical video receiving unit. A surgical image storage device provided.

  With this configuration, it is possible to accumulate surgical images that can be easily reused. For example, it is possible to appropriately search for a position in the surgical video, specifically, a temporal position in the surgical video, using the dynamic operation information. In particular, information that indicates changes in information acquired during surgery, etc., is acquired as surgical dynamic information, and the temporal position of the surgical video is retrieved using the surgical dynamic information. Surgery images can be searched and displayed where there is a change.

  The surgical video storage apparatus according to the present invention further includes a device operation information receiving unit that receives device operation information that is output in response to an operation during surgery performed by a device used in the surgery in the surgical video storage device. The acquisition unit is a surgical image storage device that acquires dynamic operation information according to the device operation information received by the device operation information reception unit.

  With this configuration, the temporal position related to the operation of the device in the surgical video can be designated as the position where the surgical video is output.

  The surgical video storage apparatus according to the present invention is a surgical video storage apparatus in which the surgical information acquisition unit acquires dynamic operation information according to information on a surgical instrument used for surgery.

  With this configuration, the temporal position where the surgical instrument is used in the surgical image can be designated as the position where the surgical image is output.

  The surgical video storage apparatus of the present invention further includes a device output information receiving unit that receives device output information output by a device used for surgery in the surgical video storage device, and the surgical information acquisition unit receives the device output information reception It is the surgery image | video storage apparatus which acquires surgery dynamic information according to the apparatus output information which the part received.

  With this configuration, the temporal position related to the output of the device in the surgical video can be designated as the position for displaying the surgical video.

  The surgical video storage apparatus of the present invention further includes a surgical voice reception unit that receives one or more surgical voices that are voices during surgery in the surgical video storage device, and the surgical information reception unit receives the surgical information acquisition unit. This is a surgical video storage device that detects the characteristics of surgical voice and acquires surgical dynamic information according to the detected characteristics.

  With this configuration, it is possible to designate a temporal position in the surgical video where the feature of the surgical voice is detected as a position for displaying the surgical video.

  In the surgical video storage apparatus according to the present invention, in the surgical video storage device, the surgical information acquisition unit detects a change in illumination during surgery, and acquires surgical dynamic information according to the detected change in illumination. It is a video storage device.

  With this configuration, a temporal position where a change in illumination is detected in the surgical image can be designated as a position for displaying the surgical image.

  In the surgical video storage apparatus according to the present invention, in the surgical video storage device, the surgical information acquisition unit may store surgical environment information that is doctor information that is information related to a surgeon or patient information that is information related to a patient. Further, the operation information is further acquired, and the accumulation unit is an operation image accumulation device that accumulates one or more operation images received by the operation image reception unit in association with operation environment information included in the operation information.

  With this configuration, it is possible to search for surgical images to be output by designating surgical environment information.

  The surgical video storage apparatus according to the present invention includes a doctor identification information receiving unit that is stored in a terminal held by a doctor in the surgical video storage device and receives doctor identification information of a doctor transmitted from the terminal. Further, the operation information acquisition unit is an operation image storage device that acquires operation environment information including doctor information including the doctor identification information received by the doctor identification information reception unit.

  With this configuration, it is possible to simplify input of doctor identification information.

  The surgical video storage device of the present invention includes a patient identification information receiving unit that is stored in a terminal held by a patient in the surgical video storage device and receives patient identification information of a patient transmitted from the terminal. Further, the operation information acquisition unit is an operation video storage device that acquires operation environment information including patient information including patient identification information received by the patient identification information reception unit.

  With this configuration, it is possible to simplify the input of patient identification information.

  The surgical video storage device of the present invention searches the surgical video storage device for a surgical video associated with the surgical information designated by the surgical information designation receiving unit for specifying the surgical information and the surgical information designation receiving unit. The surgical video storage device further includes a surgical video search unit and an output unit that outputs the surgical video searched by the surgical video search unit.

  With this configuration, it is possible to search and output a surgical video and a temporal position in the video using the surgical information.

  According to the surgical image storage apparatus and the like according to the present invention, it is possible to store surgical images that can be easily reused.

The block diagram of the operation | movement video storage device in embodiment of this invention The block diagram which shows an example of the structure of the apparatus used for the surgery which can output apparatus operation information for demonstrating the operation | movement video storage device The block diagram which shows an example of the structure of the apparatus used for the surgery which can output apparatus output information for demonstrating the surgery video storage device Flow chart for explaining the operation of the surgical image storage device Schematic of a surgical video storage system having the surgical video storage device The figure for showing the terminal with which a doctor and a patient were equipped for explaining the operation picture storage device The figure which shows an example of the surgery image | video which the imaging | photography part of the surgery image storage device image | photographed The figure which shows an example of apparatus operation information for demonstrating operation | movement of the surgery image | video storage apparatus. The figure which shows an example of surgical instrument image management information for demonstrating operation | movement of the surgery image | video storage device The figure which shows an example of 1 flame | frame of an operation | movement image | video for demonstrating operation | movement of the operation | movement image storage device. The figure which shows an example of apparatus output information for demonstrating operation | movement of the surgery image | video storage apparatus. Schematic diagram showing surgical audio for explaining the operation of the surgical video storage device The figure which shows the surgery image for demonstrating operation | movement of the surgery image storage device The figure which shows the surgery information management information which matches and manages the surgery image | video and surgery information for demonstrating operation | movement of the surgery image | video storage device. The figure which shows the operation | movement dynamic information management file which manages the correspondence of the operation | movement image | video and operation dynamic information for demonstrating operation | movement of the operation image | video accumulation | storage device. Schematic diagram showing an example of a surgical video file for explaining the operation of the surgical video storage device The figure which shows the input screen which inputs the search conditions for searching the operation | movement image | video for demonstrating operation | movement of the operation | movement image | video storage device. The figure which shows the search result of the flame | frame in an operation | movement image | video for demonstrating operation | movement of the operation image | video storage device. The figure which shows an example of the external appearance of the computer system in embodiment of this invention The figure which shows an example of a structure of the computer system in embodiment of this invention

  Hereinafter, embodiments of a surgical image storage device and the like will be described with reference to the drawings. In addition, since the component which attached | subjected the same code | symbol in embodiment performs the same operation | movement, description may be abbreviate | omitted again.

(Embodiment)
FIG. 1 is a block diagram of a surgical video storage apparatus 1 in the present embodiment.
The surgical video storage apparatus 1 includes an imaging unit 101, a surgical video reception unit 102, a device operation information reception unit 103, a device output information reception unit 104, a voice acquisition unit 105, a surgical voice reception unit 106, a doctor identification information reception unit 107, a patient An identification information receiving unit 108, a surgical information acquisition unit 109, a storage unit 110, a storage unit 111, a surgical information designation receiving unit 112, a surgical video search unit 113, and an output unit 114 are provided.

  The imaging unit 101 captures an image during surgery. Then, the captured video is output to a surgical video reception unit 102 described later. The image during the operation is an image obtained by photographing a state of an operation performed by a doctor or the like on a patient or the like. The image during the operation may be, for example, an entire or a part of the operating room, or an image on or near the operating table. Moreover, the image | video which mainly image | photographed a part of patient, for example, a surgery location, may be sufficient. Moreover, the image | video which image | photographed the visual field seen from the doctor during surgery may be sufficient. Further, it may be an image of the inside of the patient during the operation taken by an endoscope or a microscope, or an image of the patient's tissue taken out during the operation. The photographing unit 101 can be realized by a camera including a photographing element such as a CCD or a CMOS, for example. The camera may be a surgical field camera installed in the operating room, an operative field camera that captures the vicinity of the surgical site, an endoscope camera that captures the field of view of the endoscope, and a field of view of the surgical microscope. It may be a surgical microscope camera. Or the camera etc. which were attached to the headset etc. in order to image | photograph a doctor's visual field may be sufficient. Here, a case where there is one photographing unit 101 is shown as an example, but a plurality of photographing units 101 may be provided. When there are a plurality of imaging units 101, information for identifying the acquired imaging unit 101 may be added to the surgical video output by each imaging unit 101. The video described here is usually a moving image. However, it may be a still image continuous in time series. The format of the video output by the imaging unit 101 may be, for example, non-compressed, or MPEG-4 AVC / H. H.264, MPEG-2, MPEG-1, motionJPEG, DV, etc. The photographing unit 101 adds a normal time code to the photographed video. Further, identification information of the photographing unit 101 prepared in advance, information indicating a photographing region and a photographing target that the photographing unit 101 photographs may be added to the photographed video. When the imaging unit 101 encodes and outputs the captured video, the imaging unit 101 may include an encoder or the like (not shown) that performs encoding processing. The aspect ratio, the number of pixels, the number of colors, the color mode, and the like of the image captured by the imaging unit 101 are not limited. In this case, the term “in operation” refers to a period in which a doctor or the like is in a room such as an operating room to perform an operation, or an actual operation is performed on a patient. You can think of it as a period of time. Here, the case where the surgical video storage device 1 includes the imaging unit 101 has been described as an example. However, the surgical video storage device 1 may not include the imaging unit 101, and an image during surgery. It is also possible to receive a surgical image taken by an external photographing unit that photographs the image.

  The surgical video reception unit 102 receives one or more surgical videos, which are videos during surgery. Specifically, a surgical image output from the imaging unit 101 is received. In the case where a plurality of imaging units 101 are provided, the surgical video accepting unit 102 individually receives the surgical video output from each imaging unit 101. The surgery video reception unit 102 may add identification information to each surgical video received individually. The surgical video reception unit 102 and each imaging unit 101 may be connected by a signal line or the like, or may be connected by a bus such as USB or IEEE1394. In addition, the surgery video reception unit 102 and each imaging unit 101 may be connected via a network such as a LAN or the Internet. The connection may be a wired connection or a wireless connection. The acceptance described here is reception of information transmitted via a wired or wireless communication line. The surgical image receiving unit 102 can be realized by a driver for input / output control, a communication unit, a driver for communication control, or the like. Further, the surgery video reception unit 102 may be considered as an interface for connecting the imaging unit 101 or the like.

  The device operation information receiving unit 103 receives device operation information that is output by one or more devices used in the operation in accordance with the operation during the operation. The device used for the surgery described here is, for example, a device that operates by a user's operation, such as a microscope or an endoscope. Further, a defibrillator, a laser surgical device, or the like may be used. The configuration of the device used for the operation will be described later. The device operation information is, for example, information indicating that a device used for surgery is turned on or off, or information indicating that a knob or the like is rotated. In the case of a microscope, an endoscope, or the like, information indicating an enlargement ratio or the like, information indicating that the enlargement ratio has been changed, or the like. Alternatively, information such as device settings, information indicating that settings have been changed, and the like. In addition, the device operation information is, for example, only when a predetermined operation, for example, power on / off, a predetermined switch on / off, or the like is performed on a device used for surgery. Device operation information indicating that a previously designated operation has been performed may be output. In this case, the device operation information may or may not include information that can identify the content of the operation. The device operation information received by the device operation information receiving unit 103 may include device identification information for identifying the device. The reception described here is, for example, reception of an input signal transmitted from another device or the like, reading of information from a recording medium or the like. If the device operation information output by the device does not include information indicating the time when the operation was performed, the device operation information receiving unit 103 indicates information indicating the received time with respect to the received device operation information. May be obtained and given from a clock or the like (not shown). The device operation information receiving unit 103 can be realized by, for example, a driver for input / output control, a communication unit, a driver for communication control, or the like.

  Here, an example of the configuration of a device used in surgery that can output device operation information and can be used in the present embodiment will be described.

  FIG. 2 is a block diagram showing an example of the configuration of a device used for surgery that can output device operation information. The device 50 includes an operation reception unit 51, a device identification information storage unit 52, and a transmission unit 53. Note that the device 50 appropriately includes a processing unit, an input unit, an output unit, and the like for performing the original function of the device according to the operation received by the operation receiving unit 51, but the description thereof is omitted here. is doing.

  The device 50 may be any device such as the above-described endoscope or microscope as long as it is a device used during surgery.

  The operation reception unit 51 receives an operation on the device 50. The operation may be any operation such as an operation of turning on the device 50. The reception described here is, for example, reception from an input unit such as a switch, button, or knob.

  In the device identification information storage unit 52, device identification information that can identify the device 50 is stored. The device identification information is, for example, a device name, a device serial number, or a device ID.

  The transmission unit 53 transmits device operation information according to the operation received by the operation reception unit 51. Specifically, when the operation accepting unit 51 accepts an operation, information indicating the accepted operation stored in advance in a storage unit (not shown) or the like, for example, operation identification information, and a device according to the accepted operation The device identification information stored in the identification information storage unit 52 is read out, and the device operation information including the information indicating the operation and the device identification information is transmitted to the surgical image storage device 1. When there is only one device 50, the device identification information storage unit 52 can be omitted. Alternatively, the transmission unit 53 determines whether or not the operation received by the operation reception unit 51 is a predesignated operation by comparing it with information for identifying a preliminarily prepared operation. If the operation is an operation, device operation information including information indicating that a predesignated operation has been performed and device identification information stored in the device identification information storage unit 52 is transmitted to the surgical image storage apparatus 1. May be. When transmitting device operation information including device identification information, transmission of information indicating that a predesignated operation has been performed may be omitted. In addition, device operation information including information for identifying an operation performed on the device may be transmitted to the surgical image storage device 1. Moreover, the transmission part 53 may transmit the apparatus operation information containing the information which shows the transmission time and the time which received operation. When there is only one device 50, the device identification information storage unit 52 may be omitted.

  The transmission unit 53 and the device operation information reception unit 103 may be connected by a signal line or the like, or may be connected by a bus such as USB or IEEE1394. The transmission unit 53 and the device operation information reception unit 103 may be connected via a network such as a LAN or the Internet. The connection may be a wired connection or a wireless connection. The transmission unit 53 can be realized by a driver for input / output control, a communication unit, a driver for communication control, or the like.

  Instead of providing the transmission unit 53 in the device 50, the operation receiving unit 51 or the like acquires device operation information similar to the above and stores it in a storage medium such as a buffer (not shown) in the device 50. The device operation information may be acquired by directly reading out the operation video storage device 1 or the like. Alternatively, the device 50 acquires, by a processing unit (not shown), the same device operation information as described above in association with, for example, information indicating the time when the operation corresponding to the device operation information is performed. The device operation information may be stored in a removable portable storage medium (not shown) (not shown) configured by, for example, a flash memory, an optical disk, a hard disk, or the like. Then, the portable storage medium is detached from the device 50 and attached to the surgical image storage device 1, and the device operation information receiving unit 103 reads out the device operation information stored in the portable storage medium and information indicating the time. May be used for acquiring device operation information. The information indicating the time in this case is used, for example, when associating, that is, synchronizing the operation video with the device operation information.

  The device output information receiving unit 104 receives device output information output by a device used for surgery. The device output information received by the device output information receiving unit 104 is, for example, information output by the device regardless of the operation of the device by the user, unlike the above-described device operation information. The device used in the surgery described here may be the same device as the device used in the above-described surgery, but the device here is preferably a device that operates regardless of the user's operation. The device used for the surgery described here is, for example, a biological information measuring device that measures biological information of a patient. The biological information measuring device is a device that measures, for example, pulse, heartbeat, respiration, body temperature, blood pressure, and the like over time. The device output information receiving unit 104 receives device output information output by a device used for surgery. The device output information here is information indicating, for example, a value measured at a constant or indefinite timing by a device used for surgery. Further, the device output information may be, for example, information output to indicate that the device used for the surgery has exceeded or decreased the measured value in advance. Further, the device output information may be information indicating a determination result that is output when the device used for the operation determines that the condition of the patient has changed due to a change in measured value or the like. The device output information may include identification information of the device to be output. Further, the device output information may include information on the time at which the device used for the operation transmits the device output information. Alternatively, when the device output information includes information obtained by measuring biological information or the like, information on each time when the biological information is measured may be included. The device output information may include information indicating what kind of output information the device has. For example, if the device is a sphygmomanometer, it may include identification information indicating that the information is related to a blood pressure value. The reception described here is, for example, reception of an input signal transmitted from another device or the like, reading of information from a recording medium or the like. When the device operation information output by the device does not include information indicating the time when the operation is performed, the device output information receiving unit 104 displays information indicating the received time with respect to the received device operation information. May be obtained and given from a clock or the like (not shown). The device output information receiving unit 104 can be realized by, for example, a driver for input / output control, a communication unit, a driver for communication control, or the like.

  Here, an example of the configuration of a device used for surgery that can output device output information that can be used in the present embodiment will be described.

  FIG. 3 is a block diagram illustrating an example of the configuration of a device used for surgery. The device 60 includes an output information acquisition unit 61, a device identification information storage unit 62, and a transmission unit 63. Note that the device 60 appropriately includes a processing unit, an input unit, an output unit, and the like for performing an original function as a device, but description thereof is omitted here.

  The device 60 is a device that is used during surgery and can be any device such as the above-described biological information measuring device as long as it can output device output information without any user operation. Also good.

  For example, the output information acquisition unit 61 acquires output information that is output information at a timing specified in advance. For example, if the device 60 is a biological information measurement device, biological information such as a pulse rate and a respiratory rate is acquired from a patient. The output information acquisition unit 61 is a sensor or the like when the device 60 is a biological information measuring device or the like.

  In the device identification information storage unit 62, device identification information capable of identifying the device 60 is stored. The device identification information is, for example, a device name, a device serial number, or a device ID.

  The transmission unit 63 transmits device operation information according to the output information acquired by the output information acquisition unit 61. Specifically, device output information including the output information acquired by the output information acquisition unit 61 and the device identification information read from the device identification information storage unit 62 is transmitted to the surgical image storage apparatus 1. In addition, you may make it add the information which shows what kind of information for output to apparatus output information. When there is only one device 60, the device identification information storage unit 62 can be omitted. Alternatively, the transmission unit 63 determines whether the output information acquired by the output information acquisition unit 61 is information within a previously specified range, and when the information is within the specified range, Device output information including information indicating that information within a pre-designated range has been acquired by the device 60 and device identification information stored in the device identification information storage unit 62 is transmitted to the surgical video storage apparatus 1. . When transmitting device operation information including device identification information, transmission of information indicating that information within a pre-specified range has been acquired may be omitted. Moreover, you may transmit the apparatus output information containing the identification information etc. which show what kind of information is in the range designated beforehand to the surgery image | video storage apparatus 1. FIG. The range designated in advance is, for example, a range indicating an abnormal value. Moreover, the transmission part 63 may transmit the apparatus output information containing the information which shows the transmission time and the time which acquired the information within the range designated beforehand. When there is only one device 60, the device identification information storage unit 62 may be omitted. The transmission here may be considered to include an output via a signal line or the like of output information that is an electrical signal acquired by the output information acquisition unit 61.

  The transmission unit 63 and the device output information reception unit 104 may be connected by a signal line or the like, or may be connected by a bus such as USB or IEEE1394. The transmission unit 63 and the device output information reception unit 104 may be connected via a network such as a LAN or the Internet. The connection may be a wired connection or a wireless connection. The transmission unit 63 can be realized by a driver for input / output control, a communication unit, a driver for communication control, or the like.

  Instead of providing the transmission unit 63 in the device 60, the output information acquisition unit 61 or the like acquires the same device output information as described above and stores it in a storage medium such as a buffer (not shown) in the device 60. The device output information thus obtained may be acquired by directly reading out the surgical image storage device 1 or the like. Alternatively, the device 60 acquires the same device output information as described above in association with, for example, information indicating the time when the operation corresponding to the device output information is performed by a processing unit (not shown). The device output information may be stored in a removable portable storage medium (not shown) (not shown) configured by, for example, a flash memory, an optical disk, a hard disk, or the like. Then, the portable storage medium is detached from the device 60 and attached to the surgical video storage device 1, and the device output information receiving unit 104 reads out the device output information stored in the portable storage medium and information indicating the time. May be used for acquiring device operation information. The information indicating the time in this case is used, for example, when associating, that is, synchronizing, the operation video with the device output information.

  The voice acquisition unit 105 acquires one or more surgical voices that are voices during surgery. The voice during surgery is voice of a doctor or a patient uttered in the operating room during surgery. The sound acquisition unit 105 is specifically a microphone. The voice acquisition unit 105 outputs the acquired surgical voice to the surgical voice reception unit 106 described later. The audio format acquired by the audio acquisition unit 105 may be any format such as uncompressed, AAC, MP3, WAV, and the like. When the voice acquired by the voice acquisition unit 105 is encoded and output, the voice acquisition unit 105 may include an encoder. Here, a case where there is one voice acquisition unit 105 is shown as an example, but a plurality of voice acquisition units 105 may be provided. When there are a plurality of sound acquisition units 105, information for identifying the acquired sound acquisition unit 105 may be added to the surgical sound output by each sound acquisition unit 105.

  The surgical voice receiving unit 106 receives one or more surgical voices that are voices during surgery. Specifically, the operation voice output from the voice acquisition unit 105 is received. When a plurality of sound acquisition units 105 are provided, the sound acquisition unit 105 individually receives a surgical video output from each sound acquisition unit 105. Identification information may be given to the surgical voices received individually. The surgical sound reception unit 106 and each sound acquisition unit 105 may be connected by a signal line or the like, or may be connected by a bus such as USB or IEEE1394. In addition, the operation sound reception unit 106 and each sound acquisition unit 105 may be connected via a network such as a LAN or the Internet. The connection may be a wired connection or a wireless connection. The acceptance described here is reception of information transmitted via a wired or wireless communication line. The surgical image receiving unit 102 can be realized by a driver for input / output control, a communication unit, a driver for communication control, or the like. Further, the surgical sound reception unit 106 may be considered as an interface or the like for connecting the sound acquisition unit 105.

  The doctor identification information receiving unit 107 receives doctor identification information of a doctor transmitted from a terminal (not shown) held by the doctor. The identification information of each doctor is stored in advance in a terminal held by each doctor. The terminal described here is specifically a portable terminal. For example, the terminal is a passive or active RFID tag, a card having a tag, or a small device. Moreover, a mobile phone, PHS, PDA, a smart phone, etc. may be sufficient. Alternatively, a notebook computer or the like may be used. The doctor identification number may be any information that can be identified by the doctor, such as a doctor name, a doctor identification number, or an identification symbol. It may be a doctor's registration number. When the terminal is an RFID tag, the doctor identification number is stored in advance in the RFID tag, for example. For example, the terminal may read and output the doctor identification information in accordance with information requesting the doctor identification information transmitted from the doctor identification information receiving unit 107 to the terminal. The doctor identification information may be output by operating a button or a switch.

  The doctor identification information receiving unit 107 is usually realized by a wireless or wired communication means, but may be realized by a means for receiving a broadcast. When the terminal is an RFID tag, the doctor identification information receiving unit 107 may or may not include an RFID tag reader / writer. If not provided, doctor identification information received by an external RFID tag reader / writer or the like may be received. Note that a configuration for reading and writing information of the RFID tag is a known technique, and thus description thereof is omitted here.

  The patient identification information receiving unit 108 receives patient identification information of a patient transmitted from a terminal held by the patient. The identification information of each patient is stored in advance in a terminal held by each patient. Since the configuration, terminal, patient identification information, etc. of the patient identification information receiving unit 108 are the same as the configuration, terminal, doctor identification information, etc. of the doctor identification information receiving unit 107 described above, description thereof is omitted here. . If patient identification information stored in the patient's terminal can be received when entering the operating room or before starting the operation, the patient does not need to hold the terminal during the operation.

  The operation information acquisition unit 109 acquires operation information including operation dynamic information that is information related to information that can be acquired during an operation. The information that can be acquired during the operation is, for example, information that can be acquired as the operation progresses. In addition, the information dynamically changes in accordance with a surgical signal. Surgery dynamic information is, for example, information output during surgery by a device used for surgery. Or it is the information obtained by performing a predetermined | prescribed process with respect to the information which the surgery image | video, the audio | voice acquired during the surgery, the apparatus etc. which are used for a surgery output. However, the predetermined process may be performed after the operation.

  The operation dynamic information is preferably information indicating a dynamic change during the operation. Surgical dynamic information includes, for example, the state of the patient during the operation, the state of the operating room, the state of medical staff such as doctors, the state of the surgical instruments and equipment used in the surgery, and the usage status. Information indicating a dynamic change, for example, information indicating that a change has occurred, information indicating the content of the change, and the like. In addition, the operation dynamic information may be considered as information indicating a change in the operation scene or information indicating a change in the operation process. In addition, the operation dynamic information may be considered as information indicating that a situation that is highly likely to be considered medically important during the operation has occurred. Further, the operation dynamic information may be information indicating that the treatment content in the operation has changed. Specifically, the operation dynamic information is information indicating a pre-designated change detected in pre-designated information acquired during the operation. For example, the surgical dynamic information is information indicating a change designated in advance that is detected from a surgical image, biological information, audio information, or the like acquired during the surgery. For example, if the dynamic operation information is information indicating that a dynamic change during the operation has occurred, the operation dynamic information may be information indicating that the change has occurred. Information indicating what kind of change has occurred, that is, information indicating the content of the change may be used. For example, when a dynamic change is detected in real time during surgery, the surgery information acquisition unit 109 acquires surgery dynamic information indicating that a change has occurred when the change is detected. In addition, the surgery information acquisition unit 109 may acquire surgery dynamic information including information indicating a point in time and a period when a dynamic change during surgery has occurred. In addition, the surgery information acquisition unit 109 may acquire surgery dynamic information including information indicating the contents of dynamic changes. In addition, the dynamic operation information may include information indicating what kind of change, information associated with the dynamic change, for example, information indicating a situation at the time when the change occurs, and the like. For example, information for identifying an object whose change has been detected may be given. For example, when a dynamic change is detected in the lighting in the operating room, for example, identification information “illumination” may be added to the operating dynamic information. The operation dynamic information may be considered as information indicating a time point when a dynamic change occurs in the operation or information for indicating a period.

  The operation dynamic information may be information used when searching a part of the operation video. For example, the operation dynamic information may be considered as information (hereinafter referred to as index information) used as an index used when searching a part of an operation video. Further, the index information may be considered as information for indicating a position in a surgical video used for cueing a surgical video, for example, a chapter position. The position in the surgical image is, for example, a temporal position in the surgical image. The position in the surgical video may be considered as a position on the timeline in the surgical video, for example, or may be considered as a position indicating a time, a frame, or the like in the surgical video. The position here may be considered as one point in the surgical video, or may be considered as a period in which a portion in the surgical video, that is, a start position and an end position are specified. Further, the index information may be information indicating the attribute of the change that has occurred at these positions, specifically the content of the change. The index information may be flag information given to the surgical video. The index information may be considered as a tag. By adding index information or the like to the surgical video, it is possible to search for a desired temporal position in the surgical video using the index information and to read the surgical video from the position to which the index information is added. The assignment here is a concept including association. In addition, the dynamic operation information may be considered as information that can be used for searching among information acquired during the operation.

  The operation information acquisition unit 109 may acquire operation dynamic information in any way. For example, information output by a device in the operating room may be acquired as surgical dynamic information. Moreover, the dynamic change which generate | occur | produced during an operation | movement may be detected using the information etc. which the apparatus etc. in an operating room output, and the operation dynamic information which shows that this change generate | occur | produced may be acquired. Moreover, you may acquire the surgery dynamic information which shows the content of a change. In addition, for example, the surgical video received by the surgical video receiving unit 102 may be analyzed to detect a dynamic change that has occurred during the surgery, and to acquire surgical dynamic information indicating that this change has occurred. . That is, the surgical dynamic information acquired in this case is information indicating a change designated in advance detected in the surgical video. In addition, information indicating that a dynamic change has occurred during the operation, which is output by a device in the operation room, may be acquired as the operation dynamic information.

  Hereinafter, an example of acquiring surgical dynamic information will be shown.

(1) When acquiring surgical dynamic information according to device operation information The surgical information acquisition unit 109 acquires, for example, surgical dynamic information according to device operation information received by the device operation information receiving unit 103. Specifically, the operation dynamic information including at least a part of the device operation information received by the device operation information receiving unit 103 is acquired. At least a part of the device operation information is, for example, information indicating an operation accepted by the device, device identification information, or the like. In addition, the surgery information acquisition unit 109 may acquire surgery dynamic information indicating that a dynamic change has occurred when the device operation information reception unit 103 receives some device operation information. For example, the device operation information is output only when one device (for example, an endoscope or the like) receives a pre-specified operation (for example, turning on the power or operating the electric knife at the distal end of the endoscope). In such a case, when device operation information is received, dynamic operation information indicating that a device has been used and a change has occurred during the operation is acquired. Further, when the device operation information receiving unit 103 receives the device operation information, the device operation information receiving unit 103 determines whether or not the device operation information is device operation information specified in advance, and is the device operation information specified in advance. In this case, surgical dynamic information indicating that a dynamic change has occurred may be acquired. Alternatively, if the device operation information is not designated in advance, surgical dynamic information indicating that a dynamic change has occurred may be acquired. For example, when the device operation information output by one device received by the device operation information receiving unit 103 is device operation information for determining whether or not the device operation information for turning on the power of the one device is ON. In addition, the device operation information indicating that the use of one device that has not been used until now is started, that is, there is a dynamic change during the operation, is acquired. In addition, when the device operation information received by the device operation information receiving unit 103 changes, dynamic operation information may be acquired. For example, surgical dynamic information may be acquired when a change occurs in one set value indicated by device operation information received from one device. The surgical dynamic information acquired here may be information including, for example, device identification information, for example, endoscope identification information, information indicating a change detected in the device, for example, “power on”, and the like. It may be information indicating that information has been acquired during the operation or information indicating that some dynamic change has occurred in the operation.

(2) When acquiring surgical dynamic information according to information on a surgical instrument The surgical information acquisition unit 109 acquires, for example, surgical dynamic information according to information on a surgical instrument used for surgery. Specifically, dynamic operation information including information on a surgical instrument used for surgery is acquired. However, including the information of the detected surgical instrument here means including at least a part of the detected information of the surgical instrument. Specifically, a surgical instrument is a tool used for surgery. The surgical instrument is, for example, a scalpel, an electric scalpel, scissors, tweezers, forceps and the like. Specifically, the operation information acquisition unit 109 acquires operation dynamic information when it is detected that a surgical instrument designated in advance is used in an operation, or that the use has been completed. The information on the surgical instrument is, for example, identification information on the surgical instrument. However, the information on the surgical instrument may be information indicating that one of the surgical instruments designated in advance is used. The surgical dynamic information acquired here includes, for example, identification information of the detected surgical instrument. However, it may be information indicating that one of the surgical instruments specified in advance has been used, information indicating that information has been acquired during the operation, or that some dynamic change has occurred in the operation. It may be information indicating.

  The surgical information acquisition unit 109 acquires information on the surgical instrument in which the surgical instrument is used, for example, by performing image processing on the surgical video received by the surgical video reception unit 102, for example. Specifically, an image of the surgical instrument to be detected is prepared as an image for detecting the surgical instrument, and is associated with identification information of the surgical instrument, for example, the name of the surgical instrument, the manufacturing number of the surgical instrument, the manufacturer name, etc. In advance, it is stored in a storage medium (not shown). Then, image recognition processing for detecting an image that matches the detection image is performed on the operation image received by the operation image receiving unit 102, and a surgical instrument is detected from the operation image. Note that image recognition processing for detecting a surgical instrument is performed by, for example, pattern matching for moving an input image while comparing detection images, and detecting matching or approximate images, or image contours. The structure of the input image is digitized by extracting the edges and feature points, and the structure identification method for recognizing images from the similarity between the digitized ones, the statistical processing of the feature points extracted from the images, A method of determining whether images are similar can be used. And when the identification information of a surgical instrument is required, the identification information of the surgical instrument corresponding to the detected surgical instrument is acquired. However, when the identification information of the surgical instrument is unnecessary, it is not necessary to acquire the identification information of the surgical instrument, and it is only necessary to acquire information indicating that the surgical instrument has been detected. In the case of pattern matching, for example, when the surgical video is a moving image, the pattern matching may be performed individually for all the frames, or may be performed for each frame at a predetermined interval. Since surgical instruments are usually held and operated by hand, the handle etc. will be hidden by hand. Therefore, pattern matching is performed only for parts that are not hidden by hand (for example, scalpel blades or scissors). Also good. Pattern matching may be performed in any way. For example, a region including a color close to the color of the surgical instrument may be detected from the surgical video to be subjected to pattern matching, and pattern matching may be performed using the image in the region and the detection image. Further, an image showing the contour of the surgical instrument may be prepared as the detection image, and a contour having a high correlation with the contour may be detected in the surgical video.

  In addition, for example, a two-dimensional barcode indicating information for identifying the surgical instrument is attached to the surgical instrument in advance, and the two-dimensional barcode is detected from the surgical image, and the detected two-dimensional barcode is By reading, it may be used for determining whether or not a pre-designated surgical instrument has been used. Since processing for detecting and reading a two-dimensional barcode from an image is a known technique, description thereof is omitted here. Instead of the two-dimensional barcode, the surgical instrument name, identification number, etc. may be assigned to the surgical instrument, and the surgical instrument name, identification number, etc. may be read from the surgical video. In addition, a mark having a shape, color, or a combination thereof for identifying the surgical instrument is attached to the surgical instrument, and these marks are detected from the surgical image, and what kind of mark is detected. It is also possible to perform pattern matching, color detection, and the like to determine whether the surgical instrument is associated with the matching pattern or color. For example, a purple mark may be attached to a scalpel or the like in advance, and surgical dynamic information indicating that the scalpel has been used may be acquired when a purple mark is detected in the surgical image.

  The processing for acquiring surgical instrument information from the surgical video described above may be acquired by performing image processing using a surgical video stored in the storage unit 111 and the like described later after the surgery.

  In addition, instead of determining whether or not a surgical instrument has been used using the surgical image received by the surgical image receiving unit 102, information that can be used to detect whether or not the surgical instrument has been used is acquired as surgical information. The unit 109 may accept, and it may be determined whether or not the surgical instrument is used based on the accepted information. For example, an RFID tag storing identification information associated with each surgical instrument is attached to the surgical instrument, and an RFID tag reader / writer (not shown) is used using an antenna (not shown) installed in the operating room or the like. The surgical information acquisition unit 109 receives the identification information of the tag read out) and detects whether or not the surgical instrument has been used using the identification information of the RFID tag. Specifically, the reading range of the RFID tag reader / writer and the position of the antenna are adjusted in advance so that only the RFID tag located near the place where surgery is performed or only on the operating table can be read. . Then, the RFID tag reader / writer reads the identification information of the RFID tag attached to the surgical instrument that falls within this range, and transmits it to the surgical information acquisition unit 109. The surgery information acquisition unit 109 determines whether the identification information of the RFID tag transmitted from the RFID tag reader / writer is one of the identification information designated in advance, and if it is determined that the identification information is one, It detects that a pre-designated surgical instrument has been used. Instead of determining whether the identification information of the RFID tag transmitted from the RFID tag reader / writer is one of the identification information designated in advance, the surgical instrument is used every time new identification information is detected. It may be judged that it was done. In addition, management information for managing the identification information of the surgical instrument in advance in association with the identification information of the RFID tag attached to the surgical instrument is stored in a surgical instrument identification information storage unit configured by a storage medium (not shown). The surgical information acquisition unit 109 may acquire the identification information of the surgical instrument corresponding to the identification information of the RFID tag transmitted from the RFID tag reader / writer using the information. And the surgery information acquisition part 109 may acquire the surgery dynamic information containing the identification information of this surgical instrument. In this case, it is necessary to provide a receiving means (not shown) for receiving identification information output from the RFID tag reader / writer. As this receiving means, for example, the same means as other receiving means can be used.

(3) When acquiring surgical dynamic information according to device output information The surgical information acquisition unit 109 acquires surgical dynamic information according to the device output information received by the device output information receiving unit 104. Specifically, the operation information acquisition unit 109 acquires operation dynamic information including at least a part of the device output information received by the device output information reception unit 104. At least a part of the device output information is, for example, information measured by the device or device identification information. However, the dynamic operation information acquired here may be information indicating that information has been acquired during the operation or information indicating that some dynamic change has occurred in the operation. For example, the surgery information acquisition unit 109 may acquire the dynamic operation information when the device output information reception unit 104 receives some device output information. For example, when one device is to output device output information when some change is detected, the device output information receiving unit 104 receives the device output information, and when the one device operates during operation, Surgery dynamic information is acquired by determining that a general change has been detected. This operation dynamic information includes, for example, identification information of a device that has detected a dynamic change. For example, when a heart rate monitor that outputs device output information indicating that an abnormality has occurred in the heart rate is detected when a heart rate smaller than a predetermined threshold value is detected, the device output information receiving unit 104 Acquires device dynamic information when device output information is received. This surgical dynamic information may include, for example, identification information of a heart rate monitor. Alternatively, the operation dynamic information may be only the identification information of the heart rate monitor. The operation dynamic information may be information indicating that the information has been acquired during the operation or information indicating that some dynamic change has occurred. In addition, when the device output information receiving unit 104 receives the device output information, the device output information determination unit 104 determines whether or not the device output information indicates a dynamic change during the operation, and shows the dynamic change. In the case of output information, surgical dynamic information indicating that a dynamic change has occurred may be acquired. For example, it is determined whether or not the value indicated by the device output information is within a range of values specified in advance, and indicates that there has been a dynamic change when it is within the range of values specified in advance. Surgery dynamic information may be acquired. Alternatively, if it is not within the range of values designated in advance, surgical dynamic information indicating that a dynamic change has occurred may be acquired. For example, when the value indicated by the device output information output by the sphygmomanometer received by the device output information receiving unit 104 is a value equal to or lower than a predetermined lower limit value, the surgery information acquiring unit 109 Surgery dynamic information is acquired by determining that a dynamic change has occurred. This surgical dynamic information may be composed of, for example, surgical dynamic information including identification information of a sphygmomanometer or only identification information of a heart rate monitor. In addition, this dynamic operation information may be information indicating that information has been acquired during the operation, information indicating that some dynamic change has occurred in the operation, or there is an abnormality in blood pressure. It may be information indicating that

(4) When acquiring surgical dynamic information using surgical voice The surgical information acquiring unit 109 detects the characteristics of the surgical voice received by the surgical voice receiving unit 106, and operates dynamic information according to the detected features. To get. Specifically, the surgery information acquisition unit 109 detects the feature of the surgery voice received by the surgery voice reception unit 106, and acquires surgical dynamic information including information indicating the detected feature. However, the dynamic operation information acquired here may be information indicating that information has been acquired during the operation or information indicating that some dynamic change has occurred in the operation. For example, the surgery information acquisition unit 109 acquires the surgery dynamic information when a feature is detected or when a predesignated feature is detected. The information indicating the detected feature may be, for example, identification information indicating the detected feature or information indicating that some feature has been detected for the surgical sound. Specifically, the features of the operation voice detected by the operation information acquisition unit 109 are information such as the presence / absence of sound, the strength / length of the sound, and the height of the sound. Or it is the information of whether it is the audio | voice which the doctor uttered. Or the presence or absence of the sound of the character string designated beforehand may be sufficient. For example, the operation information acquisition unit 109 detects a feature that there is sound if the sound level of the operation sound is equal to or higher than a predetermined threshold value, and if there is less than the threshold value, there is no sound. Detect features. In this case, for example, dynamic surgery information including identification information indicating that there is a voice or no voice is acquired. Alternatively, if the sound level of the surgical sound is a value equal to or higher than a threshold value specified in advance, a feature that there is strong sound is detected. In this case, for example, surgical dynamic information including identification information indicating that there is a strong voice is acquired. Further, if the frequency of the voice is detected and the height of the frequency is equal to or higher than the threshold value in advance, a feature that the voice is high is detected. In this case, for example, surgical dynamic information including identification information indicating that there is a high-pitched sound is acquired. Further, the voice voiceprint is compared with a voiceprint of a doctor or the like registered in advance, and if they match, the feature that the voice is a doctor's voice is detected. In this case, dynamic surgery information indicating that there is a doctor's voice may be acquired, or the doctor's identification information is acquired from the doctor's identification information or the like associated with the voiceprint information used for comparison. Also good. In addition, the operation voice is recognized and converted into a character string, and it is determined whether or not the character string matches a character string designated in advance. If the character string matches, the voice indicating the character string designated in advance is used. Detect the feature of being. In this case, for example, surgical dynamic information including identification information indicating that there is a voice of a designated character string is acquired. Or you may acquire the surgery dynamic information containing the character string detected from the surgery voice. Since the voice recognition process is a known technique, a detailed description thereof is omitted here.

  The surgical information acquisition unit 109 detects the feature of the received surgical voice in real time when the surgical voice reception unit 106 receives the surgical voice, and acquires the dynamic operation information according to the detected feature. Alternatively, after the operation sound received by the operation sound reception unit 106 is temporarily stored in a storage medium such as a hard disk, the operation sound is detected by detecting the characteristics of the operation sound by batch processing using the stored operation sound. Information may be acquired. In this case, for example, the operation voice reception unit 106 receives operation sound including time code information that can be synchronized with the operation video, and information on the time at which the feature of the operation sound on the time code is detected The information acquisition unit 109 may acquire the operation dynamic information obtained by the batch processing as information on the temporal position at which the operation dynamic information is acquired, and may be used when making temporal association with the operation video. In this case, the operation sound may be automatically deleted after the batch processing. In addition, the surgery information acquisition unit 109 may erase the surgical sound, or other means (not shown) (for example, the erasing means) may erase the surgical sound.

(5) When acquiring surgical dynamic information according to a change in illumination The surgical information acquisition unit 109 detects a change in illumination during surgery, and acquires surgical dynamic information according to the detected change in illumination. Specifically, the operation information acquisition unit 109 acquires operation dynamic information indicating a change in illumination during an operation. Illumination conversion includes turning on or off illumination, changing illuminance, and the like. Surgery dynamic information indicating a change in illumination during surgery specifically includes surgery dynamic information including identification information indicating a change in illumination (for example, “illumination on”, etc.), and dynamic changes in illumination. Surgical dynamic information including information indicating that there has been (for example, “illumination change”) and the like. In addition, when the changed illumination can be specified, information for identifying the changed illumination may be included. However, the dynamic operation information acquired here may be information indicating that information has been acquired during the operation or information indicating that some dynamic change has occurred in the operation.

  For example, the surgery information acquisition unit 109 detects a change in illumination by image processing using the surgery video received by the surgery video reception unit 102. For example, whether or not the illumination has been turned on or off or the illuminance has been changed due to a change in brightness of the whole or part of the two or more frame images of the surgical video along the time series of the surgical video. to decide. Or you may judge whether the illumination was turned on or off from the change of the color of the illumination fixture in the surgery image | video which has imaged the lighting fixture. In addition, you may make it acquire the process which acquires the operation | movement dynamic information about illumination from an operation | movement image | video by performing an image process using the operation | movement image | video accumulated in the storage part 111 grade | etc., Mentioned later after an operation | movement.

In addition, the operation information acquisition unit 109 receives information output by an apparatus or the like that controls the illumination in the operating room according to an operation for controlling the illumination, and from this information, whether the illumination in the operating room is on or off. Or whether the illuminance is changed or the like may be detected. In this case, the apparatus that controls the illumination may transmit information including identification information of the controlled illumination to the surgery information acquisition unit 109.
The operation information acquisition unit 109 receives information indicating the measurement result of illuminance output from an illuminance meter or the like installed in the operation room, and the illumination of the operation room is turned on or off based on the change in the information indicating the illuminance. Or whether the illuminance is changed or the like may be detected. In this case, the illuminometer may transmit information indicating the measurement result of illuminance to the surgery information acquisition unit 109.

  In addition to this, the operation information acquisition unit 109 may detect the movement of a medical staff such as an operating room and acquire operation dynamic information according to the movement. Medical staff is a concept that includes doctors and nurses. For example, the magnitude of the movement of the medical staff is detected by image processing using the surgical video received by the surgical video accepting unit 102, and whether the magnitude of the movement is large or not is fast. Depending on the situation, surgical dynamic information may be acquired. For example, the operation dynamic information is acquired when the movement is large and the speed is high. This operation dynamic information may include information indicating that it is acquired with respect to the magnitude and speed of movement, information indicating that it is acquired during the operation, or any movement in the operation. It may be information indicating that a general change has occurred. The motion of the medical staff in the image can be detected by a motion detection technique used in MPEG or the like. The magnitude and speed of the motion can also be calculated from a vector indicating the motion. The surgery information acquisition unit 109 determines, for example, whether or not the magnitude or speed of the detected movement is greater than or equal to a threshold value prepared in advance. A detection result such as is obtained. The surgical dynamic information acquired here may include, for example, information on the movement of the medical staff such as the magnitude of the movement being large and the speed of the movement being fast, or the movement of the medical staff has changed. May be included. The movement may be detected from, for example, a change in position information of an RFID tag (not shown) that includes medical staff identification information and the like held by the medical staff, instead of being acquired from the surgical image. In addition, the process of acquiring the operation dynamic information about the above-described movement from the operation image may be acquired by performing image processing using an operation image accumulated in the storage unit 111 or the like to be described later after the operation. good.

  The operation information acquisition unit 109 may further acquire operation information further including operation environment information that is information including doctor information that is information about a doctor who performs an operation or patient information that is information about a patient. Surgery environment information may be considered as information about the entire situation in which surgery is performed, for example, with almost no change within one surgery. The surgical environment information may be considered as static information about the surgery. The doctor information described here is information regarding the name, specialty, job title, career, affiliation, and the like of a surgeon (for example, a surgeon). The doctor information preferably includes at least doctor identification information for identifying the doctor. By preparing a database or the like (not shown) that manages doctor identification information and information related to doctors in association with each other, information about the doctor can be obtained as a result using the doctor identification information. It is possible. The doctor identification information may be a doctor's name or a character string (including a numeric string) individually assigned to each doctor. The patient information includes the name, age, height, weight, current or past biological information of the patient to be operated, the affected area to be operated, a medical history, and the like. The patient information may be medical chart information. The patient information preferably includes at least patient identification information for identifying the patient. By preparing in advance a database or the like (not shown) that manages patient identification information and information related to the patient in association with each other, information about the patient can be obtained as a result using the patient identification information. It is possible.

  The operation information acquisition unit 109 acquires operation environment information including doctor information including doctor identification information received by the doctor identification information reception unit 107, for example. Specifically, surgical environment information having doctor information including only the doctor identification information received by the doctor identification information receiving unit 107 may be acquired. In addition, the surgery information acquisition unit 109 uses the doctor identification information received by the doctor identification information receiving unit 107, and prepares the doctor identification information as described above and information related to the doctor in association with each other. May be used to obtain information about a doctor by searching, obtain doctor information having the obtained information and doctor identification information, and obtain surgical environment information having the doctor information.

  In addition, the surgical information acquisition unit 109 acquires surgical environment information including patient information including patient identification information received by the patient identification information reception unit 108, for example. Specifically, surgical environment information having patient information including only the patient identification information received by the patient identification information receiving unit 108 may be acquired. In addition, the surgery information acquisition unit 109 uses the patient identification information received by the patient identification information reception unit 108, a database that manages the patient identification information prepared in advance and the information related to the patient in association with each other. May be used to acquire information about a patient by search, acquire patient information having the acquired information and patient identification information, and acquire surgical environment information having the patient information.

  The surgery information acquisition unit 109 can be usually realized by an MPU, a memory, or the like. The processing procedure of the surgery information acquisition unit 109 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The accumulating unit 110 associates the operation dynamic information with the portion corresponding to the operation dynamic image of the one or more operation images received by the operation image receiving unit 102, and accumulates the operation image and the operation information. The portion corresponding to the operation dynamic image is, for example, the temporal position of the operation image captured by the image capturing unit 101 or received by the operation image receiving unit 102 when the operation dynamic information is acquired. Specifically, it is a frame in the surgical video, a time code indicating the frame, or the like. The accumulating unit 110 associates the surgical dynamic information indicating the dynamic change with the portion indicating the dynamic change in the one or more surgical videos received by the surgical video receiving unit 102 so that the surgical video and the surgical information are stored. And may be accumulated. The operation dynamic information stored in the storage unit 110 may be considered as index information used when searching for a part of the operation video or searching for a head. Alternatively, it may be considered as flag information for search or cueing. The association here may be considered as synchronization. Specifically, the accumulation unit 110 accumulates the operation video and the operation information in the storage unit 111 described later. The storage unit 110 may store the portion indicating the dynamic change of the surgical image and the surgical dynamic information in association with each other. For example, the operation dynamic information may be multiplexed and stored at a temporal position corresponding to the operation dynamic information in the operation video. For example, since it is possible to embed data in the case of an image such as MPEG, the operation dynamic information may be embedded in a portion indicating the dynamic change of the operation video information. For example, the identification information of the frame of the surgical video accepted by the surgical video accepting unit 102 at the time when the surgical information acquisition unit 109 acquires the surgical dynamic information or before and after that, the text information of the frame group (for example, GOP), etc. The acquired surgical dynamic information may be embedded in an area where the information can be embedded. Similarly, the acquired surgical dynamic information may be embedded in a text track or the like where a change in the surgical video is detected. In addition, instead of embedding surgical dynamic information in association with the portion where the surgical video has changed, management information for managing the correspondence between the portion where the surgical video has changed and the surgical dynamic information indicating this change Or the like may be prepared, and this information may be associated with each other. For example, the accumulating unit 110 identifies the frame of the surgical video received by the surgical video receiving unit 102 at the time when the surgical information acquiring unit 109 acquires the surgical dynamic information or before and after that, and the identification information of the frame group (for example, GOP). For example, management information for managing the frame number and the like in association with the surgical dynamic information acquired by the surgical information acquisition unit 109 is created and stored. This management information may be managed in association with the surgical video information, or may be embedded in the surgical video information. In addition, for example, the storage unit 110 includes information indicating a time at which the operation information acquisition unit 109 acquired the operation dynamic information or a time before and after the operation dynamic information, specifically, a clock (not shown) or an operation video reception unit. Management information that is acquired from the surgical video received by 102 and that manages this time code in association with the surgical dynamic information acquired by the surgical information acquisition unit 109 may be created and stored. The time code may be information indicating an absolute time such as a time, or may be information indicating a relative time such as an elapsed time from the start of photographing. This management information may be managed in association with the surgical video information, or may be embedded in the surgical video information. The storage unit 110 may divide the surgical video every time the surgical dynamic information is acquired, and store the divided surgical video in association with the surgical dynamic information. If there is the same continuous operation dynamic information, only the oldest operation dynamic information and the newest operation dynamic information are accumulated, and the oldest operation dynamic information is stored in one dynamic change period. May be stored as surgical dynamic information indicating the end of one dynamic change period. Further, only the oldest continuous operation dynamic information may be accumulated. In addition, when there are a plurality of surgical images received by the surgical video reception unit 102, the surgical dynamic information may be associated with each surgical video, or only a part of the surgical video, for example, one surgical video may be used. Information may be associated.

  Further, the storage unit 110 may appropriately encode or decode when storing the surgical image.

  In addition, the storage unit 110 stores one or more surgical videos received by the surgical video receiving unit 102 and surgical environment information included in the surgical information in association with each other. For example, the storage unit 110 may manage the association between the identification information such as the file name of the surgical video and the management information that manages the surgical environment information in association with each other. Further, the surgical environment information may be embedded in a text track or a comment area in the surgical video. Moreover, the timing which matches an operation | movement image | video and operation environment information does not ask | require.

  The storage unit 110 can be usually realized by an MPU, a memory, or the like. The processing procedure of the storage unit 110 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The storage unit 111 stores one or more surgical videos and surgical information in association with each other. The storage unit 111 may be physically configured with one storage medium or may be connected with a plurality of storage media. The storage unit 111 may be a network-connected storage unit. The storage unit 111 is preferably a nonvolatile recording medium, but can also be realized by a volatile recording medium.

  The surgery information designation receiving unit 112 receives designation of surgery information. The designation of the surgery information is, for example, designation of information serving as a search key for searching for surgery information. The designation of operation information may be operation information stored in association with an operation video or information that can specify a part of the operation information. The designation of the surgical information accepted here may be considered as a search key used for searching for a position in the surgical video accumulated in the storage unit 111, for example. The position here is a temporal position in the surgical video, and may be considered as a period in the surgical video or a single point. Further, the position here may be considered as a position on the time axis, a position on the timeline of the operation video, time, or time. It may also be considered as a frame in a surgical image. For example, the operation information designation receiving unit 112 receives one or more pieces of information that can be accumulated as operation information. The designation of the surgical information may be designation of surgical dynamic information, designation of surgical environment information, or both. Here, it is preferable that the designation of the surgical information received by the surgical information designation receiving unit 112 includes at least one or more surgical dynamic information designations. The designation of the operation information may be a keyword, for example. The designation of the surgery information may be considered as information for designating the condition of the surgery information. This surgical information may be one or more of the information used as the above-described surgical dynamic information, or may be one or more of the information used as the surgical environment information. In addition, when the operation dynamic information includes numerical data or the like, it may be information designating the range, magnitude, etc. of the data value. Or the designation | designated of surgery information may be information which designates the accumulation | storage order among the surgery dynamic information previously designated accumulate | stored along the time series. The pre-designated surgical dynamic information here may be all the surgical dynamic information, may be the surgical dynamic information having a pre-designated attribute, or a part of the surgical dynamic information Surgery dynamic information excluding information may be used. For example, it may be information specifying the surgical dynamic information with the oldest stored order. Moreover, the information which designates the next old operation dynamic information of a certain operation dynamic information, or one new operation dynamic information may be sufficient. Here, the pre-designated surgical dynamic information may be all the surgical dynamic information, may be the surgical dynamic information having a pre-designated attribute, or a part of the surgery Surgery dynamic information excluding dynamic information may be used. Further, desired surgical information may be directly designated from the surgical information displayed in a list or the like.

  The means for inputting the designation of the operation information may be anything such as a numeric keypad, a keyboard, a mouse, or a menu screen. The surgery information designation receiving unit 112 can be realized by a device driver of an input unit such as a numeric keypad or a keyboard, control software for a menu screen, or the like.

  The surgical video search unit 113 searches for a surgical video associated with the surgical information received by the surgical information designation receiving unit 112. Specifically, the operation information indicated by the operation information received by the operation information designation receiving unit 112 is detected from the operation information accumulated in association with the operation video in the storage unit 111, and is associated with the detected operation information. Detected surgical images. In particular, when the operation information specification received by the operation information specification receiving unit 112 includes operation dynamic information specification, the temporal position in the operation video associated with the operation dynamic information indicated by the specification, for example, Detect frames etc. For example, when the designation of the surgical information is a character string, the surgical dynamic information detects the surgical dynamic information that matches the character string, and the time in the surgical video associated with the detected surgical dynamic information The correct position. The match here may be a complete match or a partial match. For example, if the operation information is information that specifies “scissors” that is operation dynamic information for one operation image, the temporal position associated with the operation dynamic information that matches “scissors” Are detected in one surgical image. In addition, when the operation dynamic information specification received by the operation information specification receiving unit 112 specifies the order in which the operation dynamic information is stored, the operation dynamic information is associated with the operation dynamic information in the specified order. The position of the operation image, for example, a frame is searched. For example, when information specifying the oldest surgical dynamic information in the accumulated order is received, the temporal position of the surgical video associated with the oldest surgical dynamic information, such as a frame, is searched. In the case of designating the next operation dynamic information of a certain operation dynamic information, the temporal position of the operation image associated with the next operation dynamic information next to the one operation dynamic information, such as a frame, is indicated. To detect. Further, when the operation information is information for specifying a condition such as a range of values of the operation dynamic information, the temporal information in the operation video associated with the operation dynamic information having a value satisfying the condition is satisfied. The position may be detected. The search here is to detect the temporal position where some kind of surgical dynamic information is added, for example, the position where the dynamic surgical information serving as an index is added regardless of the content, It is also considered as a search to detect a flag indicating the position where the information etc. acquired in the change is changed.

  The surgical image search unit 113 can be usually realized by an MPU, a memory, or the like. The processing procedure of the surgical image search unit 113 is usually realized by software, and the software is recorded in a recording medium such as a ROM. However, it may be realized by hardware (dedicated circuit).

  The output unit 114 outputs the surgical video searched by the surgical video search unit 113. Specifically, the surgical video is read from the temporal position in the surgical video searched by the surgical video search unit 113 and output. When outputting the surgical video, the surgical video may be appropriately decoded. The output here is a concept including displaying a surgical image using a monitor or the like and transmitting information on the surgical image to other devices via a network or the like.

  The output unit 114 may be considered as including or not including an output device such as a display. The output unit 114 may be implemented by output device driver software, or output device driver software and an output device.

  Although omitted in the above description, the operation sound acquired by the sound acquisition unit 105 may be stored in the storage unit 111 in synchronization with the operation video. Further, the output unit 114 may read out and output the operation sound synchronized with the operation image when outputting the operation image.

  Next, an example of operation | movement of the surgery image | video storage device 1 is demonstrated using the flowchart of FIG.

  (Step S401) The surgical video storage apparatus 1 determines whether an instruction to acquire and store a surgical video is received from a user or the like via a reception unit (not shown). If accepted, the process proceeds to step S402. If not accepted, the process proceeds to step S425.

  (Step S402) The doctor identification information receiving unit 107 determines whether doctor identification information has been received. If received, the process proceeds to step S403. If not received, the process proceeds to step S404.

  (Step S403) The surgery information acquisition unit 109 acquires doctor information including the doctor identification information received in step S402. The acquired doctor information is temporarily stored in a storage medium (not shown). Then, the process proceeds to step S404.

  (Step S404) The patient identification information receiving unit 108 determines whether or not patient identification information has been received. If received, the process proceeds to step S405. If not received, the process proceeds to step S406.

  (Step S405) The surgery information acquisition unit 109 acquires patient information including the patient identification information received in step S404. The acquired patient information is temporarily stored in a storage medium (not shown). Then, the process proceeds to step S406.

  (Step S406) The surgery video reception unit 102 determines whether or not a surgery video has been received. The surgical video reception unit 102 receives, for example, a series of surgical videos acquired by the imaging unit 101 in units of frames or seconds. The received surgical video is, for example, a surgical video composed of a frame following the surgical video received immediately before. This determination process may be performed at a predetermined time interval. For example, it may be determined whether or not it is time to receive a surgical video, and it may be determined whether or not a surgical video has been received when the timing for receiving it is reached. If accepted, the accepted surgical image is temporarily stored in a storage medium or the like (not shown), and the process proceeds to step S407. If not accepted, the process proceeds to step S424. Note that the storage unit 110 may store the data directly in the storage unit 111 instead of temporarily storing it in a storage medium (not shown).

  (Step S407) The device operation information receiving unit 103 determines whether device operation information has been received. If accepted, the process proceeds to step S408, and if not accepted, the process proceeds to step S409.

  (Step S408) The operation information acquisition unit 109 acquires operation dynamic information about device operation. For example, the operation dynamic information including the device identification information included in the device operation information received in step S407 is acquired. Then, it is temporarily stored in a storage medium or the like (not shown).

  (Step S409) The surgery information acquisition unit 109 performs a process of acquiring information about the surgical instrument using the surgery video received in step S406. Specifically, for the surgical image received in step S406, an image similar to the image of one or more surgical instruments prepared in advance on a storage medium (not shown) is searched by image recognition processing such as pattern matching. When a similar image, that is, an image with a high degree of similarity is detected, information on a surgical instrument prepared in advance in association with the surgical instrument image, such as a surgical instrument name, is acquired. In addition, when an image similar to any image of the surgical instrument cannot be detected in the surgical video, information on the surgical instrument is not acquired.

  (Step S410) The surgical information acquisition unit 109 determines whether or not information on the surgical instrument has been acquired in step S409. If acquired, the process proceeds to step S411. If not acquired, the process proceeds to step S412.

  (Step S411) The surgical information acquisition unit 109 acquires surgical dynamic information about the surgical instrument. For example, surgical dynamic information including information on the surgical instrument acquired in step S409 is acquired.

  (Step S412) The device output information receiving unit 104 determines whether device output information has been received. If accepted, the process proceeds to step S413. If not accepted, the process proceeds to step S416. Here, it is assumed that the device output information is data such as measurement results output by the device in the operating room.

  (Step S413) The surgery information acquisition unit 109 acquires the device output information received in Step S412 as the surgery dynamic information. The acquired surgical dynamic information is temporarily stored in a storage medium (not shown).

  (Step S414) The surgery information acquisition unit 109 determines whether or not the device output information received in Step S412 is information within a predesignated range. For example, it is compared with a threshold value prepared in advance in association with the device identification information included in the device output information to determine whether or not the threshold value is exceeded. It is determined that the information is within the specified range. Or you may judge whether it is below a threshold value. If the information is within the range specified in advance, the process proceeds to step S415. If the information is not within the range, the process proceeds to step S416.

  (Step S415) The surgery information acquisition unit 109 acquires surgery dynamic information indicating that the device output information is information within a range specified in advance. For example, the operation dynamic information including device identification information included in the device output information and information (for example, abnormality) prepared in association with the device identification information is acquired. The acquired surgical dynamic information is temporarily stored in a storage medium (not shown) or the like. Then, the process proceeds to step S416.

  (Step S416) The surgical sound reception unit 106 receives a surgical sound. The received operation sound is temporarily stored in a storage medium (not shown).

  (Step S417) The surgery information acquisition unit 109 performs a process for detecting features of the surgery voice received in step S416. For example, it is determined whether or not the volume of the operation voice, that is, the volume level is equal to or higher than a predetermined threshold value. And if it is more than a threshold value, it will detect that there exists the characteristic about a volume level. Alternatively, whether or not a specific phrase is included in the surgical voice is determined by voice recognition processing, and if it is included, it is detected that the phrase has a feature.

  (Step S418) The operation information acquisition unit 109 determines whether or not a feature is detected for the operation sound in step S417. If a feature is detected, the process proceeds to step S419. If a feature is not detected, the process proceeds to step S420.

  (Step S419) The operation information acquisition unit 109 acquires operation dynamic information about the characteristics of the operation sound.

  (Step S420) The surgery information acquisition unit 109 performs a process of detecting a change in illumination during surgery. For example, an average luminance value, a color temperature of the brightest part of the image, and the like are calculated and compared between the surgical image received in step S406 and the surgical video stored immediately before by the storage unit 110, and the difference of the difference is calculated. It is determined whether or not the absolute value is greater than or equal to a value specified in advance. If the absolute value is greater than or equal to the value, it is determined that the illumination has changed.

  (Step S421) The surgery information acquisition unit 109 determines whether or not a change in illumination has been detected. If a change in illumination is detected, the process proceeds to step S422, and if not detected, the process proceeds to step S423.

  (Step S422) The operation information acquisition unit 109 acquires operation dynamic information regarding the change in illumination.

  (Step S423) The storage unit 110 includes surgical environment information including the doctor information acquired by the surgical information acquisition unit 109 in step S403 and the patient information acquired in step S405, and the surgical information acquisition unit 109 performs steps S408, S411, The operation dynamic information acquired in step S413, step S415, step S419, and step S422 and the operation image received in step S406 are associated with each other and accumulated in the storage unit 111. If the surgical information cannot be acquired, the surgical video is accumulated except the surgical information. In addition, accumulation | storage of the surgery information here may be the multiplexing with respect to the surgery image | video of surgery information. Then, the process returns to step S402.

  (Step S424) The surgical image receiving unit 102 determines whether or not a predetermined time has passed since the last reception of the surgical video. If the predetermined time has elapsed, the process returns to step S401, and if not, the process returns to step S406. Instead of performing this process, it may be determined whether or not an instruction to end the reception of the surgical video has been received by an unillustrated reception unit or the like.

  (Step S425) The surgery information designation receiving unit 112 determines whether or not the designation of surgery information has been received. If accepted, the process proceeds to step S426. If not accepted, the process returns to step S406.

  (Step S426) The surgical image search unit 113 searches a temporal position corresponding to the designation of the surgical information in the surgical video corresponding to the designation of the surgical information accepted in Step S425. Specifically, the operation corresponding to the designation of the surgical dynamic information included in the designation of the surgical information in the surgical image associated with the surgical environment information corresponding to the designation of the surgical environment information included in the designation of the surgical information The temporal position associated with the dynamic information is detected. Here, the temporal position in the surgical video may be considered as a frame associated with the surgical dynamic information in the surgical video. Alternatively, it may be a range having a start point and an end point. The position here may be considered as a position on the time axis or time.

  (Step S427) The output unit 114 sequentially reads out and outputs from the temporal position in the surgical video searched in Step S426. Then, the process returns to step S401.

  In the flowchart of FIG. 4, the operation information acquisition unit 109 performs a process of acquiring operation information such as operation dynamic information at a predetermined time interval such as an interval of 1 second or an interval of a plurality of frames. good.

  In the flowchart of FIG. 4, the process ends when the power is turned off or the process ends.

  Hereinafter, a specific example of the present embodiment will be described.

  FIG. 5 is a schematic diagram of a surgical video storage system having the surgical video storage device 1 of the present embodiment. Here, the situation in the operating room 400 in which the surgical image storage system is installed is schematically shown. Here, the imaging unit 101 is assumed to be a camera that images a patient's surgical site. Further, it is assumed that the voice acquisition unit 105 is a microphone installed in the operating room 400. The endoscope 510 is connected to a control device 511 that functions as an input interface for information output from the endoscope 510 and a monitor 512 that displays an image of the endoscope. Thus, an endoscope system is configured. This endoscope system corresponds to the device shown in FIG. The control device 511 is connected to the surgical image storage device 1, and when a doctor performs an operation designated in advance on the endoscope 510, information indicating this operation is stored from the control device 511 in the surgical video storage device. Sent to device 1. In addition, the surgical image storage device 1 is connected to a heart rate monitor 500 that measures the heart rate of the patient. The heart rate monitor 500 corresponds to the device shown in FIG.

  In addition, a terminal storing doctor identification information is attached to a doctor in the operating room, and similarly, a terminal having patient identification information attached to the patient is brought into the operating room. To do.

  FIG. 6 is a diagram illustrating a terminal attached to a doctor and a patient. The terminals 521 and 522 are card-type terminals in which passive RFID tags are embedded. The terminal 521 is a terminal that stores doctor identification information, and is worn by the doctor 523. Here, the doctor identification information stored in the terminal 521 is assumed to be “DR10012” as an example. The terminal 522 is a terminal that stores patient identification information, and is worn by the patient 524. Here, the patient identification information stored in the terminal 522 is assumed to be “PT50033” as an example. However, it may be removed during the operation because it is in the way. Here, as an example, the names of the doctor 523 and the patient 524 are described on the surfaces of the card type terminal 521 and the terminal 522, and the name tags of the doctor 523 and the patient 524 are also used.

  An RFID tag reader integrated with an antenna for receiving doctor identification information and patient identification information transmitted from the terminal 521 and the terminal 522 attached to the doctor and the patient is provided at the entrance of the operating room 400 illustrated in FIG. A writer 520 is installed.

  Here, as an example, it is assumed that the surgery information acquisition unit 109 performs a process of acquiring surgery information at intervals of one second indicated by a time code included in the surgery video received by the surgery video reception unit 102. For example, when the value of the number of frames indicated by the time code becomes “01”, for example, “00:12: 11.01”, the operation information is acquired. Here, the time code indicates “hour: minute: second.frame”. If the frame is omitted, “hour: minute: second” is used.

  First, in order to record a video during surgery, if the user gives an instruction to store the surgical video to the surgical video storage device 1, the surgical video storage device 1 captures the surgical video. To start. The surgical video reception unit 102 starts receiving a surgical video output from the imaging unit 101. Then, the accumulation unit 110 sequentially accumulates the operation images received by the operation image reception unit 102 in the storage unit 111.

  In addition, the voice acquisition unit 105 starts acquiring voice during surgery. The surgical sound reception unit 106 starts receiving the surgical sound acquired by the sound acquisition unit 105. Here, as an example, the accumulation unit 110 sequentially accumulates the operation sound received by the operation sound reception unit 106 in the storage unit 111 in synchronization with the operation video described above.

  Next, when the doctor and the patient enter and leave the operating room 400, when the doctor and the patient terminal 521 and the terminal 522 are held over the antenna of the RFID tag reader / writer 520, the RFID tag reader / writer 520 is stored in the terminal 521 and the terminal 522, respectively. The doctor identification information and patient identification information being read are read out and transmitted to the surgical image storage apparatus 1. The doctor identification information receiving unit 107 of the surgical image storage device 1 receives the doctor identification information transmitted from the RFID tag reader / writer 520. Here, for example, doctor identification information “DR10012” is received. Further, the patient identification information receiving unit 108 of the surgical image storage apparatus 1 receives patient identification information transmitted from the RFID tag reader / writer 520. Here, for example, patient identification information “PT50033” is received.

  The surgery information acquisition unit 109 acquires doctor information including the doctor identification information received by the doctor identification information reception unit 107. Here, doctor identification information “DR10012” itself is acquired as doctor information.

  In addition, the surgery information acquisition unit 109 acquires patient information including the patient identification information received by the patient identification information reception unit 108. Here, it is assumed that the patient identification information “PT50033” itself is acquired as patient information.

  Then, the accumulation unit 110 accumulates the operation image received by the operation image reception unit 102 in the storage unit 111 in association with operation environment information including doctor information and patient information. Here, as an example, it is assumed that the surgical environment information is not associated with each frame of the surgical image, but is associated with the entire surgical image. For example, a surgical video file is stored in association with surgical environment information including doctor information and patient information.

  FIG. 7 is a diagram schematically illustrating a surgical image captured by the imaging unit 101, that is, a surgical image received by the surgical image receiving unit 102. The surgical video is composed of a plurality of frames, and here, it is assumed that a time code is given to each frame. Here, it is assumed that the time code indicates the date and time when the surgical video was taken. For example, the surgical image receiving unit 102 sequentially receives images of one frame or a plurality of frames of the surgical video as shown in FIG.

  Here, for example, in order to use the endoscope 510, the doctor can turn on the power to the control device 511, the monitor 512, and the endoscope 510 constituting the endoscope system, and use the endoscope 510. Suppose that The control device 511 of the endoscope system detects that the endoscope 510 is turned on, and identifies device operation information indicating that the endoscope 510 is turned on, for example, operation of the device The operation identification information “Power ON” and the device identification information “Endoscope ES002” of the endoscope 510 are transmitted to the surgical image storage apparatus 1.

  The device operation information receiving unit 103 of the surgical image storage device 1 receives device operation information transmitted from the control device 511 of the endoscope system, and temporarily stores the received device operation information in a storage medium (not shown).

  FIG. 8 is a diagram illustrating an example of device operation information received and temporarily stored by the device operation information receiving unit 103. The device operation information has items of “device ID” and “operation ID”. “Device ID” is identification information of a device that has received an operation. The “operation ID” is operation identification information for identifying an operation performed on the device.

  The operation information acquisition unit 109 acquires operation dynamic information including device operation information received by the device operation information reception unit 103 as shown in FIG. Here, it is assumed that the surgical dynamic information to be acquired is information representing the device operation information as shown in FIG. 8, for example, “endoscope ES002: power ON”.

  The storage unit 110 associates “endoscope ES002: power ON”, which is the surgical dynamic information acquired by the surgical information acquisition unit 109, with the surgical video frame received by the surgical video reception unit 102 in the storage unit 111. accumulate. Note that the frame of the operation video here is a frame acquired when the operation information acquisition unit 109 acquires the operation dynamic information. The association here may be that the surgical dynamic information is arranged in a text track corresponding to the frame received when the surgical dynamic information of the surgical video is acquired. The operation dynamic information associated with the time code corresponding to the frame received when the information is acquired, the frame number assigned to the frame, and the like may be stored in association with the operation video. Here, in particular, since the operation dynamic information is acquired in units of one second, the time code with the number of frames omitted may be associated with the surgical dynamic information.

  In addition, the surgical information acquisition unit 109 performs processing for sequentially acquiring information about surgical instruments on the surgical video sequentially received by the surgical video reception unit 102.

  FIG. 9 shows surgical instrument image management information for managing still images of main parts of surgical instruments designated in advance. The surgical instrument image management information is accumulated in, for example, a storage medium (not shown). The surgical instrument image management information includes items of “instrument image” and “instrument ID”. The “instrument image” is image information indicating the main part of the surgical instrument. The data format of image information is not limited. “Instrument ID” is identification information such as an instrument name of a surgical instrument. Here, it is assumed that the name of the instrument.

  FIG. 10 is a diagram illustrating an example of one frame of a surgical video received by the surgical video receiver 102.

  The operation information acquisition unit 109 sequentially reads the information on the “instrument image” shown in FIG. 9, and the “instrument image” shown in FIG. 9 for one frame of the operation image received by the operation image receiving unit 102 as shown in FIG. 10. ”Is performed, for example, pattern matching. When a suitable “appliance image” is detected, the value of “appliance ID” associated with this “appliance image” is read out.

  For example, it is assumed that an image that matches the “instrument image” in which the “instrument ID” illustrated in FIG. 9 is “female” is detected in the region 91 of one frame of the surgical image illustrated in FIG. 10 by the image recognition process. . The surgical information acquisition unit 109 reads “female”, which is “instrument ID” corresponding to the “instrument image”, from the surgical instrument image management information, and adds a character string “surgical instrument:” prepared in advance to the head. As a result, “surgical instrument: scalpel” which is surgical dynamic information is acquired.

  The accumulation unit 110 accumulates “surgical instrument: female”, which is the surgical dynamic information acquired by the surgical information acquisition unit 109, in the storage unit 111 in association with the surgical video frame received by the surgical video reception unit 102. Note that the frame of the operation video here is a frame acquired when the operation information acquisition unit 109 acquires the operation dynamic information. The association here is the same as in the case of device operation information.

  Here, the heart rate monitor 500 acquires the heart rate of the patient as needed, and device output information that is information in which the acquired heart rate value is associated with the identification information of the device of the heart rate monitor 500 itself. Assume that the data is output at a timing designated in advance, for example, at a timing when a heart rate value is acquired. For example, it is assumed that the heart rate monitor 500 outputs information obtained by combining the heart rate “42” and the “heart rate monitor HB 5001” that is the device identification information of the heart rate monitor 500 at a certain output timing.

  The device output information receiving unit 104 receives device output information transmitted from the heart rate monitor 500, and temporarily stores the received device output information in a storage medium (not shown).

  FIG. 11 is a diagram illustrating an example of device output information received and temporarily stored by the device output information receiving unit 104. The device output information has items of “device ID” and “output value”. “Device ID” is identification information of a device that has output device output information. The “output value” is a value output by the device that has output the device output information, for example, a measured value, a value calculated using the measured value, or the like.

  The surgery information acquisition unit 109 first acquires the device output information received and temporarily stored by the device output information reception unit 104 as one of the dynamic operation information. Specifically, the “output value” included in the device output information is acquired as the output value of the device indicated by the “device ID”. That is, the operation value “heart rate monitor HB5001: 42” indicating that the output value of the “heart rate monitor HB5001”, that is, the heart rate is “42” is acquired.

  Further, the surgery information acquisition unit 109 has an abnormality prepared in advance, in which the “output value” of the device output information received and temporarily stored by the device output information receiving unit 104 is associated with the “device ID” included in the device output information. It is determined whether or not a condition for detecting a value is satisfied. For example, here, a condition that “the output value is 50 or less” is stored in advance as a condition for determining an abnormality of the output value of the device including the character string “device ID” “heart rate”. If the medium is prepared, the surgery information acquisition unit 109 reads “heart rate monitor HB5001”, which is the “device ID” included in the device output information received and temporarily stored by the device output information receiving unit 104. As a condition associated with “device ID” that partially matches “heart rate HB5001”, “output value is 50 or less” associated with “device ID” including the character string “heart rate”. The condition is read from the storage medium or the like by searching. Then, it is determined whether or not the value of the output value included in the device output information is “50” or less. Here, the “output value” included in the device output information is determined to be “42” and “50” or less. Thereby, the surgery information acquisition unit 109 determines that the “output value” of the device whose “device ID” included in the device output information is “heart rate HB5001” is abnormal. Then, the character string prepared when the determination result is “abnormal” prepared in advance in association with the above-described conditions and the like is read from a storage medium (not shown). For example, the character string to be read is “heart rate: abnormal”. The operation information acquisition unit 109 acquires this character string as operation dynamic information. When the “output value” does not satisfy the condition associated with “device ID”, only the value of this “output value” is acquired as the operation dynamic information.

  The accumulation unit 110 receives the output value “42” of the “heart rate monitor HB5001” that is the operation dynamic information acquired by the operation information acquisition unit 109 and “heart rate: abnormal” by the operation image reception unit 102. The image is stored in the storage unit 111 in association with the video frame. Note that the frame of the operation video here is a frame acquired when the operation information acquisition unit 109 acquires the operation dynamic information. The association here is the same as in the case of device operation information. When the operation dynamic information “heart rate: abnormal” cannot be acquired, only the output value “42” of “heart rate monitor HB5001” which is one of the operation dynamic information is associated with the frame of the operation video. Accumulate in the storage unit 111.

  FIG. 12 is a schematic diagram illustrating the surgical sound received by the surgical sound receiving unit 106.

  The surgical information acquisition unit 109 is designated in advance by voice recognition processing or the like from the surgical voice received by the surgical voice receiving unit 106 when the surgical video receiving unit 102 receives a surgical video (for example, a frame of a surgical video). A process for detecting a voice indicating the specified condition is performed. For example, here, in order to detect a sound indicating that an abnormality has occurred, it is determined whether or not a sound having a volume larger than a previously designated volume has been output. Specifically, it is determined whether or not a sound volume of 80 dB abnormality is output. Usually, when an abnormality occurs, it is considered that a human voice increases. Therefore, it is determined whether an abnormality has occurred based on the volume of the sound.

  For example, if the volume of the surgical sound when receiving the surgical video is “92 dB”, the surgical information acquisition unit 109 determines that the surgical voice is a voice indicating that an abnormality has occurred. Then, “speech: abnormal”, which is a character string prepared in advance in association with the previously specified condition, is acquired as surgical dynamic information.

  The accumulation unit 110 accumulates “sound: abnormality”, which is the operation dynamic information acquired by the operation information acquisition unit 109, in the storage unit 111 in association with the frame of the operation image received by the operation image reception unit 102. Note that the frame of the operation video here is a frame acquired when the operation information acquisition unit 109 acquires the operation dynamic information. The association here is the same as in the case of device operation information.

  In addition, when the surgical image receiving unit 102 receives a new surgical video, the surgical information acquisition unit 109 compares the newly received surgical video frame with the surgical video frame received immediately before. Detect changes in lighting. For example, the average luminance of the pixels constituting the frames of the surgical video to be compared is calculated, and it is determined whether or not illumination has been added depending on whether or not the difference is equal to or greater than a predetermined value. The surgical video received immediately before the surgical video received by the surgical video receiving unit 102 may be read from the storage unit 111, or the same surgical video as the surgical video accumulated immediately before is stored in a storage medium such as a memory for comparison. The surgical image may be read out as it is stored.

  FIG. 13 is a diagram illustrating a surgical image for explaining processing for detecting illumination from the surgical image. FIG. 13A shows a frame of a new surgical image received by the surgical video receiving unit 102, and FIG. 13B shows a frame of a surgical video received immediately before by the surgical video receiving unit 102.

  For example, the pixel average luminance value calculated from the surgical video frame in FIG. 13A is “178”, and the pixel average luminance value calculated from the surgical video frame in FIG. ”. However, the value of the average luminance here is assumed to be between “0” and “256”. In this case, the difference in average luminance is 178−120 = 58. And it is detected whether this difference is more than the value "30" which shows having added the illumination. Here, since the difference is “30” or more, illumination is added. For this reason, the surgery information acquisition unit 109 acquires the surgery dynamic information “illumination change: addition” prepared in advance as the surgery dynamic information acquired when the difference is “30” or more.

  In addition, it is detected whether or not the difference in average luminance in a pre-specified area or the difference in luminance at an arbitrary position in one image is greater than or equal to a predetermined value. In some cases, for example, dynamic operation information indicating that a spotlight is turned on may be acquired.

  The accumulation unit 110 accumulates “illumination change: addition”, which is the operation dynamic information acquired by the operation information acquisition unit 109, in the storage unit 111 in association with the frame of the operation image received by the operation image reception unit 102. Note that the frame of the operation video here is a frame acquired when the operation information acquisition unit 109 acquires the operation dynamic information. The association here is the same as in the case of device operation information.

  As described above, by repeatedly performing the process of acquiring the surgical environment information and the dynamic surgical information and the process of storing the surgical information including the surgical environmental information and the dynamic dynamic information in association with the surgical video, Surgery videos associated with information can be stored in the storage unit 111.

  In this specific example, every time the operation information acquisition unit 109 acquires one piece of operation dynamic information, the storage unit 110 associates the operation image received by the operation image reception unit 102 with the operation dynamic information. However, in practice, a plurality of surgical dynamic information acquired almost simultaneously immediately after the surgical video reception unit 102 receives a surgical video (for example, a frame) is received at the same time. The operation image may be stored in association with a surgical image (for example, a frame) received by the unit 102.

  Here, an example in which the surgical image and the surgical information accumulated in the storage unit 111 are managed using management information for managing the correspondence relationship is shown.

  FIG. 14 is a diagram illustrating surgical information management information for managing surgical videos and surgical information stored in the storage unit 111 in association with each other. The surgery information management information includes items of “surgery video”, “file name”, “creation date”, “surgery environment information”, and “surgery dynamic information file”. The “surgical environment information” further has items of “doctor information” and “patient information”. “Surgery image” is an accumulated operation image. It is assumed that a time code is embedded or associated with the surgical video. “File name” is a file name of an operation video, and for example, a combination of a character string prepared in advance, a serial number, a date, or the like is used. Here, as an example, it is assumed that an operation video acquired during one operation constitutes one file. The “creation date” is the latest date on which the surgical video file was created. “Doctor information” is doctor information acquired by the surgery information acquisition unit 109. “Patient information” is patient information acquired by the surgery information acquisition unit 109. “Surgery dynamic information” is a file name of a surgical dynamic information management file that manages the correspondence between the surgical images accumulated in the storage unit 111 and the surgical dynamic information acquired by the surgical information acquisition unit 109. is there.

  The surgical video is managed in association with surgical environment information including doctor information and patient information by the surgical information management information shown in FIG. The surgical video and the surgical dynamic information acquired for the surgical video are managed by the surgical dynamic information management file.

  FIG. 15 is a diagram illustrating a surgical dynamic information management file for managing a correspondence relationship between surgical videos accumulated in the storage unit 111 and surgical dynamic information. This surgical dynamic information management file is assumed to be a surgical dynamic information management file “OP001_MF” associated with a surgical video whose “file name” is “OP001” in FIG. The operation dynamic information management file includes “time code” and “operation dynamic information”. The “time code” is a time code synchronized with the time code of the surgical video, and corresponds to the surgical video frame received by the surgical video accepting unit 102 when the surgical information acquisition unit 109 acquires the dynamic operation information. It is the attached time code. Here, since the process of acquiring the operation dynamic information is performed at an interval of 1 second, the time code stored here is a time code in which the number of frames is omitted. “Surgery dynamic information” is surgery dynamic information acquired by the surgery information acquisition unit 109. Here, various types of surgical dynamic information associated with the same time code are displayed together, but surgical dynamic information that is acquired differently may be managed using different items. .

  14 and 15 show an example in which the operation dynamic information is managed in association with the operation video using the operation dynamic information management file. The area where data such as the text of the surgical video (frame) received by the surgical video receiving unit 102 when the information is acquired can be arranged, and the temporal position on the text track of the surgical video file corresponding to this frame It may be arranged.

  FIG. 16 is a schematic diagram illustrating an example of a surgical video file in which surgical dynamic information is arranged in a data area of a frame. That is, it is a diagram showing an example of a surgical video file in which surgical dynamic information is multiplexed. In the figure, an operation image is arranged in the image area 150, and operation dynamic information is arranged in the data area 151 in association with the frame of the operation image. In the time code area 152, the time code of each frame is arranged. Here, for convenience of explanation, images of frames are shown at intervals of 1 second. Usually, however, there are a plurality of frames between the frames, and the surgical dynamic information is a frame within an interval of 1 second. Is associated with at least one of the first frames, for example, the first frame of each second.

  Here, a description will be given of a case where a process for searching for and displaying a surgical video is performed after the process of acquiring a surgical video is completed.

  FIG. 17 is a diagram illustrating an input screen for inputting a search condition for searching for a surgical image.

  First, for example, a doctor whose doctor identification information is “DR10012” first edits and summarizes a main part of a video of the contents of an operation performed on a patient whose patient identification information is “PT50033”. Suppose you want to check the video of the part.

  First, the doctor displays an input screen 160 as shown in FIG. 17 on a monitor or the like, and doctor identification information “DR10012” and patient identification are displayed in a field 161 for specifying doctor information and a field 162 for specifying patient information. It is assumed that information “PT50033” is input and the search button 163 is pressed by operating a mouse or the like.

  When the search button 163 is pressed, the surgery information designation receiving unit 112 receives the doctor identification information “DR10012” and the patient identification information “PT50033” input in the fields 161 and 162, respectively.

  The surgical image search unit 113 has “doctor information” that matches “DR10012” that is the designation of doctor information received by the surgical information designation receiving unit 112 in the surgical information management information shown in FIG. The designation receiving unit 112 detects a record having “patient information” that matches “PT50033” that is a designation of patient information received, and acquires a file name of the detected record and a preview image. As the preview image, for example, an image of a frame associated with pre-designated surgical dynamic information in the surgical video is acquired. Here, it is assumed that the surgical dynamic information designated in advance is surgical dynamic information including any of the character string “surgical instrument:”, the character string “abnormal”, and the character string “illumination change:”. . The preview image may be considered as a thumbnail image.

  Here, for example, it is assumed that a record having “doctor information” matching “DR10012” and “patient information” matching “PT50033” is detected, and the file name is “OP001”. The surgical image retrieval unit 113 includes the file name “OP001” and the surgical dynamic information associated with the surgical video, the character string “surgical instrument:”, the character string “abnormal”, and “illumination change”. The operation dynamic information including any of the character strings “:” is searched, and an image of a frame associated with the detected operation dynamic information is acquired.

  The output unit 114 reads the file name “OP001” detected by the surgical video search unit 113 and the frame image detected by the surgical video search unit 113 from the storage unit 111 and displays them on a monitor or the like in a matrix. Note that the time code associated with the read frame and the operation dynamic information may also be read and displayed.

  Further, in the retrieved surgical video, the doctor wants to browse only the surgical video of the location where the heart rate is abnormal and the location with the scalpel, and displays the same input screen for searching as in FIG. Assume that a character string “heart rate: abnormal, female” is input to the dynamic condition field 165, which is a field for designating a condition for narrowing the operation dynamic information. Note that “,” is a character string indicating an OR condition. Assume that the search button 163 is pressed by operating a mouse or the like.

  When the search button 163 is pressed, the surgery information designation accepting unit 112 accepts the information input in the dynamic condition field 165 as a condition for narrowing the surgery dynamic information.

  The surgical image search unit 113 uses the “dynamic heart rate: abnormal” or “female” in the surgical dynamic information management file as shown in FIG. 15 associated with the surgical video whose file name is “OP001” detected earlier. A record having “surgery dynamic information” partially matching any one of “” is detected, and a time code of the detected record is acquired. However, here, as an example, when the same surgical dynamic information appears continuously, only the time code corresponding to the oldest of the continuous surgical dynamic information is acquired. For example, if all of the time codes from “00:12:12” to “00:12:16” are associated with dynamic operation information “heart rate: abnormal”, it corresponds to “heart rate: abnormal”. Only “00:12:12” may be acquired as the time code.

  For example, in the surgical dynamic information management file shown in FIG. 15, since the “operative dynamic information” when the time code is “00:05:37” includes the surgical dynamic information “female”, this time code is To be acquired.

  The output unit 114 reads out an image of a frame corresponding to the time code detected by the surgical video search unit 113 from the surgical video whose file name is “OP001” stored in the storage unit 111 as a preview image, and outputs it to a monitor or the like. Display them in a matrix.

  FIG. 18 is a diagram illustrating a search result of frames in the surgical video. Each frame is an image of a frame associated with surgical dynamic information partially matching with either “heart rate: abnormal” or “female”. These frame images may be considered as images showing main scenes in the surgical video. Although an example of performing a narrow search is shown here, the search may be performed using information that designates surgical environment information and surgical dynamic information input by the user from the beginning.

  For example, when a user such as a doctor selects and reproduces an image of one frame via a reception unit (not shown) or the like, the output unit 114 displays a time code corresponding to the designated frame. The surgical image is read from the temporal position of, and the reproduction of the surgical image is started.

  Also, instead of the above search, the surgical information designation receiving unit 112 accepts the surgical dynamic information in the accumulation order specified by the information specifying the cumulative order of the pre-designated surgical dynamic information. It is also possible to search the temporal position of the operated surgical video and output the surgical video from this temporal position. For example, a surgical image starting from a frame associated with surgical dynamic information designated in advance, for example, surgical dynamic information indicating a temporal position at which a surgical condition has changed, is instructed by a user such as a doctor, You may enable it to read and display in order from the oldest accumulation order. For example, for each frame associated with surgical dynamic information serving as an index, surgical images may be cued and output in order from the oldest storage order.

  In the above specific example, as the designation of the surgical information for the search, character string information that is considered to be included in the surgical dynamic information, for example, a keyword is input. A range of values may be specified. For example, the operation information for designating the state in which the “output” of “heart rate” is “60 or less” is designated, and the operation motion including the character string matching “heart rate” and “output” is designated. Surgical information including a numerical value of “60” or less is retrieved, and a time code associated with the detected surgical dynamic information is acquired to detect a frame of the surgical video. Anyway.

  As described above, according to the present embodiment, by storing surgical dynamic information in association with a surgical video, it is possible to store a surgical video that can be easily reused. For example, by searching for operation dynamic information, it is possible to appropriately search and output a temporal position in an operation video.

  Thereby, for example, it is possible to easily search for a medically significant portion in the surgical image, the convenience of the surgical image is improved, and the value of the surgical image can be increased.

  In particular, information indicating changes such as information acquired during the operation is acquired as the operation dynamic information, and the temporal position of the operation image is searched using the operation dynamic information. It is possible to search and display the operation video of the place where the change has occurred, and to efficiently use the operation video.

  In the present embodiment, when the surgical video reception unit 102 receives a surgical video, processing for acquiring surgical information such as surgical dynamic information in real time and processing for associating the surgical information with the surgical video are not necessarily performed. There is no need to do it. For example, when information such as a time code that can be synchronized with a surgical video is included in the device operation information input to the surgical video storage device 1, the operation dynamic information acquired from the device operation information by using this time code Therefore, once the information such as device operation information is temporarily stored in the storage unit 111 or a storage medium (not shown), the operation dynamics are similar to the above using the stored device operation information. Information may be acquired, and the acquired operation dynamic information may be stored in association with the operation video using a time code or the like.

  When the surgical information acquisition unit 109 acquires surgical dynamic information indicating that there has been a change in information acquired during surgery, the storage unit 110 indicates that the surgical dynamic information indicates that there has been a change. May be stored in association with the surgical video only when the first is acquired or when the change is completed. For example, the occurrence or end of a change can be determined by whether or not the same surgical dynamic information has been acquired immediately before.

  In each of the above embodiments, each process (each function) may be realized by centralized processing by a single device (system), or by distributed processing by a plurality of devices. May be.

  Further, in each of the above embodiments, it goes without saying that two or more communication means (such as an information transmission unit) existing in one apparatus may be physically realized by one medium.

  In the above embodiment, information related to processing executed by each component, for example, information received, acquired, selected, generated, transmitted, and received by each component. In addition, information such as threshold values, mathematical formulas, addresses, etc. used by each component in processing is retained temporarily or over a long period of time on a recording medium (not shown) even when not explicitly stated in the above description. It may be. Further, the storage of information in the recording medium (not shown) may be performed by each component or a storage unit (not shown). Further, reading of information from the recording medium (not shown) may be performed by each component or a reading unit (not shown).

  In each of the above embodiments, the case where the surgical video storage device is a stand-alone has been described. However, the surgical video storage device may be a stand-alone device or a server device in a server / client system. . In the latter case, the output unit or the reception unit receives an input or outputs a screen via a communication line.

  In each of the above embodiments, each component may be configured by dedicated hardware, or a component that can be realized by software may be realized by executing a program. For example, each component can be realized by a program execution unit such as a CPU reading and executing a software program recorded on a recording medium such as a hard disk or a semiconductor memory.

  In addition, the software which implement | achieves the surgery image | video storage device in each said embodiment is the following programs. That is, this program uses a computer to obtain surgical information including a surgical video reception unit that receives one or more surgical videos that are videos during surgery and surgical dynamic information that is information related to information that can be acquired during surgery. Functions as a storage unit that stores surgical video and surgical information by associating surgical dynamic information with a portion corresponding to surgical dynamic information in one or more surgical videos received by the acquisition unit and the surgical video receiving unit It is a program to make it.

  In the program, the functions realized by the program do not include functions that can be realized only by hardware. For example, a function that can be realized only by hardware such as a modem or an interface card in an acquisition unit that acquires information or an output unit that outputs information is not included in the function realized by the program.

  Further, the computer that executes this program may be singular or plural. That is, centralized processing may be performed, or distributed processing may be performed.

  FIG. 19 is a schematic diagram showing an example of the external appearance of a computer that executes the program and realizes the surgical image storage apparatus according to the embodiment. The above-described embodiment can be realized by computer hardware and a computer program executed on the computer hardware.

  In FIG. 19, a computer system 900 includes a computer 901 including a CD-ROM (Compact Disk Read Only Memory) drive 905 and an FD (Floppy (registered trademark) Disk) drive 906, a keyboard 902, a mouse 903, a monitor 904, and the like. Is provided.

  FIG. 20 is a diagram illustrating an internal configuration of the computer system 900. 20, in addition to the CD-ROM drive 905 and the FD drive 906, a computer 901 is connected to an MPU (Micro Processing Unit) 911, a ROM 912 for storing a program such as a bootup program, and the MPU 911. A RAM (Random Access Memory) 913 that temporarily stores program instructions and provides a temporary storage space, a hard disk 914 that stores application programs, system programs, and data, and an MPU 911 and a ROM 912 are interconnected. And a bus 915. The computer 901 may include a network card (not shown) that provides connection to the LAN.

  A program for causing the computer system 900 to execute the functions of the surgical image storage device and the like according to the above-described embodiment is stored in the CD-ROM 921 or the FD 922, inserted into the CD-ROM drive 905 or the FD drive 906, and the hard disk 914. May be forwarded to. Instead, the program may be transmitted to the computer 901 via a network (not shown) and stored in the hard disk 914. The program is loaded into the RAM 913 when executed. The program may be loaded directly from the CD-ROM 921, the FD 922, or the network.

  The program does not necessarily include an operating system (OS), a third party program, or the like that causes the computer 901 to execute the functions of the surgical image storage apparatus according to the above-described embodiment. The program may include only a part of an instruction that calls an appropriate function (module) in a controlled manner and obtains a desired result. How the computer system 900 operates is well known and will not be described in detail.

  The present invention is not limited to the above-described embodiments, and various modifications are possible, and it goes without saying that these are also included in the scope of the present invention.

  As described above, the surgical image storage device according to the present invention is suitable as a device for storing surgical images, and is particularly useful as a device for storing surgical images so as to be used efficiently.

DESCRIPTION OF SYMBOLS 1 Surgical image storage apparatus 50, 60 Equipment 51 Operation reception part 52, 62 Equipment identification information storage part 53, 63 Transmission part 61 Output information acquisition part 101 Imaging part 102 Surgery video reception part 103 Equipment operation information reception part 104 Equipment output information Reception unit 105 Voice acquisition unit 106 Surgery voice reception unit 107 Doctor identification information reception unit 108 Patient identification information reception unit 109 Surgery information acquisition unit 110 Storage unit 111 Storage unit 112 Surgery information designation reception unit 113 Surgery image search unit 114 Output unit

Claims (12)

A surgical video reception unit that receives one or more surgical videos that are videos during surgery;
A surgery information acquisition unit that acquires surgery information including surgery dynamic information that is information about information that can be acquired during surgery;
A storage unit for storing the surgical video and the surgical information by associating the surgical dynamic information with a portion corresponding to the surgical dynamic information of one or more surgical videos received by the surgical video receiving unit. Surgical image storage device equipped. A device used for surgery further includes a device operation information receiving unit that receives device operation information to be output in response to an operation during the operation,
The surgical image storage apparatus according to claim 1, wherein the surgical information acquisition unit acquires dynamic operation information according to device operation information received by the device operation information reception unit. The surgical image storage device according to claim 1, wherein the surgical information acquisition unit acquires dynamic surgical information according to information on a surgical instrument used for surgery. A device output information receiving unit for receiving device output information output by a device used for surgery;
The surgical image storage device according to any one of claims 1 to 3, wherein the surgical information acquisition unit acquires dynamic operation information according to device output information received by the device output information reception unit. A surgical sound reception unit that receives one or more surgical sounds that are voices during surgery;
The operation image according to any one of claims 1 to 4, wherein the operation information acquisition unit detects a feature of the operation sound received by the operation sound reception unit, and acquires operation dynamic information according to the detected feature. Storage device. The surgical image accumulating device according to any one of claims 1 to 5, wherein the surgical information acquisition unit detects a change in illumination during surgery and acquires dynamic operation information according to the detected change in illumination. The operation information acquisition unit acquires operation information further including operation environment information that is information having doctor information or patient information that is information about a patient, which is information about a doctor who performs an operation,
The surgical image storage device according to any one of claims 1 to 6, wherein the storage unit stores one or more surgical videos received by the surgical video reception unit and surgical environment information included in the surgical information in association with each other. A doctor identification information receiving unit that is stored in the terminal held by the doctor and receives the doctor identification information of the doctor transmitted from the terminal;
The surgical image storage device according to claim 7, wherein the surgical information acquisition unit acquires surgical environment information including doctor information including doctor identification information received by the doctor identification information receiving unit. A patient identification information receiving unit which is stored in the terminal held by the patient and receives the patient identification information of the patient transmitted from the terminal;
The surgical image storage device according to claim 7 or 8, wherein the surgical information acquisition unit acquires surgical environment information including patient information including patient identification information received by the patient identification information reception unit. A surgery information designation receiving unit for designating the surgery information;
A surgical video search unit for searching for a surgical video associated with the surgical information received by the surgical information designation receiving unit;
The surgical video storage apparatus according to claim 1, further comprising an output unit that outputs a surgical video searched by the surgical video search unit. A surgical video storage method performed using a surgical video reception unit, a surgical information acquisition unit, and a storage unit,
A surgical video reception step in which the surgical video reception unit receives one or more surgical videos, which are videos during surgery;
The surgical information acquisition unit acquires surgical information including surgical dynamic information that is information related to information that can be acquired during surgery; and
The storage unit associates the surgical dynamic information with a portion corresponding to the surgical dynamic information of one or more surgical videos received in the surgical video reception step, and stores the surgical video and surgical information. An operation video accumulation method comprising: an accumulation step. Computer
A surgical video reception unit that receives one or more surgical videos that are videos during surgery;
A surgery information acquisition unit that acquires surgery information including surgery dynamic information that is information about information that can be acquired during surgery;
Functions as a storage unit that stores the surgical video and the surgical information by associating the surgical dynamic information with a portion corresponding to the surgical dynamic information of one or more surgical videos received by the surgical video receiving unit. Program to let you.

Images