JP2006302057A - Medical voice information processing apparatus and medical voice information processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a medical voice information processing apparatus capable of detecting a specific language expression or the like from a conversation or instruction exchanged in a treatment room and detecting an event corresponding to a treatment action.
SOLUTION: A voice processing device 1 includes a processing device main body 4 that performs processing such as recording medical voice information 3, a plurality of microphones 6 for acquiring generated voice as voice information 5, and treatment action presentation information 21. Is mainly composed of a liquid crystal display 8 and a speaker 9 for presenting a message. Then, from the voice information 5 acquired by the microphone 6, a characteristic language expression 25 for the treatment action stored in the language database 28 is detected in a predetermined time width, and an event indicating a main part of the treatment action is detected. . As a result, various events occurring in the treatment practice that is continuously performed can be detected in real time and recorded in the event index 2.
[Selection] Figure 1

Description

  The present invention relates to a medical speech information processing apparatus and a medical speech information processing program, and in particular, extracts a specific language expression included in an instruction, a conversation, or the like at the time of a treatment action, and is implemented according to a standard treatment program. The present invention relates to a medical voice information processing apparatus and a medical voice information processing program capable of detecting a treatment action or event.

  Conventionally, in a medical field such as emergency lifesaving medical care, a procedure of a series of treatment actions has been previously flowcharted in order for a doctor to accurately determine the state of a transported patient and perform appropriate and prompt treatment, Standard treatment programs (for example, ACLS and JATEC, etc.) that specify specific contents of the treatment action and clear conditions for transitioning to or branching to the next treatment action are adopted and used. The therapist confirms the patient's condition, the response to the treatment, the result, and the like, and determines and executes the next treatment action based on the transition or branch condition defined in the standard treatment program. At this time, it is possible to prevent the therapist from making a mistake and to perform the treatment promptly by providing the therapist with accurate information for transitioning from a certain treatment to the next treatment. When transitioning from one therapeutic action to the next, the therapist is based on information such as vital signs such as heart rate, pulse rate, and blood pressure, presence / absence of consciousness level, trauma, and radiographic results. Judgment is made in accordance with transition conditions defined in advance in the standard treatment program.

  At this time, instructions from the doctor to other doctors, nurses, or the like, or will indications such as confirmation from the nurses, are mainly performed by voice. That is, the voice in the treatment room is one of important communication tools for sharing the mutual recognition. In addition, since voices such as conversations uttered in the treatment room need to be dealt with quickly and appropriately, they are performed with simple and accurate language expressions such as words and short phrases.

  On the other hand, the therapist, after performing general medical care and treatment, or treatment according to the standard treatment program as described above, the specific content of each treatment and treatment, and the patient's past history, etc. In order to record information, a chart for each patient is created. For the purpose of simplifying the work of creating a medical chart performed after this, an electronic medical chart system that captures the medical information converted into data by capturing the state of the treatment room as video information is being constructed. Furthermore, a system and an apparatus that can acquire details of treatment contents as audio information together with the video information have already been disclosed (see, for example, Patent Document 1 and Patent Document 2). In addition, a system has been developed in which a tag is attached in advance to audio included in a recorded video and the video or audio is extracted in association with the tag (see, for example, Non-Patent Document 1). As a result, it is possible to easily summarize videos taken continuously in time.

Japanese Patent No. 3074769 JP 2002-215797 A Katsuhiro Hashisako, 2 others, “Audio tag extraction for video summarization”, Proceedings of 10th Image Sensing Symposium, Image Sensing Technology Study Group, 2004, p179-184

  When performing general medical care and treatment, or treatment according to the standard treatment program as described above, the therapist records the specific details of each treatment and treatment, and the patient's past history, A chart was created for each patient. In particular, in lifesaving medical care, it is a matter of course that patient life is given the highest priority. Therefore, all descriptions of medical treatments and records of medical treatment results (medical records) must be written after the fact. It was usual. For this reason, the medical chart preparation relied on the memory of the past, in other words, while remembering the details of the treatment he had performed each time. Therefore, the efficiency of chart preparation is poor, and inaccurate or unclear portions may remain or omissions may occur.

  In order to assist information not described in the medical record, video input equipment such as a video camera is installed in the treatment room, and the flow from when the patient is transported to the completion of all treatment actions is recorded as video information. There was also a case. However, although these video input devices are installed near the ceiling of the treatment room and capture the whole treatment room as a bird's-eye view image, the shooting angle is often fixed, but treatment is often performed. Necessary images may not always be acquired due to the influence of a shield such as a person who performs the treatment or a medical device. At this time, by using a plurality of video input devices, measures have been taken to eliminate the above-mentioned adverse effects, but although the overall flow of treatment actions can be grasped, the movement of each treatment action and the state of the affected part, It was very difficult to confirm the judgment of the treatment technique of the therapist. In addition, there is a method that uses a video input device with variable shooting angle and zoom ratio and controls it manually. However, it is difficult to control when there are multiple people and video input devices. There wasn't. Furthermore, it was necessary to check the contents of recorded video and audio for inputting medical charts, etc., but the operation required a great deal of labor. For this reason, there is a known function that improves the workability by providing a function for searching for a target location using key information, but the provided key information is not systematically classified and is efficient. It was bad.

  In particular, when a medical chart or the like is created after the treatment is performed, images and sounds from the start to the end of the treatment are sequentially reproduced, and scenes necessary for creating the medical chart may be found in a state close to manual work. In addition, it is possible to record audio information (medical audio information) that facilitates subsequent analysis, search processing, etc. in conjunction with recording of audio information and standard treatment programs with prescribed treatment procedures. There was almost nothing. In addition, when recognizing a speech or the like during a treatment action, performing a similar recognition process on all the speech requires very high recognition accuracy and processing capability of the apparatus body. For this reason, there is a need for a language expression that has a high possibility of appearing in advance and that can improve the detection accuracy of a treatment action or an event corresponding to the treatment action.

  Therefore, in view of the above situation, a medical speech information processing apparatus capable of detecting a specific language expression or the like from conversations or instructions exchanged in a treatment room and detecting a key action corresponding to the treatment action or treatment action, and It is an object to provide a medical voice information processing program.

  In order to solve the above-described problems, the medical audio information processing apparatus according to the present invention provides a “treatment for presenting treatment action presentation information corresponding to the treatment action based on a standard treatment program in which the procedure of the treatment action is standardized. Action presentation means, treatment action recording means for recording and recording the treatment actions in an event index in time series in synchronization with the presented treatment action presentation information, and voice information generated during the treatment actions as audio information Voice information acquisition means for acquiring, a characteristic language expression indicating the treatment action, and a language database storage means for storing, in a language database, statistical information calculated for each treatment action, the expression possibility of the language expression; Extraction means for extracting the linguistic expression stored in the language database from the acquired voice information; Treatment event detection means for detecting the treatment action of the standard treatment program or an event corresponding to the treatment action using statistical information, the detected treatment action or the event, the extracted language expression, And voice information recording means for recording medical voice information including at least one of the expression times of the language expressions in the event index in a structured manner.

  Here, the standard treatment program is standardized according to a series of flows, as typically shown in ACLS (Advanced life support) used in the field of emergency medicine. Judgment criteria such as branching or transition conditions are defined between each treatment action. ACLS is a technique for cardiopulmonary resuscitation that is adopted by medical institutions such as emergency lifesaving centers and performed by medical doctors or those who have received sufficient training using medical equipment and medicines under the guidance of doctors. It is now recognized as the global standard for cardiopulmonary resuscitation. The standard treatment program currently established is limited to cardiopulmonary resuscitation, but is also being established for other cases such as impaired consciousness, severe burns, gas / drug addiction, trauma, etc.

  Based on the standard treatment program, the treatment is performed, and the specific treatment content and method, the medical device to be used, the type of medicine to be used and the administration condition, and the patient's response to the treatment, etc. Information etc. are recorded. The treatment action presentation information is information for executing a treatment action based on a standard treatment program, for example, treatment content data indicating the current treatment contents, and determining the next treatment action to be performed. It includes condition data indicating the judgment criteria. This treatment action presentation information may be formed in the form of treatment action group information obtained by connecting these pieces of information for each treatment action.

  Furthermore, the treatment action presentation means specifically presents the contents of the standard treatment program selected by the therapist to the practitioner who actually carries out the treatment action. For example, the treatment action presentation means Treatment content data and condition data are displayed on the display so that the therapist can perform the treatment while confirming the treatment. Furthermore, it is also possible to present information such as the elapsed time after being transported to the treatment room and the time since the transition to the treatment action. Furthermore, the content of the treatment action may be transmitted through the hearing of the therapist via a speaker or the like.

  Here, the event index is a record in which various types of events detected from video information, audio information, text information, sensor information, etc. are integrated according to time series, and the types of information and event contents are structured and recorded. It is. Furthermore, by utilizing medical expertise, it is possible to record the relevance between events. Then, by referring to the event index, it becomes possible to classify events, select various information at the time of reproduction, and the like, making it easy to use a variety of information.

  Furthermore, with the voice input device, the number and location of installation of voice input devices composed of microphones and the like can be selected according to the condition of the treatment room. At this time, when the voices of individual therapists are identified and acquired individually, or installed in advance at a plurality of locations in the treatment room, the voices of conversations and instructions of the therapists and the devices emitted from the medical equipment in the treatment room When operating sound, alarm sound, and the like are collectively acquired as one sound information, they can also be acquired in combination.

  In addition, the language database is a language expression that is highly likely to occur during treatment, such as “tracheal intubation”, “tracheotomy”, “circular thyroid ligament puncture”, or “defibrillation”. Names of medical devices and instruments used for treatment such as “Instrument”, “Echo”, “X-ray”, etc., “Blood pressure ○○”, “Pulse △△”, “Heart arrest” etc. Reports of vital values, etc., or phrases and terms such as names of drugs to be administered are recorded together with statistical information indicating the possibility of frequent occurrence. Here, statistical information is set high for languages such as “tracheal intubation” and “tracheotomy” in the case of treatment aimed at securing the airway, for example, “echo” and “radiography” It is set low for such languages. On the other hand, when the patient complains of pain in the abdomen and the consciousness and breathing are stable, the language such as “tracheal intubation” and “tracheotomy” described above is very unlikely. There is a high possibility that the name of a medical device such as “echo” for confirming the bleeding status will be expressed during treatment. For this reason, statistical information indicating the possibility of expression calculated statistically in advance is recorded for each language so as to correspond to each language.

  Therefore, according to the medical audio information processing apparatus of the present invention, treatment action presentation information is presented based on a standard treatment program, and records of the performed treatment actions are structured and recorded in an event index in a time series. . At this time, the voice of the therapist or the like uttered during the treatment is acquired as voice information through a voice input device such as a microphone. Then, a linguistic expression characteristic of the treatment action included in the acquired speech information is extracted from a language database stored together with statistical information obtained by quantifying the expression possibility for the linguistic expression and the treatment action. That is, since the extraction process is limited to language expressions that are likely to be detected in advance, the recognition accuracy is improved. Then, the linguistic expression extracted corresponding to the treatment action or the like is structured and recorded in synchronization with the event index. This makes it possible to easily perform analysis and verification work for the subsequent treatment action.

  Furthermore, the medical speech information processing apparatus according to the present invention has, in addition to the above configuration, “a plurality of language expressions extracted within a predetermined time range before and after the language expression is expressed. May be detected based on at least one of the combination of the above, the number of extractions of the language expression, and the expression order of the language expression.

  Therefore, according to the medical sound information processing apparatus of the present invention, the treatment event detection means performs the treatment action based on the combination of the language expressions included in the predetermined time width, the number of extractions of the language expressions, the expression order, and the like. Or an event is detected. For example, within a minute after the first language expression “tracheal intubation” is detected for a predetermined time span, the same “tracheal intubation” language expression is repeatedly detected three times or more, or “tracheal intubation”. The case where “tracheostomy” is detected subsequently is mentioned. As a result, the specific linguistic expression included in the speech information is detected only once, and the treatment action, event, or the like is not detected, but is detected a plurality of times in consideration of the expression order. As a result, the detection accuracy of the treatment action and the like is improved, and erroneous information is less likely to be recorded in the event index. Note that the determination of the predetermined time width, the order of expression, and the like may be determined in advance based on medical knowledge.

  Further, the medical audio information processing apparatus according to the present invention has, in addition to the above-described configuration, “the treatment event detection unit selects and detects one of a plurality of treatment actions or event candidates, and is not selected. The treatment action or the event held in the undetected information from the treatment action or the event detected after the event, further comprising undetected information holding means for holding the treatment action or the event as undetected information. It may also be provided with a correcting means for correcting "."

  Therefore, according to the medical sound information processing apparatus of the present invention, a plurality of candidates are listed as treatment actions or events, and a particularly suitable one is recorded as a treatment action or event. At this time, at the time of the search, information regarding treatment actions and events that are not selected can be temporarily stored and used for an auxiliary role in detecting subsequent treatment actions and the like. That is, even if the treatment action or event is determined once, it may be found that an incorrect treatment action has been selected in the previous stage as a result of the subsequent treatment action. In such a case, instead of returning to the post-event and re-detecting the treatment action etc., the undetected information recorded by the non-detected information holding means held at the time of selection is changed to the subsequent status. The closest treatment action is detected. Thus, it is possible to quickly correct a prior treatment action without requiring a new treatment action or event detection process. As a result, the load on the medical sound information processing apparatus is reduced, and correct medical sound information is recorded in the event index by the correction.

  Furthermore, in addition to the above-described configuration, the medical audio information processing apparatus according to the present invention includes “first scoring means for scoring the extracted linguistic expression, and video acquired corresponding to the treatment action or the event. And second scoring means for scoring at least one modality information of information, text information, and sensor information, wherein the treatment event detection means integrates the scored language expression and the modality information. And detecting the therapeutic action or the event ”.

  Here, modality information refers to video information obtained by photographing the inside of a treatment room obtained via a video input device such as a camera, treatment results and treatment actions obtained via a text input device such as a console. This includes text information that has been converted into text and sensor information sensed by a medical device or a non-medical device, and integrates all information related to the treatment action other than the voice information described above.

  Furthermore, the first scoring means and the second scoring means calculate the degree of match for the treatment action with respect to the detected linguistic expression, and digitize it (score it). The first score obtained from the information and the second score obtained from the other modality information are integrated, and the treatment event is detected based on the combined numerical value.

  Therefore, according to the medical audio information processing apparatus of the present invention, both the extracted linguistic expression and the modality information including other images are scored (digitized), and the treatment action or event is detected based on the result. It becomes possible to do. In other words, compared to treatment actions or events detected only by voice information, by combining modality information such as other images, treatment actions can be taken from detailed actions that could not be detected from the linguistic expression of voice information. It becomes possible to recognize the contents in particular.

  On the other hand, the medical audio information processing program according to the present invention provides a treatment action presentation means for presenting treatment action presentation information corresponding to the treatment action based on a standard treatment program in which the procedure of treatment action is standardized. Treatment information recording means for recording and recording the treatment actions in an event index in time series in synchronization with the treatment action presentation information, voice information acquisition means for obtaining sound generated during the treatment actions as voice information, Language database storage means for storing, in a language database, characteristic language expressions indicating the treatment action, and statistical information obtained by calculating the expression possibility of the language expression for each treatment action, from the acquired voice information, the language Extracting means for extracting the language expression stored in the database, using the statistical information from the extracted language expression, Treatment event detection means for detecting the treatment action of the standard treatment program or an event corresponding to the treatment action, the extracted language expression of the recognized treatment action or event, and expression of the language expression It is mainly composed of what makes the medical audio information processing apparatus function as audio information recording means for structuring and recording medical audio information including at least one of the times in the event index.

  Furthermore, in addition to the above configuration, the medical speech information processing program according to the present invention includes a “combination of a plurality of language expressions extracted within a predetermined range of time before and after the expression of the language expression, the language expression. The medical speech information processing apparatus further functions as the event recognition means for detecting the treatment action or the event based on at least one of the number of times of extraction and the order of expression of the language expression ” It doesn't matter.

  In addition to the above configuration, the medical audio information processing program according to the present invention may be configured such that “one of a plurality of the therapeutic actions or the event candidates is selected and recognized, and the therapeutic actions or events that are not selected are selected. As the unrecognized information holding means for holding as unrecognized information and the correcting means for correcting the therapeutic action or event held in the unrecognized information from the therapeutic action or event recognized after the fact, The voice information processing apparatus may further function ”.

  Further, the medical audio information processing program according to the present invention is, in addition to the above configuration, “video information acquired in response to the first scoring means for scoring the extracted language expression, the treatment action or the event. Second scoring means for scoring at least one modality information of text information and sensor information, and integrating the scored language expression and modality information to detect the treatment action or the event The medical sound information processing apparatus may further function as the treatment event detection means.

  Therefore, according to the medical audio information processing program of the present invention, it is possible to cause the medical audio information processing apparatus to perform the above-described operation by executing the program.

  As an effect of the present invention, a record relating to each treatment action executed based on the standard treatment program is recorded in synchronization with the event index, and from the conversations and instructions exchanged during the treatment, it is characteristic of the treatment action. Extract linguistic expressions and detect the content and events of treatment. In particular, the detection accuracy of the language expression and the event are detected by statistical information preliminarily stored in the language database in consideration of the possibility of the treatment action, so that the detection accuracy is remarkably improved. Furthermore, since the detection of the language expression is performed corresponding to a predetermined time width, the detection of the event or the like is clearly performed by the combination of the language and other language expressions, the expression order of the language expressions, and the like. As a result, the detection accuracy of the detected event and the like is improved, and subsequent operations such as search and reproduction can be performed efficiently.

  Next, a medical voice information processing apparatus 1 (hereinafter simply referred to as “voice processing apparatus 1”) according to the present embodiment will be described with reference to FIGS. Here, FIG. 1 is a block diagram showing a schematic configuration of the voice processing device 1 of the present embodiment, and FIGS. 2 and 3 are explanatory diagrams schematically showing voice recognition by the voice processing device 1, FIG. FIG. 5 is a flowchart showing the processing flow of the audio processing apparatus 1, and FIG. 6 is an explanatory diagram showing an example of the event index 2.

  As shown in FIGS. 1 and 2, the audio processing device 1 of the present embodiment is disposed in a treatment room and is connected to a processing device main body 4 that performs processing such as recording of medical audio information 3, and the processing device main body 4. A plurality of microphones 6 for acquiring voice generated in the treatment room as voice information 5 and treatment action presentation information 21 indicating the content of each treatment action and transition conditions performed in accordance with the standard treatment program 20 It is mainly composed of a liquid crystal display 8 and a speaker 9 for presenting to a therapist through video and audio. In addition, the voice processing device 1 has other configurations such as an operation input console 11 for inputting the treatment results and findings for the therapeutic action as text information 10 and inputting the text information 10, and various types for grasping the state of the treatment room. Medical device 13 for acquiring sensor information 12 (for example, an electrocardiogram monitor for measuring a vital sign such as a heart rate of a patient) or a non-contact type for detecting a motion of a therapist or the like in a treatment room The various sensors 14 and the like are connected to a video input device 16 that takes a picture of the treatment room and acquires video information 15. Here, the sensor information 12 acquired by the various sensors 14 which are the medical device 13 or the non-medical device, the text information 10 input from the console 11, and the video information 15 input from the video input device 16 are the modality information 7. Are integrated and recorded. Here, the microphone 6 corresponds to the voice information acquisition means in the present invention, and the liquid crystal display 8 and the speaker 9 correspond to a partial function of the information presentation means 19 described later.

  Further, the processing device main body 4 has, as its functional configuration, the treatment action presentation information 21 describing the content of each treatment action, the transition conditions, and the like based on the standard treatment program 20 in which the procedure of the treatment action is standardized. The program storage means 22 to be stored, the information presentation means 19 for presenting the treatment action presentation information 21 stored in the program storage means 22 via the liquid crystal display 8 and the speaker 9, and the treatment action presentation information 21 to be presented are synchronized. The treatment action recording means 23 for recording the treatment action and the treatment record related to the treatment action in a time series in the event index 2 and the voice generated during the treatment action as audio information 5 through the microphone 6 Speech information acquisition means 24 to be acquired, dictionary information 26 for storing characteristic language expressions 25 representing therapeutic actions, and treatment of the language expressions 25 Language database storage means 29 for storing statistical information 27 in which the expression possibility for each purpose is calculated in advance in the language database 28, and language expression that matches the dictionary information 26 stored in the language database 28 from the acquired speech information 5 25, extraction means 30 for extracting 25, treatment event detection means 31 for detecting a treatment action corresponding to the standard treatment program or an event for the treatment action based on the statistical information 27 for the extracted language expression 25, and detected Audio information structured and recorded in the event index 2 as medical audio information 3 including the data of the extracted language expression 25 and the expression time 51 (see FIG. 6) when the language expression is expressed. The recording unit 34 is mainly configured. The information presenting means 19 corresponds to the therapeutic action presenting means in the present invention.

  Here, when the characteristic event 25 is extracted from the language database 28, the treatment event detection unit 31 is related to the number of extractions of the predetermined time width T extracted from the language expression 25 and the language expression 25. It includes a time width detecting means 35 for detecting a therapeutic action or event according to the extraction (combination) of the language expressions 25 to be performed or the order of expression of the plurality of language expressions 25.

  In addition, as another functional configuration, the processing apparatus body 4 is configured such that when one of a plurality of candidate events 36 is selected in the treatment event detection unit 31 for detecting a treatment action or event (selected event), Undetected information recording means 39 for recording information on other non-selected events that have not been selected as undetected information 38, and non-selected events included in the undetected information 38 according to the progress of treatment action, etc. Re-detection and correction means 40 for correcting the treatment action or event, first scoring means 41 for digitizing the extracted language expression 25 based on statistical information 27 and the like, and recorded in the processing device body 4 The second scoring means 42 for scoring the modality information 7 such as the video information 15 and the text information 10 excluding the audio information 5 is mainly configured. That. Information digitized by the first scoring means 41 and the second scoring means 42 is recorded as score information 45. The processing device body 4 used in the sound processing device 1 of the present embodiment has functions for recording and managing various data and selectively controlling various recording devices for acquiring information. Here, it is constructed by applying a general-purpose personal computer. Therefore, these processing device main bodies 4 include an arithmetic circuit such as a CPU, an interface circuit and an interface device for transmitting and receiving various signals, and a storage medium such as a hard disk and a semiconductor memory for recording and storing the acquired information. It is constructed by a hardware configuration including

  Here, the treatment event detection means 31, the undetected information recording means 39, and the correction means 40 of the present embodiment will be described in more detail. In accordance with the standard treatment program 2, as schematically shown in FIG. The therapeutic action performed along the time series and the events that occur with respect to the therapeutic action are extracted using the audio information 5. Here, the arrows directed downward in the drawing in FIG. 2 (and FIG. 3) indicate the flow along the time series of the standard treatment program 20. Then, for example, in FIG. 2, when the therapeutic action A0 (corresponding to the upper stage in FIG. 2) is performed, the specific language expression 25 detected in the specific time width T of the therapeutic action A0 A plurality of candidate events 36 (corresponding to the lower part in FIG. 2) having a possibility of being implemented are presented. In FIG. 2, a figure represented by a square shape represents each event. Then, from the detection of the language expression 25, a plurality of events A1 to AX are presented for the treatment action A0. Each event is scored by integrating the language expression 25 and other modality information 7 and is stored in association with the score value 46. In FIG. 2, a numerical value “<5>” is added to the event name along with the event name. It is also illustrated with ~ “<0>”. In this embodiment, the score value 46 is expressed in six stages from “<5>” to “<0>”, and the score “<5>” may match the language expression 25. A high therapeutic action or event is indicated, while a “<0>” indicates that the therapeutic action or event is less likely to match. The score obtained by scoring the statistical information 27 included in the language database 28 described above, the extracted speech information 5, the implementation status of the standard treatment program 20, the modality information 7 and the like from the candidate events 36 described above. Based on the information 45, a treatment action or event that is likely to be performed next is searched and determined as a selected event (shown in a double frame). In other words, event B is selected as the selected event in FIG. On the other hand, events that have not been selected (candidate events 36 other than the selected event) are listed as non-selected events, and the information is stored as undetected information 38.

  At this time, when it is recognized that the previously detected treatment action or the like is wrong by the subsequent detection, the event detected by the correcting means 40 of the present embodiment is corrected. Specifically, if an event detected immediately before (or before) event B in FIG. 3 (here, event A1 is equivalent) is determined to be a false detection, A correct event (corresponding to event A3) is detected again from the non-selected events. Then, the audio information 5 is further recognized from the event A3, and an event that may be implemented is selected from the candidate events 36 and determined (corresponding to the event B3). That is, even if the event has been determined once, if the determination is inappropriate after the fact, the stored undetected information 38 and the correction means 40 correct the event. Therefore, it is expected that the detection accuracy of the event based on the audio information 5 is increased.

  Next, the flow of processing of the speech processing apparatus 1 of the present embodiment will be described based on FIGS. 4 and 5. Here, step S1 to step S16 in the flowcharts of FIGS. 4 and 5 correspond to the medical audio information processing program of the present invention.

  First, various devices such as the processing device main body 4 of the voice processing device 1 and the microphone 6 connected to the processing device main body 4 are activated and put into operation (step S1). Then, the therapist diagnoses the condition of the patient transported to the treatment room, and makes a diagnosis of the patient based on the findings immediately after the transport. The standard treatment program to be used is then selected. At this time, the voice processing device 1 receives an input of an instruction to execute the selected standard treatment program 20 via the console 11 connected to the processing device main body 4 and executes the standard treatment program 20. (Step S2). As a result, the processing apparatus body 4 presents the treatment action presentation information 21 presented for each treatment action stored in the program storage means 22 to the therapist through the liquid crystal display 8 and the speaker 9 (step S3).

  Thereafter, a treatment action is performed on the patient by the therapist who has confirmed the content of the presented treatment action presentation information 21. At this time, the voice processing device 1 acquires voices such as conversations and instructions between the therapists uttered in the treatment room as the voice information 5 by the plurality of microphones 6 installed in the treatment room (step S4). Furthermore, at the same timing, various information is acquired and recorded by other devices connected to the processing apparatus body 4 (step S5). More specifically, the video information 15 obtained by photographing the state of the treatment room is acquired by the connected video input device 16, and the text information 10 such as the result of the treatment action and the findings obtained by using the console 11 is acquired. Then, the sensor information 12 is acquired from various sensors 14 including non-medical devices such as medical devices 13 or position sensors arranged in the treatment room.

  Thereafter, the processing device main body 4 extracts a language expression 25 related to a treatment action or a characteristic (key) event corresponding to the treatment action included in the acquired audio information 5 based on the language database 28 ( Step S6). At this time, in the language database 28, the dictionary information 26 that stores the language expression 25 related to the treatment action, and the statistical information that the expression possibility of the language expression 25 about the treatment action is statistically calculated and the result is recorded. 27. Therefore, the treatment action corresponding to the treatment action presentation information 21 presented by the standard treatment program 20 is recognized, and the language expression 25 is extracted based on the statistical information 27. At this time, as described above, the linguistic expression 25 may be extracted by detecting and extracting one linguistic expression 25 included in the speech information 5 using speech recognition or word spotting technology. In order to detect the treatment action or event with higher accuracy, the following method is used.

  That is, as shown in FIG. 2 or FIG. 3, when detecting a language expression 25a characteristic of an event or the like included in the audio information 5, the related language expression 25b related to the language expression 25a within a predetermined time width T. , 25c is included, the detection accuracy of events and the like is improved by recognizing these plural language expressions 25a. That is, since there are many other language expressions in the audio information 5 other than those related to the treatment action or the event, it is the process of the audio processing device 1 that recognizes all of them and performs the detection process associated with the event or the like. The burden on the apparatus main body 4 becomes excessive. Therefore, it is determined whether or not the language expression 25a detected for each treatment action presented by the standard treatment program 20 is recognized under a certain condition within a predetermined time width T. That is, by this, for example, the language expression 25a of “tracheal obstruction” is extracted within a predetermined time width T, and thereafter the name of a medical device such as “intratracheal tube” and a tracheal intubation for securing the airway are obtained. When the related language expressions 25b, 25c, etc. relating to the requested “treatment instruction” are extracted, the event is detected as related to the “tracheal intubation” treatment action or event, and the treatment action flow according to the standard treatment program is detected. It is recognized together. Note that a condition may be set such that, for example, when one language expression 25a is detected a plurality of times (for example, three times or more), the language expression 25a is first extracted. In addition, in the case of the expression order of each language expression 25a, for example, “language expression 25a → related language expression 25b or related language expression 25c” is recognized, while “expression of related language expression 25b → language expression 25a”. It may be associated with the expression order such as not recognizing.

  Then, the extracted language expression 25a and the like are scored so as to correspond to the statistical information 27 and the treatment action presentation information 21 of the standard treatment program 20 (step S7). Further, the state of the standard treatment program 20 (including the presented treatment action and a record of the treatment action) when the language expression 25a is extracted from the voice information 5 and the video information acquired in addition to the voice information 5 The modality information 7 such as 15 is scored simultaneously (steps S8 and S9). Then, the score values 46 that have been scored are integrated (step S10). At this time, the score value 46 is calculated according to a standard defined in advance based on the language expression 25a and the like, the statistical information 27 corresponding to the language expression 25a and the like, the relevance to the treatment action, and the like.

  Then, based on the integrated score information 45, a treatment action or event at that time is detected (step S11). At this time, if the detected event or the like does not match the event or the like detected in the previous stage in a time-series flow, that is, if the event detected in the previous stage is determined to be a false detection (YES in step S12). ) From the unselected events recorded as the undetected information 38 at the time of event detection in the previous stage, an event detection process is performed again (see step S13: two-dot chain line in FIG. 3). On the other hand, when there is no false detection (NO in step S12), the process of step S13 is cancelled.

  Thereafter, a record of the event for the treatment action is structured and recorded in the event index 2 in synchronization with the record of the treatment action (step S14). Here, an example of the event index 2 is shown in FIG. 6. The event item 50 related to the detected treatment action or event, the extracted language expression 25, the expression time 51 of the language expression, the score information 46, etc. are structured. It is recorded. As a result, since the items such as each event are structured and recorded in the work of verifying the treatment action after the fact, the audio information 5 and other related modalities can be specified by specifying the event item 50, the language expression 25, etc. Information 7 can be easily reproduced. Although not shown, the modality information 7 is also structured and recorded in synchronization with the event index 2.

  Then, after recording various information in the event index 2, the processing device body 4 detects the presence or absence of an instruction for determining whether or not to continue acquiring new audio information 5, and does not continue acquiring audio information 5. In the case (YES in step S15), the acquisition of the audio information 5 and the event detection process by the audio processing device 1 are terminated (step S16). On the other hand, when the acquisition of the voice information 5 is continued (NO in step S15), the process returns to step S4 and the acquisition of the voice information 5 acquired through the microphone 6 and the extraction of the language expression 25a and the like are continued.

  As described above, according to the voice processing device 1 of the present embodiment, a treatment action performed based on the standard treatment program or an event for the treatment action is detected based on the voice information 5 acquired through the microphone 6. However, it can be recorded in synchronization with the event index 2. In particular, by extracting linguistic expressions 25 pre-stored in the language database 28 and using statistical information 27 that statistically indicates the possibility of expression, the detection accuracy of events for each treatment action is improved. improves. In addition to the voice information 5, the modality information 7 such as other video information 15 connected to the voice processing device 1 is scored and integrated with the extracted language expression 25 score, thereby further improving the event detection accuracy. Can be increased.

  In addition, it is possible to reproduce other modality information 7 structured and recorded in the event index 2 in conjunction with the type of event that occurred (event item 50) and onset time 51, and confirm the treatment action after the fact. In the case of creating medical records and the like, reproduction work is promptly performed. Furthermore, the detected event or the like can be used as tag information and applied to a search for similar cases or the like.

  The present invention has been described with reference to preferred embodiments. However, the present invention is not limited to these embodiments, and various modifications can be made without departing from the spirit of the present invention as described below. And design changes are possible.

  That is, in the speech processing apparatus 1 according to the present embodiment, the detection of an event or the like based on a plurality of language expressions 25 extracted during a predetermined time width T is shown, but the present invention is not limited to this. The event or the like may be directly detected from the language expression 25 included in the audio information 5. Thereby, although the detection accuracy slightly decreases, the burden on the audio processing device 1 can be reduced. Therefore, in a situation where the complicated language expression 25 is not included, the detection accuracy may be switched and only the single language expression 25 may be extracted.

  Furthermore, in the audio processing device 1 of the present embodiment, the detection of an event or the like is shown by scoring the audio information 5 and the modality information 8 including the other video information 15 to determine an event or the like that integrates each. However, the present invention is not limited to this, and an event or the like may be extracted using only the audio information 5.

It is a block diagram which shows schematic structure of the audio processing apparatus of this embodiment. It is explanatory drawing which shows typically the speech recognition by a speech processing unit. It is explanatory drawing which shows typically the speech recognition by a speech processing unit. It is a flowchart which shows the flow of a process of an audio processing apparatus. It is a flowchart which shows the flow of a process of an audio processing apparatus. It is explanatory drawing which shows an example of an event index.

Explanation of symbols

1 Voice processing device (medical voice information processing device)
2 Event index 3 Medical voice information 5 Voice information 6 Microphone (voice input device)
7 Modality information 8 Liquid crystal display (information presentation means, treatment action presentation means)
9 Speaker (Information presentation means, treatment action presentation means)
10 Text information (modality information)
12 Sensor information (modality information)
15 Video information (modality information)
19 Information presentation means (treatment action presentation means)
20 Standard treatment program 21 Treatment action presentation information 23 Treatment action recording means 24 Audio information acquisition means 25 Audio information 25, 25a Language expression 25b, 25c Related language expression (language expression)
27 Statistical information 28 Language database 29 Language database storage means 30 Extraction means 31 Treatment event detection means 32 Onset time 34 Audio information recording means 35 Time width detection means 38 Undetected information 39 Undetected information recording means 40 Correction means 41 First scoring Means 42 Second scoring means T Time width

Claims (8)

A treatment action presentation means for presenting treatment action presentation information corresponding to the treatment action based on a standard treatment program in which the procedure of the treatment action is standardized;
A treatment act recording means for structuring and recording the treatment action in an event index in time series in synchronization with the presented treatment action presentation information;
Voice information acquisition means for acquiring voice generated during the treatment as voice information;
Language database storage means for storing, in a language database, characteristic linguistic expressions indicating the treatment action, and statistical information calculated for each treatment action of the expression possibility of the language expression;
Extraction means for extracting the language expression stored in the language database from the acquired voice information;
Treatment event detection means for detecting the treatment action of the standard treatment program or an event corresponding to the treatment action using the statistical information from the extracted language expression;
Audio information recording means for structuring and recording medical audio information including at least one of the detected treatment action or event, the extracted language expression, and the expression time of the language expression in the event index A medical voice information processing apparatus. The treatment event detection means includes
Based on at least one of a combination of a plurality of language expressions extracted within a time range of a predetermined range before and after the expression of the language expression, the number of extractions of the language expression, and the order of expression of the language expression The medical sound information processing apparatus according to claim 1, wherein the medical sound information processing apparatus is detected. The treatment event detection means includes
A non-detected information holding means for holding one of a plurality of the therapeutic actions or the event selected and detected, and holding the non-selected therapeutic action or the event as undetected information;
The said treatment action or the said event detected after the fact further comprises the correction means which corrects the said treatment action or the said event hold | maintained in the said undetected information, The claim 1 or Claim 2 characterized by the above-mentioned. Medical voice information processing equipment. First scoring means for scoring the extracted language expression;
Second scoring means for scoring at least one modality information of video information, text information, and sensor information acquired corresponding to the treatment action or the event,
The treatment event detection means includes
The medical speech information processing apparatus according to any one of claims 1 to 3, wherein the scored language expression and the modality information are integrated to detect the treatment action or the event. A treatment action presentation means for presenting treatment action presentation information corresponding to the treatment action based on a standard treatment program in which the procedure of the treatment action is standardized;
A therapeutic action recording means for recording the therapeutic action in an event index in time series in synchronization with the presented therapeutic action presentation information;
Voice information acquisition means for acquiring voice generated during the treatment as voice information;
Language database storage means for storing, in a language database, characteristic linguistic expressions indicating the treatment action, and statistical information calculated for each treatment action of the possibility of expression of the language expression;
Extraction means for extracting the language expression stored in the language database from the acquired voice information;
Treatment event detection means for detecting the treatment action of the standard treatment program or an event corresponding to the treatment action using the statistical information from the extracted language expression;
And a voice for structuring and recording medical voice information including at least one of the extracted linguistic expression and the expression time of the linguistic expression in the event index as the recognized therapeutic action or the event. A medical speech information processing program for causing a medical speech information processing apparatus to function as information recording means.   Based on at least one of a combination of a plurality of language expressions extracted within a time range of a predetermined range before and after the expression of the language expression, the number of extractions of the language expression, and the order of expression of the language expression 6. The medical sound information processing program according to claim 5, wherein the medical sound information processing apparatus is further caused to function as the event recognition means for detecting the treatment action or the event. Unrecognized information holding means for selecting and recognizing one of a plurality of candidates for the treatment action or the event, and holding the treatment action or the event not selected as unrecognized information;
The medical sound information processing apparatus is further caused to function as a correcting means for correcting the treatment action or the event held in the unrecognized information from the treatment action or the event recognized after the fact. The medical sound information processing program according to claim 5 or 6. First scoring means for scoring the extracted language expression;
Second scoring means for scoring at least one modality information of video information, text information, and sensor information acquired corresponding to the treatment action or the event;
The medical speech information processing apparatus is further caused to function as the treatment event detecting means for integrating the scored language expression and the modality information and detecting the treatment action or the event. The medical speech information processing program according to any one of claims 5 to 7.

Images