JP2009111506A - Apparatus for generating monitor information - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To generate display information which allows a guardian to easily confirm whether an event has occurred in the past and event contents upon detection of the event, without requiring permanent stationing of the guardian. <P>SOLUTION: An apparatus includes: a reference data storage part 608 which prestores reference sound data in which typical sound data on the occurrence of the event is associated with event content data showing contents of the event; a sound analysis part 609 which, when reference sound data pertinent to sound data being an analysis object received from a microphone 20 is present, extracts the associated event content data; a sound information generation part 610 which generates sound information display data for displaying sound pressure level value of sound data received by the time before a prescribed period, on a time axis as a graph and relates event content data with a pertinent position on the time axis; and a data superposing part 612 which superposes the generated sound information display data on image data received from a monitoring camera 10 to generate display data for display. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a monitoring information generating apparatus that generates display information for displaying video information and audio information in a monitoring area.

  In recent years, a network-type monitoring system using a communication network such as Ethernet (registered trademark) has rapidly spread as a transmission medium for transmitting digital video signals and digital audio signals captured by a monitoring camera to a display device or a recording device. ing.

  In such a network type surveillance system, it is possible to superimpose and transmit video signals and audio signals captured by a plurality of surveillance cameras using a single cable.

  Therefore, it is possible to arbitrarily change the number of surveillance cameras to be installed later by arranging network cables in advance in the facility to be monitored.

  Also, in such a network type surveillance system, it is possible to remotely transmit a control signal for controlling the photographing direction and scale of the surveillance camera such as pan, tilt and zoom using a network cable. There is an advantage that it is not necessary to separately provide a wiring for transmitting a control signal unlike a monitoring system that transmits an analog signal.

  Conventionally, in such a monitoring system, monitoring has mainly been performed only with video information, but there is an increasing demand for adding audio information in order to know in more detail the situation of the location to be monitored.

  By adding audio information to the object to be monitored, there is a high possibility that information on an area that is not shown in the video, for example, information when a glass is broken by a suspicious person in an area that is not shown in the video can be detected. Here, there are techniques described in Patent Documents 1 and 2 as a technique related to a monitoring system to which voice information is added.

  In Patent Document 1, not only video information and audio information obtained from a monitoring area are presented to a monitor, but also video information to be displayed when abnormal sound is detected by analyzing the audio information. There is described a monitoring monitor device that switches to video information taken by a monitoring camera at an abnormal sound occurrence location. With this monitoring monitor device, it is possible to easily check with an image when abnormal sound occurs in the monitoring area.

  However, with regard to the monitoring and monitoring apparatus of Patent Document 1, it is necessary for the supervisor to always reside in front of the monitoring apparatus, and it is difficult to leave information regarding events that occur when the supervisor is absent.

Patent Document 2 describes a monitoring system device that displays audio information acquired from the past to the present using a graph on a monitor together with display of video information. According to this monitoring system apparatus, it is possible for a monitor to check the history of voice information acquired in the past without being resident in front of the monitor.
JP 2004-139261 A JP 2001-307252 A

  According to the monitoring monitor device described in Patent Document 1, the monitor can know that some abnormality has occurred by switching the monitor display, but how long has passed since the abnormality has occurred. There is a problem that there is no means for knowing whether there is any abnormality, or no means for knowing what kind of abnormality has occurred when an image relating to the occurrence of abnormal sound is not shown.

  In addition, the technique of Patent Document 2 merely shows how the audio information has changed, and in order to obtain significant information regarding the occurrence of an abnormality from the displayed information, a work for confirming the monitoring video is separately required. There was a problem of being.

  Therefore, the present invention has been made in view of the above problems, and does not require the presence of a monitor, and easily confirms the presence or absence of an event that occurred in the past and the event contents when the event is detected. An object of the present invention is to provide a monitoring information generation device that generates display information that can be displayed.

  In order to achieve the above object, the monitoring information generating device (60-1 to 60-3) of the present invention includes a monitoring camera (10-1 to 10-3) for photographing a monitoring area, and the monitoring area and its vicinity. A monitoring system including a microphone (20-1 to 20-3) for collecting sound, a display device (70-1 to 70-3), and a monitoring information generating device (60-1 to 60-3) 1), the monitoring information generation device (60-1 to 60-3), the video data of the monitoring area image captured by the monitoring camera (10-1 to 10-3), and the microphone (20). -1 to 20-3), a data receiving unit (601) for receiving the sound data of the monitoring area and the sound in the vicinity thereof, and typical sound data corresponding to the sound generation event at a predetermined frequency Reference frequency indicated by sound pressure level value for each band Reference data storage unit (608) for storing in advance a plurality of reference voice data composed of collection data and event contents data which is text data indicating the contents of the voice generation event, and the received voice data A sound pressure level value calculation unit (607) that obtains and calculates a sound pressure level value for each predetermined period of the sound data, obtains the received sound data, and uses the sound data as a sound pressure for each frequency band. The reference frequency features corresponding to the analysis frequency feature data are compared with each reference frequency feature data stored in the reference data storage unit (608) converted into analysis frequency feature data indicated by level values for each band. A voice analysis unit (609) for extracting event content data corresponding to the reference frequency feature data when there is data; Sound information level data for acquiring a sound pressure level value from a predetermined time to a predetermined period and displaying it in a graph on the time axis is generated, and any event content data is extracted by the sound analysis unit (609) In this case, a mark indicating the occurrence of an event is attached to the position on the time axis corresponding to the extracted time, and the audio information generating unit (610) for associating the extracted event content data with the mark, and the reception A data superimposing unit (612) that superimposes the generated audio information display data on the generated video data to generate display display data for display; and the generated display display data is transmitted to the display device (70-). 1 to 70-3) and a video output unit (613).

  In addition, the monitoring information generation device (60-1 to 60-3) according to another aspect of the present invention includes a monitoring camera (10-1 to 10-3) for photographing the monitoring area, and the sound in the monitoring area and the vicinity thereof. Monitoring system (2) including microphones (20-1 to 20-3) for collecting sound, display devices (70-1 to 70-3), and monitoring information generation devices (60-1 to 60-3) ) In the monitoring information generation device (60-1 to 60-3), the video data of the video of the monitoring area taken by the monitoring camera (10-1 to 10-3), and the microphone (20-). A data receiving unit (601) for receiving the sound data of the monitoring area collected in 1-20-3) and the sound in the vicinity thereof, and typical sound data corresponding to a sound generation event for a predetermined period Reference variation pattern showing variation pattern of sound pressure level value A reference data storage unit (608) for storing a plurality of reference voice data composed of data and event contents data which is text data indicating the contents of the voice generation event, and the received voice data A sound pressure level value calculation unit (607) that obtains and calculates a sound pressure level value for each predetermined period of the sound data, and obtains the calculated sound pressure level value as analysis target data. Each reference variation pattern data stored in the reference data storage unit (608) is compared for each predetermined time width, and when there is the reference variation pattern data corresponding to the analysis target data, the reference variation pattern data A sound analysis unit (609) for extracting event content data corresponding to the sound pressure level value from the present calculated time to a predetermined period before When the obtained audio information display data for graph display on the time axis is generated and any event content data is extracted by the audio analysis unit (609), it corresponds to the extracted time. An audio information generation unit (610) for attaching an event occurrence mark to the position on the time axis and associating the event content data extracted by the audio analysis unit (609) with the mark; and the received video data A data superimposing unit (612) that superimposes the generated audio information display data to generate display display data for display, and the generated display display data to the display devices (70-1 to 70-). And a video output unit (613) for outputting to 3).

  According to the monitoring information generation device of the present invention, it is not necessary for the supervisor to be resident, and it is possible to easily confirm the presence or absence of a sound generation event that has occurred in the past and the event contents when the sound generation event is detected. The display information that can be generated can be generated.

<< First Embodiment >>
The monitoring information generation apparatus according to the first embodiment of the present invention determines whether or not a specific situation (referred to as an event or a sound generation event) in which sound is generated has occurred based on the frequency characteristics of sound in the monitoring area and the vicinity thereof. In addition, when it is determined that an event has occurred, the content of the event is specified, and the information is displayed on a display device.

<Configuration of Monitoring System According to First Embodiment>
A configuration of a monitoring system to which the monitoring information generating apparatus according to the present embodiment is applied will be described with reference to FIG.

<Monitoring system according to the first embodiment>
The monitoring system 1 according to the present embodiment includes a plurality of monitoring cameras 10-1 to 10-3, microphones 20-1 to 20-3 connected to these monitoring cameras 10-1 to 10-3, and a monitoring camera. The network hub 30 connected to 10-1 to 10-3, the network hub 50 connected to the network hub 30 via the network 40, and connected to the network hub 50 to the surveillance cameras 10-1 to 10-3 A plurality of monitoring information generation devices 60-1 to 60-3 that correspond to each other, and displays 70-1 to 70-3 as display devices respectively connected to these monitoring information generation devices 60-1 to 60-3, and Speakers 80-1 to 80-3 are provided.

  Each of the monitoring cameras 10-1 to 10-3 shoots each monitoring area and generates video data, and then compresses the video data in a format such as JPEG. The monitoring cameras 10-1 to 10-3 are respectively connected via the network hub 30, the network 40, and the network hub 50. It transmits to the corresponding monitoring information generation apparatus 60-1 to 60-3.

  In addition, the monitoring cameras 10-1 to 10-3 acquire audio data from the microphones 20-1 to 20-3 connected thereto, and compress the audio data in a format such as MP3 so that the network hub 30 and the network are similar to the video data. 40 and the corresponding network information 50 to the corresponding monitoring information generation devices 60-1 to 60-3.

  Each of the microphones 20-1 to 20-3 acquires the sound in the monitoring area and the vicinity thereof, converts the sound into sound data, and transmits the sound data to the connected monitoring cameras 10-1 to 10-3.

  As shown in FIG. 2 (referred to as the monitoring information generating device 60 in FIG. 2), each of the monitoring information generating devices 60-1 to 60-3 includes a data receiving unit 601, a video decoding unit 602, and a video memory unit. 603, a voice decoding unit 604, a voice memory unit 605, a voice output unit 606, a sound pressure level value calculation unit 607, a reference data storage unit 608, a voice analysis unit 609, a voice information generation unit 610, The audio information storage unit 611, the data superimposing unit 612, and the video output unit 613 are included.

  The data receiving unit 601 receives the compressed video data and audio data transmitted from the monitoring cameras 10-1 to 10-3. The video decoding unit 602 decompresses the compressed video data received by the data receiving unit 601. The video memory unit 603 stores the video data decoded by the video decoding unit 602.

  The audio decoding unit 604 decodes the compressed audio data received by the data reception unit 601. The audio memory unit 605 stores the audio data decoded by the audio decoding unit 604. The audio output unit 606 acquires the digital audio data stored in the audio memory unit 605 at a certain period, converts it into analog audio data, and transmits the analog audio data to the connected speakers 80-1 to 80-3.

  The sound pressure level value calculation unit 607 acquires the sound data decoded by the sound decoding unit 604, and calculates a sound pressure level value for each predetermined period.

  In the reference data storage unit 608, a sound pressure level value for each predetermined frequency band is a typical sound data when an event occurs, such as a sound when the glass is broken or a sound when the door is closed. A plurality of reference audio data composed of the reference frequency feature data indicated by (2) and event content data which is text data indicating the content of the event are stored in advance.

  The voice analysis unit 609 acquires the voice data decoded by the voice decoding unit 604 and converts it into analysis frequency feature data that is analysis target data indicated by sound pressure level values for each frequency band similar to the reference frequency feature data. The reference frequency feature data stored in the reference data storage unit 608 is compared for each band, and if there is reference frequency feature data corresponding to the converted analysis frequency feature data, event content data corresponding to the reference frequency feature data To extract. An FFT (Fast Fourier Transform) method can be used to convert the audio data into the analysis frequency feature data.

  The voice information generation unit 610 acquires the past sound pressure level value calculated by the sound pressure level value calculation unit 607, generates voice information display data for displaying the graph on the time axis, and the voice analysis unit 609 When any event content data is extracted, a mark indicating the occurrence of the event is attached to the position on the time axis corresponding to the extracted time, and the event content data extracted by the voice analysis unit 609 is added to this mark. Associate.

  The voice information storage unit 611 stores the voice information display data generated by the voice information generation unit 610.

  The data superimposing unit 612 acquires the video data stored in the video memory unit 603 and the audio information display data stored in the audio information storage unit 611, and displays the audio information display data superimposed on the acquired video data. Display display data for use.

  The video output unit 613 performs D / A conversion on the display display data generated by the data superimposing unit 612 and transmits it to any of the connected displays 70-1 to 70-3.

  The displays 70-1 to 70-3 acquire and display display display data transmitted from the connected monitoring information generation devices 60-1 to 60-3.

  The speakers 80-1 to 80-3 acquire and output the audio data transmitted from the audio output unit 606.

<Operation of Monitoring System According to First Embodiment>
Next, the operation of the monitoring system 1 according to this embodiment will be described.

  Each of the monitoring cameras 10-1 to 10-3 of the monitoring system 1 according to the present embodiment shoots each monitoring area to generate video data, and further compresses the video data in a format such as JPEG.

  Further, in the microphones 20-1 to 20-3 connected to the monitoring cameras 10-1 to 10-3, respectively, the monitoring area and the sound in the vicinity thereof are converted into audio data, and the connected monitoring camera 10-1 is connected. To 10-3. And it compresses in formats, such as MP3, in the surveillance cameras 10-1 to 10-3.

  The compressed video data and audio data are sent from the monitoring cameras 10-1 to 10-3 to the corresponding monitoring information generation devices 60-1 to 60-3 via the network hub 30, the network 40, and the network hub 50, respectively. Sent.

  Next, when the monitoring information generation devices 60-1 to 60-3 receive the compressed video data and audio data, and generate display data to be displayed on the connected displays 70-1 to 70-3. This process will be described with reference to the flowchart of FIG.

  First, when the data receiving unit 601 of the monitoring information generating devices 60-1 to 60-3 receives the compressed video data and audio data (S1), the compressed video data is decompressed by the video decoding unit 602. Is stored in the video memory unit 603. The compressed audio data received by the data receiving unit 601 is decoded by the audio decoding unit 604 and stored in the audio memory unit 605 (S2).

  The sound data decoded by the sound decoding unit 604 is also acquired by the sound pressure level value calculation unit 607, and the sound pressure level value is calculated every predetermined period (for example, every second) (S3).

  The audio data decoded by the audio decoding unit 604 is also acquired by the audio analysis unit 609, and the analysis target data in which the audio data for each predetermined period is indicated by the sound pressure level value for each frequency band similar to the reference frequency feature data. Is converted to analysis frequency feature data.

  The converted analysis frequency feature data is compared for each band with each reference frequency feature data stored in the reference data storage unit 608 in the voice analysis unit 609, and there is reference frequency feature data corresponding to the analysis frequency feature data. It is determined whether or not (S4).

  A comparison process between the analysis frequency feature data and the reference frequency feature data in the voice analysis unit 609 will be described with reference to the flowchart of FIG. 4 and the example of the frequency feature data of FIG.

  The reference data storage unit 608 stores a plurality of nth reference frequency feature data. In FIG. 5A, three of the first to third reference frequency feature data are stored (n = 1 to 3).

  These reference frequency characteristic data are indicated by sound pressure level values for each divided frequency band (m). In FIG. 5A, the sound pressure for each of the six frequency bands (1) to (6) is shown. It is indicated by a level value (m = 1 to 6).

  FIG. 5B is an example of analysis frequency feature data converted by the voice analysis unit 609. Here, it is assumed that the analysis frequency feature data is also represented by a sound pressure level value for each frequency band (m) (m = 1 to 6) similar to the reference frequency feature data.

  First, when comparison processing between analysis frequency feature data and reference frequency feature data is started in the voice analysis unit 609, one n-th reference frequency feature data in the reference frequency feature data is acquired. The initial value of n is 1. First, the first reference frequency feature data is acquired, and the initial value of m becomes 1 (S11, S12).

  When the first reference frequency feature data is acquired, the first reference frequency feature data and the analysis frequency feature data are compared for one frequency band (m) in the data. The initial value of m is 1. First, the first reference frequency feature data and the analyzed frequency feature data are compared for the frequency band (1).

  As a result of the comparison, it is determined whether or not the sound pressure level value of the frequency band (1) of the analysis frequency feature data falls within a certain variable range of the sound pressure level value of the frequency band (1) of the reference frequency feature data. (S13).

  The constant variable range of the sound pressure level value in the frequency band of the reference frequency feature data is an allowable range for recognizing the sound data of the frequency band (m) due to the same event. A value within ± 5% of the sound pressure level value is determined as a value within the variable range.

  In FIG. 5, it is determined that the sound pressure level value in the frequency band (1) of the analysis frequency feature data is not included in the variable range of the frequency band (1) of the first reference frequency feature data (“NO” in S13). ]), And moves to step S14.

  In step S14, the process proceeds to the processing of the second reference frequency feature data, and the process returns to step S12 to acquire the second reference frequency feature data (S14, S15, S12).

  When the second reference frequency feature data is acquired, the second reference frequency feature data and the analysis frequency feature data are compared for the frequency band (m). The initial value of m is 1, and first, the second reference frequency feature data and the analysis frequency feature data are compared for the frequency band (1).

  As a result of the comparison, it is determined whether or not the sound pressure level value of the frequency band (1) of the analysis frequency feature data falls within a certain variable range of the sound pressure level value of the frequency band (1) of the reference frequency feature data. (S13).

  In FIG. 5, it is determined that the sound pressure level value in the frequency band (1) of the analysis frequency feature data is not included in the variable range of the frequency band (1) of the second reference frequency feature data (“NO” in S13). ]), And moves to step S14.

  In step S14, it moves to the process of 3rd reference frequency feature data, returns to step S12, and 3rd reference frequency feature data is acquired (S14, S15, S12).

  When the third reference frequency feature data is acquired, the third reference frequency feature data and the analysis frequency feature data are compared for the frequency band (m). The initial value of m is 1, and first, the third reference frequency feature data and the analysis frequency feature data are compared for the frequency band (1).

  As a result of the comparison, it is determined whether or not the sound pressure level value of the frequency band (1) of the analysis frequency feature data falls within a certain variable range of the sound pressure level value of the frequency band (1) of the reference frequency feature data. (S13).

  In FIG. 5, it is determined that the sound pressure level value in the frequency band (1) of the analysis frequency feature data is included in the variable range of the frequency band (1) of the third reference frequency feature data (“YES” in S13). Move to step S16.

  In step S16, the process moves to the frequency band (2), and the process returns to step S13, where the sound pressure level value of the frequency band (2) of the analysis frequency feature data is the sound pressure of the frequency band (2) of the third reference frequency feature data. It is determined whether the level value falls within a certain variable range (S16, S17, S13).

  In FIG. 5, it is determined that the sound pressure level value in the frequency band (2) of the analysis frequency feature data is included in the variable range of the frequency band (2) of the third reference frequency feature data (“YES” in S13). Move to step S16.

  In FIG. 5, by repeating the processes of steps S13 and S16 up to the frequency band (6) (m = 6), the sound pressure level value of the analysis frequency feature data is the sound of the third reference frequency feature data in all frequency bands. It is determined that the pressure level value falls within the variable range (S13, S16, S17), and the third reference frequency feature data is determined to be data corresponding to the analysis frequency feature data (S18).

  In step S13, it is determined that the sound pressure level value of the frequency band (m) of the analysis frequency feature data is not included in the variable range of the frequency band (m) of the nth reference frequency feature data, and the reference data storage is performed. When the comparison processing for all the reference frequency feature data in the unit 608 is completed (“YES” in S15), it is determined that there is no reference frequency feature data corresponding to the analysis frequency feature data (S19).

  Above, description about the comparison process of the analysis frequency feature data and reference frequency feature data in the audio | voice analysis part 609 is complete | finished.

  Returning to the description of FIG. 3, when it is determined in step S4 that there is reference frequency feature data corresponding to the analysis frequency feature data, event content data corresponding to the reference frequency feature data is extracted (S5).

  When event content data is extracted in step S5, or when it is determined in step S4 that there is no reference frequency feature data corresponding to the analysis frequency feature data (“NO” in S4), the sound information generation unit 610 generates a sound. The past sound pressure level value calculated by the pressure level value calculation unit 607 is acquired, and voice information display data for graph display on the time axis is generated (S6). The sound pressure level value acquired here is preferably about several tens of minutes to several hours in the past.

  Here, when any event content data is extracted in the audio analysis unit 609, the time axis of the graph data corresponding to the time when the event content data is extracted from the audio information display data by the audio information generation unit 610 A mark indicating the occurrence of an event is attached to the upper position, and the event content data extracted by the voice analysis unit 609 is associated with this mark.

  The voice information display data generated by the voice information generation unit 610 is stored in the voice information storage unit 611 (S7).

  Next, the video superimposing unit 612 acquires the video data stored in the video memory unit 603 and the audio information display data stored in the audio information storage unit 611, and the audio information display data is superimposed on the acquired video data. Display display data for display is generated (S8).

  The display display data generated by the data superimposing unit 612 is D / A converted by the video output unit 613 and transmitted to any of the connected displays 70-1 to 70-3.

  The display display data transmitted from the video output unit 613 is displayed on each of the displays 70-1 to 70-3. An example of display display data displayed on the displays 70-1 to 70-3 is shown in FIG.

  FIG. 6A shows display display data generated by the monitoring information generation device 60-1 based on the video data captured by the monitoring camera 10-1 and the audio data acquired by the microphone 20-1. FIG. 6B shows an example that is generated by the monitoring information generation device 60-2 based on video data captured by the monitoring camera 10-2 and audio data acquired by the microphone 20-2. FIG. 6C shows an example in which display display data is displayed on the display 70-2. FIG. 6C shows generation of monitoring information based on video data captured by the monitoring camera 10-3 and audio data acquired by the microphone 20-3. In this example, display display data generated by the device 60-3 is displayed on the display 70-3.

  Each display display data is composed of video data 71-1 to 71-3 and audio information display data 72-1 to 72-3.

  In addition, in the audio information display data 72-1 to 72-3, marks 73-1 to 73-3 attached to the time positions at which the events are recognized are displayed, and these marks 73 are displayed. When the cursor is brought close to any of -1 to 73-3, event content data associated with the mark is displayed.

  In FIG. 6B, the text data 74-2 of “glass breaking sound” which is event content data associated with the mark 73-2 is displayed when the cursor is brought close to the mark 73-2. .

  The audio data stored in the audio memory unit 605 is D / A converted by the audio output unit 606, transmitted to the connected speakers 80-1 to 80-3, and output from the speakers 80-1 to 80-3. (S9).

  According to the monitoring system 1 of the first embodiment described above, the presence / absence of an event is detected based on the frequency characteristics of audio data acquired from the monitoring area and its vicinity, and the contents of this event when an event is further detected. Therefore, it is possible to perform appropriate monitoring for a predetermined period without requiring a monitor to be resident.

  In the first embodiment, when the analysis frequency feature data and the reference frequency feature data are compared in the voice analysis unit 609, the sound pressure level value of the analysis frequency feature data is the reference frequency feature in all the set frequency bands. The case where it is determined that the reference frequency feature data is data corresponding to the analysis frequency feature data when it is determined that the data is included within the variable range of the sound pressure level value of the data has been described. When there are many (several tens to several hundreds, etc.), the sound pressure level value of the analysis frequency feature data is the sound pressure level value of the reference frequency feature data in a certain proportion (for example, 95%) of the set frequency band. If it is included in the variable range, the reference frequency feature data may be determined as data corresponding to the analysis frequency feature data.

<< Second Embodiment >>
The monitoring information generating apparatus according to the second embodiment of the present invention determines whether or not an event has occurred based on the sound level fluctuation pattern of the sound in the monitoring area and the vicinity thereof, and when it is determined that the event has occurred, The contents of the event are specified and provided to the observer by displaying the information.

<Configuration of Monitoring System According to Second Embodiment>
Since the configuration 2 of the monitoring system according to the present embodiment is the same as the configuration of the monitoring system 1 according to the first embodiment, detailed description of the components having the same functions is omitted.

  The reference data storage unit 608 of the monitoring information generating apparatuses 60-1 to 60-3 in the present embodiment has a reference fluctuation pattern indicating a fluctuation pattern of a sound pressure level value of typical voice data when an event occurs for a predetermined period. A plurality of reference audio data composed of data and event content data which is text data indicating the content of the event is stored in advance.

  The voice analysis unit 609 acquires the sound pressure level value calculated by the sound pressure level value calculation unit 607 as analysis target data, and the fluctuation data of the sound pressure level value for each predetermined period of the analysis target data is a reference data storage unit. Each reference variation pattern data stored in 608 is compared. If there is reference variation pattern data corresponding to the analysis target data, event content data corresponding to the reference variation pattern data is extracted.

<Operation of Monitoring System According to Second Embodiment>
Next, the operation of the monitoring system 2 according to the present embodiment will be described.

  Also in the monitoring system 2 of the present embodiment, the video data captured by the monitoring cameras 10-1 to 10-3 and the microphones 20-1 to 20-3 are acquired as in the case described in the first embodiment. The audio data of the monitored area and the sound in the vicinity thereof are compressed and transmitted to the corresponding monitoring information generation devices 60-1 to 60-3 via the network 40.

  The monitoring information generating devices 60-1 to 60-3 receive compressed video data and audio data and generate display data for display on the connected displays 70-1 to 70-3. 3 is the same as that of the first embodiment except for the comparison process using the data stored in the reference data storage unit 608 in step S4 of the flowchart of FIG.

  In the present embodiment, the sound pressure level value calculated in step S3 is acquired as analysis target data by the voice analysis unit 609, and the fluctuation data of the sound pressure level value for each predetermined period of the analysis target data is the reference data storage unit. It is compared with each reference variation pattern data stored in 608, and it is determined whether or not there is reference variation pattern data corresponding to the variation data of the analysis target data (S4).

  The reference variation pattern data and the analysis target data are the same as the processing shown in the flowchart of FIG. 4 for each unit time (m) when the time width of the reference variation pattern data is divided by a preset number. The average sound pressure level value is compared with each other to determine whether there is reference variation pattern data corresponding to the analysis target data.

  Again, the average sound pressure level value per unit time divided in the reference variation pattern data has a variable range, and the average sound pressure level value per unit time of the analysis target data is the reference variation pattern data. If they are included in the variable range of the corresponding average sound pressure level value per unit time, it is recognized that the sound data is the same event.

  Hereinafter, the processing in steps S5 to S9 is the same as the processing described in the first embodiment, and thus detailed description thereof is omitted.

  According to the monitoring system 2 of the second embodiment described above, the presence / absence of an event is detected based on the fluctuation data of the voice acquired from the monitoring area and its vicinity, and when the event is further detected, the content of this event is displayed. Since it can be easily confirmed, it is possible to perform appropriate monitoring for a predetermined period without requiring a monitor to be resident.

  In the second embodiment, when the analysis target data and the reference variation pattern data are compared in the voice analysis unit 609, the sound pressure level value of the analysis target data is the reference variation pattern data in all divided unit times. The case where it is determined that the reference variation pattern data is data corresponding to the analysis target data when it is determined that the reference variation pattern data falls within the variable range of the sound pressure level value has been described. When the number of divisions is large (several tens to several hundreds, etc.), the sound pressure level value of the analysis target data is the sound pressure level value of the reference variation pattern data at a certain percentage (eg, 95%) of the divided unit time. If it is included in the variable range, the reference variation pattern data may be determined to be data corresponding to the analysis target data.

  Moreover, it is also possible to make the said computer function as a monitoring information generation apparatus by programming and integrating the functional structure of the monitoring information generation apparatus of each said embodiment into a computer.

It is a general view which shows the structure of the monitoring systems 1 and 2 using the monitoring information generation apparatus 60 which is 1st Embodiment and 2nd Embodiment of this invention. It is a block diagram which shows the structure of the monitoring information generation apparatus 60 which is 1st Embodiment and 2nd Embodiment of this invention. It is a flowchart which shows operation | movement of the monitoring information generation apparatus 60 which is 1st Embodiment and 2nd Embodiment of this invention. It is a flowchart which shows operation | movement of the audio | voice analysis part 609 of the monitoring information generation apparatus 60 which is 1st Embodiment and 2nd Embodiment of this invention. An example (a) of reference frequency feature data stored in the reference data storage unit 608 of the monitoring information generation device 60 according to the first embodiment of the present invention, and an example of analysis frequency feature data converted by the speech analysis unit 609 (B). It is an example (a)-(c) of the display data displayed on the display 70-1 to 70-3 connected to the monitoring information generation apparatus 60 which is 1st Embodiment and 2nd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 ... Surveillance system 2 ... Surveillance system 10 ... Surveillance camera 20 ... Microphone 30 ... Network hub 40 ... Network 50 ... Network hub 60 ... Surveillance information generation apparatus 70 ... Display 71 ... Video data 72 ... Audio | voice information display data 73 ... Mark 74 ... Text data 80 ... Speaker 601 ... Data receiving unit 602 ... Video decoding unit 603 ... Video memory unit 604 ... Audio decoding unit 605 ... Audio memory unit 606 ... Audio output unit 607 ... Sound pressure level value calculation unit 608 ... Reference data storage unit 609 ... voice analysis part 610 ... voice information generation part 611 ... voice information storage part 612 ... data superposition part 613 ... video output part

Claims (2)

A monitoring information generation device in a monitoring system comprising a monitoring camera that images a monitoring area, a microphone that collects sound in the monitoring area and its vicinity, a display device, and a monitoring information generation device,
A data receiving unit for receiving video data of the video of the monitoring area captured by the monitoring camera, and audio data of the monitoring area collected by the microphone and sound in the vicinity thereof;
It is composed of reference frequency characteristic data indicating typical sound data corresponding to a sound generation event with a sound pressure level value for each predetermined frequency band, and event content data which is text data indicating the content of the sound generation event. A reference data storage unit for storing a plurality of reference audio data in advance,
A sound pressure level value calculating unit for obtaining the received sound data and calculating a sound pressure level value for each predetermined period of the sound data;
Each of the reference frequency feature data stored in a reference data storage unit obtained by acquiring the received voice data, converting the voice data into analysis frequency feature data indicated by a sound pressure level value for each frequency band, and A voice analysis unit that compares each band and extracts event content data corresponding to the reference frequency feature data when there is the reference frequency feature data corresponding to the analysis frequency feature data;
The calculated sound pressure level value from the present time to a predetermined period before is acquired and voice information display data for displaying the graph on the time axis is generated, and any event content data is extracted by the voice analysis unit A mark indicating event occurrence at a position on the time axis corresponding to the extracted time, and an audio information generation unit associating the extracted event content data with the mark;
A data superimposing unit that superimposes the generated audio information display data on the received video data to generate display display data for display;
A video output unit for outputting the generated display display data to the display device;
A monitoring information generation apparatus comprising: A monitoring information generation device in a monitoring system comprising a monitoring camera that images a monitoring area, a microphone that collects sound in the monitoring area and its vicinity, a display device, and a monitoring information generation device,
A data receiving unit for receiving video data of the video of the monitoring area captured by the monitoring camera, and audio data of the monitoring area collected by the microphone and sound in the vicinity thereof;
It is composed of reference fluctuation pattern data indicating a fluctuation pattern of a sound pressure level value for a predetermined period of typical voice data corresponding to a voice generation event, and event content data which is text data indicating the contents of the voice generation event. A reference data storage unit for storing a plurality of reference audio data in advance,
A sound pressure level value calculating unit for obtaining the received sound data and calculating a sound pressure level value for each predetermined period of the sound data;
The calculated sound pressure level value is acquired as analysis target data, the analysis target data is compared with each reference variation pattern data stored in the reference data storage unit for each predetermined time width, and the analysis target A voice analysis unit that extracts event content data corresponding to the reference variation pattern data when there is the reference variation pattern data corresponding to the data;
The calculated sound pressure level value from the present time to a predetermined period before is acquired and voice information display data for displaying the graph on the time axis is generated, and any event content data is extracted by the voice analysis unit A mark indicating the occurrence of an event at a position on the time axis corresponding to the extracted time, and an audio information generation unit associating the event content data extracted by the audio analysis unit with the mark; ,
A data superimposing unit that superimposes the generated audio information display data on the received video data to generate display display data for display;
A video output unit for outputting the generated display display data to the display device;
A monitoring information generation apparatus comprising:

Images