WO2019053894A1 - Monitoring assistance apparatus and monitoring assistance system - Google Patents

Abstract

A monitoring assistance apparatus according to the present invention is provided with a reproduction unit and a history storage unit. In the case where an event, the occurring period of which is determined on the basis of the amount of change over time in a video image represented by video image data, has been selected by a monitoring person, the reproduction unit transmits, on the basis of an event identification information set selected by the monitoring person from among event identification information sets for identifying events, a part of the video image data which is stored in association with the event identification information set and which corresponds to the period in which the event is occurring, to a display device that displays a video image represented by received video image data. The history storage unit stores, in association with the event identification information of the event, monitoring assistance data that includes information indicating whether or not the reproduction unit has transmitted, to the display device, the part of the video image data corresponding to the period in which the event is occurring. Accordingly, the monitoring assistance apparatus is able to, without subdividing the video image data into parts each serving as a unit subjected to confirmation work by the monitoring person, store information of results of the confirmation work for the parts.

Description

Monitoring support device and monitoring support system

The present invention relates to a monitoring support apparatus and a monitoring support system.

Patent Document 1 discloses an example of a monitoring support system. The monitoring support system includes a monitoring support device. The monitoring support apparatus calculates the amount of movement of an object in the video represented by the video data. When the calculated movement amount is larger than the set value, the monitoring support apparatus extracts the corresponding part of the video data. The display device displays the video represented by the portion of the extracted video data based on the selection of the observer. The observer performs a visual check on the image displayed on the display device.

Japanese Patent Laid-Open No. 2004-254141

However, the monitoring support device disclosed in Patent Document 1 manages the information of the results of the confirmation operation in file units of video data. For this reason, in order to store the information of the track record of the check operation for each unit serving as the unit of the check operation by the supervisor, it is necessary to divide the video data into a file for each unit.

The present invention has been made to solve such problems. It is an object of the present invention to provide a monitoring support apparatus and a monitoring support system capable of storing information on the performance of confirmation work for each part without dividing video data into parts that are units of a target of confirmation work by a supervisor. It is to provide.

The monitoring support apparatus according to the present invention is an event identification information for identifying an event when the monitoring person selects an event whose period is determined based on the amount of change with time of the image represented by the video data. A portion of video data stored in association with the event identification information based on the event identification information of the event selected by the observer, and a portion of the video data corresponding to a period in which the event is occurring A monitoring unit that includes a reproduction unit that transmits the image represented by the data to the display device, and information that indicates whether the reproduction unit transmits the portion of the video data corresponding to the period in which the event occurs to the display device. A history storage unit that stores support data in association with event identification information of the event.

In the monitoring support system according to the present invention, the monitoring device is provided with a display device for displaying a video represented by the received video data, and an event for which a period occurring is determined based on the amount of change with time of the video represented by the video data. A part of video data stored in association with the event identification information based on the event identification information of the event selected by the observer among the event identification information identifying the event when it is selected, and the event occurs The monitoring support data including information indicating whether or not the portion of the video data corresponding to the time period is transmitted to the display device and the portion of the video data corresponding to the period during which the event occurs is transmitted to the display device And a monitoring support device that stores the event identification information of an event in association with the event identification information.

According to these inventions, the monitoring support device is a portion of the video data stored in association with the event identification information based on the event identification information of the event selected by the observer, and the event is occurring A portion of the video data corresponding to the period is transmitted to the display device. The monitoring support apparatus stores monitoring support data including information indicating whether a portion of video data corresponding to a period in which an event has occurred to the display device in association with event identification information of the event. As a result, the monitoring support apparatus can store the information on the result of the confirmation work for each of the parts, without dividing the video data into the parts that are the units of the object of the confirmation work by the supervisor.

BRIEF DESCRIPTION OF THE DRAWINGS It is a block diagram which shows the whole structure of a monitoring system provided with the monitoring assistance system which concerns on Embodiment 1 of this invention. It is a block diagram which shows the structure of the monitoring assistance apparatus which concerns on Embodiment 1 of this invention. It is a figure which shows the hardware constitutions of the principal part of the monitoring assistance apparatus which concerns on Embodiment 1 of this invention. It is a figure which shows the example of the setting screen of the designation | designated area | region which concerns on Embodiment 1 of this invention. It is a figure which shows the structure of the event classification data which concern on Embodiment 1 of this invention. It is a figure which shows the structure of the event data which concern on Embodiment 1 of this invention. It is a flowchart which shows the example of the operation | movement which determines the period which the event of the monitoring assistance apparatus which concerns on Embodiment 1 of this invention has generate | occur | produced. It is a figure which shows the example of the screen which displays the list of events which concern on Embodiment 1 of this invention. It is a figure which shows the structure of the monitoring assistance data which concern on Embodiment 1 of this invention. It is a flowchart which shows the example of the operation | movement which produces the list of the events of the monitoring assistance apparatus which concerns on Embodiment 1 of this invention. It is a figure which shows the example of the screen of the image | video in which the event which the display apparatus which concerns on Embodiment 1 of this invention displays is generate | occur | producing. It is a figure which shows the structure of reproduction | regeneration log | history data which concern on Embodiment 1 of this invention. It is a flowchart which shows the example of operation | movement which transmits the image | video in which the event of the monitoring assistance apparatus which concerns on Embodiment 1 of this invention has generate | occur | produced to a display apparatus. It is a figure which shows the structure of the monitoring result data which concern on Embodiment 1 of this invention. It is a figure which shows the example of the period couple | bonded as digest video data which concern on Embodiment 1 of this invention. It is a flowchart which shows the example of the operation | movement which produces digest video data of the monitoring assistance apparatus which concerns on Embodiment 1 of this invention. It is a figure which shows the structure of the supervisor data which concerns on Embodiment 1 of this invention. It is a figure which shows the other example of the screen of the image | video in which the event which the display apparatus which concerns on Embodiment 1 of this invention displays is generate | occur | producing. It is a flowchart which shows the example of operation | movement of the continuous reproduction | regeneration function of the monitoring assistance apparatus which concerns on Embodiment 1 of this invention. It is a block diagram which shows the structure of the monitoring assistance apparatus which concerns on the modification of Embodiment 1 of this invention. It is a block diagram which shows the structure of the monitoring assistance apparatus which concerns on the modification of Embodiment 1 of this invention. It is a block diagram which shows the structure of the monitoring assistance apparatus which concerns on the modification of Embodiment 1 of this invention.

DETAILED DESCRIPTION OF THE INVENTION Embodiments of the present invention will be described with reference to the attached drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals, and overlapping descriptions will be appropriately simplified or omitted.

Embodiment 1
FIG. 1 is a block diagram showing an overall configuration of a monitoring system provided with a monitoring support system according to the present embodiment.

The monitoring system 1 is a system for monitoring a monitoring target location.

The monitoring target location is a location to be remotely monitored. Below, the case where a monitoring object location is in an elevator car is described as an example.

The video data is data in which the situation of the monitoring target location that changes with time is recorded as video. In the following, it is assumed that the video data is in a format in which each frame, which is data representing an image captured at an interval of a fixed time, is given a continuous number. Note that the present embodiment is not limited to the specific data format of video data.

A portion of the video data is extracted from the original video data. The extracted video data portion is video data different from the original video data.

The monitoring system 1 includes a plurality of imaging devices 2, a video storage device 3, and a monitoring support system 4.

The imaging device 2 is a device that captures an image of a monitoring target location. The imaging device 2 is, for example, a network camera. The photographing device 2 transmits the photographed video as video data via the network 5. The imaging device 2 may encode video data before transmitting the video data.

The video storage device 3 is a device that stores the received video data as a file. The video storage device 3 receives video data transmitted from each of the photographing devices 2. The video storage device 3 is, for example, a NAS (Network Attached Storage).

The monitoring support system 4 is a system that supports visual confirmation of a monitoring target location.

The monitoring support system 4 includes a monitoring support device 6, a monitoring terminal 7, a display device 8, and an input device 9.

The monitoring support device 6 and the monitoring terminal 7 are connected to the network 5. The display device 8 and the input device 9 are connected to the monitoring terminal 7.

The monitoring support device 6 is a device that supports visual confirmation of the monitoring target location by data representing information on the performance of the confirmation operation of the supervisor.

The monitoring support device 6 receives the video data transmitted from each of the photographing devices 2. The monitoring support device 6 analyzes each of the received video data. The monitoring support device 6 transmits the analysis result as confirmation data.

The confirmation data includes video data representing a video in which an event has occurred.

An event is an event that occurs in a video represented by video data. The event is, for example, an event in which an elevator car door opens and closes, an event in which a person moves in and out of an elevator car, or an event in which a person on the elevator car makes an unnatural motion.

The amount of change of the image is a numerical value representing the change of the image. The amount of change is calculated based on the difference between video data for which the amount of change is calculated and video data as a reference.

The monitoring terminal 7 transmits video data included in the confirmation data received from the monitoring support device 6. The monitoring terminal 7 is, for example, a personal computer.

The display device 8 is a device that displays a video represented by video data received from the monitoring terminal 7. The display device 8 is, for example, a display.

The input device 9 is a device that transmits the input of the operation by the supervisor to the monitoring terminal 7. The input device 9 is, for example, a keyboard or a mouse. The monitoring terminal 7 transmits the received input to the monitoring support device 6. The monitoring support device 6 operates based on the received input.

Subsequently, the configuration of the monitoring support apparatus 6 according to the present embodiment will be described. FIG. 2 is a block diagram showing the configuration of the monitoring support apparatus according to the present embodiment. In the present embodiment, the network 5 and the monitoring terminal 7 mainly mediate communication between the devices, and therefore the illustration is omitted in the following figures.

The monitoring support device 6 includes a determination unit 60, an event storage unit 61, a reproduction unit 62, a history storage unit 63, a history management unit 64, a digest video processing unit 65, and a digest video storage unit 66. .

The determination unit 60 determines a period in which an event occurs based on the amount of change with respect to time of the video.

The determination unit 60 receives video data from the imaging device 2 or the video storage device 3. When the video data is received from the imaging device 2, the determination unit 60 acquires a path in which the video data file is stored in the video storage device 3. When the video data is received from the video storage device 3, the determination unit 60 acquires the path in which the file of the video data is stored. The path of the video data file is an example of information of a place where the video data file is located.

The determination unit 60 analyzes the received video data, and calculates the amount of change with time of the video represented by the video data. The determination unit 60 calculates the amount of change for a preset designated area of the video.

The designated area is an area for designating a part of the video set in advance by the supervisor.

The determination unit 60 has a plurality of methods for calculating the amount of change. The method of calculating the plurality of change amounts corresponds to the change of the plurality of features of the image.

The determination unit 60 determines a period in which an event occurs based on the calculated amount of change. The period in which the event occurs is the period between the start and the end of the period. After determining the start time of the period in which the event occurs, the determination unit 60 determines the end time of the period in which the event occurs based on the calculated amount of change.

The determination unit 60 determines, as the start time of the period in which the event occurs, the time when the value of the calculated change amount becomes larger than the value of the start threshold. The start threshold is a preset value.

After determining the start time of the period in which the event is occurring, the determination unit 60 determines the time when the value of the calculated change amount becomes smaller than the end threshold as the end time of the period. The end threshold is a preset value. The end threshold is a value equal to or less than the start threshold.

After determining the start time point of the period in which the event is occurring, the determination unit 60 determines that it is within the time period if the value of the calculated change amount is equal to or greater than the end threshold.

The determination unit 60 determines, for each event type, a period in which an event has occurred. The event type is a type that classifies an event based on a calculation method of a change amount used for determination of a occurring period, a designated area, a start threshold, and an end threshold.

The determination unit 60 stores information of the event type as event type data. The determination unit 60 stores the event type data in association with the event type number. The event type number is information for identifying an event type.

The determination unit 60 has an event type list having an event type number of the event type to be determined as an element. The determination unit 60 determines the period in which the event has occurred for all the event types in the event type list.

The event number stored in the event type list is designated by the subscript number of the element. The elements stored in the list are specified by the index numbers of the elements in the following other lists.

The determination unit 60 acquires a new event number when it determines the start time of the period in which the event is occurring. The event number is an example of event identification information that identifies an event.

The part of the video data corresponding to the period in which the event occurs is video data representing the video in which the event occurs. The start frame number of the event is the number of the first frame of the portion of the video data. The end frame number of the event is the number of the last frame of the portion of the video data. The set of the start frame number and end frame number of the event and the information of the location where the file of the video data representing the video in which the event is occurring has a portion of the video data corresponding to the period in which the event occurs. It is an example of information to identify.

The determination unit 60 transmits event data when it determines the end time of the period in which the event is occurring. The event data includes an event number of the event and information specifying a portion of the video data corresponding to a period in which the event occurs.

The event storage unit 61 stores event data. The event storage unit 61 stores the event data received from the determination unit 60 in association with the event number of the event data.

That is, the event storage unit 61 stores the information specifying the portion of the video data corresponding to the period in which the event occurs in association with the event number of the event.

The playback unit 62 creates a list of events occurring during the selected period selected by the observer.

The reproduction unit 62 acquires, from the event storage unit 61, the event number of the event occurring during the selection period. The reproduction unit 62 acquires event data from the event storage unit 61 based on the acquired event number. The reproduction unit 62 transmits, to the display device 8, data representing a list of events created based on the acquired event data. The display device 8 displays a list of events represented by the received data.

When creating a list of events, the playback unit 62 creates monitoring support data for each of the acquired events. The monitoring support data includes a monitoring support number that identifies the monitoring support data.

The reproduction unit 62 newly acquires a monitoring support number when creating the monitoring support data. After creating the monitoring support data, the reproduction unit 62 transmits the event number and the monitoring support data corresponding to the event identified by the event number to the history storage unit 63.

The history storage unit 63 stores the received event number and the monitoring support data in association with each other.

The playback unit 62 transmits video data corresponding to the event selected by the monitor to the display device 8.

The playback unit 62 acquires, from the event storage unit 61, event data stored in association with the event number selected by the monitor when the event is selected by the monitor. The reproduction unit 62 transmits the portion of the video data corresponding to the period in which the event is occurring to the display device 8 based on the acquired event data.

That is, when an event is selected by the observer, the playback unit 62 stores an image corresponding to a time period in which the event number is stored in association with the event number based on the event number of the selected event. A portion of the data is sent to the display device 8.

When transmitting the portion of the video data to the display device 8, the reproduction portion 62 represents the video in which the monitoring assistance display corresponding to the event is superimposed on the position corresponding to the designated area of the video represented by the portion of the video data. Create video data. The reproduction unit 62 transmits the created data to the display device 8.

The monitoring assistance display is, for example, a marker image displayed at a position where an event in the video has occurred. The marker image is displayed at the position of relative coordinates set in advance with reference to the coordinates corresponding to the position where the event occurs in the video.

When transmitting the video represented by the portion of the video data corresponding to the period in which the event is occurring to the display device 8, the playback unit 62 creates playback history data of the event.

The reproduction history data includes a reproduction history number identifying the reproduction history data and a time when the portion of the video data corresponding to the reproduction unit 62 is transmitted to the display device 8. The reproduction unit 62 newly acquires a reproduction history number when creating reproduction history data.

After creating the reproduction history data, the reproduction unit 62 transmits the event number of the corresponding event and the reproduction history data of the event to the history storage unit 63.

The monitoring support data stored in the history storage unit 63 includes a list of playback history numbers.

The history storage unit 63 stores reproduction history data. That is, the history storage unit 63 stores the received reproduction history data. Thereafter, the history storage unit 63 adds the reproduction history number of the stored reproduction history data to the list of the reproduction history numbers of the monitoring support data stored in association with the received event number.

When the reproduction unit 62 does not transmit the part of the video data corresponding to the period in which the event occurs to the display device 8, the list of reproduction history numbers of the monitoring support data corresponding to the event has one element. It will be an empty list not included. The list of reproduction history numbers is an example of information indicating whether or not the reproduction unit 62 has transmitted the portion of the video data corresponding to the period in which the event has occurred to the display device 8.

The monitoring support data includes a list of reproduction history numbers, which is information for identifying reproduction history data. Therefore, the monitoring support data includes information indicating whether the reproduction unit 62 has transmitted the portion of the video data corresponding to the period in which the event has occurred to the display device 8.

That is, the history storage unit 63 performs monitoring support data including information indicating whether or not the reproduction unit 62 transmits to the display device 8 a portion of the video data corresponding to a period in which the event occurs, the event of the event Store in association with the number.

The history management unit 64 creates monitoring result data based on the input performed by the monitor via the input device 9.

The monitoring result data includes a monitoring result number identifying the monitoring result data. The monitoring result data includes the result of the monitoring person judging about the presence or absence of abnormality in the event after the monitoring operation of the image displayed on the display device 8 by the visual observation of the event is performed by the monitoring person. The monitoring result data includes a video storage code.

The video storage code indicates the result of the monitoring person's judgment as to whether or not storage of a portion of video data corresponding to the event is necessary.

The history management unit 64 newly acquires a monitoring result number when creating monitoring result data. After creating the monitoring result data, the history management unit 64 transmits the event number of the corresponding event and the monitoring result data of the event to the history storage unit 63.

The monitoring support data stored in the history storage unit 63 includes a list of monitoring result numbers.

The history storage unit 63 stores monitoring result data. That is, the history storage unit 63 stores the received monitoring result data. The history storage unit 63 adds the monitoring result number of the stored monitoring result data to the monitoring result number list of the monitoring support data stored in association with the received event number.

When the result of the visual confirmation work of the video in which the event has occurred is not input, the list of monitoring result numbers of the monitoring support data corresponding to the event is an empty list. The list of monitoring result numbers is an example of the information of the result of the visual check operation performed by the monitor for the event. The monitoring support data includes a list of monitoring result numbers, which is information for identifying monitoring result data.

The digest video processing unit 65 creates digest video data.

The digest video processing unit 65 acquires from the event storage unit 61 the event number of the event occurring during the selected period selected by the observer. The digest video processing unit 65 acquires event data from the event storage unit 61 based on each of the acquired event numbers. The digest video processing unit 65 acquires monitoring support data from the history storage unit 63 based on each of the acquired event numbers.

The digest video processing unit 65 determines, based on the video storage code of each of the acquired monitoring support data, whether or not storage of the portion of the video data to which each of the acquired event numbers corresponds is necessary. The digest video processing unit 65 creates an event list storing the event numbers corresponding to the portion of the video data determined to be required to be stored.

The creation of the event list based on the selection period and the video storage code is an example of the selection of one or more events by the observer.

The digest video processing unit 65 combines, for each of the events identified in each of the elements of the event list, a portion of video data representing a video in which the event is occurring as digest video data. When there is an overlapping period in the period in which each of the events is generated, the digest video processing unit 65 combines portions of the video data corresponding to the overlapping periods as digest video data without overlapping. The digest video processing unit 65 does not combine parts of video data corresponding to a period in which no event has occurred.

The digest video processing unit 65 corrects the start frame number and the end frame number of the event in the original video data based on the number of frames corresponding to the period in which the digest video data is not combined. The digest video processing unit 65 calculates the start frame number and the end frame number of the event in the digest video data by the correction. The start frame number and the end frame number of the event in the digest video data are examples of information for specifying the portion of the digest video data corresponding to the period in which the event is occurring.

The digest video processing unit 65 creates archive data. The archive data includes digest video data, and a start frame number and an end frame number of an event in the digest video data. The archive data includes an event list and monitoring support data stored in the history storage unit 63 in association with the event numbers stored in the list. The digest video processing unit 65 transmits the archive data.

The digest video storage unit 66 stores archive data. That is, the digest video storage unit 66 stores the archive data received from the digest video processing unit 65. Thereafter, the digest video storage unit 66 associates the event number with the digest video data of the stored archive data. Thereafter, the digest video storage unit 66 associates, with the digest video data of the stored archive data, information identifying the portion of the digest video data corresponding to the period in which the event occurs. Thereafter, the digest video storage unit 66 associates monitoring support data with the digest video data of the stored archive data.

The start time point to the end time point of the video represented by the digest video data is included in the selection period. The digest video processing unit 65 deletes the portion corresponding to the video data selection period from the video storage device 3.

The monitoring support device 6 stores information on the monitor as monitor data. The monitoring support device 6 identifies the monitoring person who is performing the confirmation work via the monitoring terminal 7 by the monitoring person number transmitted from the monitoring person.

The monitoring person number is information for identifying a monitoring person who performs a visual check operation. The monitor number is transmitted from the monitor when, for example, connection with the monitor terminal 7 is started to operate the monitor support device 6. The session is a period in which the connection between the monitoring support device 6 and the monitoring terminal 7 is continuing. The monitoring support device 6 stores the monitoring person number until the session ends.

Subsequently, an example of the monitoring support apparatus according to the present embodiment will be described. FIG. 3 is a diagram showing a hardware configuration of main parts of the monitoring support apparatus according to the present embodiment.

Each function of the monitoring support device 6 can be realized by a processing circuit. The processing circuit comprises at least one processor 6b and at least one memory 6c. The processing circuit may comprise at least one dedicated hardware 6a together with or as an alternative to the processor 6b and the memory 6c.

When the processing circuit includes the processor 6 b and the memory 6 c, each function of the monitoring support device 6 is realized by software, firmware, or a combination of software and firmware. At least one of software and firmware is described as a program. The program is stored in the memory 6c. The processor 6 b implements each function of the monitoring support device 6 by reading and executing the program stored in the memory 6 c.

The processor 6 b is also referred to as a central processing unit (CPU), a central processing unit, a processing unit, an arithmetic unit, a microprocessor, a microcomputer, or a DSP. The memory 6c is composed of, for example, a nonvolatile or volatile semiconductor memory such as a RAM, a ROM, a flash memory, an EPROM, an EEPROM, a magnetic disk, a flexible disk, an optical disk, a compact disk, a mini disk, a DVD or the like.

When the processing circuit includes dedicated hardware 6a, the processing circuit is realized by, for example, a single circuit, a complex circuit, a programmed processor, a parallel programmed processor, an ASIC, an FPGA, or a combination thereof.

Each function of the monitoring support device 6 can be realized by a processing circuit. Alternatively, each function of the monitoring support device 6 can be realized collectively by a processing circuit. The respective functions of the monitoring support apparatus 6 may be partially realized by dedicated hardware 6 a and the other portions may be realized by software or firmware. Thus, the processing circuit implements each function of the monitoring support device 6 with the hardware 6a, software, firmware, or a combination thereof.

Subsequently, a function of determining a period in which an event is occurring based on the amount of change with respect to time of the video will be described. The amount of change is calculated based on a preset designated area of the video. FIG. 4 is a view showing an example of a setting screen of a designated area according to the present embodiment.

The setting screen of the designated area is displayed on the display device 8 and operated by the observer via the input device 9. The setting screen of the designated area displays a background image 80 and a box 81.

The background image 80 is an image of a monitoring target location where an event has not occurred. Background image 80 is, for example, an image in an elevator car without a passenger. The background image 80 is an example of video data as a reference.

The designated area is the inside of the box 81 displayed superimposed on the background image 80. The box 81 can be moved, enlarged and reduced by operating the anchor 81a. The anchor 81 a is operated by the observer via the input device 9. The background image 80 and the designated area are set in advance by the observer.

The determination unit 60 switches the reception source of the video data based on the input of the operation by the supervisor. For example, in a normal case, the determination unit 60 receives video data from the imaging device 2. The determination unit 60 receives video data from the video storage device 3 when, for example, again determining a period in which an event has occurred from past video data.

When the video data is encoded, the determination unit 60 decodes the video data before calculating the amount of change.

When receiving the video data from the imaging device 2, the determination unit 60 determines a period in which an event has occurred for each of the video data transmitted from each of the plurality of imaging devices 2. Each of the photographing devices 2 that transmit the video data is identified by the video number.

The determination unit 60 sequentially reads the frames of the video data into the work area. The work area is, for example, the memory 6c.

When receiving the video data from the photographing device 2, the determination unit 60 reads the frame photographed by the photographing device 2 into the work area when the operation of determining the period in which the event is occurring is started. When receiving the video data from the video storage device 3, the determination unit 60 reads the first frame to the last frame of the video data into the work area.

The determination unit 60 calculates the amount of change for each of the read frames.

The determination unit 60 calculates the difference between the luminance information of each pixel in the frame read into the work area and the background image 80. The determination unit 60 calculates the average value of the absolute values of the calculated differences in luminance information for the pixels corresponding to the designated area. The determination unit 60 sets the calculated average value as the amount of change of the frame.

The determination unit 60 calculates the coordinates of the change position when calculating the change amount. The determination unit 60 calculates a weighted average of coordinates with respect to the pixels corresponding to the designated area, using the absolute value of the difference in luminance information from the background image 80 as a weight. The determination unit 60 sets the weighted average of the calculated coordinates as the coordinates of the change position of the frame. The coordinates of the change position are an example of coordinates corresponding to the position where the event occurs in the image.

Then, the determination method of the period which the event has generate | occur | produced is demonstrated. FIG. 5 is a diagram showing the structure of event type data according to the present embodiment.

The elements listed under each item in FIG. 5 are examples of data format. Parentheses represent fixed length pairs of elements. Square brackets represent a variable length set of lists or elements.

The event type data 70 includes an event type number, a change amount calculation method number, a start threshold, an end threshold, and a designated area.

The change amount calculation method number identifies the method by which the determination unit 60 calculates the amount of change of the frame.

The determination unit 60 determines that the time when the amount of change of the frame becomes larger than the threshold for start is the start time of the period in which the event is occurring. The amount of change of the frame at the start of the event occurrence period is greater than the start threshold. The amount of change of the frame immediately before the frame at the start of the period in which the event occurs is equal to or less than the start threshold.

The determination unit 60 determines the time when the amount of change of the frame becomes smaller than the threshold for start as the start time of the period in which the event occurs. The amount of change of the frame at the end of the event occurrence period is smaller than the end threshold. The amount of change of the frame immediately before the frame at the end of the period in which the event occurs is equal to or greater than the threshold for the end.

The determination unit 60 determines the period in which the event is occurring based on the event type data 70.

The determination unit 60 calculates the change amount of the frame using the change amount calculation method identified by the change amount calculation method number included in the event type data 70.

The event type data 70 further includes a monitoring type number, relative coordinates of the marker, and marker image data.

The monitoring type number is a number that classifies the confirmation work based on what kind of video change the monitoring person should watch for monitoring. The monitoring type number is associated with the event type number so that the observer can pay attention to the feature of the video to which attention should be directed according to the event type. The monitoring type number includes information as to whether or not visual confirmation is necessary.

The marker image data is data representing a marker image displayed superimposed on the video in which the event has occurred.

The relative coordinates of the marker are data for specifying the position of the marker image displayed superimposed on the video in which the event has occurred. The relative coordinates of the marker are relative coordinates based on the coordinates of the change position.

Subsequently, event data will be described. FIG. 6 is a diagram showing the structure of event data according to the present embodiment. The event data 71 includes an event number, an event occurrence time, an event type number, a video number, a path of a file of video data, and a start frame number and an end frame number of a period in which the event is generated. . The event data 71 includes a list of change amounts from the start frame to the end frame and a list of coordinates of change positions.

The event occurrence time is the time of the video represented by the frame corresponding to the start time of the period in which the event occurs.

The event data 71 is created as follows.

The determination unit 60 newly creates the event data 71 when it determines the start time of the period in which the event is occurring. The determination unit 60 newly acquires an event number. Thereafter, the determination unit 60 acquires the number of the current frame as the start frame number. Thereafter, the determination unit 60 acquires the time of the video represented by the current frame as the event occurrence time. Thereafter, the determination unit 60 creates a list of change amounts and a list of coordinates of change positions as an empty list. Thereafter, the determination unit 60 adds the amount of change of the current frame to the list of amounts of change. Thereafter, the determination unit 60 adds the list of coordinates of the change position of the current frame to the list of coordinates of the change position.

The determination unit 60 additionally adds to the event data 71 when it is determined that the event is occurring. The determination unit 60 adds the amount of change of the current frame to the list of amounts of change. Thereafter, the determination unit 60 adds the coordinates of the change position of the current frame to the list of coordinates of the change position.

When the determination unit 60 determines the end time of the period in which the event is occurring, the determination unit 60 additionally stores the event data 71. The determination unit 60 acquires the number of the current frame as the end frame number. Thereafter, the determination unit 60 adds the amount of change of the current frame to the list of amounts of change. Thereafter, the determination unit 60 adds the coordinates of the change position of the current frame to the list of coordinates of the change position. After that, the determination unit 60 combines the information acquired or produced above, the event type number of the event for which the occurring period has been determined, the video number of the video for which the determination has been made, and the path of the video data file. Create event data.

After creating the event data 71, the determination unit 60 transmits the event data 71 to the event storage unit 61. The event storage unit 61 stores the received event data 71.

Subsequently, an operation of determining a period in which an event has occurred will be described. FIG. 7 is a flowchart showing an example of an operation of determining a period during which an event occurs in the monitoring support apparatus according to the present embodiment. The monitoring support device 6 performs the operation shown in FIG. 7 on each of the video data transmitted from each of the plurality of photographing devices 2.

The determination unit 60 reads one frame from the received video data into the work area. Thereafter, the determination unit 60 sets the current subscript number to the subscript number corresponding to the first element of the event type list (S101).

The determination unit 60 acquires one event type number designated by the current subscript number from the event type list. Thereafter, the determination unit 60 reads the event type data 70 based on the acquired event type number. Thereafter, the determination unit 60 calculates the change amount of the frame read into the work area and the coordinates of the change position based on the change amount calculation method number of the event type data 70 and the designated area (S102).

The determination unit 60 determines whether the change amount of the immediately preceding frame is equal to or less than the start threshold (S103). When the determination result is Yes, the operation of the monitoring support apparatus 6 proceeds to S104. When the determination result is No, the operation of the monitoring support apparatus 6 proceeds to S105.

The determination unit 60 determines whether the change amount of the current frame is larger than the start threshold (S104). When the determination result is Yes, the operation of the monitoring support apparatus 6 proceeds to S106. When the determination result is No, the operation of the monitoring support apparatus 6 proceeds to S109.

The determination unit 60 determines whether the change amount of the current frame is smaller than the end threshold (S105). When the determination result is No, the operation of the monitoring support apparatus 6 proceeds to S107. When the determination result is Yes, the operation of the monitoring support apparatus 6 proceeds to S108.

The determination unit 60 determines that the current frame is the start time of the period in which the event occurs. The determination unit 60 newly creates the event data 71 (S106).

The determination unit 60 determines that the current frame is within a period in which an event has occurred. The determination unit 60 adds the event data 71 (S107).

The determination unit 60 determines that the current frame is the end time of the period in which the event occurs. The determination unit 60 appends to the event data 71. Thereafter, the event storage unit 61 saves the event data 71 (S108).

The determination unit 60 determines whether the current subscript number is the suffix number at the end of the event type list (S109). If the judgment result is No, the current subscript number is taken as the next subscript number. Thereafter, the operation of the monitoring support apparatus 6 proceeds to S102. When the determination result is Yes, the determination unit 60 sets the current frame as the next frame. Thereafter, the operation of the monitoring support apparatus 6 proceeds to S101.

Next, the function of creating a list of events will be described. FIG. 8 is a diagram showing an example of a screen displaying a list of events according to the present embodiment. The example of the screen shown in FIG. 8 is displayed on the display device 8 when, for example, a session is started.

The screen displaying the list of events displays a start time input field 82a, an end time input field 82b, a list display button 83, a display area 84, a digest video creation button 85, and a continuous play button 86. .

The selection period is set by the supervisor inputting the start time and the end time in the start time input field 82a and the end time input field 82b.

The event occurring during the selection period is, for example, a period during which the time corresponding to the start time of the period during which the event is occurring is after the start time of the selection period and the event during which the event is occurring This is an event in which the time corresponding to the end time is before the end time of the selection period.

The operation of the function of creating the list of events starts when the list display button 83 is pressed by the observer. When the list of events is created, the display device 8 displays the list of events created in the display area 84.

The display area 84 displays a list of events occurring during the selection period. The display area 84 when the session starts may be blank. The display area 84 displays information stored in association with the event number. The display area 84 displays, for example, a monitoring support number, an event number, an event occurrence time, an event type number, a video number, a reproduction history number, and a monitoring result number. The display area 84 displays a play button 84a and a monitoring result input button 84b.

The operation of the function of transmitting the video in which the event is occurring to the display device starts, for example, when the play button 84a corresponding to the event is pressed by the observer.

The operation of the function of inputting the monitoring result starts, for example, when the monitoring result input button 84b corresponding to the event is pressed by the observer.

The operation of the function of creating digest video data starts, for example, when the digest video creation button 85 is pressed by the observer.

The monitoring support device 6 may have a function of continuously transmitting to the display device 8 the video generated by each of the plurality of events. The operation of the function of continuously transmitting to the display device 8 starts, for example, when the continuous play button 86 is pressed by the observer.

Subsequently, monitoring support data will be described. FIG. 9 is a diagram showing the structure of monitoring support data according to the present embodiment.

The monitoring support data 72 includes a monitoring support number, an event number, a monitoring type number, a list of reproduction history numbers, a list of monitoring result numbers, and link information to video data.

The history storage unit 63 stores monitoring support data. That is, the history storage unit 63 stores the monitoring support number and the event number in association with each other.

The link information to the video data is information for identifying a portion of the video data corresponding to a period in which the event is occurring, for the event associated with the monitoring support data. The link information to the video data is, for example, a set of a start frame number and an end frame number of the event and a path of the corresponding video data.

The reproduction unit 62 creates monitoring support data as follows.

The reproduction unit 62 acquires, from the event storage unit 61, the event number of the event occurring within the selection period, and each of the event data 71 associated with each of the event numbers. The playback unit 62 creates an event list that stores the acquired event number as an element.

The reproduction unit 62 acquires the event type data 70 based on the acquired event type number of the event data 71. The reproduction unit 62 acquires the monitoring type number included in the acquired event type data 70. The reproducing unit 62 determines the necessity of visual confirmation based on the acquired monitoring type number.

When the reproducing unit 62 determines that visual confirmation work is necessary, the reproducing unit 62 determines whether the corresponding monitoring support data has already been created. When the reproducing unit 62 determines that the corresponding monitoring support data is not created, it starts creating the monitoring support data.

The reproduction unit 62 newly acquires a monitoring support number when starting creation of the monitoring support data. Thereafter, the playback unit 62 creates a list of playback history numbers as an empty list. Thereafter, the reproduction unit 62 creates a list of monitoring result numbers as an empty list. The reproduction unit 62 creates link information to video data based on the acquired event data 71.

The reproduction unit 62 combines the information acquired or created above to create the monitoring support data 72. After creating the monitoring support data, the reproduction unit 62 transmits the created monitoring support data and the corresponding event number to the history storage unit 63.

Subsequently, an operation of creating a list of events will be described. FIG. 10 is a flowchart showing an example of an operation of creating a list of events of the monitoring support apparatus according to the present embodiment.

The supervisor inputs the selection period into the start time input field 82a and the end time input field 82b. Thereafter, the supervisor presses the list display button 83. Thereafter, the monitoring support device 6 starts an operation of creating a list of events.

The playback unit 62 creates an event list of all the events occurring in the selection period. Thereafter, the reproducing unit 62 sets the current subscript number to the subscript number corresponding to the top element of the event list (S201).

The playback unit 62 acquires one event number designated by the current subscript number from the event list. The reproduction unit 62 acquires the event data 71 from the event storage unit 61 based on the acquired event number (S202).

Based on the acquired event data 71, the reproduction unit 62 determines whether a visual confirmation operation is necessary (S203). When the determination result is Yes, the operation of the monitoring support apparatus 6 proceeds to S204. When the determination result is No, the operation of the monitoring support apparatus 6 proceeds to S206.

The reproducing unit 62 determines whether monitoring support data corresponding to the current event number has already been created (S204). When the determination result is No, the operation of the monitoring support apparatus 6 proceeds to S205. When the determination result is Yes, the operation of the monitoring support apparatus 6 proceeds to S206.

The reproduction unit 62 creates monitoring support data. Thereafter, the reproduction unit 62 transmits the current event number and the created monitoring support data. Thereafter, the history storage unit 63 stores the received monitoring support data (S205).

The reproducing unit 62 determines whether the current subscript number is the suffix number at the end of the event list (S206). If the judgment result is No, the current subscript number is taken as the next subscript number. Thereafter, the operation of the monitoring support apparatus 6 proceeds to S202. When the determination result is Yes, the operation of the monitoring support apparatus 6 proceeds to S207.

The reproduction unit 62 transmits data representing a list of events to the display device 8. Thereafter, the display device 8 displays a list of events in the display area 84 (S207).

Subsequently, a function of transmitting a video in which an event is occurring to the display device will be described. FIG. 11 is a diagram showing an example of a screen of an image in which an event displayed by the display device according to the present embodiment has occurred. An example of the screen shown in FIG. 11 is displayed on the display device 8 when the monitor presses the play button 84a.

The video display screen displays a monitoring image 87, a marker image 88, and a box 89.

The surveillance image 87 is an image represented by each frame of video data in which the situation of the surveillance target location is recorded. The monitoring image 87 is, for example, an image in an elevator car on which a passenger is riding.

The marker image 88 is displayed superimposed on the monitoring image at the position of relative coordinates set in advance with reference to the coordinates of the change position. The marker image 88 is an example of a monitoring assistance display.

The reproduction unit 62 acquires the event data 71 of the event from the event storage unit 61 based on the event number of the event selected by the observer from the list of events displayed in the display area 84.

The reproduction unit 62 acquires the event type data 70 stored in the determination unit 60 based on the acquired event type number of the acquired event data.

The reproduction unit 62 acquires the monitoring support data 72 of the event from the history storage unit 63 based on the event number of the event selected by the monitor from the created list.

The reproduction unit 62 reads in order from the start frame to the end frame of the portion of the video data corresponding to the period in which the selected event is occurring, based on the acquired event data 71.

Based on the acquired event data 71, the reproduction unit 62 sets the coordinates of the change position corresponding to the read frame as the reference of the position where the marker image is displayed. The reproduction unit 62 calculates the position at which the marker image is displayed, based on the relative coordinates of the marker included in the acquired event type data 70. For example, if the relative coordinates of the marker are (0, 0), the marker image is displayed at the coordinates of the change position.

A box 89 is displayed at the boundary of the designated area of the event type to which the selected event corresponds. Box 89 is another example of the monitoring assistance display.

The reproduction unit 62 changes the form of the monitoring assistance display based on the acquired monitoring support data.

The form of the monitoring assistance display is, for example, the color, transparency, size or line type of the monitoring assistance display.

For example, when the list of playback history numbers of the monitoring support data of the selected event is empty, the playback unit 62 turns the color of the box 89 red. For example, when the list of playback history numbers of the monitoring support data of the selected event is not empty, the playback unit 62 sets the color of the box 89 to blue.

Subsequently, reproduction history data will be described. FIG. 12 is a diagram showing the structure of reproduction history data according to the present embodiment.

The reproduction history data 73 includes a reproduction history number, an observer number, an event number, a reproduction start time, a reproduction start frame, and a reproduction end frame.

The surveillance person number is a surveillance person number corresponding to the surveillance person who is performing visual confirmation work.

The reproduction start time is the time when the monitor selects an event to be confirmed.

The number of the reproduction start frame is the number of the first frame transmitted to the display device 8 by the reproduction unit 62. The number of the reproduction end frame is the number of the last frame transmitted to the display device 8 by the reproduction unit 62.

The reproduction history data 73 is created as follows.

When transmitting the start frame of the portion of the video data corresponding to the period in which the selected event is occurring, the reproduction unit 62 newly acquires a reproduction history number. Thereafter, the reproduction unit 62 acquires the number of the reproduction start frame. Thereafter, the reproduction unit 62 acquires the current time as the reproduction start time. Thereafter, the reproduction unit 62 acquires the surveillance person number of the surveillance person who has logged in in the current session.

When transmitting the end frame of the portion of the video data to the display device 8, the reproduction unit 62 acquires the number of the reproduction end frame.

The reproduction unit 62 creates reproduction history data 73 by combining the information acquired above and the event number.

After creating the playback history data 73, the playback unit 62 transmits the event number and the created playback history data 73 to the history storage unit 63.

Subsequently, an operation of transmitting a video in which an event is occurring to the display device will be described. FIG. 13 is a flowchart showing an example of an operation of transmitting a video in which an event is occurring to the display device according to the present embodiment.

The observer selects the corresponding event by pressing the play button 84a. When the play button 84 a is pressed, the monitoring support device 6 starts the operation of the function of transmitting the video in which the selected event has occurred to the display device 8.

The reproduction unit 62 acquires event data and monitoring support data based on the event number of the selected event. The playback unit 62 sets the current frame as the start frame (S301).

The reproduction unit 62 sets the form of the monitor assistance display based on the acquired monitor support data 72 (S302).

The playback unit 62 reads the current frame into the work area (S303).

The reproduction unit 62 writes the monitoring assistance display set to be displayed superimposed on the image represented by the frame read into the work area (S304).

The reproduction unit 62 transmits the data stored in the work area to the display device 8 (S305).

The playback unit 62 determines whether the current frame number is the end frame number (S306). When the determination result is No, the reproduction unit 62 sets the current frame to the next frame. Thereafter, the operation of the monitoring support apparatus 6 proceeds to S303. When the determination result is Yes, the reproduction unit 62 acquires the number of the reproduction end frame. Thereafter, the operation of the monitoring support apparatus 6 proceeds to S307.

The reproduction unit 62 creates reproduction history data 73. Thereafter, the reproduction unit 62 transmits the reproduction history data 73 and the event number to the history storage unit 63. Thereafter, the history storage unit 63 stores the reproduction history data (S307).

Then, the function to input the result of the confirmation operation | work by visual observation is demonstrated.

The history management unit 64 creates monitoring result data based on the input of the result of the checking operation by the supervisor. FIG. 14 is a diagram showing the structure of monitoring result data according to the present embodiment.

The monitoring result data 74 includes a monitoring result number, an event number, a monitoring person number, a monitoring support number, a list of reproduction history numbers, a confirmation date and time, a monitoring result code, a video storage code, special notes, including.

The confirmation date is the time when the result of the confirmation operation is input by the supervisor.

The monitoring result code represents the result determined by the monitor as to whether or not there is an abnormality in the event after the observer performs a visual confirmation operation of the image in which the event is generated displayed on the display device 8. It is an example of information. As the monitoring result number, 0 is input when it is determined by the observer that there is no abnormality as a result of the visual confirmation operation.

The video storage code represents the result of the judgment by the observer as to whether or not it is necessary to combine and save the video data portion corresponding to the event as digest video data. As the image storage code, 0 is input when it is determined by the observer that there is no need to store the corresponding portion of the image data as a result of the visual check operation.

The special item is a free description item in which an explanation about the monitoring result is input as character string data, such as the basis of the judgment of the monitoring person, for example.

Subsequently, the operation of the function of inputting the result of the visual check operation will be described.

The observer selects the corresponding event by pressing the monitoring result input button 84b. When the monitoring result input button 84 b is pressed, the monitoring support device 6 starts the operation of the function of inputting the monitoring result of the confirmation operation for the selected event.

The monitoring support device 6 requests the supervisor to input the monitoring result. The request for the input of the monitoring result is made, for example, interactively. The monitoring support device 6 requests the monitoring person to input one by one in the order of the monitoring result code, the video storage code, and the special item.

When the input of the monitoring result for the selected event is completed, the history management unit 64 creates the monitoring result data 74.

The monitoring result data 74 is created as follows. When creating the monitoring result data 74, the history management unit 64 newly acquires the monitoring result number. The history management unit 64 acquires the monitoring support data 72 from the history storage unit 63 based on the event number of the selected event. The history management unit 64 acquires the monitoring support number included in the acquired monitoring support data 72.

The history management unit 64 acquires the surveillance person number of the surveillance person who has logged in in the current session. The history management unit 64 acquires the reproduction history number of the reproduction history common to the selected event among the reproduction histories created in the current session.

The history management unit 64 acquires the current time as the confirmation date and time.

The history management unit 64 acquires the monitoring result code, the video storage code, and the special item based on the input of the monitoring person.

The history management unit 64 creates the monitoring result data 74 by combining the information acquired above.

After creating the monitoring result data 74, the history management unit 64 transmits the selected event number and the created monitoring result data to the history storage unit 63.

Subsequently, a function of creating digest video data will be described.

FIG. 15 is a diagram showing an example of a period combined as digest video data according to the present embodiment. In FIG. 15, the horizontal axis represents time.

The digest video processing unit 65 acquires, from the event storage unit 61, the event number of the event occurring within the selection period, and the event data 71 associated with each of the event numbers. The digest video processing unit 65 creates an event list that stores the acquired event number as an element.

The digest video processing unit 65 acquires, from the history storage unit 63, each of the monitoring support data 72 stored in association with each of the event numbers stored in the event list. The digest video processing unit 65 acquires the monitoring result data 74 from the history storage unit 63 based on the monitoring result number stored in the acquired monitoring result number list of the monitoring support data 72. The digest video processing unit 65 determines whether it is necessary to combine and save video data corresponding to the event as digest video data based on the video storage code of the acquired monitoring result data 74.

The digest video processing unit 65 sets an event determined to require combining and storing as digest video data as an event requiring storage. The digest video processing unit 65 sets an event determined not to require combining and storing as digest video data as a storage unnecessary event. The digest video processing unit 65 removes the event number corresponding to the storage unnecessary event from the event list. The event represented by the event number stored in the event list is an example of one or more events selected by the observer.

In FIG. 15, a line segment having a circle at each end represents a period during which each event occurs. The left end of the line segment indicates the start time of the period in which the event is occurring. The right end of the line segment indicates the end time of the period in which the event occurs. A solid line segment having a circle at both ends represents an event 90 which needs to be stored. A broken line segment having a circle at each end represents an event 91 which does not require storage.

The digest video processing unit 65 sequentially reads from the frame number corresponding to the start time of the selection period to the frame number corresponding to the end time of the original video data. The digest video processing unit 65 determines whether or not the point in time corresponding to the read frame number is in a period in which any of the storage-needed events 90 has occurred. When it is determined that the time point corresponding to the read frame number is within the period, the digest video processing unit 65 combines the frame designated by the frame number as digest video data.

As a result, when there is an overlapping period in the period in which each of the storage-required events 90 occurs, the digest video processing unit 65 does not overlap the portion of the video data corresponding to the overlapping period. Combine as data. The digest video processing unit 65 does not combine portions of video data corresponding to a period in which no event 90 requiring storage occurs, as digest video data.

In FIG. 15, a line segment having a rhombus at each end represents a storage period 92 corresponding to the digest video data created by the digest video processing unit 65. The retention period 92 is a period including a period in which one or more selected events occur. The retention period 92 may not be one continuous period.

When combining the original video data with the video data corresponding to the discontinuous storage period 92, the digest video processing unit 65 corrects the frame number so that the frame numbers in the digest video data are continuous. The digest video processing unit 65 performs correction by subtracting the number of frames not included in the storage period 92 from the frame numbers of the frames subsequent to the frame. The digest video processing unit 65 similarly corrects the frame number for the data of the frame number included in the acquired event data 71 and the reproduction history data 73. The assigned frame number is the frame number in the digest video data.

The digest video processing unit 65 is a part of the original video data corresponding to a period including the earliest point of the start point of the save required event 90 to the latest point of the end time of the save required event 90 Is deleted from the video storage device 3.

In FIG. 15, a line segment having triangles at both ends represents a deletion period 93 for which the digest video processing unit 65 is to be deleted. The digest video processing unit 65 may delete the portion of the original video data corresponding to the entire selection period including the deletion period 93 from the video storage device 3.

The digest video processing unit 65 creates digest video data for each imaging device 2 having a different video number.

Subsequently, an operation of creating digest video data will be described. FIG. 16 is a flowchart showing an example of an operation of creating digest video data of the monitoring support apparatus according to the present embodiment. The monitoring support device 6 performs the operation shown in FIG. 16 on each of the video data transmitted from each of the plurality of photographing devices 2.

The monitoring support device 6 starts the function of creating digest video data when the digest video creation button 85 is pressed by the observer.

The digest video processing unit 65 creates an event list storing the event numbers of the events 90 that need to be stored. Thereafter, the digest video processing unit 65 sets the current frame number to the frame number corresponding to the start time of the selection period. Thereafter, the digest video processing unit 65 empties the output work area (S401).

The digest video processing unit 65 sets the current subscript number as the leading subscript number of the event list (S402).

The digest video processing unit 65 acquires the start frame number and the end frame number from the event data 71 associated with the event number specified by the current subscript number. Thereafter, the digest video processing unit 65 corrects the frame number (S403).

The digest video processing unit 65 determines whether the current frame number is within the period specified by the acquired start frame number and end frame number (S404). When the determination result is Yes, the operation of the monitoring support apparatus 6 proceeds to S405. When the determination result is No, the operation of the monitoring support apparatus 6 proceeds to S406.

The digest video processing unit 65 reads the current frame, adds it to the working area for output together with the corrected frame number, and writes it (S405).

The digest video processing unit 65 determines whether the current subscript number is the suffix number at the end of the event list (S406). If the judgment result is No, the current subscript number is taken as the next subscript number. Thereafter, the operation of the monitoring support apparatus 6 proceeds to S403. When the determination result is Yes, the operation of the monitoring support apparatus 6 proceeds to S407.

The digest video processing unit 65 updates the number of frames not included in the storage period 92 (S407).

The digest video processing unit 65 determines whether the current frame number is a frame number corresponding to the end time of the selection period (S408). If the determination result is No, the current frame number is set to the next frame number. Thereafter, the operation of the monitoring support apparatus 6 proceeds to S402. When the determination result is Yes, the operation of the monitoring support apparatus 6 proceeds to S409.

The digest video processing unit 65 combines the contents of the frames stored in the output work area as digest video data. Thereafter, the digest video processing unit 65 creates archive data. Thereafter, the digest video processing unit 65 transmits the created archive data. Thereafter, the digest video storage unit 66 stores the received archive data. Thereafter, the digest video processing unit 65 deletes the portion of the video data corresponding to the deletion period from the video storage device 3 (S409).

Subsequently, supervisor data will be described. FIG. 17 is a diagram showing the structure of supervisor data according to the present embodiment. The monitoring support device 6 stores the supervisor data 75. The monitor data includes a monitor number and a monitor name. The observer data 75 may include information on the attributes of the observer, such as the affiliation of the observer, the title, or the surveillance experience.

As described above, the monitoring support system 4 supports the monitoring work of the supervisor as follows. The playback unit 62 transmits the portion of the video data corresponding to the period in which the event is occurring to the display device 8 based on the event number selected by the observer. The display device 8 displays a video represented by the received video data. The supervisor performs a visual confirmation operation on the video displayed on the display device 8. The history storage unit 63 stores the monitoring support data 72 in association with the event number. The monitoring support data 72 includes reproduction history data 73 which is information indicating whether the reproduction unit 62 has transmitted the portion of the video data to the display device 8. That is, the monitoring support device 6 does not subdivide the video data for each part as a part serving as a unit of a target of confirmation work by the supervisor as a part of the video data corresponding to the period in which the event occurs. It is possible to store information on the results of the confirmation work for each part.

The determination unit 60 determines a period in which an event is occurring, based on the amount of change with respect to time of the video. The monitoring support device 6 extracts video data representing a video in which an event has occurred. As a result, the observer can perform the confirmation operation only on the video in which the extracted event has occurred. Therefore, the burden of the visual confirmation work of the observer is reduced.

The history storage unit 63 stores the monitoring support data 72 including the monitoring result data 74 in association with the event number. The history storage unit 63 can store the information of the result of the check operation for each of the portions without subdividing the video data for each of the portions of the video data corresponding to the period in which the event occurs. Thereby, it is suppressed that the information of the result of several monitoring work is dissipated. Therefore, the supervisor can easily manage the information of the result of the confirmation work.

The history management unit 64 creates the monitoring result data 74 based on the input made by the monitoring person himself via the input device 9. Thereby, as compared with the case where the history storage unit 63 stores the monitoring support data including only the reproduction history data 73, the monitoring support device 6 can improve the accuracy of the information as to whether the monitoring person actually performed the confirmation work or not. .

The determination unit 60 calculates the amount of change based on the designated area in the video. As a result, the observer can flexibly set the area to be checked in the video. Also, the observer can set a designated area for each event. As a result, the history storage unit 63 can store information on the result of the confirmation work by the supervisor for each event in which the time and the area in the video data are specified.

The reproduction unit 62 transmits, to the display device 8, data in which a monitoring assistance display indicating information such as an area in which an event has occurred and an event type is superimposed on a video of an event. As a result, the monitoring support system 4 can clearly show the observer the portion or region to be focused in the video. Therefore, the possibility that the observer overlooks the matter to be confirmed in the confirmation operation is reduced.

The reproduction unit 62 changes the form of the monitoring assistance display based on the monitoring support data. As a result, the monitoring support system 4 can highlight and show the part requiring special attention in the confirmation operation, such as a part of the unconfirmed video data or a part where a problematic event has occurred. Therefore, the possibility that the observer overlooks the part requiring attention in the confirmation operation is reduced.

The digest video processing unit 65 combines video data corresponding to a period in which an event occurs as digest video data. As a result, the observer can easily perform confirmation work of a plurality of events collectively.

The digest video processing unit 65 deletes the portion of the original video data corresponding to the portion combined as digest video data from the video storage device 3. Thereby, the monitoring system 1 can effectively use the storage capacity of the video storage device 3.

The digest video storage unit 66 stores digest video data extracted from the original video data and the monitoring support data 72 in association with each other. As a result, the monitor can collectively check the presence or absence of the confirmation work results for a plurality of events occurring in the video represented by the digest video data. Therefore, the occurrence of the leakage of the confirmation work is reduced. Also, the monitoring support system 4 can superimpose the monitoring auxiliary display on the digest video data based on the archive data and show it to the monitor.

Subsequently, another example of the function of transmitting the video in which the event is occurring to the display device will be described. FIG. 18 is a diagram showing another example of the screen of the video in which the event displayed by the display device according to the present embodiment has occurred.

The monitoring support device 6 may have a continuous reproduction function as another example of the function of transmitting an image in which an event is occurring to the display device 8. The continuous reproduction function is an example of the function of the reproduction unit 62 when a plurality of events occurring in the selection period are selected by the observer in a video shot by one shooting device 2 as a target. The target imaging device 2 is designated by the video number selected by the supervisor.

The video display screen displays a monitoring image 87, a marker image 88, and a box 89. The video display screen displays a marker image 88 or a box 89 for each of a plurality of events when a plurality of events occur simultaneously.

The operation of the continuous play function starts when the continuous play button 86 is pressed by the observer.

The playback unit 62 acquires, from the event storage unit 61, the event number of the event of the selected video number occurring within the selection period, and the event data 71 associated with each of the event numbers. The playback unit 62 creates an event list that stores the acquired event number as an element.

The reproduction unit 62 acquires the start frame number and the end frame number of each event based on each of the acquired event data 71.

The reproduction unit 62 acquires each of the event type data 70 based on the event type number of each of the acquired event data.

The reproduction unit 62 acquires each of the monitoring support data 72 from the history storage unit 63 based on each of the event numbers stored in the event list. The monitoring support apparatus 6 may start the operation of the function of creating the list of events before starting the operation of the continuous playback function when the list of events is not created.

The reproduction unit 62 sets the form of the monitoring assistance display for each of the events occurring in the selection period, based on the event number stored in the event list and the acquired monitoring assistance data 72.

The reproduction unit 62 sequentially reads the frames from the frame corresponding to the start time of the selection period to the frame corresponding to the end time into the work area.

When transmitting the frame corresponding to the start time point of the selection period to the display device 8, the reproduction unit 62 newly acquires a reproduction history number. Thereafter, the reproduction unit 62 acquires the number of the reproduction start frame. Thereafter, the reproduction unit 62 acquires the current time as the reproduction start time. Thereafter, the reproduction unit 62 acquires the surveillance person number of the surveillance person who has logged in in the current session.

When transmitting the frame corresponding to the end time of the selection period to the display device 8, the reproduction unit 62 acquires the number of the reproduction end frame.

In the continuous reproduction function, the reproduction unit 62 creates a reproduction history data 73 by combining event lists instead of event numbers because confirmation work of a plurality of events is performed at one time.

After creating the playback history data 73, the playback unit 62 transmits the event list and the created playback history data 73 to the history storage unit 63.

The history storage unit 63 stores the reproduction history data 73. That is, the history storage unit 63 stores the received reproduction history data 73. After that, the history storage unit 63 sets the reproduction history number included in the stored reproduction history data in the list of reproduction history numbers included in the monitoring support data stored in association with each of the event numbers stored in the received event list. to add.

Subsequently, the operation of the continuous reproduction function will be described as another example of the operation of transmitting the video in which the event is occurring to the display device. FIG. 19 is a flow chart showing an example of the operation of the continuous reproduction function of the monitoring support apparatus according to the present embodiment.

The supervisor selects a target video number. By pressing the continuous play button 86, the supervisor selects in a batch the events of the selected video number occurring within the selection period. When the continuous play button 86 is pressed, the monitoring support device 6 starts the operation of the continuous play function.

The playback unit 62 creates an event list of events occurring within the selection period. After that, the reproduction unit 62 sets the form of the monitoring auxiliary display for each of the events in the event list. Thereafter, the reproduction unit 62 sets the number of the current frame to the number of the frame corresponding to the start time of the selection period (S501).

The playback unit 62 reads the current frame into the work area. Thereafter, the reproduction unit 62 sets the current subscript number to the subscript number corresponding to the first element of the event list (S502).

The playback unit 62 determines whether the current frame is within the period of the event specified by the current subscript number (S503). When the determination result is Yes, the operation of the monitoring support apparatus 6 proceeds to S504. When the determination result is No, the operation of the monitoring support apparatus 6 proceeds to S505.

The playback unit 62 writes a monitoring assistance display of the event specified by the current subscript number, superimposed on the image represented by the frame read into the work area (S504).

The reproducing unit 62 determines whether the current subscript number is the suffix number at the end of the event list (S505). When the determination result is No, the reproducing unit 62 sets the current subscript number as the next subscript number. Thereafter, the operation of the monitoring support apparatus 6 proceeds to S503. When the determination result is Yes, the operation of the monitoring support apparatus 6 proceeds to S506.

The reproduction unit 62 transmits the data stored in the work area to the display device 8 (S506).

The reproducing unit 62 determines whether the current frame number is a frame number corresponding to the end time of the selection period (S507). When the determination result is Yes, the operation of the monitoring support apparatus 6 proceeds to S508. When the determination result is No, the reproduction unit 62 sets the current frame to the next frame. Thereafter, the operation of the monitoring support apparatus 6 proceeds to S502.

The reproduction unit 62 creates reproduction history data 73. Thereafter, the reproduction unit 62 transmits the reproduction history data 73 and the event list to the history storage unit 63. Thereafter, the history storage unit 63 stores the received reproduction history data 73 (S508).

When the display device 8 is displaying a video in which an event for which the result is to be input is generated, the function of inputting the result of the visual check operation is a marker image 88 displayed superimposed on the video. The action may be started when is selected by the observer. The selection by the observer may be, for example, when the input device 9 is a mouse, a click on the monitor assistance display by the observer.

The present embodiment is not limited to the method of calculating the change amount of the determination unit 60 and the coordinates of the change position.

The determination unit 60 may use the difference in luminance information from the immediately preceding frame for calculation of the amount of change instead of the difference in luminance information from the background image 80. Thus, the determination unit 60 can use the speed of change of the image to determine the occurrence of an event. When the unnatural motion of the person in the video is to be the target of the confirmation operation, the determination unit 60 may calculate the amount of change using the difference with the adjacent frame.

When the video data is encoded, the determination unit 60 may perform the processing without decoding the encoded data. The determination unit 60 may use the coefficients of the discrete cosine transform to calculate the amount of change. Thereby, the determination unit 60 can reduce the process related to decoding when calculating the amount of change.

The determination unit 60 may calculate the degree of coincidence of the face between the specific person and the person in the image as the change amount by a known image recognition technology. When there are a plurality of persons in the video, the determination unit 60 may set the maximum value of the degree of coincidence as the variation. Thus, the determination unit 60 can use whether or not a specific person appears in the image to determine the period in which the event occurs.

The determination unit 60 may have only one method of calculating the amount of change.

The designated area in the calculation of the amount of change is not limited to the rectangular area. The designated area may be a closed area surrounded by line segments connecting three or more anchors. The designated area may be a circular area within a range of a predetermined length from one point in the image.

The amount of change may be calculated based on the designated area. That is, the amount of change may be calculated only for the entire area of the video data.

The determination unit 60 may determine, as the start time of the period in which the event occurs, a time that is a preset time before the time when the value of the change amount becomes larger than the start threshold. The determination unit 60 may determine, as the end time of the period in which the event is generated, a time that is a predetermined time after the time when the value of the change amount becomes smaller than the end threshold. As a result, the observer can perform the confirmation work including the situation before and after the period in which the change in the image occurs.

The playback unit 62 does not have to superimpose the monitoring assistance display on the video and transmit it to the display device 8. The playback unit 62 may switch display or non-display of the monitoring auxiliary display by the operation of the monitor.

The reproduction unit 62 may not change the form of the monitoring assistance display based on the monitoring support data. Thereby, the supervisor can concentrate attention on the image when the content of the monitoring support data is not important when performing the confirmation work.

The digest video processing unit 65 does not have to create only digest video data and not archive data. This reduces the load on the process of creating the digest video data.

The digest video processing unit 65 may not delete the original video data from the video storage device 3 when creating the digest video data.

The digest video processing unit 65 needs video storage if there is data that requires video storage even if there is one or more pieces of monitoring result data corresponding to one event and the respective video storage codes do not match. It may be determined as

The digest video processing unit 65 may determine the necessity of video storage based on the event type data or the reproduction history data. For example, the digest video processing unit 65 may determine that video storage is necessary when there is no reproduction history data corresponding to the event.

FIG. 20 is a block diagram showing a configuration of a monitoring support apparatus according to a modification of the present embodiment. The monitoring support device 6 may not include the digest video processing unit 65. As a result, when the storage capacity of the video storage device 3 can be sufficiently secured, or when the video data is not stored for a long time, the monitoring support system 4 is simplified and maintainability is improved.

FIG. 21 is a block diagram showing a configuration of a monitoring support apparatus according to a modification of the present embodiment. The monitoring support device 6 may not include the history management unit 64. Thus, when management of reproduction history data is sufficient, the monitoring support system 4 is simplified and maintainability is improved.

FIG. 22 is a block diagram showing a configuration of a monitoring support apparatus according to a modification of the present embodiment. The monitoring support apparatus 6 may not include both the digest video processing unit 65 and the history management unit 64. As a result, the monitoring support system 4 is simplified and maintainability is improved.

The monitoring support device 6 may not include the determination unit 60 and the event storage unit 61. The monitoring support apparatus 6 may create and store monitoring support data based on event data created by another monitoring support system or the monitoring support apparatus.

The monitoring support system 4 may perform communication between devices or parts via a buffer. The buffer is, for example, the memory 6c. For example, the video storage device 3 may transmit video data to the buffer. The determination unit 60 may receive the video data from the buffer.

The monitoring system 1 may include only one imaging device 2. The monitoring system 1 may not include the imaging device 2. The monitoring system 1 may target only video data recorded by another monitoring system that has already been stored as a file in the video storage device 3.

The monitoring system 1 may not include the video storage device 3. The imaging device 2 may transmit the video data directly to the determination unit 60. The event storage unit 61 may store a portion of video data corresponding to a period in which an event occurs.

The monitoring system 1 may not have the monitoring terminal 7. In the monitoring system 1, the photographing device 2, the video storage device 3, the monitoring support device 6, the display device 8, and the input device 9 may be connected without the network 5 or the monitoring terminal 7. The monitoring system 1 may be realized by a single device arranged at one place.

The video data according to the present embodiment is not limited to the format in which the frames are sequentially numbered. The video data is, for example, MPEG (Motion Picture Expert Group) -2 or H.264. It may be in the form of moving image data encoded by a video encoding method such as H.264.

The portion of the video data may differ from the original video data in the format of the specific data. The portion of the video data may be data temporarily stored in the work area without being stored as a file.

Each of the data according to the present embodiment is not limited to the structure exemplified for the explanation. Each of the data according to the present embodiment may be, for example, normalized data as a relational database according to the type, format or size of the data to be applied.

In the data according to the present embodiment, the information for identifying data such as an event number is not limited to data consisting of characters or numeric values. The information identifying the data may be data indicating an address at which the data is stored.

The monitoring support apparatus and the monitoring support system according to the present invention can be applied to a system that performs visual monitoring remotely by video.

DESCRIPTION OF SYMBOLS 1 monitor system, 2 imaging | photography apparatus, 3 image storage apparatus, 4 monitor support system, 5 network, 6 monitor support apparatus, 7 monitor terminal, 8 display apparatus, 9 input apparatus, 60 determination part, 61 event storage part, 62 reproduction part , 63 history storage unit, 64 history management unit, 65 digest video processing unit, 66 digest video storage unit, 6a hardware, 6b processor, 6c memory, 70 event type data, 71 event data, 72 monitoring support data, 73 playback history Data, 74 monitoring result data, 75 observer data, 80 background image, 81 box, 81a anchor, 82a start time input field, 82b end time input field, 83 list display button, 84 display area, 84a playback button, 84b monitoring result input button, 85 digest video creation button, 86 continuous playback button, 87 monitoring image, 88 marker image, 89 box, 90 storage required event, 91 storage unnecessary event, 92 storage Period, 93 deletion period

Claims (12)

When an event whose period of occurrence is determined based on the amount of change with time of the image represented by the video data is selected by the observer, the event selected from among the event identification information for identifying the event is selected by the observer An image showing a portion of the video data which is a portion of the video data stored in association with the event identification information based on the event identification information and which corresponds to a period in which the event is occurring. A reproduction unit that transmits to a display device that displays
A history of storing monitoring support data including information indicating whether the reproduction unit has transmitted to the display device a portion of the video data corresponding to a period in which the event is occurring, in association with the event identification information of the event A storage unit,
A monitoring support device comprising: The history storage unit stores information of the result of the visual check operation performed by the supervisor with respect to the event in the monitoring support data stored in association with the event identification information of the event. The monitoring support device described. A determination unit that calculates the amount of change and determines a period in which an event occurs based on the amount of change;
An event storage unit that stores information identifying a portion of the video data corresponding to the period determined by the determination unit in association with event identification information of the event;
The monitoring support apparatus according to claim 1 or 2, comprising The monitoring support apparatus according to claim 3, wherein the determination unit determines a time when the change amount becomes larger than a preset value as a start time of a period in which an event occurs. 5. The monitoring support apparatus according to claim 3, wherein the determination unit determines that the time when the amount of change is smaller than a preset value is the end time of the period in which an event occurs. The monitoring support apparatus according to any one of claims 3 to 5, wherein the determination unit calculates the amount of change for a preset designated area of the video. The reproduction unit is a video data representing a video in which a monitoring assistance display corresponding to the event is superimposed on a position corresponding to the designated area of the video represented by the portion of the video data corresponding to a period in which the event occurs. The monitoring support device according to claim 6, which transmits the information to the display device. The monitoring support apparatus according to claim 7, wherein the reproduction unit changes the form of the monitoring support display based on the monitoring support data. A digest video processing unit that combines portions of the video data corresponding to a period including all periods in which one or more events selected by the observer occur, as digest video data;
The monitoring support apparatus according to any one of claims 1 to 8, comprising: The reproduction unit is stored in association with the event identification information based on the event identification information of the event selected by the observer among the events occurring in the video represented by the video data stored in the video storage device. Transmitting to the display device a portion of the video data that is a portion of the video data and that corresponds to a period during which the event is occurring;
10. The monitoring support device according to claim 9, wherein the digest video processing unit deletes a portion of the video data corresponding to a period including a start time point to a finish time point of the video represented by the digest video data, from the video storage device. . The digest video data is stored in the history storage unit in association with the event identification information of the event, information identifying the portion of the digest video data corresponding to the period in which the event is occurring, and the event identification information. 11. The monitoring support apparatus according to claim 9, further comprising: a digest video storage unit that stores the monitoring support data in association with each other. A display device for displaying a video represented by the received video data;
The event of the event selected by the monitor among the event identification information identifying the event when the monitor selects an event whose period is determined based on the amount of change with time of the video represented by the video data The part of the video data that is the part of the video data stored in association with the event identification information based on the identification information and corresponds to the period in which the event is occurring is transmitted to the display device, and the event A monitoring support apparatus that stores monitoring support data including information indicating whether or not a portion of the video data corresponding to an occurrence period has been transmitted to the display device in association with event identification information of the event;
Monitoring support system provided with

Images