JP2019170180A - Pet moving image analyzer, pet moving image analysis system, pet moving image analysis method and program - Google Patents

Abstract

A pet moving picture analyzing apparatus, a pet moving picture analyzing system, a pet moving picture analyzing method, and a program are provided. A pet moving image analyzing apparatus includes: a moving image acquiring unit that acquires a moving image of a pet; and a plurality of frames that constitute the moving image acquired by the moving image acquiring unit. And the number of the pet frames extracted by the pet frame extracting unit 220 is greater than the number of non-pet frames that are frames excluding the pet frame among the plurality of frames. A summary moving image generation unit 27 that generates a summary moving image that is a moving image based on the moving image acquired by the moving image acquisition unit 20, and a summary moving image output unit that outputs the summary moving image generated by the summary moving image generation unit 27 290. [Selection] Figure 2

Description

  The present invention relates to a pet video analysis device, a pet video analysis system, a pet video analysis method, and a program.

  While the owner is away from home, it is desirable to watch the pets left at home. For example, watching the appearance of a pet itself is a pleasure for the owner, and it may be tempting to look at the appearance of his / her pet while on the go. In addition, for example, the owner may want to check whether the pet is eating properly, whether the pet is pranking, or that the pet is not abnormal while absent.

  In order to watch over the pets left at home, a technology is generally available in which a commercially available network camera is installed at the home and the owner checks the image using a smartphone or the like outside the home. In addition, products that detect the movement of a pet left at home using moving object detection using infrared rays and notify the owner at the timing when the camera shooting range has moved are generally available.

  As such a technique, for example, an interactive system for pets based on a wireless communication network is known (Patent Document 1). The pet interactive system described in Patent Document 1 includes a wireless camera, a pet toy, and a control terminal installed in a room. The wireless camera transmits the captured image to the control terminal. The pet toy includes a wireless communication module for communicating with the control terminal, an utterance module for reproducing the voice from the control terminal immediately received by the wireless communication module, and a pet command based on a control command issued by the control terminal. A control module for controlling the running of the toy is attached.

JP 2016-220671 A

  When a moving image in which a pet left at home is imaged on a terminal device via a wireless communication network, the moving image is required to have a high proportion of scenes in which the pet is imaged. Also, when a pet takes a specific action at a specific place, it is required to leave a notification or record that a moving image has been captured.

In the above-mentioned general distribution products and the interactive system for pets described in Patent Document 1, the owner user needs to search for a place where the pet appears in the video imaged using the camera, which is inconvenient. It is. When a camera that captures pets is equipped with a computing device that performs computations to detect locations where the pet is well reflected, high accuracy is required to detect locations where the pet is often reflected. It is required to have high performance. For this reason, the cost of the camera may be increased.
Thus, conventionally, it has not been possible to watch over a pet efficiently.

  The present invention has been made in view of the above points, and provides a pet video analysis device, a pet video analysis system, a pet video analysis method, and a program capable of efficiently watching a pet.

  The present invention has been made to solve the above problems, and one aspect of the present invention comprises a moving image acquisition unit that acquires a moving image in which a pet is imaged and the moving image acquired by the moving image acquisition unit. A pet frame extraction unit that extracts a pet frame that is a frame in which a pet is imaged from a plurality of frames, and the number of pet frames extracted by the pet frame extraction unit is the pet frame out of the plurality of frames. A summary video generation unit that generates a summary video that is larger than the number of non-pet frames that are excluded frames based on the video acquired by the video acquisition unit, and the summary video generated by the summary video generation unit Is a pet video analysis device including a summary video output unit for outputting.

  Further, according to one aspect of the present invention, in the pet moving image analysis apparatus, the pet frames that are continuous in the moving image acquired by the moving image acquiring unit are determined from the pet frames extracted by the pet frame extracting unit. A continuous pet frame determination unit, wherein the summary video generation unit reduces the number of continuous pet frames included in the scene video composed of the pet frames determined by the continuous pet frame determination unit to a predetermined number or less. The summary video is generated based on the video.

  According to another aspect of the present invention, in the pet moving image analysis apparatus, the pet frame corresponding to a thumbnail image is selected from the continuous pet frames included in the scene moving image.

  Further, according to one aspect of the present invention, in the pet moving image analysis apparatus described above, a storage control unit that stores the scene moving image in a storage unit, and a scene that outputs the scene moving image stored in the storage unit by the storage control unit A video output unit;

  Further, according to one aspect of the present invention, in the pet moving image analysis apparatus, the frame in which the pet frame determined by the continuous pet frame determination unit is capturing an event indicating a pre-registered behavior of the pet is captured. And an event determination unit that determines whether the event frame is.

  Further, according to one aspect of the present invention, in the pet moving image analysis apparatus, the pet frame determined by the event determination unit as the event frame, and the continuous pet frame determined by the continuous pet frame determination unit An event counting unit that counts the number of occurrences of the event based on the summary video, and the summary video output unit further includes the number of occurrences counted by the event counting unit in addition to the summary video generated by the summary video generation unit. Output.

  Further, according to one aspect of the present invention, in the pet video analysis device, the summary video generation unit may include the event determination unit when the video acquired by the video acquisition unit includes a plurality of types of events. Based on the determined event frame, the summary video is generated so that the plurality of types of events are included in the summary video.

  Further, according to one aspect of the present invention, in the pet moving image analysis apparatus described above, the event determination unit determines the type of the event based on a location captured by the pet frame extracted by the pet frame extraction unit.

  Further, according to one aspect of the present invention, in the pet video analysis apparatus, the pet determination unit that determines whether the pet imaged in the pet frame extracted by the pet frame extraction unit is a specific pet; And a pet determination result notifying unit for notifying the result determined by the pet determining unit.

  Moreover, one aspect of the present invention further includes an evaluation acquisition unit that acquires an evaluation obtained in response to notification of a result determined by the pet determination unit in the pet video analysis device, and the pet determination unit Determines whether the pet imaged in the moving image acquired by the moving image acquisition unit is a specific pet based on the evaluation acquired by the evaluation acquisition unit.

  Further, according to one aspect of the present invention, the above-described pet video analysis device, a photographing device that captures the video and supplies the pet video analysis device, and a terminal that acquires the summary video output by the pet video analysis device A pet video analysis system including the apparatus.

  One embodiment of the present invention is a frame in which a pet is imaged from a moving image acquisition procedure for acquiring a moving image in which a pet is imaged and a plurality of frames constituting the moving image acquired in the moving image acquisition procedure. A pet frame extraction procedure for extracting pet frames, and a moving image in which the number of pet frames extracted in the pet frame extraction procedure is larger than the number of non-pet frames that are frames excluding the pet frames among the plurality of frames. A summary video generation procedure for generating the summary video based on the video acquired in the video acquisition procedure, and a summary video output procedure for outputting the summary video generated in the summary video generation procedure This is a pet video analysis method.

  In one embodiment of the present invention, a moving image acquisition step of acquiring a moving image in which a pet is imaged by a computer, and a frame in which the pet is imaged from a plurality of frames constituting the moving image acquired in the moving image acquisition step. A pet frame extraction step for extracting pet frames, and the number of pet frames extracted in the pet frame extraction step is greater than the number of non-pet frames that are frames excluding the pet frame among the plurality of frames. A summary video generation step that generates a summary video that is a large number of videos based on the video acquired in the video acquisition step; a summary video output step that outputs the summary video generated in the summary video generation step; Is a program for executing

  According to the present invention, a pet can be watched efficiently.

It is a figure showing an example of a pet watching system concerning a 1st embodiment of the present invention. It is a figure which shows an example of a function structure of the pet moving image analyzer which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the analysis result of the imaging moving image which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the relationship between the event moving image and the summary moving image which concern on the 1st Embodiment of this invention. It is a figure which shows an example of the production | generation and output process of the event moving image which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the production | generation method of the event moving image which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the production | generation and output process of the summary moving image which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the production | generation method of the summary moving image which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the time series screen which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the album screen which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the active mass display screen which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of the report screen which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of a function structure of the pet moving image analyzer which concerns on the 2nd Embodiment of this invention. It is a figure which shows an example of the analysis result of the imaging moving image which concerns on the 2nd Embodiment of this invention. It is a figure which shows an example of the pet determination process which concerns on the 2nd Embodiment of this invention. It is a figure which shows an example of a function structure of the pet moving image analyzer which concerns on the 3rd Embodiment of this invention.

(First embodiment)
Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a diagram illustrating an example of a pet watching system S according to the present embodiment. The pet watching system S includes an imaging device 1, a pet video analysis device 2, and a terminal device 3.

  The imaging device 1 images a captured moving image MV that is a moving image in which the pet P is captured and supplies the captured moving image MV to the pet moving image analysis device 2. The imaging device 1 is a camera having a communication function, and is installed at the home of a user who is the owner of the pet P. The imaging device 1 images the pet P left in the home of the user who is the owner of the pet P. The imaging device 1 transmits the captured moving image MV to the pet moving image analysis device 2. The pet P is a domestic cat as an example. The pet P may be other than a domestic cat. The pet P may be, for example, a domestic dog or a domestic animal other than a cat or a dog.

  The place where the imaging device 1 is installed in the user's home is a place where the action of the pet P is expected to be taken and is determined by the user. The places where the action of the pet P is expected to be imaged are, for example, the vicinity of the pet P feeding place, the vicinity of the drinking fountain, and the vicinity of the toilet.

  The pet moving image analysis apparatus 2 is a server including a graphics processing unit (GPU) calculation unit that detects the pet P from the captured moving image MV with high accuracy. The pet moving image analysis device 2 analyzes the captured moving image MV and generates a summary moving image V after establishing encrypted communication of the moving image and sound with the imaging device 1. Here, the summary video V is a video in which the ratio of the scene where the pet is imaged is larger than the ratio of the scene where the pet is not imaged. The pet video analysis device 2 periodically notifies the terminal device 3 that the summary video V has been generated in one day.

  The pet video analysis device 2 detects the pet P from the captured video MV using machine learning. The pet moving image analyzer 2 performs machine learning using the learning data VD accumulated in advance. The pet moving image analysis apparatus 2 recognizes an object from the captured moving image MV using the result of machine learning, and further determines whether or not the recognized object is a pet P. Here, the machine learning is, for example, deep learning.

  In the pet watching system S, the captured moving image MV is analyzed not by the imaging device 1 but by the pet moving image analysis device 2 which is a cloud server for the position and movement of the pet P. As a result, the pet watching system S enables scalable and highly accurate analysis.

  The terminal device 3 acquires the summary video V output from the pet video analysis device 2. The terminal device 3 is owned by the user and receives a notification that the summary video V has been generated from the pet video analysis device 2. When there is a request for the summary video V from the terminal device 3, the pet video analysis device 2 outputs the requested summary video V to the terminal device 3.

  In the pet watching system S, only a part of the scene where the pet P is imaged is extracted from the 24-hour video MV, or a part of the scene where the pet P is imaged is connected and summarized for one day. It is possible to create a video.

  Further, in the pet watching system S, the user can specify in advance a place meaning some action by the presence of the pet P in the image captured by the imaging device 1. Thereby, in the pet watching system S, the place designated in advance and the movement of the pet P can be linked, the behavior of the pet P can be analyzed, and the behavior management of the pet P can be performed. Here, the place meaning some action due to the presence of the pet P is, for example, a food container, a drinking fountain, a toilet or the like.

  FIG. 2 is a diagram illustrating an example of a functional configuration of the pet video analysis apparatus 2 according to the present embodiment. The pet video analysis device 2 includes a video acquisition unit 20, a video storage unit 21, a scene determination unit 22, a storage unit 23, an event analysis unit 24, a thumbnail selection unit 25, a frame attribute storage unit 26, and a summary. A moving image generation unit 27, a thumbnail generation unit 28, and an output unit 29 are provided.

The moving image acquisition unit 20 acquires the captured moving image MV in which the pet P is captured from the imaging device 1. The moving image acquisition unit 20 is a communication module capable of encrypted communication.
The moving image storage unit 21 stores the captured moving image MV acquired by the moving image acquisition unit 20 while the pet moving image analysis apparatus 2 analyzes the captured moving image MV. The moving image storage unit 21 supplies the stored captured moving image MV to the scene determination unit 22, the event analysis unit 24, the thumbnail selection unit 25, and the summary moving image generation unit 27.

  The scene determination unit 22 determines a scene included in the captured moving image MV supplied from the moving image storage unit 21. Here, the scene determination unit 22 determines a scene based on the learning result 230 stored in the storage unit 23. A scene is a portion in which the content being captured in the captured moving image MV is grouped. The scene is, for example, a portion where the pet P is imaged or a portion where the pet P is not imaged.

  The scene where the pet P is imaged is, for example, a scene of drinking water at a drinking fountain, a scene of eating food in a food container, a scene of excretion in the toilet, a scene of moving in front of the camera, A scene that runs around the camera and appears to be reflected in the camera in a short time, a scene in which only the tail is reflected in the camera of the imaging device 1, a scene that is located far away from the camera and is shown in a small figure It is.

The scene determination unit 22 includes a pet frame extraction unit 220 and a continuous pet frame determination unit 221.
The pet frame extraction unit 220 extracts a pet frame from a plurality of frames constituting the captured moving image MV supplied from the moving image storage unit 21. A pet frame is a frame in which the pet P is imaged. On the other hand, a frame excluding a pet frame among frames constituting the captured moving image MV is called a non-pet frame.

  The continuous pet frame determination unit 221 determines a continuous pet frame in the captured moving image MV supplied from the moving image storage unit 21 from the pet frames extracted by the pet frame extraction unit 220. The continuous pet frame determined by the continuous pet frame determination unit 221 corresponds to a scene. A moving image composed of pet frames determined by the continuous pet frame determination unit 221 is referred to as a scene moving image CV.

  A continuous pet frame is a continuous pet frame in which the content is continuously viewed as a moving image. The continuous pet frames are, for example, a series of pet frames in which the pet frames are adjacent to each other in the captured moving image MV.

  Note that even if non-pet frames are not adjacent to each other in the captured video MV and there are non-pet frames between pet frames, if the content can be viewed continuously as a video, the pet frames and non-pet frames are combined. The scene movie CV may be used. For example, in a scene where the pet P runs around the camera and is not reflected in the camera in a short time, the pet frame and the non-pet frame may be combined into a scene video CV.

  In addition, even when the pet frame determined to be a continuous pet frame is a non-pet frame before and after the pet frame determined to be a continuous pet frame, if it is determined to be the same scene as the pet frame determined to be a continuous pet frame, A pet frame may be included in the scene video CV.

  The scene determination unit 22 supplies the processing results of the pet frame extraction unit 220 and the continuous pet frame determination unit 221 to the event analysis unit 24 and the frame attribute storage unit 26 as frame attribute information FP. The frame attribute information FP is the result of the processing of the scene determination unit 22, the event analysis unit 24, and the thumbnail selection unit 25 for the plurality of frames constituting the captured moving image MV and the plurality of frames constituting the captured moving image MV. Is the associated information.

Here, with reference to FIG. 3, processing of the pet frame extraction unit 220 and the continuous pet frame determination unit 221 will be described.
FIG. 3 is a diagram illustrating an example of an analysis result of the captured moving image MV according to the present embodiment. The captured moving image MV is composed of frames FN1 to FNn. In the example illustrated in FIG. 3, in the frames FN1 to FN8, the frame FN3, the frame FN5, and the frame FN6 are pet frames. In the frames FN1 to FN8, the frame FN1, the frame FN2, the frame FN4, the frame FN7, and the frame FN8 are non-pet frames.

  The pet frame extraction unit 220 determines whether each of the frames FN1 to FNn is a pet frame. The pet frame extraction unit 220 associates the determination results with the frame numbers of the frames FN1 to FNn and includes them in the frame attribute information FP.

The continuous pet frame determination unit 221 determines whether the frame FN3, the frame FN5, and the frame FN6 determined by the pet frame extraction unit 220 as pet frames are continuous pet frames. In the example illustrated in FIG. 3, it is determined that the frame FN5 and the frame FN6 among the frames FN3, FN5, and FN6 are continuous pet frames.
The continuous pet frame determination unit 221 associates the determination results for each frame number of the frames FN5 and FN6, and includes them in the frame attribute information FP.

Returning to FIG. 2, the description of the configuration of the pet video analyzing apparatus 2 will be continued.
The storage unit 23 stores a learning result 230 and event information 231.
The learning result 230 is a learning result by machine learning using the learning data VD. The learning result 230 includes, for example, the determination result of the position of the object imaged in the frame included in the moving image and the determination result of whether or not this object is a pet.

  In addition, the learning result 230 includes a determination result as to whether or not the content captured in the frame included in the moving image has a collection. The learning result 230 includes an activity amount value corresponding to the movement of the pet imaged in the frame included in the moving image. Further, the learning result 230 includes a determination result as to which of the frames included in the moving image is suitable for the thumbnail.

  Event information is information in which an event type is associated with location information. The location information is information indicating an area in the moving image. The area in the moving image is assigned to each specific place in the home where the pet P is left.

  For example, an event “drinking” is associated with information indicating an area near drinking. Information indicating an area near the bait is associated with the “rice” event. Information indicating an area near the toilet is associated with the “toilet” event. The event “moving in front of the camera” is associated with information indicating an area far from the water drinking, feeding, and toilet.

  Note that the information indicating the region is, for example, information representing the region on the two-dimensional plane by two-dimensional coordinates. The information indicating the region may be information representing the region in the three-dimensional space by three-dimensional coordinates.

  The event analysis unit 24 analyzes an event included in the captured moving image MV supplied from the moving image storage unit 21 based on the frame attribute information FP supplied from the scene determination unit 22. Here, the event analysis unit 24 analyzes the event based on the learning result 230 and the event information 231 stored in the storage unit 23.

Here, the event is an action of the pet P indicated by the event information 231. The event is stored in advance as part of the event information 231. That is, the event indicates an action of the pet P registered in advance.
The events include, for example, “drinking” which is the action of drinking water at the drinking fountain, “rice” which is the action of eating the food in the container, “toilet” which is the action of excretion in the toilet, and in front of the camera. Is "moving in front of the camera".

The event analysis unit 24 includes an event determination unit 240, an event counting unit 241, and an activity amount measuring unit 242.
The event determination unit 240 determines an event included in the captured moving image MV. Here, the event determination unit 240 determines whether or not the pet frame determined by the continuous pet frame determination unit 221 is an event frame that is a frame in which an event is captured. That is, the event determination unit 240 determines whether an event is included in the pet frame included in the scene video CV. A scene video CV in which an event is included in a pet frame is referred to as a pre-summary event video PEV.

  The event counting unit 241 counts the number of occurrences of events included in the captured moving image MV. Here, the event counting unit 241 counts based on the event frame determined by the event determination unit 240 and the continuous pet frame determined by the continuous pet frame determination unit 221.

The activity amount measuring unit 242 measures the amount of exercise of the pet P accompanying the event included in the captured moving image MV. Here, the activity amount measuring unit 242 performs measurement based on the captured moving image MV supplied by the moving image storage unit 21, the frame attribute information FP supplied by the event determination unit 240, and the learning result 230 stored in the storage unit 23.
The activity amount measuring unit 242 generates activity amount information A based on the measurement result. Here, the activity amount information A is information in which the frame numbers of a plurality of frames constituting the captured moving image MV and the activity amount values measured by the activity amount measuring unit 242 are associated with each other.

  The event analysis unit 24 supplies the processing result of the event determination unit 240 to the thumbnail selection unit 25 and the frame attribute storage unit 26 as frame attribute information FP.

The thumbnail selection unit 25 selects a thumbnail of the summary video V. Here, the thumbnail selection unit 25 selects based on the captured moving image MV supplied from the moving image storage unit 21, the frame attribute information FP supplied from the event analysis unit 24, and the learning result 230 stored in the storage unit 23. The thumbnail selection unit 25 includes the selection result in the frame attribute information FP and supplies it to the frame attribute storage unit 26.
A thumbnail is an image that is reduced and displayed in order to enhance visibility when the summary video V is displayed. The thumbnail is selected from the pet frames constituting the summary video V.

Here, with reference to FIG. 3 again, processing of the event determination unit 240, the event counting unit 241, the thumbnail selection unit 25, and the activity amount measurement unit 242 will be described.
In the example illustrated in FIG. 3, the event determination unit 240 indicates that the action of the pet P imaged in the frame FN5 and the frame FN6 determined by the continuous pet frame determination unit 221 as the continuous pet frame is “in front of the camera”. It is determined that the event is “moving”. The event determination unit 240 includes the determination result in the frame attribute information FP.
The event counting unit 241 counts the number of occurrences of the event determined by the event determining unit 240 in the frames FN1 to FN8 as one.

  The thumbnail selection unit 25 selects a frame corresponding to the thumbnail from the frame FN5 and the frame FN6 determined by the event determination unit 240 as event frames. The thumbnail selection unit 25 includes the selection result in the frame attribute information FP.

  The activity amount measuring unit 242 measures the activity amount of the captured pet P for each of the frames FN5 and FN6 that the event determination unit 240 determines to be an event frame. In the example illustrated in FIG. 3, the activity amount measuring unit 242 associates “3” as the activity amount value with respect to each of the frames FN5 and FN6 and sets the activity amount information A.

Returning to FIG. 2, the description of the configuration of the pet video analyzing apparatus 2 will be continued.
The frame attribute storage unit 26 stores frame attribute information FP supplied from the scene determination unit 22, event analysis unit 24, and thumbnail selection unit 25. The frame attribute storage unit 26 supplies the stored frame attribute information FP to the summary video generation unit 27, the thumbnail generation unit 28, and the output unit 29.
The frame attribute storage unit 26 stores activity amount information A supplied from the event analysis unit 24. The frame attribute storage unit 26 supplies the stored activity amount information A to the summary video generation unit 27.

The summary video generation unit 27 generates a summary video V. Here, the summary video V is a video in which the number of pet frames is larger than the number of non-pet frames. The summary video V includes an event video EV generated for each event and a summary video SV generated based on the event video EV.
The summary video generation unit 27 includes an event video generation unit 270 and a summary video generation unit 271.

The event moving image generating unit 270 generates an event moving image EV based on the captured moving image MV supplied from the moving image storage unit 21 and the frame attribute information FP supplied from the frame attribute storage unit 26. Further, the event moving image generating unit 270 calculates an activity amount for the generated event moving image EV based on the activity amount information A supplied from the frame attribute storage unit 26.
The summary video generation unit 271 generates a summary video SV based on the event video EV generated by the event video generation unit 270.

Here, with reference to FIG. 4, the relationship between the event video EV and the summary video SV will be described.
FIG. 4 is a diagram illustrating an example of the relationship between the event video EV and the summary video SV according to the present embodiment. The pet moving image analysis device 2 analyzes the captured moving image MV acquired from the imaging device 1 in a predetermined cycle. Here, the predetermined cycle is, for example, one hour.

When an event is included in the one-hour captured moving image MV, the event moving image generation unit 270 generates an event moving image EV. In the example illustrated in FIG. 4, at time t1, the event moving image generation unit 270-1 performs a plurality of events # 1-1, event # 1-2, and event # 1-3 included in the one-hour captured moving image MV. An event video EV1-1, an event video EV1-2, and an event video EV1-3 are generated. Here, the event moving image generating unit 270-1 is the event moving image generating unit 270 of FIG.
The generated event video EV1-1, event video EV1-2, and event video EV1-3 are output to the terminal device 3 by the summary video output unit 290 at time t1.

In the example illustrated in FIG. 4, at time t2, the event moving image generation unit 270-2 performs the event moving image EV2 for each of the plurality of events # 2-1 and event # 2-2 included in the next one-hour captured moving image MV. −1 and event video EV2-2. Here, the event moving image generating unit 270-2 is the event moving image generating unit 270 of FIG.
The generated event video EV2-1 and event video EV2-2 are output to the terminal device 3 by the summary video output unit 290 at time t2.

The summary video generator 271 generates a plurality of generated event videos EV1-1, event videos EV1-2, and event videos EV1-3 from the event videos EV2-1 and event videos EV2-2 in a predetermined cycle. Is generated. Here, the predetermined cycle is, for example, 8 hours.
The generated summary video SV1 is output to the terminal device 3 by the summary video output unit 290 in schedule # 1.

  In the example illustrated in FIG. 4, the period in which the pet moving image analysis apparatus 2 analyzes the captured moving image MV is 1 hour, but is not limited thereto. The period in which the pet moving image analysis device 2 analyzes the captured moving image MV may be short enough that the pet moving image analysis device 2 can sequentially transmit the event moving image EV to the terminal device 3 when an event occurs. When the event occurs, the period that is short enough to allow the pet video analysis device 2 to sequentially transmit the event video EV to the terminal device 3 is, for example, 1 minute.

In addition, the pet video analysis device 2 may output the event video EV and the summary video SV to the terminal device 3 in the same cycle. When the event video EV and the summary video SV are output in the same cycle, the cycle in which the pet video analysis device 2 analyzes the captured video MV is, for example, the cycle in which the summary video generation unit 271 generates the summary video SV (for example, , 8 hours).
Moreover, the period when the pet moving image analysis apparatus 2 analyzes the captured moving image MV may be, for example, 24 hours.

Returning to FIG. 2, the description of the configuration of the pet video analyzing apparatus 2 will be continued.
The thumbnail generation unit 28 selects the thumbnail SN based on the captured video MV supplied from the video storage unit 21, the frame attribute information FP supplied from the frame attribute storage unit 26, and the summary video SV supplied from the summary video generation unit 27. Generate.

The output unit 29 outputs the summary video V to the terminal device 3. The output unit 29 includes a summary video output unit 290 and a thumbnail output unit 291.
The summary video output unit 290 outputs the summary video V generated by the summary video generation unit 27 to the terminal device 3.

  The thumbnail output unit 291 outputs the thumbnail SN generated by the thumbnail generation unit 28 to the terminal device 3.

(Generation and output of event video)
Next, the generation and output processing of the event moving image EV will be described with reference to FIGS.
FIG. 5 is a diagram illustrating an example of generation and output processing of the event moving image EV according to the present embodiment. The process shown in FIG. 5 is executed in a predetermined cycle (for example, 1 hour).

Step S100: The moving image acquisition unit 20 acquires the captured moving image MV captured by the imaging device 1. The moving image acquisition unit 20 supplies the acquired captured moving image MV to the moving image storage unit 21.
The moving image storage unit 21 supplies the captured moving image MV supplied from the moving image acquisition unit 20 to the scene determination unit 22, the event analysis unit 24, the thumbnail selection unit 25, the summary moving image generation unit 27, and the thumbnail generation unit 28. To do.

Step S102: The pet frame extraction unit 220 extracts a pet frame from a plurality of frames constituting the captured moving image MV supplied by the moving image storage unit 21. Here, the pet frame extraction unit 220 extracts a pet frame using machine learning based on the learning result 230 stored in the storage unit 23. The pet frame extraction unit 220 determines that the pet P is captured in this frame when a cat image is captured in the frame.

  The pet frame extraction unit 220 extracts a pet frame by determining whether each of a plurality of frames constituting the captured moving image MV is a pet frame by using machine learning based on the learning result 230. Based on the learning result 230, the pet frame extraction unit 220 determines the position of the object for each of a plurality of frames constituting the captured moving image MV. The pet frame extraction unit 220 determines whether or not the determined object is the pet P based on the learning result 230.

The pet frame extraction unit 220 associates the determination results with the frame numbers of a plurality of frames constituting the captured moving image MV, and includes them in the frame attribute information FP.
The pet frame extraction unit 220 supplies the frame attribute information FP including the determination result to the continuous pet frame determination unit 221.

  Here, the captured moving image MV supplied from the moving image storage unit 21 is a moving image acquired from the imaging device 1 by the moving image acquisition unit 20. Therefore, the pet frame extraction unit 220 extracts a pet frame from a plurality of frames constituting the captured moving image MV acquired by the moving image acquisition unit 20.

Step S104: The continuous pet frame determination unit 221 determines continuous pet frames in the captured moving image MV supplied from the moving image storage unit 21, based on the frame attribute information FP supplied from the pet frame extraction unit 220. That is, the continuous pet frame determination unit 221 determines a scene video CV included in the captured video MV.
Here, the continuous pet frame determination unit 221 determines the pet frame extracted by the pet frame extraction unit 220 based on the frame attribute information FP. The continuous pet frame determination unit 221 determines a pet frame that is continuous in the captured moving image MV from the pet frames extracted by the pet frame extraction unit 220.

  The continuous pet frame determination unit 221 determines continuous pet frames using machine learning based on the learning result 230 stored in the storage unit 23. The continuous pet frame determination unit 221 may determine whether a plurality of frames including a non-pet frame in addition to a pet frame are continuous.

  The continuous pet frame determination unit 221 associates a determination result with each frame number of a plurality of frames constituting the captured moving image MV, and includes the determination result in the frame attribute information FP. The continuous pet frame determination unit 221 supplies the frame attribute information FP including the determination result to the event analysis unit 24 and the frame attribute storage unit 26.

  Here, the captured moving image MV supplied from the moving image storage unit 21 is a moving image acquired from the imaging device 1 by the moving image acquisition unit 20. Therefore, the continuous pet frame determination unit 221 determines a continuous pet frame in the captured moving image MV acquired by the moving image acquisition unit 20 from the pet frames extracted by the pet frame extraction unit 220.

Step S106: The event determination unit 240 determines an event included in the captured moving image MV supplied from the moving image storage unit 21 based on the frame attribute information FP supplied from the continuous pet frame determination unit 221. The event determination unit 240 determines an event for each pet frame constituting the scene video CV determined by the continuous pet frame determination unit 221.
The event determination unit 240 includes the determination result in the frame attribute information FP and supplies the frame attribute information FP to the event counting unit 241.

  Here, the event determination unit 240 performs determination using machine learning based on the learning result 230 stored in the storage unit 23 and event information 231 stored in the storage unit 23. The event determination unit 240 uses machine learning based on the learning result 230 to determine whether the position of the pet P is included in the area indicated by the location information associated with the event type indicated by the event information 231. . The event determination unit 240 is an event in which the event type associated with the location information indicating the area including the position of the pet P in the event information 231 is captured in the pet frame that is the event determination target. judge.

For example, when the event determination unit 240 determines that the position of the pet P is included in an area near drinking, the event determination unit 240 determines that the type of event included in the pet frame is “drinking”.
Therefore, the event determination unit 240 determines the type of the event based on the location imaged in the pet frame extracted by the pet frame extraction unit 220.

Note that the event determination unit 240 may determine the event type based on the movement of the pet P imaged in the pet frame extracted by the pet frame extraction unit 220 instead of determining the event type based on the location.
When determining the type of event based on the movement of the pet P, the event determination unit 240 analyzes the movement of the pet P imaged in the pet frame by using machine learning based on the learning result 230 stored in the storage unit 23. To do. When the event determination unit 240 determines the event type based on the movement of the pet P, the learning result 230 may include a result of learning the correspondence between the pet movement and the event type.

Step S108: The event counting unit 241 counts the number of occurrences of events included in the captured moving image MV supplied by the moving image storage unit 21 based on the frame attribute information FP supplied by the event determination unit 240. Here, the event counting unit 241 counts the number of occurrences of the event based on the pet frame determined by the event determining unit 240 as the event frame and the continuous pet frame determined by the continuous pet frame determining unit 221. The event counting unit 241 counts the number of event occurrences for each event type.
The event counting unit 241 includes the counted number of event occurrences in the frame attribute information FP. The event counting unit 241 supplies frame attribute information FP including the number of occurrences to the activity amount measuring unit 242.

Step S110: The activity amount measuring unit 242 measures the amount of exercise of the pet P accompanying the event included in the captured moving image MV supplied by the moving image storage unit 21, based on the frame attribute information FP supplied by the event counting unit 241. Here, the activity amount measurement unit 242 performs measurement based on the learning result 230 stored in the storage unit 23.

The activity amount measurement unit 242 performs activity for each event frame determined by the event determination unit 240 based on the activity amount value corresponding to the movement of the captured pet included in the learning result 230 stored in the storage unit 23. Measure the amount.
The activity amount measuring unit 242 includes the measurement result in the frame attribute information FP. The activity amount measurement unit 242 supplies the frame attribute information FP including the measurement result to the thumbnail selection unit 25 and the frame attribute storage unit 26.

  It should be noted that the activity amount measuring unit 242 measures the activity amount, instead of the activity amount value corresponding to the movement of the captured pet included in the learning result 230, the activity amount assigned in advance for each event type. The value of may be used. When the activity amount measurement unit 242 uses the activity amount value assigned in advance for each event type for the activity amount measurement, the activity amount value assigned in advance for each event type is included in the event information 231. It's okay.

Step S112: The thumbnail selection unit 25 selects an event frame corresponding to the thumbnail from the event frames for each event determined by the event determination unit 240. Here, the event frame is a continuous pet frame included in the scene video CV. Therefore, the thumbnail selection unit 25 selects the pet frame corresponding to the thumbnail image from the continuous pet frames included in the scene moving image.

  Here, the thumbnail selection unit 25 selects the summary video V based on the captured video MV supplied by the video storage unit 21, the frame attribute information FP supplied by the event analysis unit 24, and the learning result 230 stored in the storage unit 23. Select a thumbnail. Based on the learning result 230, the thumbnail selection unit 25 determines which of the event frames is suitable for the thumbnail for each event determined by the event determination unit 240.

  The thumbnail selection unit 25 includes the selection result in the frame attribute information FP. The thumbnail selection unit 25 supplies the frame attribute information FP including the selection result to the frame attribute storage unit 26.

  Note that the thumbnail selection unit 25 may select a thumbnail based on a predetermined rule instead of selecting a thumbnail based on the learning result 230. The predetermined rule is to select, for example, an event frame having the earliest time order for each event as a thumbnail. In another example, the predetermined rule is to select, as a thumbnail, an event frame having the latest time order for each event. In another example, the predetermined rule is to select an event frame having the largest proportion of the image of the pet P in the event frame image as a thumbnail.

Step S114: The frame attribute storage unit 26 executes processing for preparation for event image generation. The frame attribute storage unit 26 stores frame attribute information FP supplied from the scene determination unit 22, event analysis unit 24, and thumbnail selection unit 25. The frame attribute storage unit 26 supplies the stored frame attribute information FP to the summary video generation unit 27 and the thumbnail generation unit 28. In addition, the frame attribute storage unit 26 supplies the stored frame attribute information FP to the output unit 29.
Further, the frame attribute storage unit 26 supplies the activity amount information A supplied from the event analysis unit 24 to the summary moving image generation unit 27.

Step S116: Based on the frame attribute information FP supplied from the frame attribute storage unit 26, the event video generation unit 270 determines whether the number of occurrences of the event included in the captured video MV supplied from the video storage unit 21 is 1 or more. Determine whether.

  When the event moving image generating unit 270 determines that the number of occurrences of the event included in the captured moving image MV is 1 or more (step S116; YES), the event moving image generating unit 270 executes the process of step S118. On the other hand, if the event moving image generation unit 270 determines that the number of occurrences of the event included in the captured moving image MV is not 1 or more (step S116; NO), the process ends.

Step S118: The event moving image generating unit 270 repeats the process of generating the event moving image EV for each event.

Step S120: The event moving image generating unit 270 extracts the pre-summary event moving image PEV. Here, the event moving image generation unit 270 extracts an event frame for one event from the captured moving image MV supplied from the moving image storage unit 21 based on the frame attribute information FP supplied from the frame attribute storage unit 26. The event moving image generation unit 270 sets a moving image including event frames extracted for one event as a pre-summary event moving image PEV.

Step S122: The event moving image generating unit 270 determines whether or not the extracted time length of the pre-summary event moving image PEV is equal to or longer than a predetermined time length. Here, the predetermined time is, for example, 3 seconds.

  When determining that the length of time of the pre-summarization event video PEV is equal to or longer than the predetermined time (step S122; YES), the event video generation unit 270 executes the process of step S124. On the other hand, when determining that the length of time of the pre-summary event video PEV is not equal to or longer than the predetermined time (step S122; NO), the event video generation unit 270 ends the process.

Step S124: The event moving image generating unit 270 generates an event moving image EV. The event moving image generating unit 270 supplies the generated event moving image EV to the output unit 29.
Here, with reference to FIG. 6, a method of generating the event moving image EV by the event moving image generation unit 270 will be described.

  FIG. 6 is a diagram illustrating an example of a method for generating the event moving image EV according to the present embodiment. In the example illustrated in FIG. 6, the event moving image generating unit 270 generates an event moving image EV from the pre-summary event moving image PEV1. The pre-summary event moving image PEV1 is a moving image captured of an event with the pet P from time t10 to time t13. From time t10 to time t13, that is, the length of time of the pre-summary event video PEV1 is 3 seconds or more.

  When the time length of the pre-summary event video PEV is longer than 60 seconds, the event video generation unit 270 reduces the time length of the pre-summary event video PEV1 within 60 seconds. Here, the event video generation unit 270 shortens the time length of the pre-summary event video PEV1 by deleting the start portion and the end portion of the pre-summary event video PEV1 by a predetermined length.

Here, the event moving image generation unit 270 reduces the length of the pre-summary event moving image PEV so that the number of pet frames is larger than the number of non-pet frames. If the continuous pet frame determination unit 221 does not include a non-pet frame in the scene video CV, the non-pet frame is not included in the pre-summary event video PEV.
On the other hand, when the continuous pet frame determination unit 221 includes a non-pet frame in the scene video CV, the event video generation unit 270 causes the pre-summary event video PEV so that the number of pet frames is larger than the number of non-pet frames. Reduce the length of time.

In the example shown in FIG. 6, the partial moving image P1 is selected as the first portion, and the partial moving image P3 is selected as the end portion. The event moving image generating unit 270 deletes the partial moving image P1 and the partial moving image P3 from the pre-summary event moving image PEV to obtain a partial moving image P2. The partial moving image P2 is a moving image captured for an event with the pet P from time t11 to time t12. The length of time of the partial video P2 is the video length L11. Here, the moving image length L11 is 60 seconds.
The event moving image generating unit 270 sets the partial moving image P2 as the event moving image EV.

  As described above, the event moving image generation unit 270 is a non-pet in which the number of pet frames extracted by the pet frame extraction unit 220 is a frame excluding the pet frame among a plurality of frames constituting the moving image acquired by the moving image acquisition unit 20. An event video EV that is a video that is larger than the number of frames is generated based on the captured video MV acquired by the video acquisition unit 20.

  Here, the event moving image generation unit 270 generates the event moving image EV by setting the time length of the scene moving image CV to a predetermined length or less. Therefore, the event moving image generating unit 270 reduces the number of continuous pet frames included in the scene moving image CV (pre-summary event moving image PEV) composed of the pet frames determined by the continuous pet frame determining unit 221 to a predetermined number or less. An event video EV is generated based on the video (partial video P2).

  Note that the event moving image generation unit 270 may delete the first part of the pre-summary event moving image PEV and shorten it within 60 seconds. The event video generation unit 270 may delete the end part of the pre-summary event video PEV and shorten it within 60 seconds. The event moving image generation unit 270 may delete the middle part of the pre-summary event moving image PEV and shorten it within 60 seconds.

  In addition, the event moving image generating unit 270 may select which part of the pre-summary event moving image PEV is to be deleted and shortened within 60 seconds based on the learning result 230 stored in the storage unit 23. When the event video generation unit 270 selects, based on the learning result 230, which part of the pre-summary event video PEV is to be deleted and shortened within 60 seconds, the pet is well imaged in the learning result 230 The determination result of the moving image may be included.

  The event video generation unit 270 calculates an activity amount for the generated event video EV. Here, the amount of activity with respect to the event video EV is an amount obtained by counting the amount of activity measured for each pet frame included in the event video EV at a predetermined time interval. The predetermined time interval is, for example, 10 seconds.

Returning to FIG. 5, the description of the generation and output processing of the event moving image EV will be continued.
Step S126: The thumbnail generation unit 28 generates a thumbnail SN based on the captured moving image MV supplied from the moving image storage unit 21 and the frame attribute information FP supplied from the frame attribute storage unit 26. The thumbnail generation unit 28 supplies the generated thumbnail SN to the output unit 29.

  If the event frame selected by the thumbnail selection unit 25 is included in the portion deleted by the event video generation unit 270 in the process of shortening the pre-summary event video PEV to the event video EV, the thumbnail selection unit 25 selects the event video You may select again the flame | frame corresponding to a thumbnail from the several event frame which comprises the event moving image EV which the production | generation part 270 produced | generated. In this case, the thumbnail selection unit 25 updates the frame attribute information FP based on the selection result, and supplies it again to the frame attribute storage unit 26.

Step S128: The summary video output unit 290 outputs the event video EV supplied from the event video generation unit 270 to the terminal device 3. The thumbnail output unit 291 outputs the thumbnail SN corresponding to the event video EV supplied from the thumbnail generation unit 28 to the terminal device 3.

  In addition, the summary video output unit 290 outputs the number of event occurrences indicated by the frame attribute information FP supplied from the frame attribute storage unit 26 to the terminal device 3. Therefore, the summary video output unit 290 further outputs the number of event occurrences counted by the event counting unit 241 in addition to the summary video (event video EV) generated by the summary video generation unit 27.

The summary video output unit 290 outputs the amount of activity for the event video EV calculated by the event video generation unit 270 to the terminal device 3.
In addition to the event video EV, the summary video output unit 290 transmits information indicating the start time of the event included in the event video EV to the terminal device 3.
In addition to the event video EV, the summary video output unit 290 transmits information indicating the type of event included in the event video EV to the terminal device 3.

Step S130: The event moving image generating unit 270 ends the process of generating the event moving image EV for each event.

(Generation and output of summary video)
Next, generation and output processing of the summary video SV will be described with reference to FIGS.
FIG. 7 is a diagram illustrating an example of generation and output processing of the summary video SV1 according to the present embodiment. The process shown in FIG. 7 is executed in a predetermined cycle in which the summary video SV is generated.

Step S200: The summary video generation unit 271 acquires the event video EV generated by the event video generation unit 270. Here, the event video EV generated by the event video generation unit 270 is an event video EV generated by the event video generation unit 270 since the summary video generation unit 271 previously generated the summary video SV. When there are a plurality of event videos EV generated by the event video generator 270, the summary video generator 271 acquires a plurality of event videos EV.

Step S202: The summary video generation unit 271 generates a summary video SV based on the acquired event video EV.

Here, with reference to FIG. 8, a method of generating the summary video SV1 will be described.
FIG. 8 is a diagram illustrating an example of a method for generating the summary video SV1 according to the present embodiment. In FIG. 8A, the event moving images EV1 to PEV18 generated by the event moving image generating unit 270 are shown in time series. The time length of the event video EV1, the time length of the event video EV2, and the time length of the event video EV3 are each a video length L11 (60 seconds). The time length of the event video EV4 is the video length L12 (for example, 40 seconds).
A non-event video NEV1, a non-event video NEV2,... Exist between each of the event videos EV1 to PEV18. The non-event video NEV1, the non-event video NEV2,... Are scene video CVs that have not been determined as event video EVs.

  The event videos EV1 to PEV18 include four types of events. The event video EV1 and the event video EV10 include an event “drinking”. The event movie EV4 and the event movie EV12 include an event of “rice”. The event video EV5 and the event video EV17 include a “toilet” event. The remaining event video EV2, event video EV3, event video EV6 to PEV9, event video EV11, event video EV13 to PEV16, and event video EV18 include an event “moving in front of the camera”.

The summary video generation unit 271 generates a summary video SV having a video length L31 based on the event videos EV1 to PEV18. Here, the moving image length L31 is, for example, 60 seconds. The summary video generation unit 271 selects an event video EV for generating a summary video SV from among the event videos EV1 to PEV18. The summary video generation unit 271 shortens the selected event video EV to be equal to or less than a predetermined time length, and generates a 60-second summary video SV. Here, the predetermined length is, for example, 7 seconds.
Therefore, in the example illustrated in FIG. 8, the summary moving image generation unit 271 selects at least nine event moving images EV from the event moving images EV1 to PEV18.

  However, if the captured video MV acquired by the video acquisition unit 20 includes multiple types of events, the summary video generation unit 271 summarizes the multiple types of events based on the event frame determined by the event determination unit 240. A summary video SV is generated so as to be included in the video SV.

  In the example shown in FIG. 8B, the summary video generation unit 271 includes an event video EV1, an event video EV2, an event video EV4, an event video EV5, an event video EV9, an event video EV10, an event video EV12, an event video EV16, and The event video EV18 is selected to generate a summary video SV. Accordingly, the summary video SV includes a plurality of types of events such as “drinking”, “moving in front of the camera”, “rice”, and “toilet”.

  As shown in FIG. 8C, the summary video generation unit 271 generates a shortened event video SEV1 having a video length L21 by shortening the selected event video EV1. Here, the moving image length L21 is 7 seconds or less. Similarly, the summary video generation unit 271 shortens the remaining selected event video EV2, event video EV4,... To a length of 7 seconds or less, and generates a shortened event video SEV2, a shortened event video SEV4,. . Here, the summary moving image generating unit 271 generates the shortened event moving image SEV1, the shortened event moving image SEV2,.

  The summary video generation unit 271 includes the generated short event video SEV1, the short event video SEV2, the short event video SEV4, the short event video SEV5, the short event video SEV9, the short event video SEV10, the short event video SEV12, the short event video SEV16, and The shortened event video SEV18 is concatenated in chronological order to obtain a summary video SV1.

  Since the summary video SV1 is generated based on the event video EV, the summary video SV1 has more pet frames than non-pet frames. Therefore, the summary moving image generation unit 271 is a non-pet frame in which the number of pet frames extracted by the pet frame extraction unit 220 is a frame excluding the pet frame among a plurality of frames constituting the moving image acquired by the moving image acquisition unit 20. A summary video SV, which is more than the number of videos, is generated based on the captured video MV acquired by the video acquisition unit 20.

  Here, the summary moving image generation unit 271 generates the summary moving image SV by setting the time length of the event moving image EV to a predetermined length or less. Therefore, the summary video generation unit 271 sets the number of continuous pet frames included in the scene video CV (pre-summary event video PEV) composed of the pet frames determined by the continuous pet frame determination unit 221 to a predetermined number or less. A summary video SV is generated based on the video (short event video SEV1, short event video SEV2,...).

Returning to FIG. 7, the description of the generation and output processing of the summary video SV will be continued.
Step S204: The thumbnail generation unit 28 generates the thumbnail SN of the summary video SV generated by the summary video generation unit 271. Here, the thumbnail generation unit 28 uses the captured image MV supplied from the moving image storage unit 21 and the frame attribute information FP supplied from the frame attribute storage unit 26 for each of the event SNs included in the imaged moving image MV. Is generated. The thumbnail generation unit 28 selects the thumbnail SN of the summary video SV from the thumbnail SNs corresponding to the event video EV selected by the summary video generation unit 271 to generate the summary video SV among the generated thumbnail SNs.

For example, the thumbnail generator 28 selects the thumbnail SN of the summary video SV from the event video EV including the most types of events included in the event video EV selected by the summary video generator 271 to generate the summary video SV. Select.
In another example, the thumbnail generation unit 28, for example, from the event video EV including the longest type of event included in the event video EV selected by the summary video generation unit 271 to generate the summary video SV. Select the thumbnail SN of the video SV.

  In yet another example, the thumbnail generation unit 28 selects the thumbnail SN of the latest time-series event video EV among the event videos EV selected by the summary video generation unit 271 to generate the summary video SV. May be selected.

Step S206: The summary video output unit 290 outputs the summary video SV supplied from the summary video generation unit 271 to the terminal device 3. The thumbnail output unit 291 outputs the thumbnail SN corresponding to the summary video SV supplied from the thumbnail generation unit 28 to the terminal device 3.

In the process of step S128 of FIG. 5 and the process of step S206 of FIG. 7, the summary video output unit 290 outputs the summary video V generated on the terminal device 3 when the summary video V is generated. Although explained, it is not limited to this.
The summary video output unit 290 may first output the thumbnail SN corresponding to the summary video V to the terminal device 3 and output the summary video V when a request from the terminal device 3 is acquired.

(Output result)
With reference to FIGS. 9-12, the example of the screen of the terminal device 3 which is an output result of the pet moving image analyzer 2 is demonstrated. Examples of the screen of the terminal device 3 include a time series screen G1, an album screen G2, an activity amount display screen G3, and a report screen G4.

  FIG. 9 is a diagram illustrating an example of a time-series screen G1 according to the present embodiment. The time series screen G1 is a screen for notifying that an event has occurred, arranging the event moving images EV in the order of the time series in which the event occurred. Further, the time series screen G1 is notified that the summary video SV has been generated three times a day.

  In the time series screen G1 shown in FIG. 9, it is notified that the event moving image EV1 and the event moving image EV2 have been generated. On the time series screen G1, the event video EV1 and the event video EV2 are displayed side by side in the order of the time series in which the events occurred.

  On the time series screen G1, a time stamp T1, an icon A1, and a message M1 are displayed for the event video EV1. On the time series screen G1, a time stamp T2, an icon A2, and a message M2 are displayed for the event video EV2.

The time stamp T1 indicates the start time of the event included in the event moving image EV1. The icon A1 indicates the type of event included in the event video EV1 along with the distinction between the event video EV and the summary video SV. The message M1 is a standard text corresponding to the type of event included in the event video EV1.
The same applies to the time stamp T2, the icon A2, and the message M2.

FIG. 10 is a diagram showing an example of the album screen G2 according to the present embodiment. The album screen G2 is a screen for browsing the album AL. Here, the album AL is an accumulation of event videos EV and summary videos SV generated in the past.
On the album screen G2, thumbnails SN of each of the event video EV and the summary video SV and calendar information are displayed. The thumbnail SN1 is an example of the thumbnail SN of the event video EV. The calendar image C1 is an example of calendar information.

FIG. 11 is a diagram illustrating an example of the activity amount display screen G3 according to the present embodiment. The activity amount display screen G3 is a screen for confirming the activity amount totaled for each event video EV. On the activity amount display screen G3, a thumbnail SN of the event video EV and an activity amount graph AG indicating the daily activity amount are displayed.
The thumbnails SN are displayed side by side in the order of time series in which events have occurred. The thumbnail SN2 is an example of the thumbnail SN of the event video EV.

  FIG. 12 is a diagram illustrating an example of a report screen G4 according to the present embodiment. The report screen G4 is a screen for displaying the number of times each day of “drinking”, “rice”, and “toilet” among events in a graph. In the report screen G4, a drinking frequency graph CG1, a rice frequency graph CG2, and a toilet frequency graph CG3 are displayed.

As described above, the pet video analysis apparatus 2 according to the present embodiment includes the video acquisition unit 20, the pet frame extraction unit 220, the summary video generation unit 27, and the summary video output unit 290.
The moving image acquisition unit 20 acquires a moving image (captured moving image MV) in which the pet P is imaged.
The pet frame extraction unit 220 extracts a pet frame, which is a frame in which the pet P is captured, from a plurality of frames constituting the moving image (captured moving image MV) acquired by the moving image acquisition unit 20.
In the summary video generation unit 27, the number of pet frames extracted by the pet frame extraction unit 220 is a frame excluding the pet frame from among a plurality of frames constituting the video (captured video MV) acquired by the video acquisition unit 20. A summary video V that is a video larger than the number of pet frames is generated based on the video (captured video MV) acquired by the video acquisition unit 20.

  With this configuration, the pet video analyzing apparatus 2 according to the present embodiment can output a video in which the ratio of the scene where the pet P is captured is larger than the ratio of the scene where the pet P is not captured. You can watch efficiently.

The pet video analysis apparatus 2 according to the present embodiment further includes a continuous pet frame determination unit 221.
The continuous pet frame determination unit 221 determines a continuous pet frame in the moving image (captured moving image MV) acquired by the moving image acquisition unit 20 from the pet frames extracted by the pet frame extraction unit 220.
In addition, the summary moving image generating unit 27 is a moving image (partial moving image P2, shortened) in which the number of continuous pet frames included in the scene moving image CV composed of the pet frames determined by the continuous pet frame determining unit 221 is equal to or less than a predetermined number. A summary video V is generated based on the event video SEV1, the shortened event video SEV2,.

  With this configuration, the pet moving image analysis apparatus 2 according to the present embodiment can generate the summary moving image V for each scene included in the moving image (captured moving image MV), and thus can efficiently watch the pet P for each scene.

The pet video analysis apparatus 2 according to the present embodiment further includes a thumbnail selection unit 25.
The thumbnail selection unit 25 selects a pet frame corresponding to a thumbnail image (thumbnail SN) from continuous pet frames included in the scene video CV.

  With this configuration, the pet video analysis apparatus 2 according to the present embodiment can display a thumbnail image (thumbnail SN) for the summary video V on the terminal device 3, so that the summary video V is outlined before the summary video V is played back. Can be confirmed.

In addition, the pet video analysis device 2 according to the present embodiment further includes an event determination unit 240.
The event determination unit 240 determines whether the pet frame determined by the continuous pet frame determination unit 221 is an event frame that is a frame in which an event indicating the behavior of the pet P registered in advance is captured.

  With this configuration, the pet video analysis apparatus 2 according to the present embodiment can determine an event included in the scene video CV, and thus can efficiently monitor the pet P for each event.

In addition, the pet video analysis device 2 according to the present embodiment further includes an event counting unit 241.
The event counting unit 241 counts the number of occurrences of an event based on the pet frame determined by the event determination unit 240 as an event frame and the continuous pet frame determined by the continuous pet frame determination unit 221.
The summary video output unit 290 further outputs the number of event occurrences counted by the event counting unit 241 in addition to the summary video V generated by the summary video generation unit 27.

  With this configuration, the pet video analysis apparatus 2 according to the present embodiment can notify the number of occurrences of an event included in the scene video CV, so that the number of event occurrences can be confirmed.

  In the pet video analysis device 2 according to the present embodiment, the summary video generation unit 27 includes the event determination unit 240 when the video (captured video MV) acquired by the video acquisition unit 20 includes a plurality of types of events. Based on the determined event frame, the summary video V is generated so that a plurality of types of events are included in the summary video V.

  With this configuration, the pet video analysis apparatus 2 according to the present embodiment can include a plurality of types of events in the summary video V, so that the summary video V can be prevented from becoming monotonous.

  Further, in the pet video analysis apparatus 2 according to the present embodiment, the event determination unit 240 determines the type of event based on the location imaged in the pet frame extracted by the pet frame extraction unit 220.

  With this configuration, the pet video analysis apparatus 2 according to the present embodiment can determine the type of event based on the location imaged in the pet frame, and therefore, compared to a case where the type of event is determined using only the image recognition technology. The type of event can be easily determined.

  In the present invention, it may be used for performing pet health management such as preventing a risk of a future disease by detecting an action that is a sign of a disease of the pet P by moving image analysis. For the health management of the pet, the number of occurrences of the event displayed on the report screen G4 of FIG. 12 may be used.

(Second Embodiment)
Hereinafter, a second embodiment of the present invention will be described in detail with reference to the drawings.
In the said 1st Embodiment, the pet moving image analyzer demonstrated the case where it analyzed the moving image by which the pet (cat as an example) was imaged, without determining whether a pet is a specific pet. In the present embodiment, a case will be described in which the pet moving image analysis apparatus determines whether a pet is a specific pet (a specific cat as an example) and analyzes a moving image in which the pet is imaged.
The pet video analysis device according to the present embodiment is referred to as a pet video analysis device 2a, and the terminal device is referred to as a terminal device 3a.

  FIG. 13 is a diagram illustrating an example of a functional configuration of the pet video analysis apparatus 2a according to the present embodiment. When the pet moving image analysis apparatus 2a (FIG. 13) according to the present embodiment is compared with the pet moving image analysis apparatus 2 (FIG. 2) according to the first embodiment, the moving image storage unit 21a, the storage unit 23a, and the frame attribute storage unit 26a. The pet determination unit 30a, the evaluation acquisition unit 31a, and the output unit 29a are different. Here, the functions of other components (the moving image acquisition unit 20, the scene determination unit 22, the event analysis unit 24, the thumbnail selection unit 25, the summary moving image generation unit 27, and the thumbnail generation unit 28) are the same as those in the first embodiment. The same. The description of the same functions as those in the first embodiment will be omitted, and in the second embodiment, description will be made focusing on parts different from the first embodiment.

  The moving image storage unit 21a supplies the stored captured moving image MV to the pet determination unit 30a in addition to the scene determination unit 22, the event analysis unit 24, the thumbnail selection unit 25, and the summary moving image generation unit 27.

In addition to the learning result 230 and the event information 231, the storage unit 23a stores pet information 232a.
The pet information 232a is information indicating a specific pet P. The pet information 232a is information in which the name of the pet P is associated with the learning result by machine learning to determine the image of the pet P. Machine learning for determining an image of the pet P is executed using, for example, an image of the whole body of the pet P registered in advance by the user or an image of the face of the pet P.

The pet determination unit 30 a determines whether or not the pet imaged in the pet frame extracted by the pet frame extraction unit 220 is a specific pet P. Here, the pet determination unit 30a determines based on the pet information 232a stored in the storage unit 23a.

  In addition, the pet determination unit 30a determines whether the pet imaged in the captured moving image MV acquired by the moving image acquisition unit 20 is a specific pet P based on the evaluation acquired by the evaluation acquisition unit 31a. Here, the evaluation acquired by the evaluation acquisition unit 31a is an evaluation obtained in response to the notification of the result determined by the evaluation acquisition unit 31a. This evaluation is an evaluation performed by the user as to whether or not the pet determination result determined by the pet determination unit 30a is correct. The evaluation obtained in response to the notification of the result is, for example, the determination of individual pets P that are similar in appearance such as the same cat type (or dog type if the pet P is a dog). This is feedback for the user to input correct / incorrect when P is correctly specified or incorrect.

Here, the determination result of the pet determination unit 30a will be described with reference to FIG.
FIG. 14 is a diagram illustrating an example of an analysis result of the captured moving image MV according to the present embodiment. The pet determination unit 30a determines whether the pet imaged in the frames FN3, FN5, and FN6, which are the pet frames extracted by the pet frame extraction unit 220, is a specific pet P having the name “Tama”. judge.

  In the example illustrated in FIG. 14, the pet imaged in the frame FN3, the frame FN5, and the frame FN6 is determined to be a specific pet P having the name “Tama”. The pet determination unit 30a generates pet determination information N based on the determination result. Here, the pet determination information N is information in which the frame numbers of a plurality of frames constituting the captured moving image MV are associated with the results determined by the pet determination unit 30a.

The pet determination unit 30a may determine a plurality of pets. When the pet determination unit 30a determines a plurality of pets, the pet information 232a includes information in which a pet name is associated with a learning result by machine learning for determining a pet image for each of the plurality of pets. included.
When the pet determination unit 30a determines a plurality of pets, the pet determination information N is information in which, for example, pet names are associated with frame numbers of a plurality of frames constituting the captured moving image MV.

  Returning to FIG. 13, the description of the configuration of the pet moving image analysis apparatus 2 a will be continued.

  The evaluation acquisition unit 31a acquires, from the terminal device 3, an evaluation obtained in response to the notification of the result determined by the pet determination unit 30a.

  The frame attribute storage unit 26 supplies the pet determination information N supplied from the pet determination unit 30 a to the summary video generation unit 27.

The output unit 29a includes a pet determination result notification unit 292a in addition to the summary video output unit 290 and the thumbnail output unit 291.
The pet determination result notification unit 292a notifies the terminal device 3a of the result determined by the pet determination unit 30a.

Next, with reference to FIG. 15, the pet determination process of the pet video analyzing apparatus 2a will be described.
FIG. 15 is a diagram illustrating an example of a pet determination process according to the present embodiment. The process shown in FIG. 5 is executed in a predetermined cycle (for example, 1 hour). In addition, since each process of step S300 and step S302 is the same as each process of step S100 in FIG. 5, and step S102, description is abbreviate | omitted.

Step S304: The pet determination unit 30a determines whether or not the evaluation acquisition unit 31a has acquired an evaluation from the terminal device 3a. Here, when the evaluation is supplied from the evaluation acquisition unit 31a, the pet determination unit 30a determines that the evaluation acquisition unit 31a has acquired the evaluation from the terminal device 3a.

  When it is determined that the evaluation acquisition unit 31a has acquired the evaluation from the terminal device 3a (step S304; YES), the pet determination unit 30a executes the process of step S306. On the other hand, when it is determined that the evaluation acquisition unit 31a has not acquired the evaluation from the terminal device 3a (step S304; NO), the pet determination unit 30a executes the process of step S308.

Step S306: The pet determination unit 30a updates the pet information 232a stored in the storage unit 23 based on the evaluation supplied by the evaluation acquisition unit 31a. Here, the pet determination unit 30a updates a learning result by machine learning to determine a pet image included in the pet information 232a. The pet determination unit 30a uses the evaluation supplied by the evaluation acquisition unit 31a as teacher data and performs relearning to improve the determination accuracy.
Note that the machine learning model used by the pet determination unit 30a for determination does not necessarily have generalization performance as a single model that individually specifies every pet P, and the accuracy of determination in a specific imaging environment is not necessarily provided. It may be learned for the purpose of improving.

Step S308: The pet determination unit 30a determines that the pet determined by the pet frame extracted by the pet frame extraction unit 220 is a specific pet P based on the pet information 232a stored in the storage unit 23a. It is determined whether or not there is. Here, the specific pet P is a pet indicated by the pet information 232a.
The pet determination unit 30a generates pet determination information N based on the determination result. The pet determination unit 30a supplies the generated pet determination information N to the frame attribute storage unit 26a.

Step S310: The pet determination result notifying unit 292a notifies the terminal device 3a of the determination result of the pet determining unit 30a based on the pet determination information N supplied from the frame attribute storage unit 26a. Here, the pet determination result notification unit 292a notifies the determination result of the pet determination unit 30a in association with the summary video V (the event video EV and the summary video SV).

  In the time series screen G1 displayed on the terminal device 3a, the determination result of the pet determination unit 30a is displayed together with the summary video V (the event video EV and the summary video SV). For example, on the time series screen G1, the name of the pet P is displayed as a determination result together with the time stamp T1, the icon A1, and the like.

As described above, the pet moving image analysis apparatus 2a according to the present embodiment includes the pet determination unit 30a and the pet determination result notification unit 292a.
The pet determination unit 30 a determines whether or not the pet imaged in the pet frame extracted by the pet frame extraction unit 220 is a specific pet P.
The pet determination result notifying unit 292a notifies the result determined by the pet determining unit 30a.

  With this configuration, the pet video analyzing apparatus 2a according to the present embodiment can distinguish a specific pet from other pets, and therefore can efficiently monitor a pet for each pet.

In addition, the pet video analyzing apparatus 2a according to the present embodiment includes an evaluation acquisition unit 31a.
The evaluation acquisition unit 31a acquires an evaluation obtained in response to the notification of the result determined by the pet determination unit 30a.
The pet determination unit 30a determines whether or not the pet imaged in the moving image (captured moving image MV) acquired by the moving image acquisition unit 20 is a specific pet P based on the evaluation acquired by the evaluation acquisition unit 31a.

  With this configuration, the pet video analyzing apparatus 2a according to the present embodiment can determine whether or not the pet is a specific pet P based on the evaluation obtained in response to the notification of the result determined by the pet determination unit 30a. Therefore, the accuracy of determination can be improved.

(Third embodiment)
Hereinafter, the third embodiment of the present invention will be described in detail with reference to the drawings.
In the present embodiment, when the pet video analysis device stores a video before being shortened as a summary video or a video recorded by a user, and outputs the video to the terminal device in response to a request from the terminal device Will be explained.
The pet video analysis device according to the present embodiment is referred to as a pet video analysis device 2b, and the terminal device is referred to as a terminal device 3b.

  FIG. 16 is a diagram illustrating an example of a functional configuration of the pet moving image analysis apparatus 2b according to the present embodiment. When the pet moving image analysis device 2b (FIG. 16) according to the present embodiment is compared with the pet moving image analysis device 2 (FIG. 2) according to the first embodiment, the moving image storage unit 21b, the storage unit 23b, and the frame attribute storage unit 26b. The output unit 29b, the request acquisition unit 32b, and the archive video management unit 33b are different. Here, the functions of other components (the moving image acquisition unit 20, the scene determination unit 22, the event analysis unit 24, the thumbnail selection unit 25, the summary moving image generation unit 27, and the thumbnail generation unit 28) are the same as those in the first embodiment. The same. The description of the same functions as those in the first embodiment will be omitted, and in the third embodiment, description will be made centering on parts different from the first embodiment.

  The moving image storage unit 21b supplies the stored captured moving image MV to the archive moving image management unit 33b in addition to the scene determination unit 22, the event analysis unit 24, the thumbnail selection unit 25, and the summary moving image generation unit 27.

In addition to the learning result 230 and the event information 231, the storage unit 23b stores a scene moving image 233b and a recorded moving image 234b.
The scene video 233b is a scene video CV (pre-summary event video PEV) determined by the event determination unit 240 to include an event. The time length of the scene video 233b may be 60 seconds or more.
The recorded moving image 234b is a moving image in which the captured moving image MV is recorded when the pet moving image analysis device 2b immediately outputs the captured moving image MV acquired from the imaging device 1 to the terminal device 3b.
The scene video 233b and the recorded video 234b are collectively referred to as an archive video.

  The frame attribute storage unit 26b supplies the frame attribute information FP supplied from the event analysis unit 24 to the archive moving image management unit 33b.

The request acquisition unit 32b includes an archive video request acquisition unit 320b and a live video request acquisition unit 321b.
The archive video request acquisition unit 320b acquires an archive video request command from the terminal device 3b. Here, the archive video request command is a command for causing the pet video analysis device 2 to output an archive video. The archive video request command includes a command for outputting a list of archive video and a command for outputting a specific archive video.
When the archive video request acquisition unit 320b acquires an archive video request, the archive video request acquisition unit 320b supplies the archive video request acquisition unit 320b to the archive video management unit 33b.

  The live video request acquisition unit 321b acquires a live video request command from the terminal device 3b. Here, the live video request command is a command for outputting a live video to the pet video analysis device 2 or causing the pet video analysis device 2 to record a live video.

When the live video request acquisition unit 321b acquires the live video request command, the live video request command is sent to the archive video management unit 33b according to the type of command indicated by the live video request command. Supply.
The live video request acquisition unit 321b supplies a live video output command to the archive video management unit 33b when the live video request command indicates that the live video is output to the pet video analysis device 2.
On the other hand, the live video request acquisition unit 321b supplies the live video recording command to the archive video management unit 33b when the live video request command indicates that the pet video analysis device 2 records the live video.

  The archive video management unit 33b extracts the pre-summary event video PEV from the captured video MV supplied by the video storage unit 21a based on the frame attribute information FP supplied by the frame attribute storage unit 26b. The archive video management unit 33b stores the extracted pre-summary event video PEV in the storage unit 23b as the scene video 233b.

  Here, the pre-summary event video PEV is composed of pet frames determined by the continuous pet frame determination unit 221. Therefore, the archive moving image management unit 33b is a storage control unit that causes the storage unit 23b to store the scene moving image 233b composed of pet frames determined by the continuous pet frame determination unit 221.

In addition, the archive video management unit 33b supplies the captured video MV to the live video output unit 294b in response to the live video output command supplied from the live video request acquisition unit 321b.
Further, the archive video management unit 33b stores the captured video MV supplied by the video storage unit 21 in the storage unit 23 as the recorded video 234b in response to the live video recording command supplied from the live video request acquisition unit 321b.

  Further, the archive video management unit 33b outputs the archive video stored in the request acquisition unit 32b to the archive video output unit 293b in response to the archive video request command supplied from the archive video request acquisition unit 320b.

  Here, when the archive video request command includes a command for outputting a list of archive videos, the archive video management unit 33b displays thumbnails SN respectively corresponding to a plurality of archive videos (scene video 233b and recorded video 234b). The archive moving image list is supplied to the live moving image output unit 294b.

  On the other hand, the archive video management unit 33b, when the archive video request command includes a command for outputting a specific archive video (scene video 233b and recorded video 234b), a specific archive video (scene The video 233b and the recorded video 234b) are supplied to the live video output unit 294b.

  The output unit 29b includes an archive video output unit 293b and a live video output unit 294b in addition to the summary video output unit 290 and the thumbnail output unit 291.

  The archive video output unit 293b outputs a list of archive videos supplied by the archive video management unit 33b to the terminal device 3b. The list of archive videos may be displayed on the terminal device 3b using the album screen G2 as shown in FIG.

  The archive video output unit 293b outputs the archive video (scene video 233b or recorded video 234b) supplied by the archive video management unit 33b to the terminal device 3b. Therefore, the archive video output unit 293b outputs the scene video 233b stored in the storage unit 23b by the archive video management unit 33b, which is a storage control unit.

  The live video output unit 294b outputs the captured video MV supplied as a live video by the archive video management unit 33b to the terminal device 3b. Here, the live moving image output unit 294b outputs the captured moving image MV to the terminal device 3, for example, using streaming.

  As described above, the pet video analysis device 2b according to the present embodiment includes the storage control unit (archive video management unit 33b) and the scene video output unit (archive video output unit 293b).

  With this configuration, the pet video analyzing apparatus 2b according to the present embodiment can output the video before being shortened as the summary video V to the terminal device 3b in response to the request, so that the scene video to be confirmed firmly can be shortened. It can be confirmed in the previous length.

Note that some of the pet video analysis devices 2, 2a, and 2b in the above-described embodiment, for example, the video acquisition unit 20, the video storage units 21, 21a, and 21b, the scene determination unit 22, the storage units 23, 23a, and 23b, and the event Analysis unit 24, thumbnail selection unit 25, frame attribute storage units 26, 26a, 26b, summary video generation unit 27, thumbnail generation unit 28, output units 29, 29a, 29b, pet determination unit 30a, evaluation acquisition unit 31a, request acquisition The unit 32b and the archive moving image management unit 33b may be realized by a computer. In that case, the program for realizing the control function may be recorded on a computer-readable recording medium, and the program recorded on the recording medium may be read by the computer system and executed. Here, the “computer system” is a computer system built in the pet moving image analysis apparatuses 2, 2 a, 2 b and includes hardware such as an OS and peripheral devices. The “computer-readable recording medium” refers to a portable medium such as a flexible disk, a magneto-optical disk, a ROM, and a CD-ROM, and a storage device such as a hard disk built in the computer system. Furthermore, the “computer-readable recording medium” is a medium that dynamically holds a program for a short time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, In this case, a volatile memory inside a computer system that serves as a server or a client may be included that holds a program for a certain period of time. The program may be a program for realizing a part of the functions described above, and may be a program capable of realizing the functions described above in combination with a program already recorded in a computer system.
Moreover, you may implement | achieve part or all of the pet animation analysis apparatuses 2, 2a, 2b in embodiment mentioned above as integrated circuits, such as LSI (Large Scale Integration). Each functional block of the pet moving image analysis apparatuses 2, 2a, and 2b may be individually made into a processor, or a part or all of them may be integrated into a processor. Further, the method of circuit integration is not limited to LSI, and may be realized by a dedicated circuit or a general-purpose processor. In addition, when an integrated circuit technology that replaces LSI appears due to the advancement of semiconductor technology, an integrated circuit based on the technology may be used.

  As described above, the embodiment of the present invention has been described in detail with reference to the drawings. However, the specific configuration is not limited to the above, and various design changes and the like can be made without departing from the scope of the present invention. It is possible to

  DESCRIPTION OF SYMBOLS S ... Pet watching system, 1 ... Imaging device, 2, 2a, 2b ... Pet animation analysis device, 3, 3a, 3b ... Terminal device, 20 ... Movie acquisition part, 21, 21a, 21b ... Movie storage part, 22 ... Scene Judgment unit, 220 ... pet frame extraction unit, 221 ... continuous pet frame judgment unit, 23, 23a, 23b ... storage unit, 24 ... event analysis unit, 240 ... event judgment unit, 241 ... event counting unit, 242 ... activity amount measurement 25: Thumbnail selection unit, 26 ... Frame attribute storage unit, 27 ... Summary video generation unit, 270 ... Event video generation unit, 271 ... Summary video generation unit, 28 ... Thumbnail generation unit, 29, 29a, 29b ... Output unit 290 ... summary video output unit 290, 291 ... thumbnail output unit, 292a ... pet determination result notification unit, 293b ... archive video output unit, 294 ... Live video output unit, 30a ... Pet determination unit, 31a ... Evaluation acquisition unit, 32b ... Request acquisition unit, 320b ... Archive video request acquisition unit, 321b ... Live video request acquisition unit, 30a ... Pet determination unit, 31a ... Evaluation acquisition Part, 232a ... pet information, 292a ... pet judgment result notification part, 33b ... archive video management part, 293b ... archive video output part, 294b ... live video output part, V ... summary video, EV ... event video, SV ... summary video

Claims (13)

A video acquisition unit for acquiring a video in which a pet is imaged;
A pet frame extraction unit that extracts a pet frame, which is a frame in which a pet is imaged, from a plurality of frames constituting the video acquired by the video acquisition unit;
The video acquisition unit acquires a summary video that is a video in which the number of pet frames extracted by the pet frame extraction unit is larger than the number of non-pet frames that are frames excluding the pet frame among the plurality of frames. A summary video generation unit that generates the video based on the video,
A summary video output unit that outputs the summary video generated by the summary video generation unit;
A pet video analysis device comprising: A continuous pet frame determination unit that determines the pet frames that are continuous in the moving image acquired by the moving image acquisition unit from the pet frames extracted by the pet frame extraction unit,
The summary video generation unit generates the summary video based on a video in which the number of continuous pet frames included in a scene video composed of the pet frames determined by the continuous pet frame determination unit is less than or equal to a predetermined number. The pet video analysis device according to claim 1 to generate. The pet moving image analysis apparatus according to claim 2, further comprising: a thumbnail selection unit that selects the pet frame corresponding to a thumbnail image from among the continuous pet frames included in the scene moving image. A storage control unit for storing the scene video in a storage unit;
The pet video analysis device according to claim 2 or 3, further comprising: a scene video output unit that outputs the scene video stored in the storage unit by the storage control unit. An event determination unit that determines whether the pet frame determined by the continuous pet frame determination unit is an event frame that is a frame in which an event indicating the behavior of the pet registered in advance is captured; The pet animation analysis device according to any one of claims 2 to 4 provided. An event counting unit that counts the number of occurrences of the event based on the pet frame determined by the event determination unit as the event frame and the continuous pet frame determined by the continuous pet frame determination unit;
The pet video analysis device according to claim 5, wherein the summary video output unit further outputs the number of occurrences counted by the event counting unit in addition to the summary video generated by the summary video generation unit. The summary video generation unit, when the video acquired by the video acquisition unit includes a plurality of types of events, based on the event frame determined by the event determination unit, the events of the plurality of types The pet video analysis device according to claim 5 or 6, wherein the summary video is generated so as to be included in the summary video. The pet video analysis device according to any one of claims 5 to 7, wherein the event determination unit determines the type of the event based on a location imaged in the pet frame extracted by the pet frame extraction unit. . A pet determination unit that determines whether or not the pet imaged in the pet frame extracted by the pet frame extraction unit is a specific pet;
The pet video analysis device according to any one of claims 1 to 8, further comprising: a pet determination result notification unit that notifies a result determined by the pet determination unit. An evaluation acquisition unit that acquires an evaluation obtained in response to the notification of the determination result of the pet determination unit;
The said pet determination part determines whether the said pet imaged by the said moving image acquired by the said moving image acquisition part is a specific pet based on the said evaluation acquired by the said evaluation acquisition part. Pet video analysis device. The pet video analysis device according to any one of claims 1 to 10,
An imaging device that captures the video and supplies the pet video analysis device;
A pet video analysis system comprising: a terminal device that acquires the summary video output by the pet video analysis device. A video acquisition procedure for acquiring a video of a pet image;
A pet frame extraction procedure for extracting a pet frame, which is a frame in which a pet is imaged, from a plurality of frames constituting the video acquired in the video acquisition procedure;
In the video acquisition procedure, a summary video is a video in which the number of pet frames extracted in the pet frame extraction procedure is larger than the number of non-pet frames that are frames excluding the pet frame among the plurality of frames. A summary video generation procedure to be generated based on the acquired video;
A summary video output procedure for outputting the summary video generated in the summary video generation procedure;
A pet video analysis method comprising: A video acquisition step of acquiring a video in which a pet is imaged on a computer;
A pet frame extracting step of extracting a pet frame that is a frame in which a pet is imaged from a plurality of frames constituting the moving image acquired in the moving image acquiring step;
In the moving image acquisition step, a summary moving image is a moving image in which the number of pet frames extracted in the pet frame extracting step is larger than the number of non-pet frames that are frames excluding the pet frame among the plurality of frames. A summary video generation step for generating based on the acquired video;
A summary video output step for outputting the summary video generated in the summary video generation step;
A program for running

Images