JP2019024482A - Information processing system, information processing device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To eliminate the burden on livestock and the like to be monitored, and to make it possible to read more behavior information from the monitoring target. SOLUTION: The information processing system according to the present invention has a spatial sensor that outputs spatial information in a monitoring area, and a first method that detects object information of a plurality of moving objects to be monitored from the spatial information output from the spatial sensor. A plurality of detection means, a second detection means for detecting the arrangement information of moving objects in contact with each other among a plurality of moving objects in the object information detected by the first detection means, and a plurality of object information and arrangement information. A generation means for generating behavior information of a moving object, a judgment means for determining whether the behavior of a moving object is an abnormal behavior from the behavior information, and a notification of an abnormality when the behavior of a moving object is determined to be an abnormal behavior. It is characterized by having a notification means for. [Selection diagram] Fig. 2

Description

  The present invention relates to an information processing system, an information processing apparatus, and a program.

  Conventionally, several percent of livestock raised by livestock farmers are dead due to inability to stand up or weakness before shipment. Livestock farmers are not able to watch all the livestock in the stables 24 hours a day due to labor shortages, and it is difficult to detect early changes even if livestock changes in behavior. For this reason, there are cases where livestock is killed by suffocation while being unable to stand in the stable, or is weakened due to illness or the like, leading to death. Inexperienced new farmers, unlike skilled workers, cannot even notice changes in livestock behavior, thus further increasing the rate of livestock death.

  There is a disclosure system for reporting abnormalities in the health status of livestock. This reporting system causes a cow to be equipped with sensors that detect the number of steps, receives a step count signal from each sensor, and finds abnormal health from the number of steps. If there is an abnormality in the health information, this is notified to the user of the reporting system (see Patent Document 1). In addition, there is a sensor in which a body temperature sensor is embedded in the body of a cow, and a temperature signal from the sensor is received to detect a health condition abnormality (see Patent Document 2).

International Publication No. 2013/145329 JP 2007-296042 A

  However, the conventional mechanism for detecting abnormalities in livestock by attaching sensors to livestock places a heavy burden on livestock and may give stress to livestock. For this reason, the types and number of sensors to be mounted are limited, and there is a problem that information that can be detected from livestock is limited to those that detect a specific abnormality.

  The present invention has been made in view of the above, and is an information processing system and an information processing system that eliminates the burden on livestock to be monitored and allows more action information to be read from the monitoring target. An object is to provide a device and a program.

  In order to solve the above-described problems and achieve the object, an information processing system according to the present invention includes a spatial sensor that outputs spatial information of a monitoring area, and a plurality of objects to be monitored from the spatial information output from the spatial sensor. First detection means for detecting object information of an object, and second detection means for detecting arrangement information of moving objects in contact with each other among the plurality of moving objects in the object information detected by the first detection means Generating means for generating behavior information of a plurality of moving objects from the object information and the arrangement information, determining means for determining whether the behavior of the moving object is abnormal behavior from the behavior information, and the behavior of the moving object is abnormal behavior And a notification means for notifying abnormality when it is determined that there is an error.

  The present invention has an effect of eliminating the burden on the monitoring target and reading more action information from the monitoring target.

FIG. 1 is a diagram for explaining an embodiment of an information processing system according to the present invention. FIG. 2 is a diagram illustrating an example of a system configuration of an information processing system including a monitoring camera provided in a cow chamber. FIG. 3 is a diagram illustrating an example of a hardware configuration of the monitoring camera unit. FIG. 4 is a diagram illustrating an example of a hardware configuration of the analysis server device. FIG. 5 is a diagram illustrating an example of a table configuration of an information table related to the monitoring service. FIG. 6 is a diagram illustrating an example of setting information in which determination items for abnormal behavior are set for each breeding stage. FIG. 7 is an explanatory diagram of functions of the analysis server device. FIG. 8 is an explanatory diagram of a method for detecting the contour of a cow. FIG. 9 is a diagram illustrating an example of the auxiliary unit. FIG. 10 is an explanatory diagram of auxiliary processing by the search unit and the specifying unit. FIG. 11 is an explanatory diagram for approximating the contour of a cow to a figure. FIG. 12 is a diagram illustrating an example of an area change of a cow contour image. FIG. 13 is a diagram illustrating an example of the flow of the tracking process. FIG. 14 is a diagram illustrating an example of a flow of alarm basic data generation processing. FIG. 15 is a diagram illustrating an example of a flow of threshold determination processing. FIG. 16 is a diagram illustrating an example of a flow of cow abnormal behavior determination processing.

  Embodiments of an information processing system, an information processing apparatus, and a program according to the present invention will be described below in detail with reference to the drawings. Note that the present invention is not limited to the embodiments. For example, in the following embodiment, cattle of livestock that are dairy cattle or beef cattle are described as monitoring targets. However, if it is a “moving object”, it can be used for other livestock, animals other than livestock, or living things. You may apply. In the embodiment described below, a spatial sensor that outputs a depth image indicating the depth of each subject in the monitoring area as “spatial information” is shown as an example. However, “spatial information” is not limited to a depth image, and the monitoring area Any other object can be used as long as it can identify the area or contour of the moving subject in the camera. In the embodiment described below, the contour information is shown as an example of “object information”, but the “object information” may be a moving object region itself. In the embodiment described below, the individual contour information is shown as an example of “placement information”, but the “placement information” may be other information as long as it is information that can identify the region of the individual in the object information.

  FIG. 1 is a diagram for explaining an embodiment of an information processing system according to the present invention. FIG. 1 shows a cowhouse 90 provided in the barn and a monitoring camera (an example of a “space sensor”) 10 a that monitors the behavior of the cow 91 in the cowhouse 90. A cow bull 90 represents one cow bunch partitioned by a fence 90a. A cow 91 born at the same time is accommodated in the cow bull 90. Although FIG. 1 shows two cows 91 as an example, the number of cows 91 may be one or more. Although a drinking fountain and a feeding area are not shown, they may be inside the fence 90a or outside the fence 90a.

  As shown in FIG. 1, the monitoring camera 10a is installed above the cow 90, and the growing area 90b of the cow 90 is photographed by the monitoring camera 10a. The growing area 90b is the monitoring area of this embodiment. The height H at which the monitoring camera 10a is installed is adjusted so that the growing area 90b is within the shooting range. In the present embodiment, the monitoring camera 10a is installed at a height of about 4.2 m with respect to the growing area 90b having a width W × depth L of 5.0 m × 4.1 m.

  The growing area 90b is sensed by the monitoring camera 10a, and a frame image is output from the monitoring camera 10a at a predetermined frame rate. The behavior of the cow 91 is analyzed by the server device of the monitoring service provider based on the continuous frame images (“continuous frame images” described later). And if the result which shows the abnormal action of the cow 91 is obtained in this data analysis, the said server apparatus will notify an abnormal signal to the livestock farmer of the cow 91. FIG.

  FIG. 2 is a diagram illustrating an example of a system configuration of an information processing system including the monitoring camera 10 a provided in the cow chamber 90. As shown in FIG. 2, the information processing system 1 is configured to connect a plurality of surveillance camera units 10, a cloud 20, an analysis server device 30, a management PC 40, a user terminal 50, and the like via a network 60. Have

  The surveillance camera unit 10 is installed in a cowshed 90 of a cowshed of each livestock farmer who uses the surveillance service. Although only one cow 90 is shown in FIG. 2, the monitoring camera unit 10 is individually installed in each cow 90 in the barn. One surveillance camera unit 10 may be installed for several cows 90.

  The surveillance camera unit 10 is an image transmission unit having a surveillance camera 10a and a control PC 10b. The surveillance camera 10 a is installed above each cow 90 with the shooting direction facing downward, and takes images of the breeding area 90 b of each cow 90. The control PC 10b of the monitoring camera 10a skips the continuous frame image output from the monitoring camera 10a to the access point using WiFi or the like, and transmits the image to the cloud 20 through the network 60 from the access point.

  The cloud 20 constructs a storage network and provides a data storage service. The cloud 20 accumulates and stores user storage data (in this embodiment, continuous frame images) in the storage 21 on the storage network.

  The analysis server device 30 acquires a continuous frame image (copy) from the cloud 20 and performs an analysis process on the behavior of the cow 91 with respect to the continuous frame image. When the analysis server device 30 detects the abnormal behavior of the cow 91 by the analysis process, the analysis server device 30 notifies the user terminal 50 of the livestock farmer of the cow 91 of the signal (abnormal signal 70) for notifying the abnormal behavior of the cow 91. The notification of the abnormality signal 70 is performed by an alarm or an e-mail.

  The management PC 40 is a personal computer provided in each management farm 80 by each livestock farmer. The management PC 40 performs setting of the monitoring camera unit 10 by a corresponding setting program.

  The user terminal 50 is a mobile phone, a smartphone, a tablet terminal or the like owned by each livestock farmer. The user terminal 50 receives an abnormal signal 70 due to an alarm or the like from the analysis server device 30, and the cow 91 has an abnormal behavior due to a notification sound, light emission of an LED (Light Emitting Diode), display of a message screen, or the like. To the owner of the user terminal 50. Further, the user terminal 50 may acquire an e-mail notifying the content of the abnormal behavior of the cow 91 as the abnormal signal 70.

  Further, the user terminal 50 accesses a URL (Uniform Resource Locator) specified by the company that provides the monitoring service, and reproduces and displays the continuous frame image output from the monitoring camera unit 10 of the user's cow 90. It is also possible to do. The continuous frame image to be reproduced is a color continuous frame image output from the color camera 11c (see FIG. 3) of the monitoring camera unit 10.

  The network 60 is a wireless local area network (LAN) network, a public network, the Internet, or the like. The access point to which the control PC 10b is connected via WiFi is configured in the wireless LAN network of the network 60.

  FIG. 3 is a diagram illustrating an example of a hardware configuration of the monitoring camera unit 10. The monitoring camera 10a of the monitoring camera unit 10 includes an infrared projector 11a, an infrared camera 11b, a color camera 11c, and the like.

  The infrared projector 11 a irradiates the entire cow 90 with infrared rays. The infrared camera 11b continuously outputs an image (referred to as a “depth image”) representing the depth to each subject in the cow bull 90 at a predetermined frame rate. Specifically, the infrared camera 11b calculates a depth for each pixel at a predetermined frame rate based on infrared light reflected from each subject in the cow bull 90, and associates each depth with a predetermined pixel value. Is output continuously. For example, the distance to each subject is calculated as the depth based on the time from when the infrared light is irradiated until the reflected light from each subject is received. In the present embodiment, the continuous frame image indicating the depth is simply referred to as “continuous frame image” in distinction from the color continuous frame image.

  The color camera 11c continuously outputs a color image of the cow 90 at a predetermined frame rate. Specifically, the color camera 11c converts each subject in the cow 90 into a color image at a predetermined frame rate based on illumination light or light reflected by sunlight, and continuously outputs it. In the present embodiment, it is clearly indicated that the color is “color”.

  The monitoring camera 10a is connected to the control PC 10b via a USB (Universal Serial Bus) cable 10c. The monitoring camera 10a outputs a continuous frame image and a color continuous frame image with settings instructed from the control PC 10b.

  The control PC 10b includes a control unit 12 having a computer configuration such as a CPU (Central Processing Unit), a ROM (Read Only Memory), and a RAM (Random Access Memory). The controller 12 is connected to a USB controller 13, a WiFi communication controller 14, and the like.

  In the control unit 12, the CPU executes a ROM program and performs, for example, the following processing. The control unit 12 acquires a continuous frame image and a color continuous frame image from the monitoring camera 10 a under the control of the USB controller 13, and the acquired continuous frame image and the color continuous frame image are controlled by the communication controller 14 in the cloud 20. Send to.

  The USB controller 13 performs USB communication with a target (infrared camera 11b, color camera 11c, etc.) of the monitoring camera 10a via the USB cable 10c. Mainly, the USB controller 13 acquires a continuous frame image and a color continuous frame image from the monitoring camera 10a.

  The communication controller 14 is connected to the access point 60a via WiFi, and transmits a continuous frame image and a color continuous frame image to the cloud 20 as needed.

  FIG. 4 is a diagram illustrating an example of a hardware configuration of the analysis server device 30. The analysis server device 30 shown in FIG. 4 has a CPU 31, a ROM 32, a RAM 33, an HDD (Hard Disk Drive) 34, a communication board 35, and the like connected via a bus 36.

  The CPU 31 executes a program and performs arithmetic processing and control processing of each unit. The ROM 32 is a non-volatile memory that stores a fixed program such as BIOS. The RAM 33 is a volatile memory that the CPU 31 uses as a work area.

  The HDD 34 is an auxiliary storage device. The HDD 34 stores a basic program C1 such as an OS (Operating System), an analysis program C2 related to a monitoring service, an information table T, setting information D, and the like. The basic program C1 includes a Web (World Wide Web) server, a database server, and the like.

  The communication board 35 is a communication board for performing communication such as Ethernet (registered trademark). The communication board 35 is connected to, for example, an in-house LAN of a service provider via a communication cable.

Next, the table configuration of the information table T related to the monitoring service will be described.
FIG. 5 is a diagram illustrating an example of a table configuration of the information table T related to the monitoring service. FIG. 5A is a user information registration table T1 for registering user information of a user who starts a monitoring service. FIG. 5B shows a cow information registration table T2 for registering information about the cow 90 where the monitoring camera 10a is installed. FIG. 5C shows an individual information registration table T3 for registering individual information of the cow 91 in the cow bull 90. FIG. 5D is a recording table T4 that sequentially records monitoring data indicating the behavior of the cow 91. FIG. 5E is an alarm basic data setting table T5 for setting alarm basic data generated from monitoring data.

  The user information registration table T1 shown in FIG. 5A includes, as items, a “user ID (Identification)” t11 of the user, a “phone number” t12 of the user, and an “e-mail address” of the user. t13, the user's “URL for viewing images” t14, and the like. Here, “telephone number” t12 and “mail address” t13 are examples of information indicating a notification destination for notifying the user of the abnormality of the cow 91. The “image browsing URL” t14 is the URL from which the color continuous frame image obtained by photographing the state of the user's cow bull 90 is acquired.

  The cow information registration table T2 shown in FIG. 5B includes, as items, “user ID” t11, “cattle ID” t21 of the cow 90 to be set as a monitoring target, and cattle 91 for each cow 90. “Birth date” t22 and the like. Here, the monitoring camera unit ID (such as the MAC address of the control PC 10b) of the monitoring camera unit 10 installed in the cow 90 is registered in the “beef ID” t21. In “birth date” t22, the date of birth of the cow 91 in the cow bull 90 is registered. In this example, the case where the cow 91 born on the same day is housed in the same cow chamber 90 is shown. When cattle 91 having different birth dates are accommodated in the same cow 90, the table configuration may be modified so that the date of birth can be managed for each individual cow 91 instead of 90 units.

  The individual information registration table T <b> 3 illustrated in FIG. 5C includes items such as “cow ID” t <b> 21, “individual ID” t <b> 31 of the cow 91 in the cow chamber 90, and the like. Here, in the “individual ID” t31, individual IDs of all cows 91 housed in the cow 90 are registered for each cow 90. Individual IDs may be arbitrarily set as long as they can be distinguished from each other.

  The record table T4 shown in FIG. 5D includes “individual ID” t31, “period” t41, “monitoring data” t42, “contact flag” t43, and the like as items. Here, in “period” t41, information on the set date and time when data is set in “monitoring data” t42 is set. In the “monitoring data” t42 and the “contact flag” t43, monitoring data of the cow 91 described later and a contact flag (an example of “contact number information”) indicating contact between the cows 91 are set.

  The alarm basic data setting table T5 shown in FIG. 5E has “individual ID” t31, “period” t51, “behavior information (alarm basic data)” t52, and the like as items. Here, in the “period” t51, the date including the set date and time of the monitoring data and the morning, noon, and night periods are set. “Behavior information (alarm basic data)” t52 includes “walking distance” t521, “standing time” t522, “sitting time” t523, “number of times of contact” t524 with other cows 91, , “Recumbent time” t525, and other alarm basic data setting items for information representing the behavior of the cow 91. Here, the “alarm basic data” is basic data for determining the abnormal behavior of the cow 91 from the behavior of the cow 91. Each item of “behavior information (alarm basic data)” t52 is an example, and may be appropriately modified to other items. Here, the sitting position refers to a posture in which the knees are lowered and the chest is raised. Recumbent is a posture in which the limbs are thrown out and laid down.

  FIG. 6 is a diagram illustrating an example of the setting information D in which items for determining abnormal behavior are set for each breeding stage. The training stage is an example of “stage information”, and the setting information D is an example of “corresponding information”. The setting information D shown in FIG. 6 includes a first stage d11, a second stage d12, a third stage d13, a fourth stage d14, a fifth stage d15,. Here, as an example, the first stage d11 is “colostrum”, the second stage d12 is “feeding”, the third stage d13 is “fattening”, the fourth stage d14 is “estrus”, and the fifth stage d15 is “ It shows what is set to “Pregnancy”,. Each setting of the breeding stage is divided by the number of days from the birth of the cow 91, and thus has information indicating the number of days and the period from the birth of the cow 91. For example, in “colostrum”, information indicating a period from birth to the seventh day is set.

  Further, in the setting information D, determination item information d2 regarding abnormal behavior is associated with the breeding stage information d1. As an example, “disappear” d21, “estrus” d22, “disease sign” d23, “not standing” d24, and the like are provided as determination items, and a flag “ 1 ”. Here, as an example, the first stage d11 and the second stage d12 are flagged as “disappear” d21 and “disease sign” d23, and the third stage d13 is “disease sign” d23 and “not standing”. A flag is set for d24, a flag is set for "estrus" d22 for the fourth stage d14, and a flag is set for "not standing up" d24 for the fifth stage d15.

  In addition, you may add or delete suitably about the item shown as an example here. Moreover, you may change suitably the item made into the determination object in each cultivation stage (1st stage d11, 2nd stage d12, ...).

  Further, the breeding stage of the setting information D may be changed as appropriate. For example, the growing stage may be further divided. Further, the breeding stages may be divided according to the type of cow 91 (beef cattle, milk cow), and individual breeding stages corresponding to each may be set.

  7 to 11 are explanatory diagrams of functions of the analysis server device 30. The analysis server device 30 realizes this function by the CPU 31 appropriately reading and executing the basic program C1 and analysis program C2 of the HDD 34 in the RAM 33. This function may be realized by being assigned to one CPU core or may be realized by being distributed to a plurality of CPU cores. Further, a plurality of analysis server devices 30 may be provided, and the functions may be distributed to each CPU.

  Hereinafter, the various functions shown in FIG. 7 will be described with reference to FIGS. In addition, between each function, notification of the completion of a process etc. shall be performed by inter-process communication.

  In FIG. 7, as main functions related to the analysis processing, a table operation unit 301, a reading unit 302, a detection unit 303, an auxiliary unit 304, a monitoring data generation unit 305, an association unit 306, a setting unit 307, an alarm basic data generation unit 308, a threshold value determination unit 309, a determination unit 310, and a notification unit 311 are illustrated.

  The table operation unit 301 registers, updates, deletes, and extracts data for the information table T.

  The reading unit 302 reads frame images Mn (n = 1, 2,...) For one frame from the continuous frame images M0 of the IDs in the order in which the images are captured, for each monitoring camera unit ID. For example, the continuous frame image M0 is sequentially transmitted in the order in which the images were captured from the cloud 20 (see FIG. 2), and stored in the reception buffer of the communication board 35. The reading unit 302 reads the frame image Mn for one frame for the continuous frame image M0 in the reception buffer.

  The detection unit 303 detects the contour of the cow (contour information P1) from the read frame image Mn.

  FIG. 8 is an explanatory diagram of a method for detecting the contour of a cow. FIG. 8A shows a color image for explaining the frame image Mn which is a depth image. FIG. 8B shows a frame image Mn that is a depth image. FIG. 8C shows contour information P1 detected from the frame image Mn which is a depth image.

  FIG. 8A shows a color image at the moment when two cows 91 in the same growing period are in the standing state shown in FIG. In the standing position, the depth becomes shallow because the distance between the cow 91 and the infrared camera 11b approaches. In the image of FIG. 8A, the head is cut off from the image because the cow 91 puts his head out of a food box or the like outside the fence, but the head of the cow 91 is at a position lower than the torso. Therefore, the depth becomes deeper than the trunk. In addition, the ground is farthest from the infrared camera 11b and has the deepest depth.

  As shown in FIG. 8B, the frame image Mn of the depth image is an image showing a plurality of constant depth regions E1, E2,. Each of the regions E1, E2,... Having a constant depth represents that each pixel has the same depth in the region. In the regions E1, E2,... Having a certain depth, the depth to the subject is different between adjacent regions, and the depths are different from each other. Note that the pixel value of each pixel configured in the frame image Mn corresponds to the depth representing the distance from the subject corresponding to the position of each pixel. Here, for the sake of explanation, each region E1, E2,... Is represented by changing the hatched interval according to the depth. The narrower the diagonal line interval, the shallower the depth. Moreover, although the area | region of fixed depth is divided | segmented into the area | regions E1-E8, the number of this area | region is an example and the division | segmentation number changes according to the decomposition | disassembly capability of depth.

  When the cow 91 is in a sitting or lying state, the depth of the cow 91 becomes deeper and the depth value becomes larger. The ground is white because it is the deepest. In addition, the fence 90a, a food area, a drinking fountain, and the like are represented at a fixed depth over a continuous frame.

  The detection unit 303 selects a region having a certain depth that has changed, such as moving from a plurality of regions E1, E2,... And the contour is detected. When the adjacent constant depth region changes in conjunction, the region including the interlocking constant depth region is used as the cow region. In FIG.8 (b), since the area | region E1 changes, the area | region E1 is specified as a cow area | region. Furthermore, since the region E2 and the region E3 adjacent to the region E2 change in conjunction with each other, the region E2 and the region E3 are specified as regions of different cows.

In FIG. 8C, the two identified cow regions are indicated by rectangular frames F1 and F2. The detection unit 303 detects the cow contours f1 and f2 as the contour information P1 from the cow regions in the specified rectangular frames F1 and F2. The contours f1 and f2 of the cow are detected from a difference in depth information from other regions.
The cow region may be specified by other methods. For example, in the fence 90a, there are food boxes and drinking fountains in addition to the cattle to be monitored, but these are in fixed positions. For this reason, the area | region except the area | region shown by coordinates, such as a food box and a drinking fountain, in the frame image Mn is specified as an area | region of each cow.

  The auxiliary unit 304 performs an auxiliary process of detecting the contour of each cow (individual contour information P2) when in contact with the contour (contour information P1) detected by the detection unit 303. Auxiliary processing is processing that enables each contour of each cow 91 to be detected when the contour of the cow detected by the detection unit 303 is detected as one contour by contact of the plurality of cows 91. With this process, even when a plurality of cattle 91 being tracked are detected as a single contour by contact over the frame images Mn of the continuous frame image M0, individual identification of each cow 91 during that time is made possible.

  FIG. 9 is a diagram illustrating an example of the auxiliary unit 304. As shown in FIG. 9, the auxiliary unit 304 includes a search unit 304a and a specifying unit 304b. The auxiliary processing by the search unit 304a and the specifying unit 304b will be described with reference to FIG.

  FIG. 10 is an explanatory diagram of auxiliary processing by the search unit 304a and the specifying unit 304b. FIG. 10A shows a contour f3 of the cow when two cows 91 are in contact with each other in the frame image Mn. FIG. 10B shows an example of auxiliary processing for the cow outline f3 in FIG.

  The search unit 304a scans the contour f3 from one point (start point v0) on the detected cow contour f3 in one direction (for example, the scanning direction V1), and sequentially counts the arbitrary number on the contour f3 from the start point v0. Only the skipped points (search points v1, v2,...) Are searched, and line segments (skip contour lines g1, g2,...) Connecting adjacent search points are obtained. For example, the skip outline g1 is obtained as a line segment connecting the start point v0 and the search point v1.

  The specifying unit 304b obtains a singular point where an angle (intersection angle) formed by adjacent skip contour lines (for example, the skip contour line g1 and the skip contour line g2) rapidly changes within a predetermined angle. For example, the specifying unit 304b sequentially obtains the intersection angles of adjacent skip outlines from the start point v0 in the scanning direction V1, and within a predetermined angle from the comparison with the intersection angle obtained in the immediately preceding frame Mn-1. Determine whether the point changes rapidly. When the obtained point does not correspond to a point that changes suddenly within a predetermined angle, the intersection angle of the next adjacent skip outline in the scanning direction V1 is determined. In this example, the search point v5 is determined as a point that changes rapidly within a predetermined angle, and is specified as one of the singular points. If there is no point corresponding to a point that changes rapidly within a predetermined angle and the process returns to the start point v0, it is determined that there is no singular point in the contour, and the auxiliary process is terminated.

  When one of the singular points is specified in the scanning direction, the singular point specifying process in the scanning direction V1 is stopped, and the singular point specifying process is started in the direction opposite to the scanning direction V1 from the start point v0. . When another one singular point is specified in the reverse direction, the singular point specifying process in that direction is also stopped. In this example, the search point -v18 is determined as a point that changes abruptly within a predetermined angle, and is specified as another one of the singular points.

  The auxiliary unit 304 is brought into contact with two cows 91 by a boundary line connecting a set of singular points (in this example, search point v5 and search point -v18) specified by the specifying unit 304b. The represented contour f3 is separated into the respective contours f1 and f2 of the two cows 91 for distinction. The individual contour information P2 is information obtained by updating the contour f3 included in the contour information P1 to the contour f1 and the contour f2 separated by the auxiliary process.

  Here, as an example, the specification unit 304b scans in the scanning direction V1 to identify a singular point, and then scans in the opposite direction to identify a singular point. A set of singularities may be determined. Moreover, according to the method of contact, you may obtain | require not only one set but several sets of singular points.

  In addition, here, as an example, the case where two heads are in contact is taken as an example, but even in the case of contact with three or more heads, each boundary is obtained by specifying singular points, and the contour is separated for each cow 91 To do.

  The monitoring data generation unit 305 generates monitoring data P3 for monitoring the behavior of each cow 91 from the contour of each cow 91 (individual contour information P2). In this embodiment, the monitoring data generation unit 305 generates an approximate graphic (function data or the like) that approximates the contour of the cow to the graphic as the monitoring data P3. Specifically, when one region is divided into a plurality of regions at the boundary by the contour of each cow and the auxiliary unit 304, the monitoring data generation unit 305 performs conversion to approximate each contour to the above-described figure.

  FIG. 11 is an explanatory diagram for approximating the contour of a cow to a figure. FIG. 11A shows the individual contour information P2 of the frame image Mn in which the contour of the cow is indicated by the contour f3 (see FIG. 10A). After separating the contours f1 and f2 by the auxiliary unit 304, the monitoring data generation unit 305 converts the respective contours f1 and f2 into, for example, an ellipse approximation by a least square method. FIG. 11B shows an example of an approximate ellipse obtained by approximating the respective contours f1 and f2 to an ellipse by the least square method. Each approximate ellipse r1, r2 is obtained as data of a unique elliptic function indicating the arrangement of the frame image Mn in the secondary plane coordinate system.

  When the contour of the cow 91 is separated by the auxiliary unit 304 as in this example, a contact flag A indicating contact is added to each elliptic function data. For the contact flag A, for example, “1” or “0” is provided in the first bit.

  The association unit 306 associates the monitoring data P3 with the individual ID of the cow 91. Specifically, the associating unit 306 uses the table operation unit 301 to display an information table of the monitoring camera unit ID of the user's cow 90 and the individual ID of the cow 91 in the cow 90 at the start of the monitoring service. Information obtained from T and associated with the surveillance camera unit ID and the individual ID is set as the tracking information P4. Thereafter, when the number of contours indicating each cow matches the number of individual IDs, the associating unit 306 selects monitoring data P3 (including the contact flag A) corresponding to each cow as one of the individual IDs. Set as the initial value. Then, the monitoring data P3 (including the contact flag A) of each cow 91 obtained in chronological order from the continuous frame images is set in the tracking information P4 as needed. As for the association of the monitoring data P3 after the initial value with the individual ID, for example, priority is given to the one having a small difference (center position difference or the like) from the immediately preceding monitoring data P3 associated with the individual ID. Do. Thereby, in the tracking information P4, the monitoring data P3 of each individual ID in time series order is associated with each individual ID.

  The setting unit 307 separates the contact flag A from the monitoring data P3, and sets the monitoring data P3 and the contact flag A in the recording table T4 by the table operation unit 301. For example, the setting unit 307 triggers the set number of monitoring data P3 set in the tracking information P4, or at all predetermined time intervals (for example, every few seconds), except for the latest monitoring data P3 and the contact flag A. The monitoring data P3 and the contact flag A are set in the recording table T4. In this example, the latest monitoring data P3 is left for use in determining which individual ID is associated with the next monitoring data P3 to be obtained.

  The alarm basic data generation unit 308 generates action information as alarm basic data based on the monitoring data P3 (including the contact flag A when counting the number of times of contact) in the recording table T4. For example, the alarm basic data generation unit 308 sets each individual ID of each monitoring camera unit ID from the recording table T4 after setting by the setting unit 307 or every predetermined time (for example, every several seconds, several minutes, every several hours, etc.). The monitoring data P3 and the contact flag A are extracted by the table operation unit 301, and the behavior information of the cow 91 is generated from the monitoring data P3 and the contact flag A for each individual ID of each monitoring camera unit ID. The behavior information is, for example, the walking distance of the cow 91, the standing time, the sitting time, the lying time, the number of times of contact with other cows 91, and the like. In addition, information representing the behavior of the cow 91 may be set as appropriate.

  In general, the cow 91 repeats a posture in which the knee is lowered and the chest is raised (sitting position), and a posture in which the limb is thrown out and laid down (recumbent). During this repetitive motion, the cow 91 may be unable to stand up from being able to return from the lying position to the sitting position. The state of being unable to stand up may be caused by a fall accident. Such a standing-up state and a recumbent state may cause an accident due to death from suffocation. Therefore, in the present embodiment, lying time is set as behavior information.

  The behavior information of each cow is generated as follows, for example, in units of a predetermined period (in this example, morning, noon, and night). About the standing time and sitting time of one cow 91, the standing or sitting state is determined from the monitoring data P3 of the cow 91 in time series order within the period, and the standing or sitting state is obtained. Find the total duration from. The standing position and the sitting position are determined based on the area of the approximate figure that is the monitoring data P3. For example, the case where it is larger than a predetermined area is set as the standing position, and the case where it is smaller is set as the sitting position.

  In addition, as for the number of times of contact of the cow 91, the contact flag A of the cow 91 in chronological order is obtained as one time from contact until leaving.

  Further, the walking distance of the cow 91 is obtained from the movement locus of the coordinates of the approximate figure of the cow 91 (such as the center coordinates of the approximate ellipse).

  As for the lying time of one cow 91, the lying state is determined from the monitoring data P3 of the corresponding cow 91 in the time series in the period, and the total duration time after that state is obtained. The recumbent state is determined by the ratio of the vertical and horizontal lengths of the approximate figure that is the monitoring data P3. For example, in the case of an approximate ellipse, the lying state of the cow 91 is determined from the ratio of the lengths of the major axis and the minor axis (major axis and minor axis). When the cow 91 is in a lying state when viewed from above, the ratio of the length of the major axis to the minor axis changes greatly. The lying state of the cow 91 is determined from the change in the ratio of the lengths.

  The recumbent state may be determined based on the outline area of the approximate figure that is the monitoring data P3. Here, FIG. 12 is a diagram showing an example of a change in the area of the contour image of the cow. In FIG. 12, the vertical axis represents the area (pixel) of the cow contour image, and the horizontal axis represents time. As shown in FIG. 12, the contour area in the recumbent state is larger than the contour area in the sitting state.

  As other information representing the behavior of the cow 91, for example, regarding the number of times of drinking and the number of times of feeding, the coordinates of the head on the approximate figure of each cow 91 (one of the focal coordinates if it is an approximate ellipse), the drinking fountain and the feed It is calculated from the distance to the field coordinates. A case where the distance falls within a predetermined range and falls outside the predetermined range is determined as one time.

  Thus, the alarm basic data generation unit 308 sets the generated behavior information in the alarm basic data setting table T5.

  The threshold determination unit 309 determines a threshold that considers the behavior of the cow 91 as an abnormal behavior. In this example, one day is divided into a morning period, a noon period, and a night period, and the latest threshold value (upper limit value and lower limit value) is determined for each period. For each cow 91, a threshold is determined for each item of behavior information. The threshold value is determined by performing a moving average based on the basic alarm data for a predetermined number of days (here, the latest several days) of the corresponding item. The determined threshold is set in the threshold information P5 for each individual ID of the cow 91, for each morning, noon, and night period.

  The determination unit 310 determines the abnormal behavior of each cattle 91 (such as losing a mate, estrus, signs of illness, or standing inability) from the behavior information of each cattle 91 in a period included at the time of determination such as the day of determination.

  Specifically, the determination unit 310 determines from the setting information D the stage corresponding to the setting of the “birth date” t22 of the cow 91 for each cow 90 in the cow information registration table T2 (see FIG. 5B). The determination item of the flag “1” is acquired, and further, threshold values (upper limit value and lower limit value) of the determination period are acquired from the threshold information P5. Then, the determination unit 310 uses the table operation unit 301 to search the alarm basic data setting table T5 for each individual ID of each bovine 90, and for each acquired determination item, an individual ID exceeding the respective threshold, Individual IDs exceeding the threshold and their determination items are extracted.

  When the extraction result includes an individual ID that exceeds the threshold, the determination unit 310 determines that the cow 91 has abnormal behavior. The type of abnormal behavior is determined based on the determination items extracted together. For example, when the determination item is “number of times of contact”, since the number of times of contact with other cows 91 is larger than usual, it is determined as an abnormal action indicating “estrus”. When the determination item is “bubo”, since the recumbent time continues, it is determined that the cow 91 is in an “unable to stand” state. The determination method is not limited to this, and may be changed as appropriate. For example, when a plurality of determination items are extracted for one individual ID, the abnormal behavior may be determined by combining them.

  When the determination unit 310 determines that the notification unit 311 is abnormal behavior, the notification unit 311 acquires a destination such as an email address of the corresponding livestock farmer set in the user information registration table T1, and an alarm, an e-mail, or the like To notify the destination of the abnormal signal.

  Next, processing in the analysis server device 30 will be described. The analysis server device 30 receives user registration information from a user terminal, an administrator terminal, or the like as a preparation process for starting a monitoring service, and stores the registration information in the user information registration table T1 (FIG. 5A). (See FIG. 5B) and the individual information registration table T3 (see FIG. 5C). After the setting, the analysis server device 30 receives a signal indicating the start of the monitoring service from the user's terminal, the administrator's terminal, or the like, and performs an analysis process on the behavior of the cow 91 of each cow 90 of the user. Start.

  FIGS. 13-16 is explanatory drawing of the analysis process which analyzes the action | operation of the cow 91 in the cow chamber 90 of the continuous frame image M0 output from one monitoring camera unit 10 of a user. The analysis processing includes tracking processing, alarm basic data generation processing, threshold determination processing, abnormal behavior determination processing, and the like. Hereinafter, an example of each process will be described with reference to FIGS.

  FIG. 13 is a diagram illustrating an example of the flow of the tracking process. First, the reading unit 302 acquires the value (monitoring camera unit ID) of the “cow ID” t21 of the cow 90 of the user who has accepted the start of the monitoring service from the cow information registration table T2 by the table operation unit 301. (S1).

  Subsequently, the reading unit 302 reads frame images for one frame in the order in which the monitoring camera unit ID is captured from the continuous frame image M0 of the transmission source (S2).

  Subsequently, the detection unit 303 determines whether the number of read frames is two or more (S3). When the number of read frames is one (S3: No determination), the process returns to step S2, and the reading unit 302 reads the next frame image in the order of shooting.

  When the number of frames is two or more (S3: Yes determination), the detection unit 303 detects the contour of the cow with respect to the frame image (frame image M2) read later (S4). Specifically, the detection unit 303 identifies a region that has changed from each region having a different depth in the frame image M2 as compared to the immediately preceding frame image (frame image M1) as a cow region, and Detect contours.

  Subsequently, the auxiliary unit 304 performs an auxiliary process for detecting individual contours of the cow 91 for each detected contour (S5). Specifically, the auxiliary unit 304 obtains two singular points from the intersection angle of the skip contour line provided on the contour, and when two singular points are obtained, two heads with the line connecting the two points as a boundary. The individual contours of the cattle 91 are detected.

  Subsequently, the monitoring data generation unit 305 generates monitoring data P3 indicating the behavior of each cow from the detected individual contours of each cow (S6). In the present embodiment, the monitoring data P3 is generated as an approximate figure, and the behavior of each cow 91 can be monitored based on the center position and aspect ratio of the approximate figure.

  Subsequently, the associating unit 306 obtains the value of “individual ID” t31 associated with the value (monitoring camera unit ID) of the “cow ID” t21 from the individual information registration table T3 by the table operation unit 301. Then, the tracking information P4 in which the acquired surveillance camera unit ID and the individual ID are associated is set in the RAM 33 (S7).

  Subsequently, the associating unit 306 determines whether the number of individual IDs set in the tracking information P4 matches the number of monitoring data (S8). When the numbers do not match (S8: No determination), from step S2, the contour of the remaining cow 91 is detected for the next frame image in the order of shooting, and monitoring data is generated. In addition, about the monitoring data of the already detected cow 91, until the monitoring data of the remaining cow 91 are produced | generated, the detection part 303, the auxiliary | assistant part 304, and the monitoring data production | generation part 305 are targeted for each frame image in the meantime. Perform the same processing, and the associating unit 306 updates and manages the latest monitoring data.

  When the number of individual IDs and the number of monitoring data match (S8: Yes determination), the setting unit 307 monitors the monitoring data of each cow 91 for each individual ID with respect to the tracking information P4 (in this case, the initial value). Is assigned and set (S9). In this case, monitoring data of each cow 91 is arbitrarily assigned to each individual ID. In addition, an individual ID corresponding to each cow 91 may be assigned. When assigning the individual ID corresponding to each cow 91, for example, the individual ID of each cow is extracted from the color image by image analysis. As the individual ID, an individual identification number or the like provided on each cow 91 by a stamp or the like in a position that can be read by the monitoring camera and in a size that can be analyzed is used. Then, from the coordinates in the depth image indicated by the monitoring data of each cow 91 and the coordinates of each individual ID in the color image, the individual ID and the monitoring data having the same coordinates are associated with each other. When a color image is not used, for example, an IC tag for each individual ID attached to each cow 91 is read by a reading device fixed at a predetermined position, and the distance and arrangement of each IC tag are determined from the received intensity. Are identified, and the individual ID and the monitoring data are associated with each other based on the identified information. Alternatively, the individual ID may be arbitrarily assigned, and the user or administrator may later rewrite the association of the individual ID. The method of assigning the individual ID corresponding to each cow 91 shown here is an example, and other methods may be used.

  After the above initial value setting processing, the various functions perform the following processing. First, one frame followed by the reading unit 302 is read (S10), and the detection unit 303 detects the contour of the cow with respect to the frame image Mn (S11). The detection unit 303 detects the contour of the cow by comparing it with the immediately preceding frame image Mn-1.

  Subsequently, the auxiliary unit 304 performs an auxiliary process for detecting the individual contour of the cow 91 with respect to the contour of each cow (S12). This process is the same as the process of step S5.

  Subsequently, the monitoring data generation unit 305 generates monitoring data indicating the behavior of each cow 91 from the detected contour of each cow 91 (S13). This process is the same as the process of step S6.

  Subsequently, the associating unit 306 sets the monitoring data of each cow 91 to each individual ID of the tracking information P4 in chronological order (S14).

  Subsequently, the associating unit 306 determines whether the monitoring data associated with the tracking information P4 has reached a predetermined number (S15).

  If the predetermined number has not been reached (S15: No determination), the process is performed on the subsequent one frame from step S10.

  When the predetermined number is reached (S15: Yes determination), the setting unit 307 sets the monitoring data arranged in the time series of the tracking information P4 in the recording table T4 by the table operation unit 301 (S16). Thereafter, the processing is performed on the subsequent one frame from step S10.

  FIG. 14 is a diagram illustrating an example of a flow of alarm basic data generation processing. This process is started at a predetermined timing. For example, it is started when the number of times that the setting unit 307 sets monitoring data in the recording table T4 by the table operation unit 301 reaches a predetermined number. This process is performed for each individual ID of each surveillance camera unit ID.

  First, the alarm basic data generation unit 308 acquires individual ID monitoring data from the recording table T4 by the table operation unit 301 (S21), and generates action information of the cow 91 from the monitoring data (S22).

  Subsequently, the alarm basic data generation unit 308 sets the behavior information of the cow 91 in the alarm basic data setting table T5 by the table operation unit 301 (S23).

  In addition, since it has already demonstrated the production | generation method to action information, description here is abbreviate | omitted.

  FIG. 15 is a diagram illustrating an example of a flow of threshold determination processing. The following processing is performed for each individual ID of each monitoring camera unit ID. First, the threshold value determination unit 309 determines whether the threshold update time has come (S31). The update time is, for example, the switching time between the night and the morning of the next day (for example, AM 4:00) at the end of the day. In addition, it is good also as the switching time of morning and noon, the switching time of day and night, etc. Moreover, you may specify the time of each switching.

  When the update time is reached (S31: Yes determination), the threshold value determination unit 309 acquires the action information for each item for the most recent days of the individual ID from the alarm basic data setting table T5 by the table operation unit 301 (S32). .

  Subsequently, the threshold value determination unit 309 divides the behavior information for several days of the individual ID into items such as a moving average by dividing into morning, noon, and night periods, The threshold of the individual ID is determined for each item of behavior information and for each period (morning period, noon period, night period) (S33).

  Then, the threshold determination unit 309 updates the set of thresholds for each individual ID in the threshold information P5 to the determined value (S34). Note that a predetermined default value is set as the initial value of the threshold.

  If the update time has not been reached (S31: No determination), the process waits until the update time is reached.

  FIG. 16 is a diagram illustrating an example of a flow of abnormal behavior determination processing of the cow 91. The determination unit 310 performs the next abnormal action determination process at a predetermined time interval (for example, every 5 minutes).

  First, the determination part 310 acquires the determination item of the cow 90 which determines abnormal behavior from the setting information D (S41).

  Subsequently, for each acquired determination item, the determination unit 310 acquires a threshold value set for the solid ID of the cow 91 of the cow 90 from the threshold information P5 (S42).

  Subsequently, the determination unit 310 uses the table operation unit 301 to set the action information item in the alarm basic data setting table T5 for the period (morning, noon, or night) including the determination time in the alarm basic data setting table T5. The individual ID of the cow 91 whose other total value exceeds the threshold value of each item acquired from the threshold information P5 is searched, and the individual ID of the cow 91 exceeding the threshold value and its determination item are extracted (S43). Here, for example, when the determination time is AM 9:00 in the morning period, the aggregation value is an aggregation value in the morning period (AM 4: 00 to AM 9:00) on the determination day. Specifically, for each cow 91, the table operation unit 301 compares the total value of the behavior information with the corresponding upper limit value and lower limit value for each determination item, and the total value of the behavior information is the upper limit value and lower limit value. It is determined whether or not abnormal behavior that falls outside the range is included.

  Subsequently, the determination unit 310 determines the abnormal behavior of the cow 91 from the extraction result (S44).

  When the individual ID of the cow 91 and its determination item are extracted (S44: Yes determination), the notification unit 311 acquires a destination such as a mail address set in the user information registration table T1, and uses the destination as the destination. An abnormal signal is notified (S45). At this time, even if the individual ID of the cow 91 determined to be abnormal behavior, the detachment indicated by the determination item, the estrus, the sign of illness, the content of abnormal behavior such as a standing-up state, etc. are notified to the destination Good. Further, a veteran coping method may be stored in a database, a coping method corresponding to the content of the abnormal action may be extracted from the database, and the content may be notified.

  When the individual ID of the cow 91 and the item name of the behavior information are not extracted (S44: No determination), this process is terminated.

  The search for the individual ID of the cow 91 showing abnormal behavior is performed as follows, for example. For example, an individual ID whose sitting time is continued in the behavior information is searched, and the latest figure of the subject such as an approximate ellipse of the individual ID is acquired. Then, if the vertical / horizontal ratio of the figure is out of the predetermined range, or the contour area is larger than the predetermined area, it is determined that the cow 91 is in the lying state. As an example, when the figure is an approximate ellipse, the recumbency of the cow 91 is determined from the ratio of the lengths of the major axis and the minor axis (major axis and minor axis). When the cow 91 is in a lying state when viewed from above, the ratio of the length of the major axis to the minor axis changes greatly. The recumbency of the cow 91 is determined from the change in the ratio of the lengths. If the recumbent state continues for a predetermined time or longer, the cow 91 is determined to be in a state of being unable to stand up from the recumbent position and cannot determine to have abnormal behavior.

  In addition, the determination unit 310 searches for an individual ID having a large number of contacts in the behavior information of the most recent predetermined period, and the movement distance of the individual ID for the most recent predetermined period is greater than that of the other cattle 91. When the number of times of eating is less or when standing time is longer than other cows 91, it is determined that it is estrus, and it is determined that there is an abnormality.

  In addition, the determination unit 310 searches for an individual ID with extremely few contacts in the behavior information for the most recent predetermined period, determines that the cow 91 is out of the group, and determines that there is an abnormality.

  In addition, the determination unit 310 searches for an individual ID whose movement distance is extremely small in the behavior information of the most recent predetermined period, and determines a sign of illness from the number of times of drinking or feeding during the most recent predetermined period of the individual ID. It is determined that there is an abnormality.

  In this embodiment, monitoring data is generated for the monitoring target cow, and abnormal behavior is determined using the monitoring data. However, environmental information (for example, information such as temperature and weather) is further added to the monitoring data. May be included. For example, a temperature sensor is connected to the monitoring camera 10a, and the temperature information in the cow chamber detected by the temperature sensor is output to the cloud through the controller PC. The weather information is acquired from, for example, a weather database that provides weather information in time series. Abnormal behavior is determined by including such environmental information in the behavior information indicating the behavior of the cow.

  As described above, in this embodiment, since it is not necessary to directly attach a sensor to the cow to be monitored, the burden on the cow can be reduced or eliminated. Moreover, monitoring data including various information on cattle can be accumulated at any time, and slight changes in behavior of cattle can be found from the accumulated monitoring data in real time or with a small time lag. For this reason, if an abnormal behavior is observed in a cow, it is possible to immediately notify the livestock farmer of the cow breeder, and to promptly respond to the cow. If it is a cow in a state where it cannot stand, the cow's upper body is raised, and if it is a cow that shows signs of illness, the death rate of the cow can be reduced by treating it early.

  Inexperienced new farmers are also notified of cattle abnormalities if they have abnormal behaviors of cattle, so they are less aware of changes in cattle behavior and are not aware of changes in cattle behavior. The scrap rate can be reduced. Moreover, since it is possible to notify the subject method of the expert by e-mail by setting, it is possible to perform the same countermeasures as the expert.

  In the present embodiment, the detection unit 303 mainly corresponds to “first detection means”. The auxiliary unit 304 mainly corresponds to “second detection means”. The monitoring data generating unit 305, the associating unit 306, the setting unit 307, the alarm basic data generating unit 308, etc. mainly correspond to “generating means”. The threshold determination unit 309, the determination unit 310, and the like mainly correspond to “determination means”. The notification unit 311 mainly corresponds to “notification means”. The correspondence relationship between each means and each function is not limited to this. Combinations may be changed as appropriate according to the configuration of the information processing system, some functions may be replaced with other functions, some functions may be deleted, and other functions may be added.

  In this embodiment, an example is shown in which the CPU loads the HDD program into the RAM to realize the function. However, a part or all of the function may be configured by hardware such as an ASIC (Application Specific Integrated Circuit). Good.

(Modification)
Moreover, you may deform | transform so that a part or all of the said function can be performed by AI (artificial intelligence) which is a "machine learning means." For example, processing such as the alarm basic data generation unit 308, the threshold value determination unit 309, and the determination unit 310 is performed by AI.

  For the alarm basic data generation unit 308, processing for generating action information from the monitoring data acquired by the alarm basic data generation unit 308 is performed by AI. For example, monitoring data is given to AI as an input value, and machine learning is repeated at AI. From AI, an output value of action information calculated by a method in which an error in determination of abnormal action is optimized to a minimum is obtained.

  The threshold value determination unit 309 performs threshold value determination processing using AI. For example, action information is given to AI as an input value, and machine learning is repeated at AI. From the AI, a threshold output value calculated by a method in which an error in determination of abnormal behavior is optimized to a minimum is obtained.

  About the determination part 310, AI is utilized in the determination process of abnormal action. The result of the determination item is a calculation method in which errors such as the result of the abnormal action determination item are optimized to a minimum by giving the AI action information within the determination time period as an input value and repeating machine learning. An output value indicating is obtained.

  Note that a series of processes in which any two or more of the alarm basic data generation unit 308, the threshold value determination unit 309, and the determination unit 310 are combined may be performed by AI.

1 Information Processing System 10 Surveillance Camera Unit 10a Surveillance Camera 10b Control PC
20 Cloud 21 Storage 30 Analysis server device 40 Management PC
50 User terminal 60 Network 70 Abnormal signal 80 Management building 90 Ushibo 90b Breeding area

Claims (12)

A spatial sensor that outputs spatial information of the monitoring area;
First detection means for detecting object information of a plurality of moving objects to be monitored from the spatial information output from the spatial sensor;
Second detection means for detecting arrangement information of moving objects in contact with each other among the plurality of moving objects in the object information detected by the first detection means;
Generating means for generating behavior information of the plurality of moving objects from the object information and the arrangement information;
Determination means for determining whether the action of the moving object is an abnormal action from the action information;
Notification means for notifying abnormality when it is determined that the action of the moving object is an abnormal action;
An information processing system comprising: The object information is contour information indicating contours of the plurality of moving objects,
The second detection means detects one set or a plurality of sets of singular points included in the outlines of the plurality of moving objects, and the respective outlines of the moving objects in contact with each other are detected. Detect placement information that distinguishes the boundary connecting points,
The information processing system according to claim 1. In the spatial information output from the spatial sensor at a predetermined frame rate, the singular point has a predetermined angle within a plurality of frames formed by adjacent line segments connecting two adjacent points on the contour. A point that changes within an angle
The information processing system according to claim 2. Having correspondence information associating stage information of the plurality of moving objects and the determination item of the abnormal behavior;
The determination means determines, based on the correspondence information, whether or not the action of the moving object is an abnormal action from the action information for a determination item of abnormal action corresponding to the stage information of the plurality of moving objects.
The information processing system according to claim 1, wherein the information processing system is an information processing system. The determination unit compares a threshold value indicating abnormal behavior determined based on behavior information for a predetermined number of days nearest to the plurality of moving objects and the behavior information on the determination day of the determination unit, and determines the abnormal behavior. Determine if there is,
The information processing system according to claim 1, wherein the information processing system is an information processing system. The generating means converts the object information and the arrangement information into an approximate figure approximating an outline of the moving object, and the time-series order corresponding to the spatial information output from the spatial sensor at a predetermined frame rate. Generating the behavior information based on the approximate figure,
The information processing system according to any one of claims 1 to 5, wherein The determining means determines that the plurality of moving objects are in a state in which they cannot stand up based on a ratio of a length of a major axis to a length of a minor axis of the approximate figure.
The information processing system according to claim 6. The determination unit determines that the plurality of moving objects are in a state where they cannot stand up based on a contour area of the approximate graphic.
The information processing system according to claim 6. The generation means generates contact number information as the behavior information based on the detection of the arrangement information by the second detection means.
The information processing system according to claim 1, wherein the information processing system is an information processing system. Machine learning means for performing machine learning in a part of the generation means or the determination means;
The information processing system according to any one of claims 1 to 9, wherein First detection means for detecting object information of a plurality of moving objects to be monitored from space information output from the space sensor;
Second detection means for detecting arrangement information of moving objects in contact with each other among the plurality of moving objects in the object information detected by the first detection means;
Generating means for generating behavior information of the plurality of moving objects from the object information and the arrangement information;
Determination means for determining whether the action of the moving object is an abnormal action from the action information;
Notification means for notifying abnormality when it is determined that the action of the moving object is an abnormal action;
An information processing apparatus. Computer
First detection means for detecting object information of a plurality of moving objects to be monitored from space information output from the space sensor;
Second detection means for detecting arrangement information of moving objects in contact with each other among the plurality of moving objects in the object information detected by the first detection means;
Generating means for generating behavior information of the plurality of moving objects from the object information and the arrangement information;
Determination means for determining whether the action of the moving object is an abnormal action from the action information;
Notification means for notifying abnormality when it is determined that the action of the moving object is an abnormal action;
Program to make it function.