JP2001175868A - Method and device for human detection - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a human detecting method and its device which can discriminate mutually hidden or adjacent persons when detecting respective persons in a picked-up image and detect persons even if the direction and size of an object varies and even the same person have large shape variation in a moving picture. SOLUTION: Image feature output by an edge detection operator which has a gap expanded is made to correspond to a model shape set on the basis of the head of a person or the outline of the upper half part of the body which are hardly hidden and has small shape variation, and the maximum point of a product at each point is detected as a human area. Thus, the influence of a fine difference in outline shape is reduced and adjacent objects are separated and detected. This mentioned process is performed for an image obtained by photographing the space from above and persons in the obtained visual field are counted and totalized to actualize various information services such as analysis of a store customer distribution.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method and apparatus for detecting a person in an image, and more particularly to an image processing method and apparatus for detecting each person from an image of a large number of persons obtained in a store or a public facility. The present invention relates to an image processing method and apparatus for performing counting and displaying the degree of congestion.

[0002]

2. Description of the Related Art When detecting a person region from an image obtained by capturing a plurality of persons, conventionally, a background difference, an inter-frame difference,
Various approaches have been taken, such as edge detection, optical flow calculation, pattern irradiation with infrared spot light, and the use of range images by the stereo method. Alternatively, an energy model using an edge such as a snake as a feature amount, a method using color information such as skin color detection, or a method using spatiotemporal image analysis are also being studied.

As an example of counting a plurality of persons in an image using these techniques, Japanese Patent Application Laid-Open No. 9-281605 discloses a technique of detecting a person photographed on a photographic film using a skin color detection process and counting the number of persons counted. Discloses an apparatus for automatically determining the number of additional prints in accordance with the number of prints. As another example, JP-A-8-123935 discloses a method in which a plurality of counting lines are provided in an image, and a spatio-temporal image is created by connecting images obtained by performing background subtraction in the time direction. A method of determining a passing direction from a moving object region in the inside is disclosed.

[0004] Further, as an example of measuring the density of a person, Japanese Patent Laid-Open No. 1-244598 discloses a method in which a person image is extracted by means of a background difference, and when an unnatural distribution bias or an undesired density state is extracted, a warning is issued. Is disclosed.

[0005]

However, the above-mentioned method cannot deal with concealment between regions generated between a plurality of persons, and it is difficult to distinguish and detect persons imaged adjacently. There is a problem that it is difficult to apply the method to an image of a person.

For example, in the case of background subtraction, a plurality of overlapping persons in an image are cut out as one area, so that each person cannot be detected individually. In the case of a system that requires counting, a process for estimating the approximate number of people is usually performed by dividing the area of the extracted region by the area per person. However, in the conventional background subtraction method, the overlapping area of the persons in the image, the size of each person, etc.
Accurate counting was difficult due to large fluctuation factors.

In a situation where a large number of people are imaged, the orientation, posture, and size of the target with respect to the camera are various, and even if the same person, the shape change in a moving image is large. It was difficult to apply.

An object of the present invention is to detect and count a person in an image obtained by capturing a large number of persons, and measure the degree of congestion in a situation where there is a difference in shape or size of each object or concealment between objects exists. The present invention also provides a processing method and apparatus that can perform the above-mentioned processing and has less restrictions on imaging conditions. Furthermore, the processing method and the apparatus are used to continuously perform person counting and congestion degree measurement. Another object of the present invention is to realize an apparatus for providing various information services to a user.

[0009]

In order to solve the above-mentioned problems, a person detecting method according to the present invention is adapted to correspond to a plurality of preset points on a person shape model having a shape based on a person contour. A feature detector for detecting a feature of a person to be detected is arranged at a relative position to be configured to form a feature detector set, and the feature detector set is set to each coordinate set on the image plane. At each corresponding position, the image feature amount of the acquired image is detected, and the detection is performed based on the value of the image feature amount detected by the feature amount detector configuring the feature amount detector set at each position. A predetermined operation is performed to determine that the image feature amount is a feature amount of an image having a shape that matches the human shape model within a predetermined allowable range, and the operation result is detected. The image feature amount is In the case shown to be a feature quantity of an image having a shape that matches the Le estimates the human image to the position exists.

In the above-mentioned predetermined calculation, the product of the feature detected by each feature detector at each position of the feature detector set is calculated, the product is compared with a predetermined threshold, and the product of the product is calculated. When the value exceeds the predetermined threshold, the processing includes determining the image as a human image at the position.

In one embodiment of the above-described predetermined calculation, the predetermined calculation is performed by setting a product of the characteristic amounts detected by the respective characteristic amount detectors as an evaluation value, the image corresponding to the position of the characteristic amount detector set. An evaluation value image is generated in association with the coordinates of the surface, a local region centered on each pixel in the evaluation value image is set, a local maximum point at which the evaluation value becomes maximum is searched for in the local region, and the local maximum point is found. Among them, the processing includes the process of setting the pixel at the center of the local area where the local maximum point where the evaluation value exceeds the predetermined threshold exists as the point where the person exists.

The calculation of the product of the feature values detected by the respective feature value detectors is performed by performing, for each coordinate on the image plane, the process of calculating the product of the displacement feature values corresponding to the respective feature value detectors. Is done. However, each of the feature detectors arranged at relative positions corresponding to the positions of a plurality of preset points on the person shape model is used to detect feature amounts corresponding to preset points on the person shape model. A displacement vector from the positions of the plurality of preset points on the human shape model to a reference point arbitrarily set in the human shape model is defined as a feature amount detector corresponding to the plurality of points. The feature amount detected by each feature amount detector is associated with the coordinates of a position displaced by a feature amount displacement vector from the position where the feature amount is detected, and the feature amount detection vector is defined. Is defined as a displacement feature at the coordinates corresponding to the container.

[0013] A person detecting apparatus according to the present invention includes: an image acquiring apparatus for acquiring an image frame photographed by a camera for photographing an area where a large number of persons are present or moving to create an image capable of image processing; Holding and extracting an image having a shape that matches the human shape model from the image that can be processed, and each extracted person exists as a person whose image is captured in the image plane. A person detection device that outputs coordinates of a point representing a position on an image plane.

[0014] The person detecting device includes a feature amount detector set including a set of feature amount detectors arranged at relative positions corresponding to a plurality of preset points on the person shape model, The feature amount detector set detects the image feature amount for each point on the image surface obtained by the image acquisition device at a relative position corresponding to each position on the human shape model, and calculates the detected image feature amount. A feature value calculating unit that outputs the feature value in association with the coordinates of the image plane; and a value of an image feature value set including a set of image feature values detected at the relative position by each feature value detector of the feature value detector set. To evaluate the degree to which the image feature set appropriately matches the human shape model, and generate an evaluation value corresponding to the position of the feature detector set on the image plane. An evaluation value calculation unit that generates an evaluation value image representing an evaluation value corresponding to each coordinate of the evaluation value image calculated by the evaluation value calculation unit. A maximum point search unit that searches for an evaluation value maximum point at which a corresponding evaluation value is maximum in a nearby area, and outputs the obtained coordinates of the evaluation value maximum point as a point where a person exists.

The feature amount calculation unit includes an operator set including a set of edge detection operators as a feature amount detector set, and the operator set uses the operator set at a relative position corresponding to a plurality of positions on the contour of the human shape model. The image contour is detected for each point on the image plane obtained by the image acquisition unit, and the detected value of the detected image contour is output in association with the coordinates of the image plane.
As a predetermined operation on the value of the image feature amount set,
For each coordinate on the image plane, the product of the displacement feature values for the edge detection values detected by the respective edge detection operators is calculated, and the local maximum point searching unit calculates each pixel in the evaluation value image obtained by the evaluation value calculation unit. Set a local area centered on, search for a local maximum point where the evaluation value is maximum in the local area,
The position of the pixel at the center of the local area where the local maximum point where the evaluation value exceeds a predetermined threshold value is output as the position of the point where the person exists.

Regarding the function of the evaluation value calculating section, a displacement vector from a position of a plurality of points set in advance on the human shape model to a reference point arbitrarily set in the human shape model is determined. A feature displacement vector of the edge detection operator at the relative position corresponding to the position is defined, and the edge detection value detected by each edge detection operator is displaced by the feature displacement vector from the position where the edge detection value is detected. In association with the coordinates of the position, it is defined as a displacement feature amount at the coordinates corresponding to the edge detection operator.

The evaluation value calculation unit has, as feature amount detectors, an edge detection operator having a small detected edge width and an edge detection operator having a large detected edge width. It is a gap expansion operator that expands the area width of the central gap without weight so that a stable detection value is output even if there is a slight change.The edge detection operator is less likely to cause occlusion and has less change in shape. It is arranged at a relative position corresponding to the part of the person.

The person detecting device of the present invention further has a person counting device for counting the number of persons present in the area being photographed based on the detection result obtained by the person detecting device. The person counting device stores a parameter value necessary for estimating the position of the person in the real space, or a lookup table describing pixels and the spatial position of the person, and initializes the measurement in advance before performing measurement. The image and position correspondence storage unit, which stores each correspondence between each pixel in the image and the spatial position and stores the correspondence in advance, and the point where the person output by the person detection means exists in the image, performs calibration in the image. The position of the head of the person image is regarded as the position of the person, and the position of the person in the space is estimated based on the information in the image / position correspondence storage device 42. A person density measuring unit that measures the number of people in the micro area based on the calculated person density.

As one of the above-mentioned calibrations,
The relationship between the positions of the object points arranged at five predetermined spatial positions and the image points obtained by imaging the object points on the image plane is represented by the global coordinates of the camera position and the posture of the camera. This is performed by describing five angles using one angle (elevation angle and azimuth angle) as a parameter, and calculating two angles representing the global coordinates of the most probable camera position and the attitude of the camera by the least squares method.

The person detecting apparatus according to the present invention may include a counting result storage device for sequentially recording the person density counted by the person counting device with the time at which the image was taken.

The person detection device of the present invention is a counting result display for processing and displaying the counting result obtained by the person counting device or the counting result of the past time stored in the counting result storage device as necessary. A counting result display device, based on the recorded data, a number of customers storage unit for storing the number of customers per unit time over all business hours, and a store visit per unit time stored in the counting result storage device. Based on the number of customers and the number of customers stored in the number-of-purchasers storage unit, the purchase rate, that is, the number of customers /
A purchase rate display unit for counting the number of customers visiting the store. As another embodiment of the counting result display device of the present invention,
The counting result display device stores the number of employees working at each time, the number of employees stored in the counting result storage means, the number of customers visiting the store per unit time, and the number of employees stored in the number of employees storing unit. Based on predetermined conditions, the number of customers per employee, that is, the number of customers coming to the store / the number of employees, can be totaled and displayed, and an average number of customers receiving display unit can be provided.

As another embodiment of the counting result display device of the present invention, the counting result display device is estimated from the number of used cash registers storing the number of cash registers used at each time and the number of customers lined up at the cash registers. The number of cash registers used is compared with the number of cash registers stored in the number of cash registers used, and whether the number of cash registers used at each time is greater than or less than the required number is counted according to a predetermined condition. And a display unit for displaying the number of cash registers.

As another embodiment of the counting result display device of the present invention, the counting result display device includes an in-store employee storage unit for storing the number of employees arranged in the counter at each time; Comparing the optimal number of employees estimated from the number of customers in the number of employees with the number of employees stored in the number of employees storage device,
It is possible to have a consultation section for the number of employees in a sales floor, which sums up and displays whether the number of employees arranged at each time is more or less than a required number according to a predetermined condition.

As still another embodiment of the counting result display device according to the present invention, the counting result display device includes a unit time processing number storage unit for storing the number of persons handled by each lift or attraction facility per unit time, and a lift or attraction facility. At the time of imaging, the customer waits for the service of the facility at the imaging time from the recorded number of customers who are lined up and the number of people that each lift or attraction facility handles per unit time stored in the unit time processing number of people storage unit. A waiting time display unit for calculating the number of customers / the number of people that the facility can handle per unit time) and displaying the calculated waiting time on a bulletin board or the like provided at each point inside or outside the game facility.

[0025]

The feature amount detectors constituting the feature amount detector set each generate a feature amount image, but each of the feature amount detectors is shifted to a position shifted by the relative position of the feature amount detector. Generate Therefore, when each feature detector generates a feature image on a separate image plane, the image plane corresponding to each feature detector is translated in a direction opposite to the relative displacement of the feature detector. When they are overlapped (combined), the same part of the feature image on each image plane overlaps.
However, different feature amount images of each image plane or different portions of the same feature amount image (hereinafter, referred to as different feature amount image portions) have their feature amount image portions even after the above-described parallel movement / synthesis processing. If they are not at a special relative position that overlaps by the parallel movement / synthesis processing, they will not overlap. In other words, only the feature amount image portions having the same relative arrangement as the feature amount detectors, that is, only the feature amount images matching the human shape model are simultaneously overlapped by the parallel movement / synthesis processing. According to the present invention, only the feature amount image of a person image is extracted from the feature amount image plane according to this principle.

The human shape model used at this time is set with the average size of the person in the image based on the contour edge information of the head and the upper body of the person whose concealment is less likely to occur and whose shape change is small. Is done. In addition, as the feature detector, a small gap-expanded edge detection operator is used for a small head, and a large gap-expanded edge detection operator is used for a large upper body contour. A person can be reliably detected.

By performing the processing using the above means, a plurality of human positions existing in the field of view of the camera are detected,
In addition, counting can be performed. In addition, since there are few restrictions on photographing conditions such as camera movement and lighting, a flexible system configuration using existing surveillance cameras and portable cameras is possible, and a wide variety of technical developments are possible.

[0028]

Next, embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is an overall configuration diagram showing the configuration of a person detection device for implementing the person detection method of the present invention. The human detection device of the present invention basically includes:
It comprises a camera 1, an image acquisition device 2, a person detection device 3, a person counting device 4, a counting result storage device 5, and a counting result display device 6.

The camera 1 is installed above a target space where a large number of people are present, and captures an image including a group of people with a single camera or by switching a plurality of cameras.
The image acquisition device 2 switches the image frames captured by the camera 1 as necessary, captures the images at predetermined time intervals, and performs well-known processing such as discretization to obtain a still image that can be subjected to image processing.

The person detecting device 3 holds a contour shape model of a person, extracts an image having a shape matching the contour shape model within a predetermined allowable range from the image captured by the image acquiring device 2, Each extracted image is taken as a person captured in the image, and coordinates of a point (hereinafter, referred to as a representative point) representing a region where the person is present are output on an image plane.

The person counting device 4 counts the number of persons existing in the field of view in the image from the person detection result obtained by the person detecting device 3. The counting result storage device 5 sequentially records the number of persons counted by the person counting device 4 in association with the time at which the image was captured. The counting result display device 6 displays the person counting result in the image counted by the person counting device 4 or the person counting result at the past time accumulated in the counting result storage device 5 as necessary. I do.

Next, the operation of this embodiment will be described. FIG.
5 is a flowchart illustrating a processing routine in the present embodiment. An image frame including a large number of people in the target space captured by the camera 1
Is switched as necessary, and is taken into the apparatus as a still image that can be discretized and processed at predetermined time intervals (step A1). For this image, the person detection device 3 performs a process of detecting each person included in the image (step A2). The coordinate sequence indicating the person area obtained by the person detecting device 3 in the process of step A2 is counted by the person counting device 4 (step A3). The counting result is associated with the time at which the image was captured in the counting result storage device 5 as the estimated number of persons captured in the image, stored (step A4), and output to the counting result display device 6. .

The person counting result counted by the person counting device 4 and the person counting result at the past time stored in the counting result storage device 5 are displayed on the counting result display device 6 as necessary or by the operator. According to the request of
For example, it is displayed after various processes such as a counting result graph with respect to time or a superimposed display on the acquired image (steps A5 and A6).

After the above process is completed, if an input image is further provided, the process returns to step A1 and the same process is repeated for the next obtained image (Y in branch A7). Ends the processing (branch A
7 N).

Next, an embodiment of the person detecting section 3 shown in FIG. 1 will be described. FIG. 3 is a block diagram showing a configuration of one embodiment of the person detection unit 3. The person detecting apparatus according to the present embodiment includes a feature amount calculating unit 301, an evaluation value calculating unit 302, and a local maximum point searching unit 3.
03.

The feature amount calculation unit 301 extracts a predetermined number (the number of types necessary for detecting a person) of image features for each point in the image obtained by the image acquisition device 2.
Each image feature is output in association with the coordinates in the image.

The feature amount calculation unit 301 holds a model of a shape based on a human contour, that is, a human shape model.
There is provided a set of feature detectors which are respectively arranged at a plurality of preset points on the human shape model. Hereinafter, this one set of feature amount detectors is referred to as a feature amount detector set. Therefore, placing the human shape model at a certain position in the image frame is equivalent to placing the feature detector set at that position. As described in the examples below,
When the outline of a person image is identified from the outline of another image, the feature amount detector set includes feature amount detectors (for example, edge detection) that are arranged at a plurality of positions of the outline of the human shape model and identify the outline. Operator). Further, for example, when an image of a person wearing a yellow hat such as a schoolchild is identified from other images, in addition to a plurality of feature amount detectors for identifying contours, a head of a person shape model is used. A feature detector for detecting yellow is disposed at a predetermined position. As described above, in the feature amount detector constituting the feature amount detector set, different types of feature amount detectors are set at different positions according to the features of the person to be extracted.

This person shape model (specifically, a set of feature detectors arranged in the person shape model) is placed at each position of an image area where a person is to be detected in a still image frame to be inspected, and a person image is placed. At the time of extraction, each feature amount detector constituting the feature amount detector set detects a person feature amount and other background image feature amounts at a relative position corresponding to each part on the person shape model.
The evaluation value calculation unit 302 detects each of the feature amount detectors in the feature amount detector set while detecting the relative position while maintaining the relative position.
A predetermined operation is performed on the values of a set of image feature amounts (hereinafter, referred to as an image feature amount set) to evaluate the degree to which the image feature amount set appropriately matches the human shape model. An evaluation value corresponding to the current position on the image frame is generated. The aforementioned “appropriately” means “within a detection allowable range” of the feature amount detector. The “position of the human shape model on the image frame” is
The coordinates of the arbitrarily set representative point of the human shape model on the image frame. In this way, the evaluation value calculation unit 302 generates an evaluation value image representing an evaluation value corresponding to each coordinate of the image frame.

The local maximum point searching section 303 includes an evaluation value calculating section 30
For each pixel of the image frame, the evaluation value corresponding to the coordinates of the pixel is searched for an evaluation value maximum point at which the evaluation value becomes the maximum in a nearby region,
The coordinates of the obtained evaluation value maximum point are output as a point where a person exists.

There are various embodiments in which the evaluation value calculation unit 302 evaluates whether or not the human shape model properly matches the human image. As one embodiment, there is a method of scanning a human shape model on an image frame. In this case, if all of the values of the feature set detected by the feature detectors of the feature detector set “at substantially the same time” are equal to or greater than a predetermined threshold, the human shape model is properly matched to the human image. Is determined to be Here, "substantially simultaneously" means "simultaneously" within a time width obtained by dividing the above-mentioned permissible detection range by the scanning speed. Further, as in a method used in an embodiment to be described later, a feature amount image generated by each of the feature amount detectors forming the feature amount detector set corresponds to a relative position between the feature amount detectors. There is also a method of combining by displacing by a displacement vector (feature amount displacement vector).

Next, a person detection process (step A in FIG. 2)
2) will be described with reference to the flowchart of FIG. In the captured image, first, the feature amount of each point is calculated in the feature amount calculation unit 301 (step B1). Next, for the obtained feature amount image, a person shape model is placed at each position on the image frame in the evaluation value calculation unit 302,
From a set of feature amounts obtained at a plurality of preset points on the person shape model, a feature amount evaluation value at each of the positions is calculated, and the feature amount evaluation value is set to coordinates on an image frame corresponding to the position. It is output as a feature amount evaluation value image in association with it (step B2).

The obtained evaluation value image is obtained by
In 3, a process of extracting a point where a person is present in the image is performed. Specifically, a local area centering on each pixel in the image is set, and a local maximum point with the maximum evaluation value is searched for in the local area (step B3). The coordinates of the center of the local area where the maximum point is obtained are output as points where the person exists (step B4).

FIG. 5 is a block diagram of an embodiment of the person detecting device of FIG. The person detecting device of the present embodiment includes an image differentiating unit 31, an evaluation value image combining unit 32, and a local maximum point searching unit 33. The person detecting apparatus according to the present embodiment uses an edge detection operator as a feature amount detector of the person detecting unit in FIG. 3, and sets a feature amount detector at a plurality of preset points on the contour of the person shape model. Each is composed of a fixed set of edge detection operators. The image differentiating unit 31, the evaluation value image combining unit 32, and the local maximum point searching unit 33
The feature amount calculation unit 301 and the evaluation value calculation unit 30 in FIG.
2. This is an embodiment of the local maximum point search unit 303.

The image differentiating section 31 includes a set of edge detection operators configured to maintain a relative position corresponding to a predetermined position on the contour of the human shape model. The edge detection operator is referred to as an operator set). In this embodiment, the operator set is configured using two types of edge detection operators, large and small, for detecting edges in the x and y directions. this house,
Large edge detection operators, like the upper body contour,
A small edge detection operator is located at a position corresponding to a relatively large and flat portion, and a small edge detection operator is located at a position corresponding to a relatively small and relatively large curvature, such as a contour of a head. You. Each edge detection operator is applied to the person image and the background image obtained by the image acquisition unit 2 and creates a differential image of these images.

Although the edge detection operators constituting the operator set each generate a differential image, each edge detection operator generates a differential image of the same image at a position shifted by the relative position of the edge detection operators. Therefore, when each edge detection operator generates a differential image on a separate screen, the screen corresponding to each edge detection operator is moved in parallel in the direction opposite to the relative displacement of the edge detection operator and is overlaid (combined). And the same part of the differential image on each screen overlaps. However, even if the different differential images on each screen or different portions of the same differential image (hereinafter, referred to as different edge portions) are subjected to the above-described parallel movement / synthesis processing, the edge parts are not moved by the parallel movement / synthesis processing. If they are not in a special relative position, they will not overlap. In other words, only the edge portion having the same relative arrangement as that of the edge detection operator, that is, only the image that matches the human shape model is simultaneously overlapped by the parallel movement / synthesis processing. In the present embodiment, only the differential image of the human image is extracted from the differential image plane based on this principle.

The evaluation value image synthesizing section 32 converts the differential image corresponding to each edge detection operator obtained by the image differentiating section 31 into a displacement vector of the edge detection operator with respect to a reference point arbitrarily set on the human shape model surface. In other words, the product of the edge value (displacement feature value) of the parallel-transferred differential image at each coordinate in each differential image plane is evaluated as the evaluation value at that coordinate. Create an image. The local maximum point searching unit 33 sets a local region centered on each pixel in the evaluation value image obtained by the evaluation value image synthesizing unit 32, searches for a local maximum point having the maximum evaluation value in the local region, Among them, the position of the pixel at the center of the local area where the maximum point exceeding the predetermined threshold exists is output as the position of the point where the person exists.

FIG. 6 is a diagram showing an edge detection operator used in this embodiment. This operator is Japanese Patent Application Hei 1
1-152381 is an edge detection operator. As shown in FIG. 6, as compared with the conventional Sobel operator, the width of the gap region having no weight at the center of the operator is expanded, and the size of the operator is also several pixels × several pixels. Greater than. The reason for expanding the width of the gap region having no center weight is that the shape changes depending on the person, and even for the same person, the image differs depending on the direction to the camera, so the position and direction of the edge detected by the operator This is to allow an image to have a tolerance so as not to detect such a change in the shape of the person or a change in the image due to the orientation with respect to the camera.

Such an edge detection operator has two types, large and small, in each of the x direction and the y direction.
Seeds are available. These operators are used according to the shape of the contour of the image, as described later with reference to FIG. By applying the operator set composed of the edge detection operators to each position of the entire image obtained by the image acquisition device 1 of FIG. 1, a differential image having a differential value at each coordinate as a pixel value is created. .

FIG. 7 is a diagram showing the arrangement of the person shape model and the edge detection operator according to the present embodiment. A small operator for detecting an edge in the y direction is arranged at a point A corresponding to the top of the person contour shown in FIG. 7, and an edge in the x direction is detected at points B and C corresponding to the left and right of the head. Small operators are arranged. At points D and E corresponding to the right and left sides of the body, large operators for detecting edges in the x direction are arranged. In this way, a small operator is placed on a contour with a small shape like the head, and a wide linear contour is placed on a part with a relatively large and flat contour like the contour of the body. A large operator to detect is arranged. 5 arranged as in FIG.
The operators operate as an operator set while maintaining this relative position.

Next, the person detecting process in this embodiment will be described. FIG. 8 is a flowchart illustrating the flow of the person detection process according to the present embodiment. The image captured by the image acquisition device 2 (see FIG. 1) is first converted by an image differentiator 31 (see FIG. 5) into the x direction and y constituting an operator set.
Differentiation processing is performed by the edge detection operator in the direction (step C1). Next, the differential images (five differential images in the embodiment of FIG. 7) created by each edge detection operator are preset in the evaluation value image combining unit 32 according to the shape and size of the person in the image. Are translated and combined in accordance with the human shape model as shown in FIG.

More specifically, first, a point A corresponding to the top of the person's contour shown in FIG. Points B and C corresponding to the left and right correspond to differential images obtained by a small operator that detects edges in the x direction, respectively, and points D and E corresponding to the body left and right detect large edges in the x direction, respectively. Are associated with the differential images obtained by the operator.

Next, from each point in the above model, the reference point O
Each differential image corresponding to each point is translated using the displacement vector toward (step C2). In the embodiment of FIG. 7, the differential image of the small operator in the y direction corresponding to the crown A is displaced by a vector toward the head center point O serving as a reference point. That is, the image is translated downward in the image (this translation is calculated by coordinate transformation).

Further, the differential images that have been translated are superimposed, and the product of the pixel values (displacement features) of the pixels existing at the same coordinates is calculated for all the coordinates in each image, and the product is calculated at the coordinates. An evaluation value image is synthesized as a pixel value (step C3).

The evaluation value image obtained by the evaluation value image synthesizing unit 32 is subjected to a process of extracting a point where a person is present in the image in a local maximum point searching unit 33. Specifically, a local region centering on each pixel in the image is set, and a local maximum point at which the evaluation value of the pixel is maximum is searched for in the local region (step C4). Among them, the coordinates of the pixel at the center of the local area having an evaluation value exceeding a predetermined threshold are output as a point where a person exists (step C5).

FIG. 9 is a conceptual diagram for explaining an embodiment of the person counting device 4, the counting result storage device 5, and the counting result display device 6 of FIG.

In this embodiment, first, in an environment in which the position and orientation of the camera are completely fixed, as shown in FIG. 9, one area to be monitored is included in an image acquired by the camera. One or more. And FIG.
It is determined whether or not the position of the person's head obtained by the person detection device 3 is within each observation area, and the number of persons is output for each observation area.

[0057] The treatment, with the person counting device 4, a collection U = {u _i of the pixel think that the center position of the head of the person obtained by the human detection device 3, v _i | i is the total number of 0 - The integer｝ up to 1 is calculated in each person observation area V _j = ｛(u, v) | t
b _j ≤ u ≤ tt _j , tl _j ≤ v ≤ tr _{j ｊ} (j is an integer from 0 to the total number of human observation areas −1, tb _j , tt _j , tl _j ,
tr _j is a downward direction, an upward direction, and a j-th area, respectively.
It is determined whether the values are within the left and right direction coordinate thresholds), and the number of persons included is determined for each region and output. The person counting device 4 counts the number of persons in each set area from the detection result obtained by the person detecting device 3.

The counting result storage device 5 associates the number of persons counted by the person counting device 4 with the time at which the image was taken for each set area and sequentially records the number. The counting result display device 6 displays the person counting result in the image counted by the person counting device 4 or the person counting result at the past time stored in the counting result storage device 5 according to each set area as necessary. And display it.

According to this embodiment, for a large facility used by many people such as a concert hall, a ball game facility, an art museum, etc., each entrance or the periphery of an exhibit is set in advance as a processing target area and gathered around the entrance. Man,
This makes it possible to count the number of people lining up at a bus stop or those who appreciate each exhibit in an art museum or the like, thereby increasing the usefulness.

FIG. 10 is a block diagram of one embodiment of the person counting device 4 of FIG. The person counting device 4 of this embodiment (see FIG. 1) includes a person position detection unit 41, an image / position correspondence storage unit 42, and a person density measurement unit 43. The image / position correspondence storage unit 42 stores a parameter value necessary for estimating the position of the person in the real space by the person position detection device 41 or a lookup table describing the spatial position of the pixel and the person. Before the measurement, calibration (calibration) is performed in advance by initial setting, and the correspondence between each pixel in the image and the spatial position is obtained and stored in advance.

The person position detecting section 41 is provided for the person detecting device 3.
The output of (see FIG. 1) is regarded as the head position in the image as it is, and the position of the person in the space is estimated based on the information of the image / position correspondence storage device 42. The person density measuring unit 43 measures the number of people in a small area based on the person position detected by the person position detecting unit 41. The counting result storage device 5 (FIG. 1)
) Sequentially records the person density measured by the person density measurement unit 43 in association with the time at which the image was captured.
The counting result display device 6 (see FIG. 1) stores the person density in the image measured by the person counting device 4 or the counting result storage unit 5.
Is processed as necessary and displayed in the past.

Hereinafter, the key performed by the image / position correspondence storage unit 42
The calibration method will be described. First, the caliber
In the preparation stage of the camera, internal parameters of the camera
(Focal length f, horizontal pixel distance d_h, Vertical pixel
Distance d_v, Image center (u_c, V _c), Distortion aberration counting)
I will ask for it in advance. This internal parameter calculation method is
There are many, but the most versatile is called the Tsai method
(Reference: Roger Y. Tsai, “A
n Efficient and Accurate
Camera Calibration Techni
que for3D Machine Visio
n ", Proceedings of IEEE Co.
nference on Computer Visi
on andPattern Recognitio
n, Miami Beach, FL, 198.
6).

Next, these internal parameters are known.
Assuming that, the camera position in space and
And posture (hereinafter referred to as camera position in space and
The posture is called an external parameter). FIG.
Figure 1 shows a model diagram of the space where camera positions and person positions are measured.
Show. The origin O at the lower right of FIG.
This is the origin, and the xyz axis is determined for convenience as shown in FIG.
Each point (x_i, Y _i, T) (where i = 1... 5)
Is a pole with a pole set up for calibration
Is the height of the center of the head of a general person (floor
140-160cm from the surface)
Decide in advance. (Separately know relative position in space
It is also possible to set the same in five points
However, from the viewpoint of improving the accuracy,
Set)) Also, as the range for estimating the person position,
11 points (x_i, Y_i, T) (where i = 1... 4)
It is an enclosed area. Cameras are lens-centered globally
(X₀, Y₀, Z₀) And the optical axis is z
Tilt by an angle α around the y-axis with respect to the positive
Around the z axis so as to form an angle β with the x axis.
It is assumed that it is in the turned angular position (posture). (u_c, v_c)
Is the image center (the point where the camera optical axis and the image plane intersect)
, (U_i, V_i) (I = 1 ... 5) is the object point (x_i,
y_i, T) (i = 1... 5) of the pixel imaged and reflected
The coordinates (coordinates of an image point on the image plane).

[0064] In this case, the point on the image plane _{(u i, v i) (} i
= 1. . The global coordinates (X _i , Y _i , Z _i ) of 5) are determined as follows. First,

[0065]

(Equation 1)

For u _i ′ and v _i ′ obtained by

[0067]

(Equation 2)

The following holds. Here, κ is a distortion aberration coefficient. (The distortion is an aberration in which an image is distorted in a barrel shape or pincushion shape when a lattice-shaped object is photographed by a camera. In a normal perspective transformation camera model, u _i ′ = ds ( u _i −u _c ) holds, but in the case of a general camera, the perspective transformation model does not always hold due to the presence of distortion. Equation (1.1) is an equation for correcting it.

(X _i , Y _i , Z _i ) obtained from equation (2)
Is the next straight line

[0070]

(Equation 3)

It is above. Therefore, the following equation holds.

[0072]

(Equation 4)

From equation (2), (X _i , Y _i , Z _i ) is

[0074]

(Equation 5)

As expressed by x ₀ , y ₀ , z ₀ , α, β, and (u _i ′, v _i ′) (i = 1, 2,...), The equation (2) Substituting into Equations (4.1) and (4.2), Equations (4.1) and (4.2) yield (x ₀ , y ₀ ,
z _0, α, β) and _{(u i ', v i'} ), (x i, y i,
t) (i = 1, 2,... 5). Of these, (x
_{Since i} , y _i , t) (i = 1, 2,... 5) are preset points, their values are known. Also, (u _i ',
v _i ′) (i = 1, 2,... 5) are represented by coordinates (x _i , y _i ,
t) By placing an object at (i = 1, 2,... 5) and taking an image in advance, the value is known. Therefore, for the five values of i, five equations (4.1) and (4.2) for (x ₀ , y ₀ , z ₀ , α, β) are obtained. From these five equations, the most probable values of the five parameters x ₀ , y ₀ , z ₀ , α and β can be obtained by the least squares method.

In order to estimate the most probable values of the position (x ₀ , y ₀ , z ₀ ) and posture (α, β) of the camera by the least square method, the square of the left side of the equation (4.1) is used. And the sum of the squares on the left side of equation (4.2)
(U _i ′, v _i ′). Five parameters x ₀ , y ₀ ,
The most probable values of z ₀ , α, and β are obtained by the least squares method using S. A typical example of such a nonlinear least squares method is the Marquart method.
iam H. Press et al. Written and translated by Haruhiko Okumura et al. "Numerical Recipe in Sea"
1993) at pages .505-510. Since the content of the nonlinear least squares method has little relevance to the present application, the description is omitted here. X obtained by the above
₀ , y ₀ , z ₀ , α, β are stored in the image / position correspondence storage device 42, and the calibration is completed.

Next, the processing of the person position detecting device 41 will be described. It is assumed that the person detection device 3 has detected a pixel (u, v) that is considered to be the center of the head of a person in the image. At this time, the coordinate value (x _p , y _p , z _p ) of this pixel in the global coordinate system is obtained by performing the same transformation as in the equations (1.1) and (1.2) below (u ′, v ′) is expressed as in equation (5).

[0078]

(Equation 6)

[0079]

(Equation 7)

Assuming that the height of this person is t, the global coordinates (x, y, t) of the person's position are given by the following equation (3).
Can be expressed as the following equation (6) obtained by modifying the equation of the straight line.

[0081]

(Equation 8)

Thus, the global coordinates of the person position are

[0083]

(Equation 9)

Is obtained. The person position detecting section 41 outputs the (x, y, t) coordinate values thus obtained to the person density measuring section 43.

Further, in order to speed up the processing of the person position detecting section 41, the calculation from (1.3) to (7) is performed in advance for all the pixels in the image, and the image / position correspondence storage section 42 is calculated. The look-up table is recorded in the table, and the global coordinates (x, y, t) of the person position are obtained by referring to the look-up table with respect to (u, v) obtained by the person detecting device 3. is there.

Next, the processing of the person density measuring section 43 will be described. FIG. 12 is a diagram illustrating an example in which the position of each person in the real space obtained by the person position detection unit 41 is projected on the xy plane and plotted. As a region for estimating the position of a person, four points in FIG. _{11 (x i, y i,} t) ( where, i =
1. . Since the area surrounded by 4) is set in advance, FIG. 12 shows only the area within the area. FIG. 13 divides this area into 36 small areas,
It shows how many people are in each area.
In this figure, the area is divided into 36 areas of substantially equal size. However, the number of areas and the size of each area can be freely set according to the application. The region for estimating the position does not need to be rectangular as shown in FIG. 13, and the size and shape of the region can be set freely according to the application. In FIG. 13, regions 1 to 36 are named to identify each region, and each region is named S _i =
{(X, y) | x l [i] ≦ x ≦ x r [i], y b [i]
≦ y ≦ y _t [i]｝ (i is an integer up to 1.36). Here, x _l [i] and x _r [i] are the x coordinates of the left end (left side) and right end (right side) of the area i, respectively. Similarly, y _b [i] and y _t [i] are y coordinates of the lower end (lower base) and the upper end (upper base) of the area i, respectively. The value of each person position in the global coordinate system is (x _j ,
y _j ) (j is a value from 1 to the total number of persons), (x _j ,
When y _j ) ∈S _i , it is assumed that (x _j , y _j ) exists in the region i. In this way, the number of persons existing in each area is calculated for each area, and the result is counted.
(See FIG. 1).

The result of the person density counted by the person counting device 4 and the result of the person counting at the past time stored in the counting result storage device 5 are displayed on the counting result display device 6 as necessary or in response to the operator's request. For example, the magnitude of the density as shown in FIG. 14 is indicated by a graph indicating the density of the image (in FIG. 14, the density is indicated by diagonal lines, but is actually displayed in color). Various processes are performed, such as graphing the change in the density of the data, and displayed.
This embodiment is useful for measuring the density of a person in a wide area as shown in FIG. 14, for example, measuring the congestion state of a platform.

Next, an embodiment of the counting result display device 6 shown in FIG. 1 will be described. FIG. 15 shows a first example of the counting result display device 6.
FIG. 4 is a block diagram of an embodiment of FIG. In the present embodiment, the image acquisition device 2 is provided by a wide-angle camera that can overlook the entire room of the store.
(See FIG. 1), an image is acquired, and the counting result storage device 5
Records the number of customers who have visited the store together with the image acquisition time.

The number-of-purchasers storage unit 602 stores the number of customers per unit time based on data recorded in the POS or the like over the entire business hours. Purchase rate display unit 6
01 is based on the number of shoppers per unit time stored in the counting result storage device 5 of FIG. 1 and the number of shoppers stored in the number of shoppers storage unit 602 according to predetermined conditions such as hourly or day of week. Aggregate the purchase rate (number of customers / number of visitors).

FIG. 16 is a block diagram of a second embodiment of the counting result display device 6. Also in the present embodiment, an image is acquired from the image acquisition device 2 (see FIG. 1) by a wide-angle camera that can overlook the entire room of the store, and the counting result storage device 5 is used.
Records the number of customers who have visited the store together with the image acquisition time.

[0091] The number of employees storage unit 612 stores the number of employees working at each time. Average customer count display section 611
Is based on the number of customers per unit time stored in the counting result storage device 5 (see FIG. 1) and the number of employees stored in the number of employees storage device 612 according to predetermined conditions such as hourly or day of week. Number of customers per employee (number of customers visiting stores)
/ Number of employees) and display.

FIG. 17 is a block diagram of a third embodiment of the counting result display device 6. In the present embodiment, when an image is obtained by photographing the inside of a store, a predetermined cash register periphery is set in advance as a processing target area, the persons in the predetermined area are counted by the person counting device 4, and the counting result storage device 5 is used. In, the person counting result in each area,
The number of customers lined up at the cash register is recorded together with the image acquisition time. The used register number storage unit 622 stores the number of cash registers used at each time. The cashier number consulting unit 621 reads the contents stored in the used cashier storage unit 622 and compares the optimal cashier number estimated from the number of customers in the cashier with the cashier number stored in the used cashier storage device. And
Whether the number of cash registers used at each time is greater than or less than the required number is totaled and displayed according to a predetermined condition such as hourly or weekly.

FIG. 18 is a block diagram of a fourth embodiment of the counting result display device 6. In this embodiment, a specific sales floor is set in advance as a processing target area, and an image is acquired by photographing the inside of the store. The person counting device 4 counts the people in a predetermined area, and the counting result storage device 5 records the person counting result in each area as the number of customers in a specific sales floor together with the image acquisition time.

The in-store employee storage unit 632 stores the number of employees placed in the above-mentioned store at each time. The consultation number 631 for the number of employees in the sales floor is the employee storage unit 6 in the sales floor.
32, and compares the optimum number of employees estimated from the number of customers in the specific store with the number of employees stored in the number-of-employees storage device. Whether the value is higher or lower is counted and displayed according to a predetermined condition such as hourly or weekday.

FIG. 19 is a block diagram of a fifth embodiment of the counting result display device 6. In this embodiment, for an image obtained by photographing the inside of a playground such as a ski resort or an amusement park, the vicinity of a predetermined lift landing or the entrance of an attraction facility is set in advance as a processing target area.

The person counting device 4 counts persons in a predetermined area. The counting result storage device 5 records the number of people counted in each area as the number of customers in a lift or an attraction facility together with the image acquisition time. The unit time processing number storage unit 642 stores the number of people that each lift or attraction facility handles per unit time. The waiting time display unit 641 reads the stored contents of the unit time processing number storage unit 642, and displays the recorded number of customers and the lifts or attraction facilities stored in the unit time processing number storage unit of the customers lined up in the lift or attraction facility. The waiting time for the service of the facility at the time of imaging (the number of customers lined up / the number of facilities that the facility handles in the unit time) is calculated from the number of people handling per unit time at the imaging time, and a bulletin board provided at each point inside and outside the play facility Display the calculated waiting time.

Each of the above embodiments can be implemented by appropriately changing the conditions. For example, the number of cameras, the weight shape and gap width of the edge detection operator, the shape of the human model, the arrangement of the points to be selected, and the like. There is no limit on the number and the like.

The above-described person detection processing by the person detection apparatus is executed under the control of a processor (not shown) in accordance with a person detection program recorded on a recording medium (not shown).

The person detection program according to the present invention includes an image acquisition process for taking in an image frame taken by a camera for taking an image of an area where a large number of people are present or moving and creating an image capable of image processing; Holding and extracting an image having a shape that matches the human shape model from the image that can be processed, and each extracted person exists as a person whose image is captured in the image plane. A person detection process for outputting coordinates of a point representing a position on the image plane, a person counting unit for counting the number of persons present in the field of view of the image plane from a person detection result obtained by the person detection unit, The number of persons counted by the counting means is associated with the time at which the image was taken,
The counting result storage processing to record sequentially, the person counting result in the image plane counted by the person counting means, or the person counting result at the past time accumulated in the counting result storage means is processed as necessary. A procedure for executing a counting result display process for displaying the count result is described. This person detection program causes the computer to execute a set of feature amounts arranged at relative positions corresponding to positions of a plurality of preset points on the person shape model in order to cause the computer to execute the person detection processing. By a feature amount detector set consisting of a detector, at a relative position corresponding to each position on the human shape model, an image feature amount is detected for each point on the image surface obtained by the image acquisition processing,
A feature amount calculation process of outputting the detected image feature amounts in association with the coordinates of the image plane is executed, and a set of image feature amounts detected by each feature amount detector of the feature amount detector set at the relative position is executed. A predetermined operation is performed on the values of the image feature amount set to evaluate the degree to which the image feature amount set appropriately matches the human shape model, and corresponds to the position of the feature amount detector set on the image plane. To generate an evaluation value, generate an evaluation value image representing an evaluation value corresponding to each coordinate of the image plane, and execute an evaluation value calculation process. For a pixel at each coordinate, a local maximum that searches for an evaluation value local maximum point at which an evaluation value corresponding to the coordinate of the pixel is maximum in a nearby area, and outputs the coordinates of the obtained evaluation value local maximum as a point where a person exists. Execute point search processing Procedure for has been written.

[0100]

The first effect is that a person in an image can be detected while allowing a slight difference or change in shape. The reason is that by using an edge detection operator with an extended gap, a high differential output value can be obtained if there is a contour in the gap portion of the operator even if the contour shape fluctuates slightly, and as a result, a high evaluation value is obtained. This is because

The second effect is that even when the target persons are adjacent to each other, each individual can be detected separately.
The reason is that even when the target person group is relatively dense, it is difficult to hide, and the feature amount detector is arranged as a model with the contour shape of the head and upper body with relatively little change in shape, and the person detection process is performed. Because it can be done.

A third effect is that the speed can be easily increased by hardware mounting. The reason is that the person detection device is mainly configured to filter the entire image,
This is because it is easy to increase the processing speed by applying image processing dedicated hardware using a DSP or the like.

The fourth effect is that there are few restrictions on the installation conditions of the system and the flexibility in operation is high.
The reason is that, since the shape of a person is used, it is hardly affected by the movement of a camera or a change in illumination unless the change in the shape of the imaged person is remarkable.

[Brief description of the drawings]

FIG. 1 is a block diagram showing a configuration of an embodiment of the present invention.

FIG. 2 is a flowchart illustrating a processing routine of the embodiment of FIG. 1;

FIG. 3 is a block diagram illustrating a configuration of a person detection device 3 of FIG. 1;

FIG. 4 is a flowchart showing a flow of a person detection processing routine of the person detection device 3;

FIG. 5 is a block diagram illustrating a configuration of a person detection device 3 of FIG. 1;

FIG. 6 is a conceptual diagram showing an edge detection operator used in the person detection device 3.

FIG. 7 is a conceptual diagram showing a person contour shape model used for a person detection process and an arrangement of edge detection operators.

FIG. 8 is a flowchart showing the contents of a person detection processing routine using an edge detection operator.

FIG. 9 is a conceptual diagram showing an embodiment in which one or a plurality of areas to be monitored are set in an image acquired by a camera.

FIG. 10 is a block diagram of one embodiment of the person counting device 4 of FIG. 1;

FIG. 11 is a diagram showing a model space for calibrating a camera arrangement and a person position.

FIG. 12 is a sample diagram in which the position of a person in a space before being processed by the person density measurement device 9 is plotted.

FIG. 13 is a diagram illustrating an example of each small area processed by the person density measurement device 9;

FIG. 14 is an example of a person density diagram displayed on the counting result display device 6;

FIG. 15 is a block diagram of a first embodiment of the counting result display device 6;

FIG. 16 is a block diagram of a second embodiment of the counting result display device 6;

FIG. 17 is a block diagram of a third embodiment of the counting result display device 6.

FIG. 18 is a block diagram of a fourth embodiment of the counting result display device 6.

FIG. 19 is a block diagram of a counting result display device 6 according to a fifth embodiment.

[Explanation of symbols]

 REFERENCE SIGNS LIST 1 camera 2 image acquisition device 3 person detection device 301 feature amount calculation device 302 evaluation value calculation device 303 local maximum point search device 31 image differentiation unit 32 evaluation value image synthesis unit 33 local maximum point search unit 4 person counting unit 41 human position detection unit 42 Image / position correspondence storage unit 43 Person density measurement unit 5 Counting result storage unit 6 Counting result display unit 601 Purchase rate display unit 602 Purchaser number storage unit 611 Average customer reception number display unit 612 Employee number storage unit 621 Number of cash register consulting unit 622 Cash register used Number storage unit 631 Consultation number of employees in sales floor 632 Storage unit of employees in sales floor 641 Waiting time display unit 642 Storage unit for processing time per unit time

Continued on the front page F term (reference) 5B057 BA02 DA13 DC09 DC25 DC33 5L096 AA02 BA08 CA04 CA05 EA15 FA06 FA69 FA81 GA02 GA38 HA08

Claims (16)

[Claims] 1. A person detection method for photographing an area in which a large number of people exist or move and acquiring an image, and performing a person detection process of extracting a person image from the acquired image, the method comprising: A feature detector for detecting a feature of a person to be detected is arranged at a relative position corresponding to the positions of a plurality of points set in advance on the shape of a human figure to constitute a feature detector set. The feature amount detector set is placed at each position corresponding to each coordinate set on the image plane, and the image feature amount of the acquired image is detected, and the feature amount detector set is configured at each of the positions. Based on the value of the image feature amount detected by the feature amount detector, the detected image feature amount is a feature amount of an image having a shape matching the human shape model within a predetermined allowable range. To judge When a predetermined calculation is performed, and the calculation result indicates that the detected image feature amount is a feature amount of an image having a shape that matches the human shape model, a person image exists at the position. Then, a person detection method to estimate. 2. The method according to claim 1, wherein, at each position of the feature amount detector set, a product of the feature amounts detected by the respective feature amount detectors is calculated, and the product is compared with a predetermined threshold value. When the value exceeds the predetermined threshold, the processing includes determining that the image is a person image at the position.
The person detection method according to claim 1. 3. The method according to claim 1, wherein the predetermined calculation is performed by associating a product of the feature amounts detected by each of the feature amount detectors with an evaluation value image in association with coordinates of an image plane corresponding to a position of the feature amount detector set. Is generated, a local area centered on each pixel in the evaluation value image is set, a local maximum point at which the evaluation value becomes maximum is searched for in the local area, and among the local maximum points, the evaluation value is a predetermined threshold. The method according to claim 2, further comprising a process of setting a pixel at the center of the local area where the maximum point exceeding the value is present as a point where a person is present. 4. A method according to claim 1, wherein each of the feature amount detectors arranged at relative positions corresponding to a plurality of preset points on the human shape model corresponds to a predetermined point on the human shape model. A feature vector is defined as a feature amount detector, and a displacement vector from a position of the plurality of preset points on the human shape model to a reference point arbitrarily set in the human shape model is a feature corresponding to the plurality of points. The feature amount is defined as a feature amount displacement vector of the amount detector, and the feature amount detected by each feature amount detector is associated with the coordinates of a position displaced by the feature amount displacement vector from the position where the feature amount is detected, and When defining a displacement feature value at the coordinates corresponding to the feature value detector, the calculation of the product of the feature values detected by the respective feature value detectors is performed for each feature value detector for each coordinate on the image plane. Corresponding changes The method according to claim 2, further comprising a process of calculating a product of the position features. 5. An image acquiring means for acquiring an image frame photographed by a camera photographing an area where a large number of persons are present or moving, and creating an image capable of image processing,
Holding a person shape model, from the image that can be image-processed, extract an image having a shape that matches the person shape model, each extracted image as a person being imaged in the image plane, each extracted A person detecting unit that outputs coordinates on the image plane of a point representing a position where a person is present, wherein the person detecting unit is configured to determine positions of a plurality of preset points on a person shape model. A feature amount detector set consisting of a set of feature amount detectors arranged at relative positions corresponding to the, at the relative position corresponding to each position on the human shape model by the feature amount detector set, The image feature amount is detected for each point on the image plane obtained by the image obtaining means,
A feature amount calculating unit that outputs the detected image feature amount in association with the coordinates of the image plane; and an image that is a set of image feature amounts detected by the feature amount detectors of the feature amount detector set at the relative position. A predetermined operation is performed on the values of the feature set to evaluate the degree to which the image feature set appropriately matches the human shape model, and the feature detector set is evaluated in accordance with the position on the image plane. An evaluation value calculating unit that generates a value and generates an evaluation value image representing an evaluation value corresponding to each coordinate on the image surface; and a pixel at each coordinate on the image surface with respect to the evaluation value image calculated by the evaluation value calculation unit. A maximum value search unit that searches for an evaluation value maximum point at which an evaluation value corresponding to the coordinates of the pixel is maximum in a nearby area, and outputs the obtained evaluation value maximum point coordinates as a point where a person is present. A person characterized by having Detection device. 6. The feature amount calculation unit includes an operator set including a set of edge detection operators as a feature amount detector set, and the operator set determines a relative position corresponding to a plurality of positions on the contour of the human shape model. Detecting an image contour for each point on the image plane obtained by the image acquisition means, outputting a detected value of the detected image contour in association with the coordinates of the image plane, From the positions of the plurality of points, a displacement vector to a reference point arbitrarily set in the human shape model is defined as a feature amount displacement vector of an edge detection operator at a relative position corresponding to the positions of the plurality of points, The edge detection value detected by each edge detection operator is displaced by a feature displacement vector from the position where the edge detection value is detected. In association with the coordinates,
When defining the displacement feature amount at the coordinates corresponding to the edge detection operator, the evaluation value calculation unit performs, as a predetermined operation on the value of the image feature amount set, the respective edge detection operators for each coordinate on the image plane. The product of the displacement feature amounts for the detected edge detection values is calculated. The local maximum point searching unit sets a local region centered on each pixel in the evaluation value image obtained by the evaluation value calculation unit, Search for the maximum point where the evaluation value is maximum, and output the position of the pixel at the center of the local area where the maximum value where the evaluation value exceeds the predetermined threshold exists as the position of the point where the person exists. Item 6. The apparatus according to Item 5. 7. The evaluation value calculation unit includes:
It has an edge detection operator with a small detection edge width and an edge detection operator with a large detection edge width, and each edge detection operator outputs a stable detection value even if there is a slight change in the shape of the human image. A gap extending operator having an increased width of a central gap portion having no weight, wherein the edge detection operator is arranged at a relative position corresponding to a part of a person where concealment hardly occurs and shape change is small. 7. The apparatus according to 6. 8. A human counting means for counting the number of persons present in the area being photographed based on the detection result obtained by the person detecting means, wherein the person counting means comprises: A parameter value required to estimate the or a look-up table that describes the pixel and the spatial position of the person and stores each pixel in the image by performing calibration in the initial settings before performing measurement. An image / position correspondence storage unit that stores the correspondence with the spatial position and stores the correspondence in advance, and regards the point at which the person output by the person detecting means is present as the position of the head of the person image in the image. Position correspondence storage device 4
Based on the information of 2 above, a person position detecting unit that estimates the position of the person in the space, and based on the person position detected by the person position detecting unit, the number of persons in the micro area is measured, and the person density is calculated. The apparatus according to claim 5, further comprising: a person density measuring unit for calculating. 9. Calibration is performed by determining the relationship between the position of an object point arranged at five predetermined spatial positions and the position of an image point obtained by imaging the object point on the image plane. The method includes describing the coordinates and two angles representing the camera posture as parameters using five equations, and calculating the most probable global coordinates of the camera and the two angles representing the camera posture by the least squares method. 8
An apparatus according to claim 1. 10. The apparatus according to claim 8, further comprising a counting result storage unit that sequentially associates the person density counted by the person counting unit with the time at which the image was captured and records the result. 11. A counting result display means for processing and displaying the counting result obtained by the person counting means or the counting result of the past time accumulated in the counting result storage means as necessary, The counting result display means stores, based on the recorded data, the number of customers per unit time over all business hours, a number of customers per unit time stored in a counting result storage device, and 9. The apparatus according to claim 8, further comprising: a purchase rate display unit that totalizes a purchase rate, that is, the number of purchasers / the number of customers visiting the store, according to predetermined conditions based on the number of purchasers stored in the number-of-customers storage unit. 12. A counting result display means for processing and displaying the counting result obtained by the person counting means or the counting result of the past time stored in the counting result storage means as necessary, The counting result display means, an employee number storage unit that stores the number of employees working at each time,
Based on the number of visitors per unit time stored in the counting result storage means and the number of employees stored in the number of employees storage unit,
9. The apparatus according to claim 8, further comprising an average number of customers displayed, which counts and displays the number of customers per employee, that is, the number of customers visiting the store / the number of employees according to predetermined conditions. 13. A counting result display means for processing and displaying, as necessary, a counting result obtained by the person counting means or a counting result of a past time stored in the counting result storage means, The counting result display means includes a used cash register number storage unit for storing the cash register number used at each time, an optimal cash register number estimated from the number of customers arranged in the cash register, and a cash register number stored in the used cash register memory device. And a register number consulting unit for counting and displaying whether the number of cash registers used at each time is greater than or less than a required number according to a predetermined condition. apparatus. 14. A counting result display means for processing and displaying the counting result obtained by the person counting means or the counting result of the past time accumulated in the counting result storage means as necessary, The counting result display means includes an in-store employee storage unit that stores the number of employees arranged in the counter at each time, and an optimum number of employees and a number of employees storage device estimated from the number of customers in a specific counter. Comparing with the stored number of employees, the number of employees arranged at each time has a consultation section for the number of employees in the sales floor which aggregates and displays whether the number of employees is more or less than the required number according to predetermined conditions, An apparatus according to claim 8. 15. Counting result display means for processing and displaying the counting result obtained by the person counting means or the counting result of the past time stored in the counting result storage means as necessary, The counting result display means includes a unit time processing number storage unit that stores the number of people that each lift or attraction facility handles per unit time, and a recorded number of unit and processing time unit number of customers who line up in the lift or attraction facility. From the stored number of people that each lift or attraction facility handles per unit time, calculate the waiting time for the service of the facility at the imaging time, that is, the number of customers that the facility handles per unit time / unit time at the imaging time, A waiting time display unit for displaying the calculated waiting time on a bulletin board or the like provided at each point inside and outside the game facility. The apparatus according to. 16. An image acquisition process for capturing an image frame captured by a camera for capturing an area where a large number of people are present or moving and creating an image that can be subjected to image processing. From the possible images, an image having a shape matching the human shape model is extracted, and each of the extracted images is taken as a person being imaged on the image plane, and a point representing a position where each of the extracted people is present is located. A person detection process for outputting coordinates on the image plane, a person counting means for counting the number of persons present in the field of view of the image plane from a person detection result obtained by the person detecting means, A count result storage process for sequentially recording the number of persons in association with the time at which the image was captured, and a person count result in the image plane counted by the person counting means, or the counting result. Recording a person detection program that describes a procedure for executing a counting result display process of processing and displaying, as necessary, a person counting result at a past time stored in the result storage means. A medium, wherein the person detection program is a program for causing a computer to execute the person detection process, wherein one set of relative positions corresponding to positions of a plurality of preset points on a person shape model is provided. The feature amount detector set, which is a feature amount detector, detects an image feature amount for each point on the image surface obtained by the image acquisition process at a relative position corresponding to each position on the human shape model, and performs detection. A feature value calculation process of outputting the obtained image feature values in association with the coordinates of the image plane, wherein each feature value detector of the feature value detector set has the relative position. A predetermined operation is performed on the value of the image feature amount set composed of a set of image feature amounts detected in the above, and the degree to which the image feature amount set appropriately matches the human shape model is evaluated. Generates an evaluation value corresponding to the position of the set on the image surface, and executes an evaluation value calculation process of generating an evaluation value image representing an evaluation value corresponding to each coordinate of the image surface. For the evaluation value image
For a pixel at each coordinate on the image plane, an evaluation value corresponding to the coordinates of the pixel is searched for a maximum evaluation value maximum point in a nearby area, and the coordinates of the obtained maximum evaluation value point are determined as points where a person exists. A recording medium on which a person detection program is recorded, in which a procedure for executing a maximum point search process to be output is described.