WO2015003287A1

WO2015003287A1 - Behavior recognition and tracking system and operation method therefor

Info

Publication number: WO2015003287A1
Application number: PCT/CN2013/000845
Authority: WO
Inventors: 方姿尹; 林家安; 褚陈翰; 莫书涵; 李东洋; 王政杰
Original assignee: 台湾色彩与影像科技股份有限公司
Priority date: 2013-07-12
Filing date: 2013-07-12
Publication date: 2015-01-15

Abstract

A behavior recognition and tracking system comprises at least one shooting apparatus (1); at least one processing apparatus (2), and a storage apparatus (3). The processing apparatus (2) is electrically connected to the shooting apparatus (1), and has a determining module (21) and a recognition module (22) electrically connected to the determining module (21), and can recognize a video and an image, record a time point, record a network card number and an IP address of the shooting apparatus (1), and further, generate recognition and analysis data. The storage apparatus (3) is electrically connected to the processing apparatus (2) and is used by the processing apparatus (2) to transmit and compare data.

Description

Behavior recognition and tracking system and its operation method

Technical field

The present invention relates to a tracking system, and more particularly to a behavior recognition and tracking system for tracking a person's flow, identifying its behavior and characteristics, and then statistically analyzing the population. Background technique

With the evolution of the times, the development of science and technology is changing with each passing day, and the dissemination and acquisition of information has become quite convenient. In modern service industries or commercial activities, in order to improve customer satisfaction and analyze customer behavior in depth, there is usually a corresponding investigation. Ways to conduct market acceptance analysis of products. In general, in order to obtain the customer's consumption information and preferences, satisfaction and other information, the store will design a questionnaire to provide customers with the survey. However, since the time point provided by the questionnaire is always after the completion of the consumer behavior, the information obtained is not enough. It is difficult to confirm the authenticity, and it is difficult to confirm the authenticity. It is quite difficult to organize, and it also takes up a lot of space. Therefore, it is difficult to classify or store it in a systematic way. Therefore, there are still many difficulties that must be overcome by using the questionnaire to complete the market survey.

In order to solve the above problems, the electronic questionnaire was designed, and the questionnaire was electronically used to allow the staff to use computers and other tools to assist in the completion. Although the inconvenience of storage and archiving was solved, the following problems were also caused:

(1) Because the questionnaire is filled in or the inquiry is a conscious behavior, the customer will reply to the answer after multiple considerations, and the customer's true thoughts cannot be fully reflected. As a result, the questionnaire is not accurate enough. In addition, the behavior of the customer in the store, including the movement of the line, the attention to the goods or advertisements, and the residence time in the store, etc., are difficult to obtain information by means of questionnaires.

(2) The identity of the customer must be verified manually. It takes a lot of time and cost to archive and classify it, which is quite inconvenient to use.

In view of this, the inventors have accumulated experience based on years of research, and with their own creativity and continuous attempts, developed a behavioral recognition that can effectively improve the defects mentioned in the above prior art. And tracking system. Summary of the invention

A first object of the present invention is to provide a behavior recognition and tracking system for tracking a person flow by using an image pickup device, and by image capture of an image pickup device, information can be collected in an unconscious manner by a subject, for example, being photographed. Face, skin color, age, gender, gesture, body shape, expression, sound, clothing color, gaze tracking, dwell time, and gaze time, etc., track the behavior of the subject to enhance the behavior of the subject The accuracy of the analysis.

A second object of the present invention is to provide a behavior recognition and tracking system that utilizes videography. The device, with the discriminating and identifying module, performs multiple intelligent image recognition on the human flow, and can simultaneously give the subject an identity for recording, for example, the consumption behavior, the time point of the shooting, and the network card number and IP address of the camera to establish The complete consumption process analyzes its behavior, so that the relevant information of the subject can be directly classified by the image, and the manual search and classification method is not necessary to organize the information.

The object of the present invention and solving the technical problems thereof are achieved by the following technical solutions. The present invention provides a behavior recognition and tracking system, comprising: at least one camera device capable of capturing an image; at least one processing device electrically connected to the camera device, and capable of processing the image captured by the camera device, And having: a discriminating module, which can determine whether the image captured by the camera device is a desired image; and an identification module electrically connected to the discriminating module, and can identify, record, and record the time and record of the image passing through the discriminating module a network card number and an IP address of the camera device, thereby generating identification analysis data; a storage device electrically connected to the processing device, wherein the processing device performs transmission and comparison of the identification analysis data, and can establish a database; The processing device compares the identification analysis data with the data in the storage device, and directly overwrites and edits if there is matching data, and directly establishes a new storage space to store the identification analysis data if there is no matching data.

The object of the present invention and solving the technical problems thereof can be further achieved by the following technical measures.

In the foregoing behavior recognition and tracking system, the discriminating module determines whether the image captured by the camera device is a human face image by using a geometric feature, a contour shape, and a feature invariant method.

The foregoing behavior recognition and tracking system, wherein the identification module can identify at least one of the following combinations: face, skin color, age, gender, gesture, body shape, expression, sound, clothing color, gaze tracking, resident Stay time and watch time.

The behavior recognition and tracking system described above, wherein the behavior recognition and tracking system further has at least one operation platform connected to the camera device.

The foregoing behavior recognition and tracking system, wherein the operating platform can selectively use at least one of the following combinations: advertising media billboards, tablets, interactive media billboards, point of sale information management (POS: point of sa les) .

The foregoing behavior recognition and tracking system, wherein the storage device is more selectively connected to any one of the following combinations: CRM: Cus tomer relating management (CRM), and business intelligence analysis (Bus) Ines s Inte l 1 igence), data mining (Da ta Mining) and massive data computing system (B ig da ta).

In the foregoing behavior recognition and tracking system, the discriminating module and the discriminating module can be respectively disposed on different processing devices.

The object of the present invention and solving the technical problems thereof are also achieved by the following technical solutions. The invention further provides a method for operating a behavior recognition and tracking system, the steps comprising: SI : intercepting at least one image by using a camera device;

S2: determining, by the discriminating module, whether the detected image is an image required by the user, and if yes, proceeding to step S3; if not, proceeding to step S1;

S3: using the identification module to identify the image, record the time point, record the network card number and IP address of the camera device, and obtain identification analysis data;

S4: the processing device compares the identification analysis data with the data in the storage device, and determines whether there is corresponding data, if yes, proceeds to step S5, and if not, proceeds to step S6;

S5: storing the identification analysis data in the same storage space of the data, and performing step S7;

S6: storing the initial identification data in the new storage space, and performing step S7;

S7: End.

The foregoing method for operating a behavior recognition and tracking system, wherein the identification module can identify at least one of the following combinations: face, skin color, age, gender, gesture, body shape, line of sight tracking, dwell time, and gaze time.

With the above technical solution, the behavior recognition and tracking system and the operation method thereof of the present invention have at least the following advantages and beneficial effects:

(1) Tracking of the flow of people by means of the camera device, by means of image capture by the camera device, information can be collected unconsciously by the subject, such as the face, skin color, age, gender, gesture, body shape of the consumer, Expressions, sounds, clothing colors, gaze tracking, dwell time, and gaze time, etc., track the behavior of the subject to improve the accuracy of consumer behavior analysis.

(2) The same is to use the camera device, with the discriminating and identification module to identify the person flow, and at the same time give the subject an identity, to record its consumption behavior, the time point of the shooting and the network card number and IP of the camera. The address is used to establish a complete consumption process to analyze its consumption behavior. In this way, the relevant information of the consumer can be directly classified, and the information is not required to be sorted by manual searching and classification methods.

The above description is only an overview of the technical solutions of the present invention, and the technical means of the present invention can be more clearly understood, and can be implemented in accordance with the contents of the specification, and the above and other objects, features and advantages of the present invention can be made more obvious. It is to be understood that the following preferred embodiments are described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

Figure 1: The behavior of the present invention. A schematic diagram of a module of a third embodiment of the present invention; FIG. 2 is a block diagram of a second preferred embodiment of the behavior recognition and tracking system of the present invention; Figure 3 is a block diagram showing a preferred embodiment of the behavioral discriminating system of the present invention; and Figure 4 is a schematic diagram showing the steps of a method for operating the behavior recognition and tracking system of the present invention.

[Main component symbol description]

1 : Camera 2: Processing unit

21: Discrimination Module 22: Identification Module

3: Storage device 4: Operating platform

A: Identification analysis data S1 - S7: Step number The best way to achieve the invention

In order to further illustrate the technical means and efficacy of the present invention for achieving the intended purpose of the present invention, a behavior recognition and tracking system and a method for operating the same according to the present invention will be described below with reference to the accompanying drawings and preferred embodiments. , structure, characteristics and their efficacy, as detailed below.

In order to more clearly describe a behavior recognition and tracking system proposed by the present invention, a preferred embodiment of the present invention will be described in detail below with reference to the drawings. Please refer to FIG. 1 , which is a block diagram of a first preferred embodiment of the behavior recognition and tracking system of the present invention. The structure includes: five camera devices 1 for capturing images, and the camera device in this embodiment. 1 is described by taking five units as an example, and the number thereof can be adjusted according to the user's needs; the processing device 2 is electrically connected to the camera device 1 and can process the image captured by the camera device 1 and The determination module 21 can determine whether the image captured by the imaging device 1 is a desired image. In the embodiment, the determination module 21 determines the imaging device by using a geometric feature, a contour shape, and a feature invariant method. Whether the captured image is a desired facial image; and the identification module 22 is electrically connected to the determining module 21, and identifies and records the time point of the image passing through the determining module 21, and records the network card number of the imaging device 1 and The IP address, in turn, generates the identification analysis data A. In addition, the recognition module 22 can further face, color, age, gender, gesture, figure, expression, sound, The color, the line of sight tracking, the dwell time and the gaze time are identified; wherein the number of the processing devices 2 can also be adjusted, as shown in FIG. 2, which is a second preferred embodiment of the behavior recognition and tracking system of the present invention. The module diagram is different from the first preferred embodiment. The second preferred embodiment provides the discriminating module 21 and the discriminating module 22 on separate processing devices 2, even if each individual processing device 1 processes a program for avoiding excessive load when the data is excessive; and a storage device 3 electrically connected to the processing device 2 for the processing device 2 to perform transmission and comparison of the identification analysis data A, and The processing device 2 compares the identification analysis data A with the data in the storage device 3, and directly overwrites and edits if there is matching data, and directly establishes a new storage space if there is no matching data. The identification analysis data A; meanwhile, please refer to FIG. 3, which is a block diagram of a third preferred embodiment of the behavior recognition and tracking system of the present invention. Where different preferred embodiment of the first embodiment in that there are further respectively connected to the platform on each imaging device 14, can be detected by providing real-time interactive operation that Platform 4 can optionally use advertising media billboards, tablets, interactive media billboards, and point-of-sale POS (POS: point of sa les).

In addition, the storage device 3 is more selectively connected to a Customer Relationship Management System (CRM), a Busyness Intel igence, a Data Mining or a Massive Data Computing System (Big). On the data, the data in the storage device 3 can be processed or analyzed.

After the structure of the first to third preferred embodiments of the present invention is clearly described, the operation method of the behavior recognition and tracking system of the present invention is finally described. Please refer to FIG. 4 as a behavior of the present invention. A schematic diagram of the steps of the operation method of the identification and tracking system, the steps of the method comprising: Step S1: intercepting at least one image by using the camera device 1;

Step S2: The image is transmitted to the processing device 2, and the determination module 21 determines whether the image is a human face image, and if so, proceeds to step S3, and if not, proceeds to step S7;

Step S3: using the identification module 22 to perform image recognition on the image, record the time point, record the network card number and IP address of the camera device 1 and record the time point, and perform face, skin color, age, gender, and the detected person. Gesture, body shape, expression, sound, clothing color, line of sight tracking, dwell time and gaze time identification, and finally obtain identification analysis data A;

Step S4: The processing device 2 compares the identification analysis data A with the data in the storage device 3, and determines whether there is corresponding data, if yes, proceeds to step S5, and if not, proceeds to step S6;

Step S5: storing the identification analysis data A in the same storage space of the data, and executing step S7;

Step S6: storing the initial identification data A in the new storage space, and executing step S7;

Step S7: End.

The above has explained the operation method of the behavior recognition and tracking system of the present invention. The following will be explained with the operation example. The operation identification and tracking system of the present invention can be used in the store, at the entrance and exit of the store, and at the checkout. The camera device 1 is installed in a place required by the user. When the consumer enters the store, the camera device 1 captures the image, and the image recognition and identification is performed by the processing device 2 to generate the identification analysis data A. The identification analysis is performed. The data A may include a time point when the consumer is photographed, a network card number or an IP address of the camera, to establish a time flow and a walking route of the consumer browsing in the store, and then, the identification data A and the storage device are identified. When the processing device 2 determines that the data in the storage device 3 matches the identification analysis data A, it can be determined that the consumer has the identity of the historical data in the storage device 3, The identification analysis data A is directly stored, but if the comparison is made, the storage device 3 is not associated with the identification analysis data A. When the data is met, the processing device 2 will give the consumer a new identity code and then store the data. 6 “υι/ υΐ /ΐυ id/ υ υ υ » t »J Next, when the customer browses in the store, the camera in the store will also perform image capture, and the identification and identification by the processing device 2 at the back end Homework, where the identification project will increase the visibility of the line of sight, dwell time and gaze time to record the line of sight, dwell time, or gaze time of a kanban and product for the camera inside the store. What kind of kanban or product dwell time or gaze time or other information the consumer is doing, this part is the detection of the unconscious situation, which can avoid the defects caused by the conscious investigation in the prior art. .

In addition, the camera device 1 inside the store can also be connected with an operation platform 4, which can selectively use advertising media billboards, tablet computers, interactive media billboards, and point-of-sale information management machines according to the needs of users (POS: point Of sa les), these operating platforms 4 can also maintain an instant interactive survey method, in which, if the operating platform of the advertising kanban is used, after identifying the consumer, the identification result according to the gender, age and gaze tracking. Or a history recorded in the storage device 3 to play advertisements or interactive information suitable for the subject to be viewed; in addition, the storage device 3 is more selectively connected to a customer relationship management system (CRM) , Business Intelligence Analysis (Data Sense) or Data Data (Big data), so that the data in the storage device 3 can be processed or analyzed by the user.

Finally, when the consumer leaves the store, ^†^t^3⁄4f will also perform image capture, and the processing device 2 will perform the identification and identification actions to complete the overall behavior recognition and tracking.

The above has been fairly fully disclosed in the present invention. In summary, it can be seen that the present invention has the following two advantages:

The above is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Although the present invention has been disclosed in the above preferred embodiments, it is not intended to limit the present invention. The skilled person can make some modifications or modifications to the equivalent embodiments by using the above-disclosed technical contents without departing from the technical scope of the present invention, but without departing from the technical solution of the present invention, according to the present invention. Technical simplifications Any simple modifications, equivalent changes and modifications made to the above embodiments are still within the scope of the technical solutions of the present invention.

Claims

Rights request

1. A behavior recognition and tracking system, characterized in that it comprises:

At least one camera device can capture images;

At least one processing device is electrically connected to the camera device, and can process the image captured by the camera device, and has:

The discriminating module can determine whether the image captured by the camera device is a desired image; and the identification module is electrically connected to the discriminating module, and recognizes the image passing through the discriminating module, records the time point, and records the network card of the camera device Number and IP address, which in turn generates identification analysis data;

The storage device is electrically connected to the processing device for the processing device to perform transmission and comparison of the identification analysis data, and the database can be established.

2. The behavior recognition and tracking system according to claim 1, wherein the determination module determines whether the image captured by the camera device is a face image by using a geometric feature, a contour shape, and a feature invariant method.

3. The behavior recognition and tracking system according to claim 1, wherein the recognition module can identify at least one of the following combinations: face, skin color, age, gender, gesture, body shape, Expressions, sounds, clothing colors, gaze tracking, dwell time, and gaze time.

4. The behavior recognition and tracking system of claim 1 , wherein the behavior recognition and tracking system further has at least one operating platform coupled to the camera device.

5. The behavior recognition and tracking system of claim 4, wherein the operating platform selectively uses at least one of the following combinations: advertising media billboards, tablets, interactive media billboards, point of sale Information management machine.

6. The behavior recognition and tracking system according to claim 1, wherein the storage device is further selectively connected to any one of the following combinations: a customer relationship management system, a business intelligence analysis, Data exploration and massive data computing system.

The behavior recognition and tracking system according to claim 1, wherein the determination module and the identification module are respectively disposed on different processing devices.

A method for operating a behavior recognition and tracking system, characterized in that the steps thereof comprise: S1: intercepting at least one image by using a camera device;

S2: the image is transmitted to the processing device, and the determination module determines whether the image is the desired image, and if so, proceeds to step S3, and if not, proceeds to step S7;

S3: using the identification module to identify and record the time point of the image, and obtain identification analysis data;

S4: the processing device compares the identification analysis data with the data in the storage device, and Determining whether there is corresponding data, if yes, proceeding to step S5, if not, proceeding to step S6; S5: storing the identification analysis data in the same storage space of the data, and performing step S7; S6: initializing the initial end The identification analysis data is stored in the new storage space, and step S7 is performed; and S7: is ended.

9 . The method according to claim 8 , wherein the identification module can identify at least one of the following combinations: face, skin color, age, gender, gesture, Body shape, line of sight tracking, dwell time and gaze time.