JP2013149050A - Target object determination device, target object determination method and program - Google Patents

Abstract

An object of the present invention is to make it possible to accurately identify an object that is attracting attention.
A whole body imaging unit 11 captures moving images of a plurality of people. The body motion amount detection unit 12 detects the body motion amount of each of the plurality of people from the image captured by the whole body imaging unit 11. The face imaging unit 13 captures a moving image of each face of the plurality of people. The imaging unit control unit 14 controls the face imaging unit 13 to detect the face of the person whose detected body movement amount is smaller than the threshold among the plurality of persons or the person whose detected body movement amount has decreased. The visual field of the face imaging unit 13 is directed to the face, and a moving image of the face is captured. The line-of-sight detection unit 15 detects the direction of the line of sight of a person whose face is reflected in the image captured by the face imaging unit 13 from the captured image. The attention object specifying unit 17 specifies an object attracting attention based on the direction of the detected line of sight.
[Selection] Figure 3

Description

  The embodiments discussed herein relate to techniques for identifying the subject of interest.

  A technique is known in which the attention level of a specific target of interest is measured and attention level data obtained by analysis can be easily grasped. This technique first measures the gaze direction of the person who has been imaged, calculates the attention position information of the person from the measured gaze direction, and uses the calculated attention position information to determine the attention target of the person. Attention level information is calculated. This technique then performs a display in which graphics information generated based on the calculated attention level information and the target image are superimposed.

  In addition, a technique for evaluating video content from human eye movements is known. In this technique, a stimulus image as an image to be evaluated is presented on a display, and an eyeball reaction of a subject who is observing the stimulus image is imaged. In this technique, the captured eyeball image signal is then binarized to obtain predetermined eye movement data, and the stimulus image and the corresponding eye movement data are displayed together on the time axis.

  In addition, when the image displayed on the image display device is a moving image, a technique for measuring the degree of attention in consideration of not only the time and number of people watching the image but also the timing of attention is known. ing. In this technology, first, an inspected individual who watches the moving image information displayed on the display unit is imaged, a person's gaze is detected from the captured image, and it is determined whether the detected gaze is gazing at the display unit. , Count the number of people watching. This technique then calculates a frame attention score indicating how much the frame of the video has received attention from the counted number of people and the frame weight given to each frame, and a moving image is calculated from the calculated frame attention score. The moving image attention score in the whole is calculated.

JP 2007-286995 A JP 2004-282471 A JP 2006-268630 A

  There are known display devices (digital signage) and product display shelves for displaying or arranging content (objects such as products) for advertisement. In the following description, these are collectively referred to as “signage”.

  In signage, a plurality of spectators determine whether there is a desired item while simultaneously viewing various contents displayed or arranged there. In such a situation, each spectator looks at various contents and moves according to the degree of attention. In such an environment where multiple spectators watch signage content at the same time, the content provider checks the attention level of the content spectators, knows which content they like, and uses that knowledge in the future. There is hope to be useful for product development.

  There is known a technique for extracting the degree of attention to a content when a plurality of viewers are viewing the content on the signage. For example, as illustrated in FIG. 1, the face images of a plurality of spectators 3 who are viewing the content 2 on the signage 1 are captured one by one with the camera 4, and each spectator 3 detected from the face image is captured. There is a technique of extracting a degree of attention to the content 2 based on line-of-sight information. In this method, the line-of-sight information is acquired by rotating one camera while capturing the face images of the spectator 3 one by one in a short imaging time of, for example, about 1 second.

  However, with this method, it is difficult to correctly determine whether or not the spectator 3 really pays attention to the content 2. This is because, in the short imaging time as described above, the spectator 3 changes to the content 2 when the content 2 happens to be viewed at the time of the imaging or when the gaze happens to be removed from the content 2 of interest. This is because it may be misunderstood whether attention is being paid.

  In general, when a person actually looks at an object in order to pay attention, the user often keeps watching (for example, about 5 seconds). Therefore, for example, if the gaze information is acquired using a face image obtained by continuously capturing a face image of, for example, a dozen seconds, it is determined whether or not the spectator 3 is paying attention to the content 2. It can be confirmed. However, if the process of extracting the degree of attention to the content 2 is focused on a specific person, the other spectators 3 may move away from the spot or the target of the content 2 of interest may change. Thus, the extraction of the attention level to the content 2 of other spectators 3 is neglected. Thus, with the method described above, it is difficult to extract the attention level to the content 2 for all the onlookers 3 who are watching the signage 1.

  As another method, as illustrated in FIG. 2, a method may be considered in which a wide-angle camera 5 having a wide angle of view is used to image the faces or bodies of all the viewers 3 who are simultaneously viewing the signage 1. However, with this method, although it is possible to confirm up to which direction each spectator 3 is facing, it is difficult to individually confirm whether each spectator 3 is paying attention to something. Further, since it is difficult to obtain the line-of-sight information of the spectator 3 from the wide-angle image obtained by the wide-angle camera 5, it is difficult to examine in detail how much each spectator 3 is paying attention to the content 2 with this method. is there.

  In addition, in order to determine whether or not each of a plurality of people is paying attention to the content, it is possible to prepare a large number of cameras and acquire the line-of-sight information by capturing each person's face image with each camera. However, the cost becomes extremely high by preparing a large number of cameras.

  In view of the above-described problems, the attention target identification device described later in this specification enables a target that is attracting attention to be accurately identified.

  One of the attention target identification devices described later in this specification includes a first imaging unit, a body movement amount detection unit, a second imaging unit, a control unit, a line-of-sight detection unit, and an attention target identification unit. There is. Here, the first imaging unit captures moving images of a plurality of people. The body movement amount detection unit detects the body movement amount of each of the plurality of persons from the captured image obtained by the first imaging unit. The second imaging unit captures a moving image of each face of the plurality of people. The control unit controls the second imaging unit to select a face of the plurality of persons whose detected body movement amount is smaller than a threshold value or a person's face whose detected body movement amount has decreased. The field of view of the second imaging unit is directed to capture a moving image of the face. The line-of-sight detection unit detects the direction of the line of sight of a person whose face is reflected in the image captured by the second image capturing unit from the captured image. Then, the attention target identification unit identifies an object that is attracting attention based on the direction of the detected line of sight.

  Further, in one of the attention target identification methods described later in this specification, first, a plurality of human moving images are captured by a first imaging unit. Next, the amount of body movement of each of the plurality of people is detected from the captured image in the first imaging unit. Next, the second imaging unit that captures a moving image of each face of the plurality of people is controlled to detect the face of the person whose detected body movement amount is smaller than the threshold among the plurality of people, or detection The visual field of the second imaging unit is directed to the face of the person whose body movement amount has been reduced, and a moving image of the face is captured. Next, the direction of the line of sight of a person whose face is reflected in the captured image in the second imaging unit is detected from the captured image. Then, an object that is attracting attention is identified based on the detected direction of the line of sight.

  One of the programs described later in this specification causes the arithmetic processing unit to perform the following processing. In this process, first, moving images of a plurality of people are imaged by the first imaging unit. Next, the amount of body movement of each of the plurality of people is detected from the captured image in the first imaging unit. Next, the second imaging unit that captures a moving image of each face of the plurality of people is controlled to detect the face of the person whose detected body movement amount is smaller than the threshold among the plurality of people, or detection The visual field of the second imaging unit is directed to the face of the person whose body movement amount has been reduced, and a moving image of the face is captured. Next, the direction of the line of sight of a person whose face is reflected in the captured image in the second imaging unit is detected from the captured image. Then, an object that is attracting attention is identified based on the detected direction of the line of sight.

  According to the attention target identification device described later in this specification, there is an effect that an object that is attracting attention can be accurately identified.

It is a figure explaining the 1st example of the method of extracting the attention degree with respect to a content. It is a figure explaining the 2nd example of the method of extracting the attention degree with respect to a content. It is the functional block diagram which illustrated the structure of one Example of an attention object specific device. It is a hardware block diagram of one Example of an attention object specific device. It is the flowchart which illustrated the process sequence of the control process performed in an attention object specific apparatus. It is explanatory drawing of a whole body imaging part control process. It is explanatory drawing of the motion of the onlooker's body in the state which is not paying attention to content. It is explanatory drawing of the motion of the onlooker's body in the state which is paying attention to content. It is explanatory drawing of drive control of the whole body imaging part by an imaging part control part. It is the flowchart which illustrated an example of the process sequence of a whole body image imaging process. 6 is a flowchart illustrating an example of a processing procedure of body movement amount detection / determination processing. It is explanatory drawing of the drive control of the face imaging part by an imaging part control part. It is the flowchart which illustrated an example of the process sequence of a face image imaging process. It is an example of the table in which the arrangement position on the signage of content is represented. It is a figure explaining the table of FIG. It is explanatory drawing of the example of the misjudgment in determination of the attention to the content based on the amount of body movements. It is the flowchart which illustrated the process sequence of the process which a face detection part performs. It is explanatory drawing of provision of a priority. It is the flowchart which illustrated the process sequence of the process which a priority provision part performs.

  In the attention target identification device according to the embodiment described below, a camera for whole-body imaging is used to determine the degree of attention to each person's content when a plurality of people are simultaneously viewing the content displayed on the signage. And two cameras for detecting the line of sight.

  In this apparatus, first, a human body motion and a facial motion are imaged using a whole-body imaging camera. As will be described in detail later, when a person is not paying attention to the content, the person moves the face and body in various directions. However, if attention is directed to something, the movement of the body and face becomes smaller. In this device, the change of the movement of the body and face is captured and the person who is paying attention is extracted.

  As the whole-body imaging camera used in this apparatus, it is preferable to use a wide-angle and high-frame-rate camera, for example, to capture a moving image at a speed of 30 fps (frame / second) or more. When imaging a plurality of people with this whole body imaging camera, the whole body may be imaged one by one while changing the direction of the camera, or the whole body of a plurality of people may be imaged at a time. Good. This apparatus is provided with a function of changing the orientation of the whole body imaging camera.

  Next, in this device, a camera for detecting the line of sight is used to capture an image of the face of a person whose face or body movement is small, or a person whose change in movement is small, and from the obtained face image The direction of the line of sight is detected.

  As a camera for line-of-sight detection, a camera with a lower frame rate than that for whole body imaging is used, and for example, a moving image of a face is captured at a speed of 1 to 20 fps (frames / second) or more. . This is because when a person is paying attention to something, he or she sees the same content continuously for a relatively long period of time, so the change in the movement of the line of sight is small, and therefore moving image capture at a low frame rate. This is because the line of sight can be detected appropriately even if the number of face images acquired is reduced. Note that when face images are picked up by a camera for line-of-sight detection, one person is picked up while changing the direction of the camera.

  The attention target specifying device according to the embodiment has a function of acquiring the attention level of a person who pays attention to the content by using two cameras, the whole body imaging camera and the line-of-sight detection camera, as described above. .

With reference to FIG. FIG. 3 is a functional block diagram illustrating the configuration of an embodiment of the target identification device.
The attention target specifying device 10 of FIG. 3 includes a whole body imaging unit 11, a body movement amount detection unit 12, a face imaging unit 13, an imaging unit control unit 14, a line-of-sight detection unit 15, a storage unit 16, and an attention target identification unit 17. Yes.

  The whole body imaging unit 11 captures moving images of a plurality of people who are viewing the content (object) displayed on the signage and are around the signage. More specifically, the whole-body imaging unit 11 is the above-described whole-body imaging camera that images a plurality of human body movements and facial movements.

The body motion amount detection unit 12 detects the body motion amount of each of the plurality of people from the image captured by the whole body imaging unit 11.
The face imaging unit 13 captures a moving image of a human face. More specifically, the face imaging unit 13 is the above-described eye-gaze detection camera that captures an image of a face of a person whose face or body movement is small or a face of a person whose change in movement is small. .

  The imaging unit control unit 14 controls the face imaging unit 13 to detect the face of the person whose body movement amount detected by the body movement amount detection unit 12 is smaller than the threshold among the plurality of persons, or the detected face. The visual field of the face imaging unit 13 is directed to the face of a person whose body movement amount has decreased, and a moving image of the face is captured. The imaging unit control unit 14 also performs drive control of the whole body imaging unit 11.

The line-of-sight detection unit 15 detects the direction of the line of sight of a person whose face is reflected in the image captured by the face imaging unit 13 from the captured image.
The storage unit 16 is a memory that stores a captured image by the whole body imaging unit 11 and the face imaging unit 13 and a history of control by the imaging unit control unit 14. Further, the storage unit 16 stores the detection results of the body movement amounts of the plurality of persons by the body movement amount detection unit 12 and the detection results of the direction of each person's line of sight by the line of sight detection unit 15.

  The attention target identification unit 17 identifies the content that is attracting attention based on the result of detection of the direction of each person's gaze by the gaze detection unit 15 in the information stored in the storage unit 16.

  In addition, when the signage is digital signage, that is, content that is drawing attention may be displayed on a display device (not shown). In this case, the attention target specifying unit 17 is based on the direction of the line of sight detected by the line-of-sight detection unit 15, and further, on the display device of the person and content whose line-of-sight direction is detected by the line-of-sight detection unit 15. The content is specified based on the relative positional relationship with the display position at.

  In this attention object specifying device 10, an imaging unit having a wider angle of view than the face imaging unit 13 may be used as the whole body imaging unit 11. In this way, it is possible to capture images of a large number of people from a captured image of one frame.

  In addition, the frame rate for capturing a moving image of a face by the face imaging unit 13 may be lower than the frame rate for capturing a moving image of a plurality of people by the whole body imaging unit 11. Even if it does in this way, detection of a gaze can be performed appropriately and it leads also to the reduction of the processing amount of a gaze detection, and the saving of the capacity | capacitance of the memory | storage part 16 as a secondary.

  Note that the attention target identification device 10 may further include a face detection unit 21 drawn using broken lines in FIG. The face detection unit 21 detects a person whose face is reflected in the captured image among a plurality of people reflected in the captured image of the whole-body imaging unit 11. In this case, the image capturing unit control unit 14 causes the face image capturing unit 13 to transmit a moving image of the face detected by the face detecting unit 21 for a person whose face appears in the captured image of the whole body image capturing unit 11. Let the image be taken. Although details will be described later, this makes it possible to suppress erroneous detection of a person who is not paying attention to the content as being focused on the content.

  Moreover, the attention object specific apparatus 10 may further be provided with the analysis part 22 drawn using the broken line in FIG. The analysis unit 22 analyzes the degree of attention to the content attracting attention. In the present embodiment, this analysis is performed based on the length of time that the person whose line-of-sight direction is detected by the line-of-sight detection unit 15 is directing his / her line of sight to the content. The analysis unit 22 performs this analysis using the line-of-sight detection result by the line-of-sight detection unit 15 stored in the storage unit 16.

  Moreover, the attention object specific apparatus 10 may further be provided with the priority provision part 23 drawn using the broken line in FIG. The priority assigning unit 23 sets the priority indicating the level of the degree of attention to the content attracting attention among the plurality of people described above relative to each of the plurality of people and the content. Is given to each of the plurality of persons based on the physical positional relationship. In this case, the imaging unit control unit 14 determines a person whose body motion amount detected by the body motion amount detection unit 12 is smaller than a threshold or a person whose detected body motion amount has decreased among a plurality of people. Then, the visual field of the face imaging unit 13 is directed in order from the highest priority to capture a moving image of the face. As will be described in detail later, by doing this, content that attracts a lot of attention is detected at an earlier stage as the degree of attention to the content is relatively high among a plurality of people watching the content. Can be identified early.

  The attention target identification device 10 in FIG. 3 is configured as described above. In this attention object identification device 10, first, whole body images of a plurality of people around the signage are acquired, and the movements (body movements) of the faces and bodies of the plurality of people are captured from the whole body images. Based on the amount of body movement, it is determined which person is paying attention to the content. In this attention object specifying device 10, it is further confirmed in detail whether or not this person is paying attention to the content by detecting the line of sight of the person determined to be paying attention to the content. In this way, the attention target identification device 10 performs a two-stage check as to whether or not attention is paid to content. In this way, the target object specifying device 10 can accurately detect which content has a high degree of attention when a plurality of people are watching the same signage at the same time. Therefore, high detection performance can be obtained when detecting the degree of attention.

Next, FIG. 4 will be described. FIG. 4 is a hardware configuration diagram of an embodiment of the target identification device.
In FIG. 4, the attention target identification device 10 includes a computer 30, a whole body imaging camera 41, a face imaging camera 42, and a camera driving device 43.

  The computer 30 includes the body movement amount detection unit 12, the imaging unit control unit 14, the line-of-sight detection unit 15, the storage unit 16, the attention target identification unit 17, the face detection unit 21, the analysis unit 22, and the priority assignment unit 23 illustrated in FIG. Provide each function.

  The whole body imaging camera 41 is a camera that captures whole body images of a plurality of people around the signage watching the content (object) displayed on the signage. The whole body imaging unit 11 in FIG. Provide functionality.

The face imaging camera 42 is a camera that captures a moving image of a human face, and provides the function of the face imaging unit 13 in FIG.
The camera driving device 43 is controlled by the computer 30 functioning as the imaging unit control unit 14 and drives the whole body imaging camera 41 and the face imaging camera 42 to change their directions, thereby changing the field of view of the camera. .

A hardware configuration of the computer 30 will be described.
The computer 30 includes an MPU 31, a ROM 32, a RAM 33, an interface device 34, and a hard disk device 35. These components are connected via a bus line 36, and various data can be exchanged under the management of the MPU 31.

An MPU (Micro Processing Unit) 31 is an arithmetic processing unit that controls the operation of the entire computer 30.
A ROM (Read Only Memory) 32 is a read-only semiconductor memory in which a predetermined basic control program is recorded in advance. The MPU 31 can control the operation of each component of the computer 30 by reading out and executing this basic control program when the computer 30 is activated.

  A RAM (Random Access Memory) 33 is a semiconductor memory that can be written and read at any time and used as a working storage area as necessary when the MPU 31 executes various control programs.

  A whole body imaging camera 41, a face imaging camera 42, and a camera driving device 43 are connected to the interface device 34. The interface device 34 manages the exchange of various information such as captured image data and control data performed with these devices.

  The hard disk device 35 is a storage device that stores various control programs executed by the MPU 31 and various data. The MPU 31 can perform various control processes by reading and executing a predetermined control program stored in the hard disk device 35. The hard disk device 35 functions as the storage unit 16 in FIG.

  As described above, the computer 30 includes the body movement amount detection unit 12, the imaging unit control unit 14, the line-of-sight detection unit 15, the storage unit 16, the attention target identification unit 17, the face detection unit 21, the analysis unit 22, and the priority in FIG. Each function of the degree giving unit 23 is provided. For this purpose, a control program for causing the MPU 31 to perform various control processes described below is created in advance, and the created control program is stored in the hard disk device 35 in advance. Then, a predetermined instruction is given to the computer 30 to cause the MPU 31 to read and execute the above-described control program from the hard disk device 35. This makes it possible for the computer 30 to function as each functional block in FIG. 3 described above.

  Further, the control program described above may be stored in a portable recording medium 51 as depicted in FIG. Examples of the recording medium 51 include a flash memory equipped with a USB (Universal Serial Bus) standard connector, a CD-ROM (Compact Disc Read Only Memory), a DVD-ROM (Digital Versatile Disc Read Only Memory), and the like. . In this case, the computer 30 is provided with the recording medium driving device 50 drawn using broken lines in FIG. 4 and connected to the bus line 36. Then, a predetermined instruction is given to the computer 30 to cause the MPU 31 to read and execute the above-described control program from the recording medium 51 via the recording medium driving device 50. This also allows the computer 30 to function as each functional block in FIG. 3 described above.

  Next, the control process performed in the attention object specific apparatus 10 is demonstrated. FIG. 5 is a flowchart illustrating the processing procedure of this control processing. This control process will be described along this flowchart.

  When the control process of FIG. 5 is started, first, in S100, the whole body image capturing process is performed by the imaging unit control unit 14 and the whole body imaging unit 11. As illustrated in FIG. 6, the whole-body image imaging process is performed by the imaging unit control unit 14 driving the whole-body imaging unit 11 to change the imaging direction of each of the plurality of spectators 3 near the signage 1. This is a process of causing the whole-body imaging unit 11 to capture the moving image. This whole body image capturing process will be described.

Usually, when a person is not paying attention to the content, the face and body are moved in various directions. However, if attention is paid to something, the movement of the body and face becomes small.
For example, as illustrated in FIG. 7A, if the spectator 3 has not yet determined which of the contents 2 on the signage 1 is to be focused on, the spectator 3 is looking around the signage 1. In this case, the body and face of the onlooker 3 are moving. Thereafter, as illustrated in FIG. 7B, when the viewer 3 determines the content 2 of interest, the body and face of the viewer 3 are directed in the direction of the content 2, and the movement of the body and face is reduced. The whole body imaging unit 11 captures a moving image of the whole body of the spectator 3 in order to capture the change in which the movement of the body and face of the spectator 3 becomes large and small.

  As illustrated in FIG. 8, the imaging unit control unit 14 drives and controls the whole-body imaging unit 11 to perform rotational movement (swinging) on the horizontal plane and movement (raising and lowering the neck) that is inclined with respect to the horizontal plane. The whole body image of the plurality of spectators 3 is taken by the whole body imaging unit 11. Here, if a camera having a wide angle of view is used as the whole-body imaging unit 11, the whole-body image of a large number of spectators 3 can be captured in one imaging frame. Can obtain full body images of everyone. Of course, the whole body imaging unit 11 may acquire whole body images of a plurality of spectators 3 one by one.

  In the above-described embodiment, the whole body imaging unit 11 acquires the whole body image of the spectator 3. However, if the change of the body movement of the spectator 3 from large to small is sufficiently captured, the spectator 3 The whole-body imaging unit 11 may acquire the image of the face or the image of the upper body.

  Here, FIG. 9 will be described. FIG. 9 is a flowchart illustrating an example of the processing procedure of the whole-body image capturing process, which is a process performed in the attention target identification device 10 as the process of S100 in the flowchart of FIG. Each process illustrated in the flowchart of FIG. 9 is based on the assumption that the whole-body imaging unit 11 is installed as illustrated in FIG.

The processing procedure of the whole body image capturing process will be described with reference to the flowchart of FIG.
In FIG. 9, first, in S101, the imaging unit control unit 14 performs a process of moving the whole body imaging unit 11 in a direction in which the field of view of [A] in FIG.

  Next, in S102, the whole-body imaging unit 11 is stopped in a direction in which the visual field of [A] in FIG. 8 can be obtained, and a moving image of the whole body of the spectator 3 is, for example, at a frame rate of 30 fps, for example, for 1 second. The imaging unit control unit 14 performs processing for causing the imaging unit 11 to perform imaging.

Next, in S103, the imaging unit control unit 14 performs drive control of the whole body imaging unit 11 and moves the whole body imaging unit 11 in a direction in which the field of view [B] in FIG. 8 is obtained.
Next, in S104, the whole-body imaging unit 11 is stopped in a direction in which the visual field of [B] in FIG. The imaging unit control unit 14 performs processing for causing the imaging unit 11 to perform imaging.

Next, in S105, the imaging unit control unit performs the same processing of S101 and S102 and the same movement and imaging control processing of the whole-body imaging unit 11 as the processing of S103 and S104 until all the spectators 3 that are imaging targets are imaged. 14 repeats. Here, when the imaging of the whole body images of all the spectators 3 in the vicinity of the signage 1 is completed, the whole body image imaging processing of FIG. 9 ends, and thereafter the processing proceeds to S200 of FIG.
The process so far is the whole-body image capturing process.

  The description of the control process in FIG. In S200 following the whole body image capturing process in S100, the body movement amount detection / determination process is performed by the body movement amount detection unit 12. In the body movement amount detection / determination process, the body movement amount of the spectator 3 is detected from the captured image acquired by the whole-body imaging unit 11 in the process of S100, and the content 2 on the signage 1 is viewed by the spectator based on the detected body movement amount. 3 is a process for determining whether or not attention is paid.

  Here, FIG. 10 will be described. FIG. 10 is a flowchart illustrating an example of the procedure of the body movement amount detection / determination process, which is a process performed in the target object identification device 10 as the process of S200 in the flowchart of FIG.

The process of FIG. 10 is performed on the whole body image of each spectator 3 included in the captured image acquired by the whole body imaging unit 11.
First, in step S <b> 201, the body movement amount detection unit 12 performs a process of extracting a whole body image (subject image) of the spectator 3 for each frame of the captured image.

  Next, in S202, the body movement amount detection unit 12 performs a process of detecting a nostril image from the subject image extracted in the process of S201 and acquiring a position of the nostril image on the captured image. In this embodiment, the position of the nostril image is used as a reference for detecting the amount of body movement of the spectator 3.

  Next, in S203, the body movement amount detection unit 12 performs a process of calculating a difference value of the position of the nostril image between the preceding frame and the subsequent frame for every two consecutive frames in the captured image. In the present embodiment, this difference value calculated in the frame order of the captured image is used as the body movement amount of the spectator 3.

  Next, in S204, the body motion amount detection unit 12 performs a process of determining whether or not the body motion amount acquired in the frame order by the processing in S203 satisfies at least one of the following two conditions. The first condition of these two conditions is that the amount of body movement acquired in frame order is monotonously decreasing in that order, and the second of these two conditions is acquired in frame order. All of the amount of body movement performed is smaller than the predetermined threshold th. When the body movement amount detection unit 12 determines that at least one of the first condition and the second condition is satisfied (when the determination result is Yes), the body movement amount detection unit 12 advances the process to S205. On the other hand, when the body movement amount detection unit 12 determines that neither the first condition nor the second condition is satisfied (when the determination result is No), the process proceeds to S206.

  In S205, a body motion amount detection unit performs a process of determining that the onlooker 3 whose body motion amount detection / determination process is being processed is a whole body image is the onlooker 3 who is paying attention to the content 2 on the signage 1. 12 does. Thereafter, the body movement amount detection unit 12 ends the body movement amount detection / determination process, and advances the process to S300 in FIG.

On the other hand, in S <b> 206, a process for determining that the onlooker 3 whose body movement amount detection / determination process is being performed is a target person who is paying attention to the content 2 on the signage 1 is determined. The detection part 12 performs. Thereafter, the body movement amount detection unit 12 ends the body movement amount detection / determination process, and advances the process to S300 in FIG.
The above processing is the body motion amount detection / determination processing.

  In the body movement amount detection / determination process in FIG. 10, the position of the nostril image on the captured image is used as a reference for detecting the body movement amount of the spectator 3. It may be used as a reference for detecting the amount of body movement. That is, as a reference for detecting the amount of body movement of the spectator 3, for example, the position of the spectator 3's eye (pupil) or the position of the mouth image on the captured image may be used. The center of the face may be obtained from the contour of the image, and the position of the center on the captured image may be used.

  The description of the control process in FIG. In S300 following the body movement amount detection / determination process in S200, the imaging unit control unit 14 and the facial imaging unit 13 perform a facial image imaging process. In the face image capturing process, the image capturing unit control unit 14 drives the face image capturing unit 13 so that the image capturing direction is directed to the target of detection of the line-of-sight direction, and the face image capturing unit 13 captures a moving image of the face. It is. Note that the subject of detection of the direction of the line of sight is a spectator 3 determined to be paying attention to the content 2 on the signage 1 by the body movement amount detection / determination process. This face image capturing process will be described.

  As illustrated in FIG. 11, the imaging unit control unit 14 drives and controls the face imaging unit 13 to perform rotational movement (swinging) on the horizontal plane and movement (raising and lowering the neck) inclined with respect to the horizontal plane. The imaging direction is directed to the detection target person 6. At this time, the imaging unit control unit 14 also adjusts the zoom amount of the face imaging unit 13 to adjust the field of view of the face imaging unit 13 to the face of the detection target person 6.

  Note that initial information (installation position, neck swing angle, elevation angle, zoom amount, etc.) between the face imaging unit 13 and the whole body imaging unit 11 is grasped in advance. The face imaging unit 13 and the whole body imaging unit 11 may be installed at the same position, or both may be installed at different positions.

  Here, FIG. 12 will be described. FIG. 12 is a flowchart illustrating an example of the processing procedure of the face image capturing process, which is a process performed in the attention target identification device 10 as the process of S300 in the flowchart of FIG.

  The process of FIG. 12 is performed for each one of the spectators 3 who are determined to be paying attention to the content 2 on the signage 1 by the body movement amount detection / determination process.

First, in S301, the imaging unit control unit 14 performs a process of referring to the whole body image of the detection target person 6.
Next, in S302, the whole body image of the detection target person 6 being referred to, the installation position information of the whole body imaging unit 11, and control information of the whole body imaging unit 11 at the time of imaging the whole body image (such as a neck swing angle and an elevation angle). Based on the above, the imaging unit control unit 14 performs a process of specifying the position of the person 6 to be detected.

  Next, in S303, based on the result of specifying the position of the detection target person 6 in the process of S302 and the installation position information of the face imaging unit 13, the face imaging unit 13 for imaging the face image of the detection target person 6 is displayed. The imaging unit control unit 14 performs processing for calculating control information (neck swing angle, elevation angle, zoom amount).

Next, in S304, the imaging unit control unit 14 performs a process of driving and controlling the face imaging unit 13 based on the control information calculated in the process of S303.
Next, in S <b> 305, the imaging unit control unit 14 performs processing for controlling the face imaging unit 13 to capture a moving image of the face of the detection target person 6. When the face image capturing is completed, the face image capturing process in FIG. 12 ends, and thereafter the process proceeds to S400 in FIG.

  The description of the control process in FIG. In S400 following the face image capturing process in S300, the line-of-sight detection unit 15 performs the line-of-sight detection process. In this process, line-of-sight information indicating the direction of the line of sight of the detection target person 6 is detected from the face image of the detection target person 6 captured by the face image capturing process. In this embodiment, the line-of-sight information is detected using the positions of the eyes and nose, the pupil of the eyes, and the corneal reflection in the face image. In addition, in order to obtain information on the pupil of the eye and corneal reflection from the face image, the detection target person 6 is irradiated with infrared light emitted using an infrared light emitting diode or the like as the face imaging unit 13. You may make it use the infrared imaging device which images the infrared image of 6 faces.

  Next, in S500, the whole body image obtained in the process of S100, the image data of the face image obtained in the process of S300, the amount of body movement obtained in the process of S200, and the line-of-sight information obtained in the process of S400. Is stored in the storage unit 16. Note that the storage unit 16 stores the image data, the amount of body movement, and the line-of-sight information in association with each spectator 3, and further, the imaging date / time data of the image data and the imaging unit control unit 14 at the time of imaging. The control information is also stored in association with each other. By doing so, it is possible to analyze the degree of attention to the content 2 later. The body movement amount and the line-of-sight information can be detected later again from the other various types of stored data, and may be excluded from the storage target in the storage unit 16.

  Next, in S600, the imaging unit control unit 14 performs a process of determining whether or not to end the data acquisition for determining the degree of attention to the content 2. The end of the data acquisition is instructed, for example, by the user of the target object specifying device 10. Here, when it is determined that the data acquisition end instruction has been received (when the determination result is Yes), the imaging unit control unit 14 advances the processing to S700. On the other hand, when it is determined that the data acquisition end instruction has not been received (when the determination result is “No”), the imaging unit control unit 14 returns the process to S100 and repeats the above-described process. Continue acquisition.

  Next, in S700, the target content specifying unit 17 performs the target content specifying process. This process is a process of identifying the content 2 that is being watched by the detection target person 6 based on the direction of the line of sight detected for the detection target person 6. This process will be described.

  The whole body imaging unit 11 and the face imaging unit 13 are fixedly installed at a position in the vicinity of the place where the signage 1 is installed, and the head is swung, the neck is raised and lowered, and zooming is performed. Accordingly, it is possible to specify which content 2 on the signage 1 is viewed based on the captured image. For this purpose, the attention target identification unit 17 performs the following processing.

  First, the distance between the person and the signage 1 is determined based on the image of the person being imaged (the height is, for example, 1.. It can be roughly determined from the number of pixels (assuming about 7 m). Further, since the distance between both eyes (between the pupils) of the person's face is approximately 65 mm, the approximate distance between the person and the signage 1 using the ratio of the distance corresponding to the distance between the eyes in the acquired image and the number of pixels. Can be requested. Furthermore, the position and direction of the signage 1 viewed from the detection target person 6 are obtained based on the imaging conditions (neck angle, elevation angle, and zoom amount) when the face imaging unit 13 captures the face image of the detection target person 6. be able to. From these, the positional relationship between the person to be detected 6 and the signage 1 is specified. The target object specifying unit 17 performs the above calculation to obtain the positional relationship information between the detection target person 6 and the signage 1.

On the other hand, the attention target specifying unit 17 is prepared in advance with a table showing the arrangement position of the content 2 on the signage 1. FIG. 13 is an example of this table.
In the table of FIG. 13, coordinate data representing an arrangement position on the signage 1 that is a display device is shown for each of the products A, B, C,. In this table, for example, for the product “A”, the “horizontal direction” coordinates are “α11 to α12”, and the “vertical direction” coordinates are “β11 to β12”. As illustrated in FIG. 14, when a two-dimensional orthogonal coordinate with the upper right end as the origin is defined on the signage 1, the product A has a range of coordinates α11 to α12 in the P direction (left and right direction), and , It is arranged in the range of coordinates β11 to β12 in the Q direction (vertical direction).

  The attention target specifying unit 17 uses the positional relationship information between the detection target person 6 and the signage 1, the above-described table, and the line-of-sight information of the detection target person 6 so that the line of sight of the detection target person 6 penetrates the content 2. Processing to determine whether or not. And the process which specifies the content 2 penetrated by the visual line of the detection subject 6 is performed.

The identification of the content 2 focused on by the person to be detected 6 is performed as described above.
In the noted content specifying process described above, a table showing coordinate data representing the arrangement position of the content 2 on the signage 1 is prepared in the noted target specifying unit 17. Instead, an image of the imaging position and imaging direction aligned with the position of the spectator 3 when viewing the signage 1 and the direction of the line of sight is captured in advance by the camera, and this captured image is prepared in the attention target specifying unit 17. To leave. And the attention object specific | specification part 17 may perform the process which specifies the content 2 which the detection object person 6 is paying attention using this image information.

  The description of the control process of FIG. When the attention object identification device 10 includes the analysis unit 22, the analysis unit 22 performs the attention degree analysis process in S800 following the attention content identification process in S700. This process is a process of analyzing the level of attention (attention level) to the content 2 that is attracting attention. In the present embodiment, the length of time during which the detection target person 6 is paying attention to the content 2 (with a line of sight) is set as the attention level, and the attention level is totaled for each detection target person 6 and for each content 2. Like that. Note that the analysis unit 22 calculates the total using the identification information of the content 2 by the attention target identification unit 17 and the imaging date / time data of the face image for each detection target person 6 stored in the storage unit 16. Do. After the attention level analysis process is completed, the control process of FIG. 5 ends.

By performing the control processing described above, the attention target specifying device 10 specifies the content 2 that is attracting attention from the onlooker 3.
In the control process described above, is the spectator 3 paying attention to the content 2 on the signage 1 based on the amount of body movement obtained from the whole body captured image of the spectator 3 captured by the whole-body imaging unit 11? Judgment is made. However, in this determination, as illustrated in FIG. 15, a spectator 3b standing with his back to the signage 1 or a spectator 3d whose line of sight deviates from the signage 1 is stopped in the same manner as the spectators 3a and 3c. , It may be erroneously determined that the content 2 is focused. The face detection unit 21 described above is provided in the target object identification device 10 in order to suppress such erroneous determination.

  Here, FIG. 16 will be described. FIG. 16 is a flowchart illustrating a processing procedure of processing performed by the face detection unit 21. This flowchart is inserted between the process of S204 and the process of S205 in the body movement amount detection / determination process flowchart of FIG.

  When the result of the determination process in S204 of FIG. 10 is Yes, first, in S211, a face image is detected from the whole body image of the person 3 to be determined, which is reflected in the captured image acquired by the whole body imaging unit 11. Processing is performed by the face detection unit 21. The face image is detected by, for example, detecting a skin-colored oval-shaped area (face) from the upper part of the whole-body image.

  Next, in S212, the face detection unit 21 performs a process of determining whether or not a face image has been detected by the detection process of S211. Here, when the face detection unit 21 determines that the face image has been detected (when the determination result is Yes), the onlooker 3 faces the signage 1 and therefore pays attention to the content 2. And the process proceeds to S205 in FIG. On the other hand, when the face detection unit 21 determines that the face image could not be detected (when the determination result is No), the onlooker 3 is not facing the signage 1, so pay attention to the content 2. If it is not determined, the process proceeds to S206 in FIG.

When the face detection unit 21 performs the above processing, erroneous determination in determination of attention to the content 2 based only on the amount of body movement is suppressed.
Note that the face detection unit 21 not only detects the face image, but also further detects the image of the facial part such as the eyes and nose, and determines whether or not the onlooker 3 is facing the signage 1. The accuracy may be further increased.

  By the way, when a plurality of onlookers 3 who are interested in the content 2 on the signage 1 are extracted by the above-described body movement amount detection / determination process, priority is given to the gaze detection process for each of the extracted onlookers 3. You may make it give a degree. The priority assigning unit 23 described above is provided in the attention target specifying device 10 in order to assign this priority to each of the extracted spectators 3.

  The priority assigning unit 23 assigns a priority set based on the relative positional relationship between the spectator 3 and the content 2 to each spectator 3. In the present embodiment, the priority assigning unit 23 gives a higher priority to the spectator 3 whose distance to the signage 1 is short, and a person whose direction with respect to the signage 1 is close to a right angle (that is, close to the front), Give high priority. In the example illustrated in FIG. 17, the distance to the signage 1 is the shortest among the spectators 3 e, 3 f, 3 g, and 3 h and is the highest for the spectator 3 g facing the signage 1. The priority is given by the priority assigning unit 23.

  Here, FIG. 18 will be described. FIG. 18 is a flowchart illustrating the contents of processing by the priority assigning unit 23. The processing procedure illustrated in this flowchart is performed following the processing of S205 in the flowchart of the body movement amount detection / determination processing in FIG.

  Following S205 of FIG. 10, in S221, priority is given to the process of acquiring the captured image captured by the whole-body imaging unit 11 and the control information for the whole-body imaging unit 11 of the imaging unit control unit 14 at the time of capturing the captured image. The degree giving unit 23 performs this. Note that the captured images acquired by this processing are for all the spectators 3 (that is, the line-of-sight detection target 6) that are determined to be focused on the content 2 by the processing of S205.

  Next, in S222, the priority provision part 23 performs the process which calculates the distance of the detection subject 6 and the signage 1. FIG. The calculation of this distance may be the same as the calculation method in the attention object specifying unit 17 described above, for example.

  Next, in S223, the priority assigning unit 23 performs a process of acquiring a relative angle between the detection target person 6 and the signage 1. In this embodiment, in this embodiment, the relative angle is obtained from the lateral neck swing angle of the whole body imaging unit 11 when the whole body image of the detection subject 6 is captured. For example, when a whole body image of the detection target person 6 is imaged and the neck swing angle of the whole body imaging unit 11 is perpendicular to the surface of the signage 1, the detection target person 6 moves to the signage 1. It is considered to be directly facing.

Next, in S224, the priority assigning unit 23 performs a process of assigning priorities to the detection target person 6 in order of increasing distance calculated by the process of S222.
Next, in S225, a process of giving priority to the detection target person 6 having the same distance calculated by the process of S222 in the order in which the relative angle obtained by the process of S223 is close to a right angle. 23 does. Then, the priority assigning unit 23 ends the process of FIG. 18 and returns the process to the body movement amount detection / determination process of FIG. Then, the body movement amount detection unit 12 ends the body movement amount detection / determination process in FIG. 10 and advances the process to S300 in FIG.

  The priority assigning unit 23 performs the above processing, so that the priority of the line-of-sight detection processing is given to each detection target person 6. The face image capturing process in S300 and the line-of-sight detection process in S400, which are executed thereafter, are performed in the order according to this priority for each detection target person 6.

In addition, the following additional remarks are disclosed regarding the embodiment described above.
(Appendix 1)
A first imaging unit that captures a plurality of moving images;
A body movement amount detection unit that detects the amount of body movement of each of the plurality of persons from a captured image in the first imaging unit;
A second imaging unit that captures a moving image of each face of the plurality of persons;
By controlling the second imaging unit, among the plurality of persons, the detected body movement amount is smaller than a threshold value of a person's face, or the detected body movement amount of the person's face is decreased. A control unit for directing the field of view of the two imaging units to capture a moving image of the face;
A line-of-sight detection unit that detects the direction of the line of sight of a person whose face is reflected in the captured image in the second imaging unit;
An object of interest identification unit for identifying an object that has attracted attention based on the direction of the detected line of sight;
A target object specifying device comprising:
(Appendix 2)
The attention object specifying device according to appendix 1, wherein the first imaging unit has a wider angle of view than the second imaging unit.
(Appendix 3)
The frame rate in capturing the moving image of the face by the second imaging unit is lower than the frame rate in capturing the moving image of the plurality of people by the first imaging unit. Attention target identification device.
(Appendix 4)
A face detection unit for detecting a person whose face is reflected in the captured image among the plurality of people reflected in the captured image of the first imaging unit;
The control unit causes the second imaging unit to capture a moving image of a face of a person whose face is reflected in the captured image of the first imaging unit;
The attention object specifying device according to any one of Supplementary notes 1 to 3, characterized in that:
(Appendix 5)
The degree of attention to the object that is attracting attention based on the length of time that the person whose direction of the line of sight is detected by the line-of-sight detection unit directs the line of sight to the object that is attracting attention The attention object specifying device according to any one of Supplementary notes 1 to 4, further comprising an analysis unit that analyzes
(Appendix 6)
The priority representing the rank of the degree of attention to the target that is attracting attention among the plurality of persons is set as a relative positional relationship between each of the plurality of persons and the target. And further including a priority assigning unit to be given to each of the plurality of people,
The control unit is configured to apply the second imaging to the person whose detected body movement amount is smaller than a threshold value or the person whose detected body movement amount has decreased among the plurality of persons in descending order of priority. Directing the part's field of view to capture a moving image of the face,
The attention object specifying device according to any one of Supplementary notes 1 to 5, characterized in that:
(Appendix 7)
The object attracting attention is displayed on a display device,
The target object specifying unit is based on the detected line-of-sight direction, and on the display device of the target that is attracting attention and the person whose line-of-sight direction is detected by the line-of-sight detection unit. Identifying the object based on the relative positional relationship with the display position at
The attention object specifying device according to any one of Supplementary notes 1 to 6, characterized in that.
(Appendix 8)
Captured images at the first imaging unit and the second imaging unit, a control history by the control unit, a detection result of the body movement amount by the body movement amount detection unit, and a detection result of the gaze direction by the gaze detection unit Is further provided with a storage unit for storing
The target object specifying unit specifies the target object using content stored in the storage unit,
The attention object specifying device according to any one of Supplementary notes 1 to 7, characterized in that.
(Appendix 9)
Capture multiple people's moving images with the first imaging unit,
Detecting the amount of body movement of each of the plurality of people from a captured image in the first imaging unit;
A second imaging unit that captures a moving image of each of the plurality of people's faces is controlled, and the detected body movement amount of the plurality of people is smaller than a threshold value or the detection The moving image of the face is captured by directing the visual field of the second imaging unit to the face of the person whose body movement amount is reduced,
Detecting from the captured image the direction of the line of sight of the person whose face is reflected in the captured image in the second imaging unit;
Identifying an object attracting attention based on the direction of the detected line of sight;
A method for identifying an object of interest characterized by that.
(Appendix 10)
Detecting a person whose face is reflected in the captured image among the plurality of persons reflected in the captured image in the first imaging unit;
When controlling the second imaging unit, the second imaging unit is made to capture a moving image of a face about a person whose face is reflected in the captured image of the first imaging unit.
The attention target specifying method according to supplementary note 9, characterized by:
(Appendix 11)
Analyzing the degree of attention to the object that is attracting attention based on the length of time that the person whose direction of the line of sight is detected is directing the line of sight to the object that is attracting attention. The attention target specifying method according to Supplementary Note 9 or 10, which is a feature.
(Appendix 12)
The priority representing the rank of the degree of attention to the target that is attracting attention among the plurality of persons is set as a relative positional relationship between each of the plurality of persons and the target. Based on each of the plurality of persons,
In the control of the second imaging unit, among the plurality of persons, the detected body movement amount is smaller than a threshold value, or the detected body movement amount is decreased. Directing the visual field of the second imaging unit to capture a moving image of the face;
The attention target specifying method according to any one of Supplementary notes 9 to 11, characterized in that:
(Appendix 13)
The object attracting attention is displayed on a display device,
When specifying the object attracting attention, the display device of the object attracting attention and the person who is detected based on the direction of the detected line of sight and further detecting the direction of the line of sight Identifying the object based on a relative positional relationship with the display position above;
The attention target specifying method according to any one of appendices 9 to 12, characterized by:
(Appendix 14)
The captured image in the first imaging unit and the second imaging unit, the control history of the second imaging unit, the detection result of the body movement amount, and the detection result of the direction of the line of sight are stored in the storage unit,
When specifying the target that is attracting attention, the target is specified using the storage content in the storage unit.
14. The attention target specifying method according to any one of supplementary notes 9 to 13, characterized by:
(Appendix 15)
Capture multiple people's moving images with the first imaging unit,
Detecting the amount of body movement of each of the plurality of people from a captured image in the first imaging unit;
A second imaging unit that captures a moving image of each of the plurality of people's faces is controlled, and the detected body movement amount of the plurality of people is smaller than a threshold value or the detection The moving image of the face is captured by directing the visual field of the second imaging unit to the face of the person whose body movement amount is reduced,
Detecting from the captured image the direction of the line of sight of the person whose face is reflected in the captured image in the second imaging unit;
Identifying an object attracting attention based on the direction of the detected line of sight;
A program that causes an arithmetic processing unit to execute processing.
(Appendix 16)
Among the plurality of people reflected in the captured image in the first imaging unit, the processing unit further performs processing for detecting a person whose face is reflected in the captured image,
The process of controlling the second imaging unit causes the second imaging unit to capture a moving image of a face about a person whose face is reflected in the captured image of the first imaging unit.
The program according to appendix 15.
(Appendix 17)
A process of analyzing the degree of attention to the target that is attracting attention based on the length of time that the person whose direction of the line of sight is detected is directing the line of sight to the target that is attracting attention The program according to appendix 15 or 16, which is further executed by the arithmetic processing unit.
(Appendix 18)
The priority representing the rank of the degree of attention to the target that is attracting attention among the plurality of persons is set as a relative positional relationship between each of the plurality of persons and the target. Based on the above, the arithmetic processing device is further executed processing to be given to each of the plurality of people,
The process of controlling the second imaging unit is performed on a person who has the detected body movement amount smaller than a threshold value or a person who has the detected body movement amount reduced among the plurality of persons. Directing the visual field of the second imaging unit in descending order to capture a moving image of the face;
The program according to any one of appendices 15 to 17.
(Appendix 19)
The object attracting attention is displayed on a display device,
The process of identifying the object that is attracting attention is based on the direction of the detected line of sight, and further, the display of the object that is attracting attention and the person who has detected the direction of the line of sight Identifying the object based on a relative positional relationship with a display position on the device;
The program according to any one of appendices 15 to 18.
(Appendix 20)
The process of storing the captured image in the first imaging unit and the second imaging unit, the control history of the second imaging unit, the detection result of the body movement amount, and the detection result of the direction of the line of sight in the storage unit Let the arithmetic processing unit further execute,
The process of identifying the object that is attracting attention is to identify the object using the storage content in the storage unit.
The program according to any one of Supplementary Notes 15 to 19.

DESCRIPTION OF SYMBOLS 1 Signage 2 Content 3, 3a, 3b, 3c, 3d, 3e, 3f, 3g, 3h A spectator 4 Camera 5 Wide-angle camera 6 Detection target 10 Attention target specific apparatus 11 Whole body imaging part 12 Body motion detection part 13 Face imaging part 14 Imaging unit control unit 15 Gaze detection unit 16 Storage unit 17 Target object identification unit 21 Face detection unit 22 Analysis unit 23 Priority assignment unit 30 Computer 31 MPU
32 ROM
33 RAM
34 interface device 35 hard disk device 36 bus line 41 whole body imaging camera 42 face imaging camera 43 camera driving device 50 recording medium driving device 51 recording medium

Claims (10)

A first imaging unit that captures a plurality of moving images;
A body movement amount detection unit that detects the amount of body movement of each of the plurality of persons from a captured image in the first imaging unit;
A second imaging unit that captures a moving image of each face of the plurality of persons;
By controlling the second imaging unit, among the plurality of persons, the detected body movement amount is smaller than a threshold value of a person's face, or the detected body movement amount of the person's face is decreased. A control unit for directing the field of view of the two imaging units to capture a moving image of the face;
A line-of-sight detection unit that detects the direction of the line of sight of a person whose face is reflected in the captured image in the second imaging unit;
An object of interest identification unit for identifying an object that has attracted attention based on the direction of the detected line of sight;
A target object specifying device comprising:   The target object specifying device according to claim 1, wherein the first imaging unit has a wider angle of view than the second imaging unit.   3. The frame rate for capturing the moving image of the face by the second imaging unit is lower than the frame rate for capturing the moving images of the plurality of people by the first imaging unit. The target object identification device described. A face detection unit for detecting a person whose face is reflected in the captured image among the plurality of people reflected in the captured image of the first imaging unit;
The control unit causes the second imaging unit to capture a moving image of a face of a person whose face is reflected in the captured image of the first imaging unit;
The target object specifying device according to any one of claims 1 to 3, wherein   The degree of attention to the object that is attracting attention based on the length of time that the person whose direction of the line of sight is detected by the line-of-sight detection unit directs the line of sight to the object that is attracting attention The target object specifying device according to any one of claims 1 to 4, further comprising an analysis unit that analyzes the information. The priority representing the rank of the degree of attention to the target that is attracting attention among the plurality of persons is set as a relative positional relationship between each of the plurality of persons and the target. And further including a priority assigning unit to be given to each of the plurality of people,
The control unit is configured to apply the second imaging to the person whose detected body movement amount is smaller than a threshold value or the person whose detected body movement amount has decreased among the plurality of persons in descending order of priority. Directing the part's field of view to capture a moving image of the face,
The target object specifying device according to any one of claims 1 to 5, wherein: The object attracting attention is displayed on a display device,
The target object specifying unit is based on the detected line-of-sight direction, and on the display device of the target that is attracting attention and the person whose line-of-sight direction is detected by the line-of-sight detection unit. Identifying the object based on the relative positional relationship with the display position at
The attention object specifying device according to any one of claims 1 to 6, wherein Captured images at the first imaging unit and the second imaging unit, a control history by the control unit, a detection result of the body movement amount by the body movement amount detection unit, and a detection result of the gaze direction by the gaze detection unit Is further provided with a storage unit for storing
The target object specifying unit specifies the target object using content stored in the storage unit,
The attention object specifying device according to any one of claims 1 to 7, wherein Capture multiple people's moving images with the first imaging unit,
Detecting the amount of body movement of each of the plurality of people from a captured image in the first imaging unit;
A second imaging unit that captures a moving image of each of the plurality of people's faces is controlled, and the detected body movement amount of the plurality of people is smaller than a threshold value or the detection The moving image of the face is captured by directing the visual field of the second imaging unit to the face of the person whose body movement amount is reduced,
Detecting from the captured image the direction of the line of sight of the person whose face is reflected in the captured image in the second imaging unit;
Identifying an object attracting attention based on the direction of the detected line of sight;
A method for identifying an object of interest characterized by that. Capture multiple people's moving images with the first imaging unit,
Detecting the amount of body movement of each of the plurality of people from a captured image in the first imaging unit;
A second imaging unit that captures a moving image of each of the plurality of people's faces is controlled, and the detected body movement amount of the plurality of people is smaller than a threshold value or the detection The moving image of the face is captured by directing the visual field of the second imaging unit to the face of the person whose body movement amount is reduced,
Detecting from the captured image the direction of the line of sight of the person whose face is reflected in the captured image in the second imaging unit;
Identifying an object attracting attention based on the direction of the detected line of sight;
A program that causes an arithmetic processing unit to execute processing.

Images