JP2010268128A - Control apparatus, imaging apparatus, imaging system, image acquisition method and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a controller, etc., that enables suitability of information which is acquired by an imaging apparatus to be recognizable by another imaging apparatus. <P>SOLUTION: The control apparatus for a second imaging apparatus between a first imaging apparatus and the second imaging apparatus which is movable, independently of each other includes an image acquisition section 32, 132 which acquires an image photographed by an imaging section; an information acquisition section 140, which acquires first imaging apparatus information as information associated with the first imaging apparatus, based on the photographed image of the second imaging apparatus; an information suitability calculation section 150 which calculates information suitability of the acquired first imaging apparatus information; and a transmission section 135 which transmits the calculated information suitability to the first imaging apparatus. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a control device, an imaging device, an imaging system, an image acquisition method, a program, and the like.

  2. Description of the Related Art In recent years, a life log that automatically captures and records a daily scene that a user encounters with a camera that can be worn on the body (hereinafter, appropriately referred to as a wearable camera) has attracted attention. For example, Patent Document 1 discloses a technique for automatically capturing images at a predetermined time interval with a wearable camera and recording a life log image.

  However, when a wearable camera is used, there is a problem that it is possible to shoot a daily landscape but not a user himself. For this reason, there is a problem that important information is lost when looking back at the log (record) such as how the user has reacted.

  In terms of recording the state of the user, for example, Patent Document 2 discloses a technique for recording the state of the driver using two in-vehicle cameras.

  In this prior art, the surrounding landscape is photographed by one of the two cameras. Then, the driver is photographed with another camera, and the driver recognizes and records the object to which the line of sight is directed. By doing so, it becomes possible to display the object watched by the driver later.

  However, in this conventional technology, because it is a limited environment of the driver's seat of the vehicle, a camera for photographing the driver can be fixedly installed, and in the life log for photographing daily scenery, It is not realistic to prepare a camera for user photography separately. For example, when a fixed camera is used as a user shooting camera, the user's reaction or the like can be shot only at a location within the field of view of the fixed camera. For this reason, when the user leaves the place, as in the prior art of Patent Document 1, important information when looking back on the log later is lost.

JP 2004-356970 A JP 2006-172215 A

  According to some aspects of the present invention, it is possible to provide a control device, an imaging device, an imaging system, an image acquisition method, a program, and the like that allow the appropriateness of information acquired by the imaging device to be grasped by another imaging device.

  One embodiment of the present invention is a control device for the second imaging device among a first imaging device and a second imaging device that can move independently from each other, and obtains a captured image captured by an imaging unit. An image acquisition unit that acquires information about the first imaging device based on the captured image of the second imaging device, and an information acquisition unit that acquires the first imaging device information. The control device includes an information suitability calculation unit that calculates the information suitability and a transmission unit that transmits the calculated information suitability to the first imaging device. Another embodiment of the present invention relates to a program that causes a computer to function as each of the above-described units or a computer-readable information storage medium that stores the program.

  According to one aspect of the present invention, the first imaging device information that is information about the first imaging device is acquired based on the captured image of the second imaging device, and the information appropriateness of the acquired first imaging device information Is calculated. The calculated information appropriateness level is transmitted from the second imaging device to the first imaging device. Accordingly, the first imaging device can grasp the appropriateness of the first imaging device information acquired by the second imaging device.

  In one aspect of the present invention, the information processing apparatus includes a reception unit that receives the information transmission request for the information appropriateness level from the first imaging device, and the transmission unit includes, as return information for the information transmission request for the information appropriateness level, The information suitability level may be transmitted to the first imaging device.

  In this way, when an information transmission request is received from the first imaging device, the information appropriateness level is transmitted to the first imaging device as the return information.

  In the aspect of the invention, the reception unit receives an information transmission request for the first imaging device information from the first imaging device that has received the information suitability, and the transmission unit receives the first imaging. The first imaging device information may be transmitted to the first imaging device as reply information to the device information information transmission request.

  In this way, for example, when the first imaging device that performed the information appropriateness evaluation process requests transmission of the first imaging device information, the first imaging device information can be transmitted to the first imaging device. It becomes like this.

  In the aspect of the invention, if the transmission unit does not receive an information transmission request for the first imaging device information within a predetermined period after transmitting the information appropriateness level, the first imaging device You may perform the process which discards information and the said information appropriateness degree.

  In this way, it is possible to prevent a situation in which the storage area or the like of the second imaging device is used wastefully.

  In one aspect of the present invention, the information acquisition unit determines whether or not the first imaging device information can be acquired, and determines that the first imaging device information can be acquired. Imaging device information may be acquired.

  In this way, only when the first imaging device information can be acquired by the second imaging device, the first imaging device information is acquired and transmitted to the first imaging device. Therefore, the situation where useless information is transmitted can be prevented.

  In the aspect of the invention, the information acquisition unit may determine whether the first imaging device is included in the field of view of the second imaging device based on the captured image. It may be determined whether or not one imaging device information can be acquired.

  In this way, it becomes possible to determine whether or not the first imaging device information can be acquired using the captured image of the second imaging device.

  In the aspect of the invention, the receiving unit receives the individual identification information of the first imaging device, and the information acquisition unit is configured so that the imaging device included in the field of view is based on the received individual identification information. Even if it is detected whether or not it is the first imaging device to be identified, and it is detected that the imaging device is the first imaging device, it is determined that the first imaging device information can be acquired. Good.

  In this way, the first imaging device information is obtained by detecting whether the imaging device included in the field of view is the first imaging device using the received individual identification information of the first imaging device. It becomes possible to determine whether or not it can be acquired.

  In the aspect of the invention, the information acquisition unit acquires positional relationship information between the first imaging device and the second imaging device, and the first imaging device information is obtained based on the positional relationship information. It may be determined whether or not acquisition is possible.

  In this way, for example, it is possible to determine whether or not the first imaging device information can be acquired from the positional relationship information of the first and second imaging devices without using the captured image of the second imaging device. become.

  In the aspect of the invention, the information appropriateness calculation unit may calculate the information appropriateness based on a shooting state of a user of the first imaging device.

  In this way, the first imaging device can grasp the shooting state of the user of the first imaging device based on the received information appropriateness level.

  In one aspect of the present invention, the information appropriateness calculation unit calculates the information appropriateness based on at least one of a face orientation and a face size of the user of the first imaging device. Also good.

  In this way, the first imaging device can grasp whether the face direction and the size of the face of the user of the first imaging device are appropriate based on the received information appropriateness level. .

  In the aspect of the invention, the information appropriateness degree calculation unit may be configured to use the information appropriateness degree based on at least one of contrast, gradation, brightness, and blur of a captured image of the second imaging device. May be calculated.

  In this way, the first imaging device can grasp the quality of the captured image of the second imaging device that images the first imaging device based on the received information appropriateness level.

  Another aspect of the present invention is a control device for the first imaging device of the first imaging device and the second imaging device that can move independently from each other, and is a captured image captured by an imaging unit. And the second imaging device acquires first imaging device information that is information related to the first imaging device based on a captured image of the second imaging device, and the first imaging device When calculating and transmitting information suitability of imaging device information, the control device includes a receiving unit that receives the information suitability and an information suitability evaluation unit that performs an evaluation process on the received information suitability To do. Another aspect of the present invention relates to a program that causes a computer to function as each of the above-described units, or a computer-readable information storage medium that stores the program.

  According to another aspect of the present invention, the information appropriateness level of the first imaging device information acquired based on the captured image of the second imaging device is transmitted from the second imaging device to the first imaging device. Information appropriateness evaluation processing is performed. Accordingly, the first imaging device can evaluate the information appropriateness of the first imaging device information acquired by the second imaging device, and can determine whether to receive the first imaging device information, for example. become.

  In another aspect of the present invention, the information processing apparatus includes a transmission unit that transmits the information transmission request for the information appropriateness level, and the reception unit receives the information suitability level as return information for the information transmission request for the information appropriateness level. May be.

  In this way, an information transmission request can be transmitted to the second imaging device, and the information appropriateness level can be received as the return information.

  In another aspect of the present invention, the transmission unit transmits an information transmission request for the first imaging device information after receiving the information suitability, and the reception unit receives information on the first imaging device information. The first imaging device information may be received as reply information to the transmission request.

  In this way, for example, the first imaging device that has performed the information appropriateness evaluation process can receive the first imaging device information from the first imaging device by requesting transmission of the first imaging device information. It becomes like this.

  In another aspect of the present invention, the transmission unit may transmit the information transmission request for the information appropriateness level by broadcast.

  In this way, it is possible to broadcast information transmission requests to a plurality of imaging devices around the first imaging device and acquire the information appropriateness level.

  According to another aspect of the present invention, the information processing apparatus includes a transmission unit that transmits an information transmission request for the first imaging device information. It may be determined whether or not to transmit an information transmission request.

  In this way, when the degree of appropriateness of information received from the second imaging device is evaluated to be high, for example, the second imaging device can be requested to transmit the first imaging device information. .

  According to another aspect of the present invention, the information processing apparatus includes a transmission unit that transmits an information transmission request for the first imaging device information. The transmission destination of the information transmission request may be determined.

  If it does in this way, transmission of the 1st image pick-up device information can be demanded to the 2nd image pick-up device which has transmitted the information appropriateness of high evaluation.

  In another aspect of the present invention, the receiving unit receives a plurality of information suitability levels from a plurality of imaging devices, and the information suitability rating unit performs the plurality of information suitability evaluation processes, When the second imaging device is selected from the plurality of imaging devices based on the evaluation results of the plurality of information suitability levels, the transmission unit performs the first imaging on the second imaging device. An information transmission request for device information may be transmitted.

  In this way, when information appropriateness is received from a plurality of imaging devices, a request for transmission of the first imaging device information is made to the second imaging device that has transmitted a highly evaluated information appropriateness. become able to.

  In another aspect of the present invention, the information processing apparatus may include a writing unit that writes the received first imaging device information in a storage unit in association with the captured image of the first imaging device.

  In this way, for example, information in the blind spot of the first imaging device can be stored in the storage unit in association with the captured image.

  Another aspect of the invention relates to an imaging device including any of the above-described control devices.

  According to another aspect of the present invention, there is provided an imaging system including a first imaging device and a second imaging device that are movable independently of each other, wherein the second imaging device is a captured image of the second imaging device. The first imaging device information that is information related to the first imaging device is acquired based on the information, the information appropriateness of the acquired first imaging device information is calculated, and the calculated information appropriateness is calculated as the first information appropriateness. The first imaging device is transmitted to an imaging device, and the first imaging device is related to an imaging system that receives the information suitability and performs an evaluation process on the received information suitability.

  According to another aspect of the present invention, there is provided an image acquisition method using a first imaging device and a second imaging device that are movable independently of each other, and acquires and acquires a captured image of the second imaging device. Obtaining first imaging device information that is information relating to the first imaging device based on the captured image of the second imaging device that has been obtained, calculating an information appropriateness level of the acquired first imaging device information, The calculated information appropriateness is transmitted from the second imaging device to the first imaging device, the information appropriateness is evaluated, a captured image of the first imaging device is acquired, and the information is acquired. The present invention relates to an image acquisition method in which the first imaging device information acquired as a result of the appropriateness evaluation process is written in a storage unit in association with the acquired captured image of the first imaging device.

Explanatory drawing of the method of this embodiment. 1 is a first configuration example of a control device and a camera (imaging device) according to the present embodiment. FIG. 3A to FIG. 3C are explanatory diagrams of a method for calculating the information appropriateness of the CM1 information and transmitting it to the camera CM1. FIG. 4A to FIG. 4E are more specific explanatory diagrams of a method for calculating the information appropriateness of the CM1 information and transmitting it to the camera CM1. FIG. 5A to FIG. 5D are explanatory diagrams of a broadcast transmission method for an information transmission request. 6A to 6C are explanatory diagrams of the degree of information appropriateness. Explanatory drawing of the transmission / reception method of information appropriateness and CM1 information. FIGS. 8A to 8C are explanatory diagrams of a method for identifying the camera CM1 using the individual identification information. 9A and 9B are explanatory diagrams of camera identification marks. FIGS. 10A to 10C are explanatory diagrams of a method for acquiring user information. FIG. 11A and FIG. 11B are explanatory diagrams of a captured image recording method. The flowchart for demonstrating the detailed process of this embodiment. The flowchart for demonstrating information acquisition availability determination processing. The flowchart for demonstrating CM1 information acquisition processing. The 2nd structural example of the control apparatus of this embodiment, and a camera (imaging device). FIGS. 16A and 16B are explanatory diagrams of a method for determining whether or not CM1 information can be acquired using positional relationship information of the cameras CM1 and CM2. The flowchart for demonstrating the information acquisition availability determination process in the 2nd structural example of this embodiment.

  Hereinafter, this embodiment will be described. In addition, this embodiment demonstrated below does not unduly limit the content of this invention described in the claim. In addition, all the configurations described in the present embodiment are not necessarily essential configuration requirements of the present invention.

1. Configuration Example In this embodiment, a camera (an imaging device in a broad sense) that can record not only the front but also information related to the camera such as user information can be provided.

  For example, in FIG. 1, a camera CM1 (first imaging device in a broad sense) and a camera CM2 (second imaging device in a broad sense) are cameras that are movable (portable) independently of each other. That is, it is not a camera that is fixedly installed like a stationary camera, but a camera that the users US1 and US2 can carry freely.

  When shooting with the camera CM1, the state of the user US1 of the camera CM1 and the like are shot with the camera CM2 of another user US2. Then, information about the camera CM1 such as user information of the camera CM1 (hereinafter referred to as CM1 information as appropriate) is acquired. Next, the information appropriateness of the acquired CM1 information is calculated, and the calculated information appropriateness is transmitted from the camera CM2 to the camera CM1.

  The camera CM1 performs an information appropriateness evaluation process received from the camera CM2, and transmits an information transmission request to the camera CM2 based on the result of the information appropriateness evaluation process. Information is received from the camera CM2. The camera CM1 also records the captured image taken by itself and the CM1 information acquired based on the result of the information appropriateness evaluation process. In this case, for example, the cameras CM1 and CM2 automatically perform information transmission requests and exchange of CM1 information without requiring the operations of the users US1 and US2.

  According to the above-described method of the present embodiment, the blind spot of the camera CM1 is photographed by the camera CM2, and thereby CM1 information that is information on a target at a position that cannot be photographed by the camera CM1 is acquired. The information appropriateness of the CM1 information is evaluated, and the CM1 information is transmitted from the camera CM2 to the CM1 based on the result of the evaluation process, and is recorded in association with the captured image of the camera CM1.

  The CM1 information is information (information in the field of view of CM2) obtained from the captured image of the camera CM2, and is information related to the camera CM1. Specifically, the CM1 information is, for example, information about the user US1 of the camera CM1 and information about the camera CM1 itself, such as the orientation of the camera CM1. Alternatively, the captured image itself of the camera CM2 may be used. Also, the user information is, for example, facial expression information (such as what kind of reaction was made) of the user US1 or gaze direction information (what the user was looking at).

  The information appropriateness is an index that indicates whether the CM1 information is good or bad. Specifically, when the CM1 information is acquired by photographing the user US1 or the camera CM1 with the camera CM2 as shown in FIG. 1, it is an index that represents the quality of the photographing state or photographing state. For example, when a captured image in which the face of the user US1 is greatly captured or a captured image in which the line-of-sight direction VD1 faces the front direction is obtained, the information appropriateness of the CM1 information acquired from the captured image is high. Be evaluated. On the other hand, when a captured image in which the face of the user US1 is small or a captured image in which the line-of-sight direction VD1 is oriented in the horizontal direction is obtained, it is evaluated that the information appropriateness of the CM1 information is low. Alternatively, when the contrast of the image captured by the camera CM2 is high, when the gradation is good, when it is bright, or when the degree of blur is small, it is evaluated that the information appropriateness of the CM1 information is high. On the other hand, when the contrast of the captured image of the camera CM2 is low, when the gradation is poor, when it is dark, or when the degree of blur is large, it is evaluated that the information appropriateness of the CM1 information is low.

  The camera CM1 is a camera used by the user US1 for photographing the life log, for example, and FIG. 1 assumes a camera of a type hanging on the neck (portable camera). Besides this, for example, a camera (wearable camera) of a type attached to the glasses used by the user US1 may be used.

  On the other hand, the camera CM2 is a camera used by a user US2 other than the user US1 of the camera CM1 for photographing a life log and the like, and has a function equivalent to that of the camera CM1 as a product image. In response to a request from the camera CM1, the camera CM2 captures the user US1 of the camera CM1, acquires user information such as a line of sight and a facial expression, and sends the user information and information appropriateness to the camera CM1. Bear.

  The names of the cameras CM1 and CM2 are distinguished depending on the roles played by both at a certain timing. For example, the camera CM1 captures the user US2 of the camera CM2 in response to a request from the camera CM2, acquires user information (CM2 information) such as line of sight and facial expression, and the appropriateness of the information, and sends it to the camera CM2. Also good.

  In addition, the imaging apparatus according to the present embodiment may be an imaging apparatus built in not only an electronic camera such as a digital camera and a video camera but also a mobile phone, a portable information terminal, a portable game machine, etc. For the sake of convenience, the imaging apparatus will be referred to as a camera.

  FIG. 2 shows a first configuration example of a control device that can realize the technique of the present embodiment of FIG. 1 and a camera (imaging device) including the control device.

  In FIG. 2, the camera CM <b> 1 (first imaging device) includes an imaging unit 20, a control device 30, an operation unit 60, a storage unit 70, a communication unit 80, a display unit 90, and an information storage medium 98. The camera CM2 (second imaging device) includes an imaging unit 120, a control device 130, an operation unit 160, a storage unit 170, a communication unit 180, a display unit 190, and an information storage medium 198. Note that various modifications such as omitting some of these components (for example, the operation unit, the display unit, and the information storage medium) and adding other components are possible. In FIG. 2, for convenience of explanation, the cameras CM1 and CM2 are shown as having different configurations. However, the cameras CM1 and CM2 can have the same configuration. For example, an information acquisition unit and an information appropriateness calculation unit may be provided in the camera CM1, or a writing unit, a time acquisition unit, and an information appropriateness evaluation unit may be provided in the camera CM2.

  The imaging units 20 and 120 are for imaging a subject (imaging target), and include an optical system such as a lens (not shown), an imaging device, an analog front end circuit (A / D conversion circuit), a focus adjustment unit, and the like. Can be included.

  The control devices (control units) 30 and 130 perform overall control of the cameras CM1 and CM2, various arithmetic processes, image processing, and the like, and can be realized by various processors such as a CPU and a DSP and a dedicated ASIC. The control devices 30 and 130 perform various processes according to the present embodiment based on programs stored in the information storage media 98 and 198, for example.

  The operation units 60 and 160 are used by a user (operator or user) to perform various settings of the cameras CM1 and CM2 and to input various information, and can be realized by various buttons, switches, dials, and the like. When the display units 90 and 190 are configured by a touch panel display, the display units 90 and 190 also function as operation units.

  The storage units 70 and 170 function as work areas for the control devices 30 and 130 and the communication units 80 and 180, and store (save) image data obtained by imaging. RAMs and HDDs (hard disks) Drive).

  The communication units 80 and 180 exchange information with an external device such as another camera or server by wireless or wired communication, and the function thereof is such as a communication ASIC or a communication processor. It can be realized by hardware or communication firmware. For example, the cameras CM1 and CM2 can communicate via wireless (wireless LAN or the like) or a wired network.

  The display units 90 and 190 are realized by, for example, an electro-optical panel such as a liquid crystal panel or an organic EL panel, an EVF (electronic viewfinder), and the like, and display various images such as a through image and a GUI (Graphical User Interface) screen. indicate.

  Information storage media 98 and 198 (computer-readable media) store programs, data, and the like, and their functions are memory cards, HDDs (hard disk drives), optical disks (CDs, DVDs), ROMs, and the like. This can be realized by using a memory. The control devices 30 and 130 perform various processes of the present embodiment based on programs (data) stored in the information storage media 98 and 198. In other words, the information storage media 98 and 198 include a program for causing a computer (an apparatus including an operation unit, a processing unit, a storage unit, and an output unit) to function as each unit of the present embodiment (for causing the computer to execute the processing of each unit). Program).

  The control device 30 of the camera CM1 includes an image acquisition unit 32, a reception unit 34, a transmission unit 35, a writing unit 36, a time acquisition unit 38, and an information appropriateness evaluation unit 50.

  The image acquisition unit 32 acquires a captured image. For example, a captured image captured (captured) by the imaging unit 20 is acquired. Here, the captured image may be a still image or a moving image. For example, a life log image of the user of the camera CM1 can be assumed as the captured image. The life log image is an image that is automatically captured and recorded at a predetermined time interval, for example, even if there is no explicit shooting instruction from the user.

  The reception unit 34 and the transmission unit 35 perform information reception processing and transmission processing, respectively. For example, processing for receiving and transmitting information to and from other devices such as the camera CM 2 is performed via the communication unit 80. That is, information to be communicated is prepared or an instruction to communicate the information is given to the communication unit 80.

  The writing unit 36 performs processing (storing processing) for writing various information such as a captured image (image data) in the storage unit 70. The time acquisition unit 38 performs time acquisition processing. For example, the current time (hour, day, month, year, etc.) is acquired.

  The information appropriateness evaluation unit 50 performs an information appropriateness evaluation process. For example, when the receiving unit 34 receives the information appropriateness calculated by the camera CM1, the received information appropriateness is evaluated.

  The control device 130 of the camera CM 2 includes an image acquisition unit 132, a reception unit 134, a transmission unit 135, an information acquisition unit 140, and an information appropriateness calculation unit 150. The image acquisition unit 132 acquires a captured image captured by the imaging unit 120. The reception unit 134 and the transmission unit 135 perform information reception processing and transmission processing, respectively. For example, information is received or transmitted to other devices such as the camera CM 1 via the communication unit 180. Process. The information acquisition unit 140 acquires various information such as information related to the camera CM1. The information acquisition unit 140 includes a camera identification mark detection unit 141, a face detection unit 142, a facial expression recognition unit 143, a gaze direction recognition unit 144, a camera direction recognition unit 145, and a gaze direction correction unit 146. The information appropriateness calculation unit 150 performs information appropriateness calculation processing (derivation processing, arithmetic processing).

  In the present embodiment, the information acquisition unit 140 of the camera CM2 acquires CM1 information (first imaging device information) that is information about the camera CM1 based on the captured image of the camera CM2 (imaging unit 120). The CM1 information may be a captured image of the camera CM2. Then, the information appropriateness calculation unit 150 calculates the information appropriateness of the acquired CM1 information, and the transmission unit 135 transmits the calculated information appropriateness to the camera CM1.

  The receiving unit 34 of the camera CM1 receives the information appropriateness level from the camera CM2. For example, the information appropriateness transmitted via the communication units 80 and 180 and a wireless network or the like is received. And the information appropriateness evaluation part 50 performs the received information appropriateness evaluation process.

  More specifically, the transmission unit 35 of the camera CM1 transmits an information transmission request. For example, a transmission request for information appropriateness is transmitted to another camera. Specifically, the transmission unit 35 transmits an information transmission request by broadcast to an unspecified number of people, for example.

  Note that the transmission unit 35 of the camera CM1 also performs transmission processing of individual identification information (ID) of the camera CM1. For example, as described later, it is assumed that an identification mark (identification information in a broad sense) for identifying the camera CM1 is set in the camera CM1. In this case, the transmission unit 35 transmits the individual identification information corresponding to the identification mark. For example, the same camera ID as the camera ID represented by the identification mark is transmitted as individual identification information. This individual identification information (camera ID) can be included in, for example, an information transmission request field.

  The receiving unit 134 of the camera CM2 receives the information transmission request from the camera CM1. Then, the information acquisition unit 140 of the camera CM2 acquires CM1 information based on the captured image of the camera CM2. Further, the information appropriateness calculation unit 150 calculates the information appropriateness of the acquired CM1 information.

  Then, the transmission unit 135 of the camera CM2 transmits the calculated information appropriateness level to the camera CM1 as reply information to the information transmission request from the camera CM1.

  The reception unit 34 of the camera CM1 receives the information appropriateness when the camera CM2 that has received the information transmission request calculates and transmits the information appropriateness. And the information appropriateness evaluation part 50 performs the received information appropriateness evaluation process.

  The transmission unit 35 of the camera CM1 determines whether or not to transmit an information transmission request for CM1 information based on the evaluation result of the information appropriateness level. For example, when the information suitability is evaluated to be high, CM1 information is transmitted, and when the information suitability is evaluated to be low, CM1 information is not transmitted. Alternatively, the transmission destination (destination) of the information transmission request for the CM1 information is determined based on the evaluation result of the information appropriateness level. Specifically, the receiving unit 34 receives a plurality of information suitability levels from a plurality of cameras (imaging devices). Then, the information appropriateness evaluation unit 50 performs a plurality of information appropriateness evaluation processes, and the transmission unit 35, when the camera CM2 is selected from a plurality of cameras based on the plurality of information appropriateness evaluation results. Then, an information transmission request for CM1 information is transmitted to the camera CM2. For example, when it is evaluated that the information appropriateness received from the camera CM2 is higher than the information appropriateness received from another camera, an information transmission request for CM1 information is transmitted to the camera CM2. In this case, the transmission of the CM1 information information transmission request is not a broadcast transmission but a transmission with the camera CM2 as the destination.

  When the reception unit 134 of the camera CM2 receives the information transmission request for CM1 information from the camera CM1, the transmission unit 135 transmits CM1 information to the camera CM1 as reply information to the information transmission request for CM1 information. Note that the transmission unit 135 may perform processing for discarding the information suitability and the CM1 information when the information send request is not received within a predetermined period after sending the information suitability. This can prevent a situation where a useless storage area is used.

  When the receiving unit 34 of the camera CM1 receives the CM1 information, the writing unit 36 (save processing unit) of the camera CM1 stores the received CM1 information (first imaging device information) in association with the captured image of the camera CM1. Write to 70.

  In this case, when the writing unit 36 does not receive the CM1 information within a predetermined period after transmitting the information transmission request, the writing unit 36 writes the captured image not associated with the CM1 information in the storage unit 70. For example, only the captured image of the camera CM1 is written in the storage unit 70. When the receiving unit 34 receives information on the user of the camera CM1 as the CM1 information, the received user information is written in the storage unit 70 in association with the captured image of the camera CM1. For example, user information is associated and written in a life log image (an image that is automatically captured and recorded every predetermined time) of the user of the camera CM1.

  Note that the information acquisition unit 140 of the camera CM2 determines whether or not CM1 information can be acquired, and acquires the CM1 information when determining that the CM1 information can be acquired. Then, the transmission unit 135 transmits the acquired CM1 information to the camera CM1.

  In this case, the information acquisition unit 140 determines whether the camera CM1 (part or all) is included in the field of view (within the range of the angle of view) of the camera CM2 based on the captured image of the camera CM2. It is determined whether CM1 information can be acquired. For example, it is determined that CM1 information can be acquired when a camera CM1 (user of CM1) is captured in an image captured by the camera CM2.

  For example, the receiving unit 134 of the camera CM2 receives the individual identification information of the camera CM1. Specifically, the individual identification information included in the information transmission request from the camera CM1 is received. Then, the information acquisition unit 140 detects whether or not the camera (imaging device) included in the field of view of the camera CM2 is the camera CM1 that is identified by the received individual identification information. When it is detected that the camera is CM1, it is determined that CM1 information can be acquired.

  Specifically, the information acquisition unit 140 detects whether or not the identification information set in the camera (imaging device) included in the field of view matches the identification information of the camera CM1 identified by the individual identification information. If they match, it is determined that CM1 information can be acquired. As this identification information, an identification mark (identification figure) such as a barcode can be used, and this identification mark detection process is performed by the camera identification mark detection unit 141.

  Further, the information acquisition unit 140 acquires information about the user of the camera CM1 as CM1 information. For example, at least one of user facial expression information (reaction information) and line-of-sight direction information (direction in which the user is looking) is acquired as user information. Further, the information acquisition unit 140 may acquire the orientation information of the camera CM1. The orientation information of the camera CM1 is used, for example, for correction processing of the user's gaze direction information. The face detection process, facial expression recognition process, and gaze direction recognition process of the user of the camera CM1 are performed by the facial expression recognition unit 143 and the gaze direction recognition unit 144, respectively. The camera CM1 orientation recognition process is performed by the camera orientation recognition unit 145, and the user's gaze direction correction process based on the camera CM1 orientation is performed by the gaze direction correction unit 146.

  The information acquisition unit 140 detects a human face in the field of view of the camera CM2. Then, among the detected one or more person faces, the person face closest to the camera CM1 is determined to be the person face of the user of the camera CM1, and user information is acquired. For example, a human face above (including substantially above) the camera CM1 is determined as the face of the user of the camera CM1. Then, the user's facial expression recognition processing and line-of-sight direction recognition processing are performed. The user's face detection process is performed by the face detection unit 142.

2. 2. Method according to this embodiment 2.1 Information appropriateness of CM1 information Next, the method according to this embodiment will be described in detail with reference to FIGS. 3 (A) to 11 (B).

  In FIG. 3A, the camera CM1 of the user US1 takes a life log image of the user US1, for example. The camera CM2 of the user US2 near the user US1 acquires CM1 information that is information related to the camera CM1 from its captured image. Specifically, it is determined whether or not CM1 information can be acquired by determining whether or not the camera CM1 and its user US1 are within the field of view of the camera CM2. If it is determined that the CM1 information can be acquired, the CM1 information such as the facial expression and line-of-sight direction of the user US1 is acquired.

  Next, as illustrated in FIG. 3B, the camera CM2 calculates the information appropriateness of the acquired CM1 information. For example, the size of the face of the user US1 shown in the captured image, the line-of-sight direction, etc. are evaluated, and the evaluation value is obtained. Then, as shown in FIG. 3C, the calculated information appropriateness level is transmitted to the camera CM1.

  Upon receiving the information appropriateness level of the CM1 information, the camera CM1 performs an evaluation process on the received information appropriateness level. Then, for example, based on the evaluation result of the information suitability, it is determined whether or not to transmit an information transmission request for CM1 information. Alternatively, when there are a plurality of cameras that are the targets of the information transmission request, the camera that is the transmission destination of the information transmission request for the CM1 information is determined based on the evaluation result of the information suitability.

  FIG. 4A to FIG. 4C are diagrams for explaining a more specific method of the present embodiment. First, as shown in FIG. 4A, the camera CM1 transmits an information transmission request in order to acquire the information appropriateness level of the CM1 information. Then, the camera CM2 of the user US2 near the user US1 receives this information transmission request.

  Next, as shown in FIG. 4B, the camera CM2 calculates (derived and calculates) the information appropriateness of the CM1 information. Then, the calculated information appropriateness level is transmitted to the camera CM 1 as reply information to the information transmission request.

  Then, as shown in FIG. 4C, the camera CM1 performs an evaluation process on the received information appropriateness level. For example, when the evaluation value of information suitability is high, an information transmission request for requesting transmission of CM1 information is transmitted to the camera CM2.

  As shown in FIG. 4D, when receiving the information transmission request, the camera CM2 transmits CM1 information as reply information to the camera CM1. Accordingly, the camera CM1 can record the CM1 information in association with the captured image of the camera CM1. For example, when the camera CM1 has taken a life log image of the user US1, it is possible to record the life log image in association with the facial expression, the line-of-sight direction, etc. of the user US1 at the time of shooting.

  As shown in FIG. 4E, after the camera CM2 transmits the information appropriateness, if the CM1 information information transmission request is not received within a predetermined period, the CM1 information and the information appropriateness are discarded. Perform the process. By doing so, it is possible to prevent a situation in which the storage area of the camera CM2 is wasted.

  Alternatively, the camera CM2 may transmit an information transmission request, and the camera CM1 that has received the information transmission request may calculate the information appropriateness of the CM2 information that is information related to the camera CM2 and transmit the information to the camera CM2. In this case, the camera CM1 transmits CM2 information to the camera CM2. The camera CM2 records CM2 information in association with the captured image.

  In the present embodiment, the cameras CM1 and CM2 are portable cameras (wearable cameras) possessed by the users US1 and US2. Therefore, unlike a stationary camera in which the arrangement position is fixed, the cameras CM1 and CM2 move independently as the users US1 and US2 move, and the positional relationship between the cameras CM1 and CM2 changes as needed.

  For this reason, if the CM1 information is unilaterally transmitted from the camera CM2 to the camera CM1, useless CM1 information that cannot be used is transmitted to the camera CM1. For example, there is a case where the shooting state of the camera CM2 is poor, the face of the user US1 shown in the captured image of the camera CM2 is small, or the line-of-sight direction is facing sideways. Alternatively, there may be cases where the image quality such as contrast, gradation, brightness, and blur of the captured image is poor. Even in such a case, if the CM1 information is transmitted from the camera CM2 to the CM1, the processing load of the camera CM1 increases or the storage area is wasted. It will be wasted.

  In this regard, in this embodiment, the camera CM2 obtains the information appropriateness level of the CM1 information. For example, the degree of information suitability obtained by quantifying the evaluation of the size and line-of-sight direction of the user US1 in the captured image of the camera CM2 is obtained. Alternatively, the degree of information suitability obtained by quantifying the quality such as the contrast of the captured image is obtained. The information appropriateness level is transmitted from the camera CM2 to the CM1, and the camera CM1 performs an evaluation process of the information appropriateness level. When the evaluation of the information suitability is high (when the evaluation value is greater than or equal to a predetermined value), the camera CM2 is requested to transmit the CM1 information.

  Thereby, the CM1 information acquired based on the captured image is transferred from the camera CM2 to the CM1 only when the face of the user US1 shown in the captured image is large, when the line-of-sight direction is the front, or when the quality of the captured image is high. Will be sent. Therefore, the camera CM1 can acquire the CM1 information while preventing the situation where the resources of the cameras CM1 and CM2 are wasted. As a result, for example, the camera CM1 can also acquire information in the blind spot of the camera CM1, and can record user information or the like in association with a life log image or the like.

2.2 Broadcast Now, as shown in FIG. 5A, it is desirable that the camera CM1 transmits the information transmission request by broadcast without designating the destination. As a result, as shown in FIG. 5A, an information transmission request is transmitted to the plurality of cameras CM2, CM3, and CM4 around the camera CM1. When a wireless LAN or the like is used for communication between cameras, an information transmission request is transmitted by broadcast to cameras within the wireless LAN communication range.

  As shown in FIG. 5B, when receiving the information transmission request, each of the cameras CM2, CM3, and CM4 acquires CM1 information that is information about the user US1 of the camera CM1. Then, the cameras CM2, CM3, and CM4 calculate the information appropriateness level of the CM1 information, and transmit the calculated information appropriateness level to the camera CM1. Then, the camera CM1 performs a plurality of information appropriateness evaluation processes received from the cameras CM2, CM3, and CM4, and identifies, for example, the camera that transmitted the information appropriateness having the highest evaluation value.

  In FIG. 5C, since the evaluation value of the information appropriateness from the camera CM2 is evaluated to be the highest, the camera CM1 transmits an information transmission request to the CM2. That is, instead of broadcasting as shown in FIG. 5A, an information transmission request for CM1 information is transmitted to the camera CM2 as a destination. As a result, as shown in FIG. 5D, the camera CM2 transmits CM1 information to the camera CM1 as return information of the information transmission request.

  As described above, if an information transmission request for information appropriateness is transmitted by broadcast, information appropriateness from a plurality of cameras CM2, CM3, and CM4 around the user US1 can be acquired. Based on the acquired information appropriateness level, a camera that requests transmission of CM1 information can be selected. Therefore, among the plurality of cameras CM2, CM3, and CM4, it is possible to identify the camera that is shooting the user US1 in the best shooting state (shooting condition) and acquire CM1 information of the camera. This makes it possible to acquire more appropriate CM1 information and record the CM1 information in association with, for example, a life log image.

  For example, FIGS. 6A to 6C show examples of captured images of the cameras CM2, CM3, and CM4. In the captured image of the camera CM2 in FIG. 6A, the face FA1 of the user US1 is greatly reflected, and the line-of-sight direction VD1 is also facing the front direction. Therefore, in this case, it is evaluated that the information appropriateness of the CM1 information acquired based on the captured image of the camera CM2 is high.

  On the other hand, in the captured image of the camera CM3 in FIG. 6B, the face FA1 of the user US1 is shown small, and the line-of-sight direction VD1 is also in the horizontal direction. Therefore, in this case, it is evaluated that the information appropriateness of the CM1 information acquired based on the captured image of the camera CM3 is low. Further, in the captured image of the camera CM4 in FIG. 6C, the size of the face FA1 of the user US1 is medium, so the information appropriateness of the CM1 information acquired based on the captured image of the camera CM4 is medium. Evaluated as being.

  Accordingly, in the case of FIGS. 6A to 6C, the CM1 information of the camera CM2 is adopted because the CM1 information of the camera CM2 has the highest information appropriateness, and is recorded in association with the captured image of the camera CM1, for example. Will come to be.

  As shown in FIG. 7, the camera CM1 transmits an information appropriateness information transmission request from another camera for a predetermined period TD1 after transmitting an information appropriateness information transmission request. Since the information appropriateness of the camera CM2 is evaluated to be the highest among the information appropriateness received during this period TD1, an information transmission request is transmitted to the camera CM2. Upon receiving this information transmission request, the camera CM2 transmits the requested CM1 information to the camera CM1.

  On the other hand, the other cameras CM3 and CM4 attempt to receive an information transmission request from the camera CM1 only for a predetermined period TD2 and TD3 after transmitting the information appropriateness level. If the information transmission request is not received within the periods TD2 and TD3, the CM1 information and the like are discarded. By doing so, it is possible to prevent a situation where useless resources are consumed.

2.3 Camera Identification Processing When the camera CM2 receives an information transmission request, it is determined whether or not the transmission source camera of the information transmission request matches the camera in the field of view of the camera CM2. There is a need to.

  For example, in FIG. 8A, not only the camera CM1 but also the camera CM3 transmits an information transmission request. If the cameras CM1 and CM3 exist within the communication range such as the wireless LAN of the camera CM2, the camera CM2 receives information transmission requests from both the cameras CM1 and CM3. Then, as shown in FIG. 8B, when the camera CM2 captures an image, it is determined whether or not the received information transmission request is an information transmission request transmitted by the camera CM1 reflected in the field of view of the camera CM2. There is a need to.

  Therefore, in this embodiment, the camera CM2 receives the individual identification information of the camera CM1. When it is detected that the camera included in the field of view is the camera CM1 identified by the received individual identification information, it is determined that information regarding the camera CM1 can be acquired.

  Specifically, as shown in FIG. 9A, an identification mark (identification information in a broad sense) for identifying the camera is set in the camera CM1. This identification mark (identification figure) is, for example, a two-dimensional barcode. However, the form of the identification mark is not limited to that shown in FIG. 8A, and various marks for identifying the camera CM1 can be used. Corresponding identification marks may also be set for the cameras CM2 and CM3.

  As shown in FIG. 9B, the information transmission request includes individual identification information of the camera. Specifically, the information transmission request includes, for example, a command code field for designating an information transmission request and a camera ID field that is individual identification information of the camera. Then, the camera CM 1 sets its own camera ID in the camera ID field of the information transmission request to be transmitted. Similarly, the camera CM 3 also sets its own camera ID in the camera ID field of the information transmission request to be transmitted.

  Upon receiving the information transmission request from the cameras CM1 and CM3, the camera CM2 extracts the camera ID included in each information transmission request. In addition, the detection process of the identification mark reflected in the captured image of the camera CM2 is performed. In FIG. 8B, the camera identification mark is detected based on the captured image of the camera CM2, and the ID of the detected camera identification mark matches the camera ID included in the information transmission request from the camera CM1. Has been detected. Therefore, in this case, it is determined that the CM1 information of the camera CM1 can be acquired.

  On the other hand, the ID of the identification mark that appears in the captured image does not match the camera ID included in the information transmission request of the camera CM 3. Accordingly, it is determined that CM3 information that is information related to the camera CM3 cannot be acquired.

  In FIG. 8C, the camera CM2 acquires CM1 information that is information related to the camera CM1, and transmits the acquired CM1 information to the camera CM1. On the other hand, the CM3 information of the camera CM3 is not acquired and is not transmitted to the camera CM3.

  In this way, CM1 information can be transmitted only to the camera CM1 within the field of view of the camera CM2, and it is possible to prevent a situation where resources are wasted due to wasted information being transmitted.

  Even if the CM3 information is transmitted to the camera CM3, since the camera CM3 is not in the field of view of the camera CM2, this CM3 information becomes useless information for the camera CM3. In this regard, in the present embodiment, such a situation that the CM3 information is transmitted unnecessarily is prevented, so that a situation where resources such as a processing load and a storage area of the camera CM3 are consumed is also prevented.

  Note that the identification method of the camera CM1 is not limited to the method using an identification mark as shown in FIG. For example, a light emitting device such as an infrared ray may be provided in the camera CM1, and the identification processing of the camera CM1 may be performed using the identification information obtained from the light emission pattern of the light emitting device. Alternatively, the camera CM1 may be provided with an RFID and identified by this RFID. Alternatively, as will be described later, a position information acquisition device such as a GPS may be provided to identify whether the camera in the field of view is the camera CM1 based on the acquired position information.

2.4 User Information In this embodiment, user information such as the user's facial expression and line-of-sight direction is transmitted from the camera CM 2 to the camera CM 1 as CM 1 information.

  For example, FIG. 10A is an example of a captured image of the camera CM2. This captured image includes images of the camera CM1 and its user US1. In this case, in the present embodiment, the expression of the user US1 and the line-of-sight direction VD1 are recognized based on the captured image of the camera CM2. For example, in FIG. 10A, it is recognized that the user US1 is a smiling expression. Further, it is recognized that the line-of-sight direction VD1 of the user US1 is the right direction (lower right direction) as viewed from the user US1. Then, the camera CM2 acquires information on the facial expression and line-of-sight direction of these users as user information and transmits it to the camera CM1.

  FIG. 10B is an example of a captured image of the camera CM1. This captured image is, for example, a life log image of the user US1. Then, based on the information on the line-of-sight direction VD1 of the user US1 recognized in FIG. 10A, it can be seen that the user US1 is paying attention to the subject OBA. Further, based on the facial expression information of the user US1, it is understood that the user US1 was laughing while looking at the subject OBA.

  The facial expression information of the user is classified into, for example, the facial expression of the user as “laughing”, “angry”, “happy”, “sad”, etc. It can be recognized by judging whether it is close to the expression of the classification.

  In addition, as shown in FIG. 10A, a plurality of persons may be shown in the captured image of the camera CM2. In this case, in the present embodiment, a human face in the field of view of the camera CM2 is detected, and among the detected one or more human faces, the human face closest to the camera CM1 is selected as the person's face of the camera CM1. Judged as a human face. For example, in FIG. 10A, the face FA1 of the user US1 is detected, and it is determined that this face FA1 is closer to the camera CM1 (for example, an identification mark) than the face of another person. In this case, the acquired user information is regarded as information on the user US1 of the face FA1.

  In order to identify the subject of interest OBA in FIG. 10B from the line-of-sight direction VD1 detected in FIG. 10A, as shown in FIG. 10C, the line-of-sight direction VD1 with respect to the direction CD1 of the camera CM1 is shown. The relative angle α is important. That is, even if only the line-of-sight direction VD1 of the user US1 is detected, if the relative angle α with respect to the direction CD1 of the camera CM1 cannot be obtained, the position of the subject of interest OBA in FIG. It becomes difficult.

  For this reason, the camera CM2 also acquires information on the orientation CD1 of the camera CM1 for the correction process of the line-of-sight direction VD1 of the user US1. If the orientation CD1 of the camera CM1 is also obtained, the relative angle α of the line-of-sight direction VD1 with respect to the orientation CD1 can be obtained, so that the subject of interest OBA of the user US1 in FIG. 10B can be easily identified. become. It should be noted that the correction process of the line-of-sight direction VD1 based on the orientation CD1 of the camera CM1 may be performed on the camera CM1 side or the camera CM2 side.

2.5 Recording Captured Image In this embodiment, the camera CM1 writes and stores CM1 information such as user information in the storage unit 70 in association with the captured image of the camera CM1.

  Specifically, as shown in FIG. 11A, for example, the user's facial expression, line-of-sight direction, and the like with respect to the headers HD1 and HD2 of the captured images IM1 and IM2 taken at predetermined time intervals. Information is written and stored in the storage unit 70. Specifically, user information is written in a header defined by EXIF in an EXIF format image file. When reproducing a life log image or the like, a reproduction image is generated using the user information written in the header.

  For example, FIG. 11B is an example of a reproduced image of an image taken by the camera CM1. This image includes subjects OBA, OBB, OBC, OBD, and the like. Based on the facial expression information of the user included in the header, it is determined that the facial expression of the user was a laughing face when the image of FIG. Accordingly, as indicated by A1 in FIG. 11B, for example, an icon mark representing a laughing face is displayed (superimposed) on the captured image.

  Further, based on the user's line-of-sight direction information included in the header, it is determined that the user was gazing in the lower left direction when the image of FIG. Accordingly, as shown in FIG. 11B, it is determined that the subject that the user is gazing out of OBA, OBB, OBC, and OBD is OBA, and image effects such as highlighting are applied to the subject OBA. Is given.

  If the reproduced image as shown in FIG. 11 (B) is displayed, the user US1 can recognize which subject he / she is paying attention to when viewing the reproduced image of the life log image, and what kind of expression the subject Can be grasped by looking at the icon mark A1 and the image effect of the subject OBA. Accordingly, it is possible to reproduce the life log image reflecting the information of the user US1 in the blind spot of the camera CM1. Note that the reflection of the user information in the reproduced image may be realized by using an avatar or the like that expresses the user's facial expression or line-of-sight direction.

3. Detailed Processing Example Next, a detailed processing example of the present embodiment will be described with reference to the flowcharts of FIGS.

  FIG. 12 is a flowchart showing the overall processing of the cameras CM1 and CM2. In the present embodiment, the camera CM2 means all cameras that can respond to the request from the camera CM1, and it is assumed that there are a plurality of cameras.

  First, the user of the camera CM1 instructs the start of life log shooting of the camera CM1 through the operation unit 60 (input unit) (step S1). The power-on of the camera CM1 may also serve as a life log shooting start instruction. Alternatively, the shooting start instruction may be realized by switching from another shooting mode to the life log shooting mode. As an input method in this case, a button, a slide switch, a rotary switch, a touch panel, an acceleration sensor, or the like can be used.

  Next, the user of the camera CM1 uses the operation unit 60 to set a shooting timing that is a time interval of life log shooting (step S2). The set time interval is stored in the storage unit 70. For example, the user sets a time interval such as “every 1 minute” or “every 5 minutes”. As an input method in this case, a button, a touch panel, or the like can be used. Note that a predetermined time interval stored in advance in the camera CM 1 may be used as it is without setting by the user.

  The time acquisition unit 38 of the camera CM1 acquires the current time. Then, the next shooting time is calculated by adding the time interval set in step S2 to the acquired current time. The calculated next shooting time is stored in the storage unit 70. The time acquisition unit 38 acquires time information via a built-in clock or other radio waves or a network.

  The camera CM1 (actually a control device; the same applies to the following) waits until the next shooting time recorded in the storage unit 70 is reached (step S3). When the shooting time is reached, the standby state of the camera CM1 is released, and the process proceeds to the shooting sequence described below.

  First, as described with reference to FIG. 5A, the camera CM1 broadcasts an information transmission request to other cameras via the communication unit 80 and the network (step S4). This information transmission request is a request to another camera to send the information appropriateness level of the CM1 information. As described with reference to FIG. 9B, the data constituting the information transmission request includes the individual identification information (camera ID) of the camera CM1 that is the request transmission source.

  The camera CM2 receives the information transmission request transmitted from the camera CM1 via the communication unit 180 and the network (step S5). Then, by analyzing the content (data), it is recognized that it is a transmission request for information appropriateness. Specifically, it is recognized by decoding the command code of FIG. Further, the individual identification information of the camera CM1 included in the information transmission request data is acquired. The acquired individual identification information of the camera CM1 is temporarily stored in the storage unit 170.

  Next, it is determined whether or not CM1 information can be acquired (step S6). If it is determined that the camera CM1 is within the field of view of the camera CM2 and the CM1 information can be acquired, CM1 information acquisition processing is performed (steps S7 and S8). If the camera CM1 is out of the field of view and the like and CM1 information cannot be acquired, for example, the process ends.

  Next, information appropriateness calculation processing is performed (step S9). Specifically, the information appropriateness calculation unit 150 of the camera CM2 evaluates the goodness of the acquired CM1 information (user information) according to the following equation (1). Such an evaluation value is called information suitability.

Information appropriateness = β × (90− | face orientation |) + γ × (face size) (1)
In Expression (1), the face size is a value acquired at the time of face detection in the CM1 information acquisition process described later. The face orientation is 0 degree for the front direction and ± 90 degrees for the right and left sides. The face size is the vertical size (number of pixels) of the detected face. β and γ are predetermined coefficients.

  The camera CM2 transmits the information appropriateness level to the camera CM1 via the communication unit 180 and the network (step S10).

  As described with reference to FIG. 7, the camera CM 1 transmits an information transmission request in step S4 and then attempts to receive the information appropriateness only for a predetermined period (step S11). When there are a plurality of cameras around the camera CM1, a plurality of information suitability levels are transmitted. However, if the information suitability levels reach within a predetermined period, all are received. The order of arrival may be up to a predetermined upper limit. That is, only a predetermined upper limit number of information suitability may be received within a predetermined time.

  The information appropriateness evaluation unit 50 of the camera CM1 performs an information appropriateness evaluation process (step S12). For example, the highest value of all received information suitability is obtained, and the camera CM2 corresponding to it is determined. The camera CM1 transmits an information transmission request only to the camera CM2 having the highest information suitability (step S13). This information transmission request is a request for requesting transmission of the CM1 information (user information) of the camera CM1 from the camera CM2 to the CM1.

  The camera CM2 tries to receive the information transmission request for the CM1 information for a predetermined period after the information appropriateness is transmitted (step S14). As described with reference to FIG. 7, when the information transmission request from the camera CM1 cannot be received within the predetermined period, the acquired CM1 information and the like are discarded. Then, only the camera CM2 that has received the information transmission request transmits the CM1 information to the camera CM1 (step S15).

  The camera CM1 transmits an information transmission request for CM1 information to a plurality of cameras that have transmitted a high degree of information appropriateness, performs an averaging process on the received CM1 information, and obtains final CM1 information. Also good. For example, when a plurality of gaze directions are obtained as the gaze direction of the user, the gaze direction is averaged to obtain the final gaze direction. Alternatively, when a plurality of candidates are obtained as candidates for the facial expression of the user, the final facial expression is determined by averaging processing.

  The camera CM1 receives the CM1 information transmitted from the camera CM2 via the communication unit 80 and the network (step S16). Note that if there is no response from the camera CM2 even after a predetermined time has elapsed after transmitting the information transmission request, it is determined that the CM1 information could not be acquired.

  The camera CM1 acquires the external state as a captured image (step S17). Then, as described with reference to FIG. 11A, the camera CM1 associates the acquired captured image with the CM1 information and stores them in the storage unit 70 (step S18). If CM1 information cannot be acquired, only the captured image is stored.

  Next, it is determined whether or not the user has instructed to end the shooting (step S19). If there is an instruction to end the shooting, the lifelog shooting is ended. On the other hand, if there is no shooting end instruction, shooting timing update processing is performed (step S20). Specifically, the camera CM 1 acquires the current time by the time acquisition unit 38. And the time interval memorize | stored in the memory | storage part 70 is acquired, both are added, and the next imaging | photography time is calculated. The calculated next shooting time is stored in the storage unit 70. Then, the process returns to step S3 (standby until shooting timing). In addition to the user's facial expression and line-of-sight direction, the camera CM2 may transmit the user's image itself (an image taken during the information acquisition availability determination process) to the camera CM1.

  FIG. 13 shows a flowchart of the information acquisition possibility determination process (camera positional relationship recognition process) in step S6 of FIG.

  First, the camera CM2 acquires the state of the outside world as a captured image (step S21). That is, information within the field of view of the camera CM2 is acquired. Here, a spatial range that can be acquired as an image by the camera CM2 is referred to as a field of view of the camera CM2. The same applies to the field of view of the camera CM1.

  Next, the camera CM2 searches the acquired image and performs the camera identification mark detection process described with reference to FIG. 9A (step S22). If not detected, it is determined that the camera CM1 is out of the field of view (step S26). On the other hand, if it is detected, the camera identification mark is decoded, and it is determined whether or not it matches the individual identification information of the camera CM1 included in the information transmission request in step S5 of FIG. Is determined to be within the field of view (steps S23, S24, S25). On the other hand, if they do not match or if the camera identification mark cannot be decoded, it is determined that the camera CM1 is out of the field of view (step S26).

  FIG. 14 is a flowchart of CM1 information acquisition processing. First, as described with reference to FIG. 10A, a face located above (including substantially above) the position where the camera identification mark of the camera CM1 is detected is detected (step S31). For example, in the case of a camera that hangs from the neck, the user's face is above the camera. Even if the camera is placed in a breast pocket, the user's face is generally upward. If a plurality of faces are detected, the face closest to the camera is regarded as the user's face.

  As a face detection technique, there is a technique such as “Paul Viola, Michael Jones“ Rapid Object Detection using a Boosted Cascade of Simple Features ”CVPR2001”.

  If no face is detected, it is determined that the user information, which is CM1 information, cannot be acquired. If detected, the direction (orientation) of the detected face is recognized (steps S32 and S33). . The detected facial expression is recognized (step S34). Further, the detected line-of-sight direction of the face is recognized (step S35).

  For example, for face orientation recognition (head pose estimation), “Koichi Kinoshita, Yoshinori Konishi, Masato Kawade et al. (OMRON)” Face feature point detection and head pose estimation “MIRU2008” by 3D model fast fitting is there. For facial expression recognition, there is "Shiro Kumano, Kazuhiro Otsuka, Junji Yamato, Eisaku Maeda, Yoichi Sato (Tokyo Univ., NTT)" "MIRU2008", a facial expression recognition method that is robust against head posture changes by combining particle filter and gradient method. . Regarding the recognition of the gaze direction, there is "Daijo Yamazoe, Akira Utsumi, Atsuko Yonezawa, Shinji Abe (ATR)" Remote eye gaze estimation with a monocular camera using a three-dimensional eyeball model "MIRU2008".

  Next, for example, the orientation of the camera CM1 is recognized from the graphic shape of the identification mark of the camera CM1 (step S36). For example, Japanese Patent Application Laid-Open No. 2006-351024 discloses a method of capturing a rectangular figure and obtaining the positional relationship between the camera and the figure based on the figure state in the image. The direction of the camera CM1 indicates the direction of the optical axis of the optical system.

  Next, as described with reference to FIG. 10C, the line-of-sight direction correction processing is performed (step S37). For example, the user's line-of-sight direction is converted into a direction based on the direction of the camera CM1 to obtain a correction value for the line-of-sight direction. The information of the user's facial expression and the corrected line-of-sight direction acquired as described above is used as the user information in this embodiment.

  According to the method of the present embodiment described above, the quality of CM1 information acquired by a plurality of cameras is ranked using the information appropriateness level. As a result, when there are a plurality of cameras, the CM1 information can always be acquired using the camera with the best state. Further, by selecting whether or not to receive the CM1 information based on the information appropriateness level, it is possible to prevent a situation where unnecessary information is sent from the camera CM2 to the CM1. This prevents a situation in which the battery and the network bandwidth are wasted.

4). Second Configuration Example FIG. 15 shows a second configuration example of the control device and camera (imaging device) of this embodiment. In the second configuration example of FIG. 15, the control device 30 of the camera CM <b> 1 further includes a positional relationship information acquisition unit 47. The information acquisition unit 140 of the camera CM 2 further includes a positional relationship information acquisition unit 147. Other components are almost the same as those in FIG. 2, and thus detailed description thereof is omitted.

  The positional relationship information acquisition unit 47 of the camera CM1 acquires positional relationship information (information for specifying the positional relationship) of the camera CM1. Specifically, for example, position information of the camera CM1 is acquired using information from a GPS provided in the camera CM1. The acquired position information is set in the field corresponding to the information transmission request in FIG. 9B, for example, and transmitted to the camera CM2.

  The positional relationship information acquisition unit 147 of the camera CM2 acquires positional relationship information of the camera CM2. For example, the position information of the camera CM2 is acquired using information from GPS or the like. Further, for example, the orientation information of the camera CM2 is also acquired using information from a gyro sensor or the like.

  The information acquisition unit 140 determines whether or not CM1 information can be acquired based on the positional relationship information of the cameras CM1 and CM2. Specifically, it is determined whether or not the CM1 information can be acquired based on the position information of the camera CM1, the position information of the camera CM2, and the orientation information of the camera CM2.

  For example, in the first configuration example, whether the CM1 information can be acquired is determined by detecting the camera identification mark MK provided in the camera CM1. However, as shown in FIG. 16A, the camera identification mark MK may not be detected, for example, hidden behind the user's US1 hand or clothes. In such a case, in the first configuration example, it is determined that the CM1 information cannot be acquired although the CM1 information should be acquired from the captured image of the camera CM2.

  Therefore, in the second configuration example, the positional relationship information of the cameras CM1 and CM2 is determined to determine whether or not the CM1 information can be acquired. For example, in FIG. 16B, the position P1 of the camera CM1, the position P2 of the camera CM2, and the orientation CD2 of the camera CM2 are acquired. Using these positions P1, P2 and orientation CD2, it can be determined whether or not the camera CM1 is in the field of view of the camera CM2. If it is determined that the camera CM1 is within the field of view, it is determined that CM1 information can be acquired, and the CM1 information acquisition process described in the first configuration example is performed. In this way, CM1 information can be acquired even in the situation shown in FIG.

  Next, the operation of the second configuration example will be described. First, the camera CM1 transmits an information transmission request in the same manner as in the first configuration example. At this time, the data constituting the information transmission request includes the position / orientation information of the camera CM1 in addition to the individual identification information of the camera CM1. This position / orientation information is acquired by the positional relationship information acquisition unit 47.

  The camera CM2 receives the information transmission request in the same manner as in the first configuration example. Furthermore, the individual identification information and the position / orientation information of the request transmission source (camera CM1) included in the information transmission request data are acquired. The acquired individual identification information and position / orientation information of the camera CM1 are temporarily stored in the storage unit 170.

  Next, the camera CM2 performs a process for determining whether or not to obtain CM1 information. FIG. 17 shows a flowchart of information acquisition availability determination processing (camera positional relationship recognition processing) in the second configuration example.

  First, the camera CM2 tries to detect a camera identification mark in the same manner as in the first configuration example (steps S41 and S42). If the camera identification mark is detected and represents the individual identification information of the camera CM1, it is determined that the camera CM1 is in the field of view (steps S43, S44, S45).

  On the other hand, if it is not determined in steps S43 and S44 that the camera CM1 is in the field of view, the process proceeds to a determination process using position / orientation information. Specifically, first, the position / orientation information of the camera CM1 is acquired from the storage unit 170 (step S46). Next, the positional relationship information acquisition unit 147 acquires the position / orientation information of the camera CM2 (step S47).

  Next, the distance between the camera CM1 and the camera CM2 is calculated and compared with a predetermined value (step S48). If the distance is not less than the predetermined value, it is determined that the camera CM1 is out of the field of view (step S52). On the other hand, when the distance is equal to or smaller than the predetermined value, the field-of-view range of the camera CM2 is calculated from the field-of-view angle and position / orientation information of the camera CM2 (step S49). For example, from the position / orientation information, the position of the imaging surface of the camera and the direction of the optical axis can be known, and the range that is expanded by the viewing angle is the viewing range. It is assumed that the viewing angle is calculated from the design value of the optical system in advance or is actually measured and recorded in the storage unit 170.

  Next, the position / orientation information of the camera CM1 is compared with the visual field range of the camera CM2, and it is determined whether or not the camera CM1 is within the visual field range of the camera CM2 (step S50). If it is within the visual field range, it is determined that the camera CM1 is within the visual field (step S51). On the other hand, if it is outside the visual field range, it is determined that the camera CM1 is out of the visual field (step S52).

  According to the above second configuration example, when the identification mark representing the individual identification information of the camera CM1 cannot be directly viewed from the camera CM2, the positional relationship of the camera is grasped using the positional information such as GPS. In this way, the user information of the camera CM1 can be acquired even when the camera CM1 is hidden by the shielding object or the identification mark representing the individual identification information cannot be clearly photographed from the camera CM2.

  Although the present embodiment has been described in detail as described above, it will be easily understood by those skilled in the art that many modifications can be made without departing from the novel matters and effects of the present invention. Accordingly, all such modifications are intended to be included in the scope of the present invention. For example, a term (camera, identification mark, etc.) described together with a different term (imaging device, identification information, etc.) in a broader sense or the same meaning at least once in the specification or the drawing, It can be replaced by that different term. Further, the configurations and operations of the control device and the imaging device are not limited to those described in the present embodiment, and various modifications can be made.

US1, US2 user, CM1, CM2 camera (imaging device), VD1 viewing direction,
20 imaging unit, 30 control device, 32 image acquisition unit, 34 receiving unit, 35 transmitting unit,
36 writing unit, 38 hour acquisition unit, 47 positional relationship information acquisition unit,
50 information suitability evaluation unit, 60 operation unit, 70 storage unit, 80 communication unit,
90 display unit, 98 information storage medium, 120 imaging unit, 130 control device,
132 image acquisition unit, 134 reception unit, 135 transmission unit, 140 information acquisition unit,
141 camera identification mark detection unit, 142 face detection unit, 143 facial expression recognition unit,
144 Gaze direction recognition unit, 145 Camera direction recognition unit, 146 Gaze direction correction unit,
147 positional relationship information acquisition unit, 150 information appropriateness calculation unit, 160 operation unit,
170 Storage Unit, 180 Communication Unit, 190 Display Unit, 198 Information Storage Medium

Claims (24)

A control device for the second imaging device of the first imaging device and the second imaging device that can move independently from each other,
An image acquisition unit that acquires a captured image captured by the imaging unit;
An information acquisition unit that acquires first imaging device information that is information related to the first imaging device based on the captured image of the second imaging device;
An information appropriateness calculating unit for calculating the information appropriateness of the acquired first imaging device information;
A transmission unit that transmits the calculated degree of appropriateness of information to the first imaging device;
The control apparatus characterized by including. In claim 1,
A receiving unit that receives an information transmission request for the information appropriateness level from the first imaging device;
The transmitter is
The control device, wherein the information suitability is transmitted to the first imaging device as reply information to the information transmission request of the information suitability. In claim 2,
The receiver is
Receiving an information transmission request for the first imaging device information from the first imaging device that has received the information suitability;
The transmitter is
A control device, wherein the first imaging device information is transmitted to the first imaging device as reply information to the information transmission request for the first imaging device information. In claim 3,
The transmitter is
If the information transmission request for the first imaging device information is not received within a predetermined period after transmitting the information appropriateness level, a process of discarding the first imaging device information and the information appropriateness level is performed. A control device characterized by. In any one of Claims 1 thru | or 4,
The information acquisition unit
A control device that determines whether the first imaging device information can be acquired, and acquires the first imaging device information when it is determined that the first imaging device information can be acquired. In claim 5,
The information acquisition unit
Determining whether the first imaging device information can be acquired by determining whether the first imaging device is included in the field of view of the second imaging device based on the captured image. A control device characterized by. In claim 6,
The receiver is
Receiving individual identification information of the first imaging device;
The information acquisition unit
When it is detected whether the imaging device included in the visual field is the first imaging device identified by the received individual identification information, and when it is detected that the imaging device is the first imaging device And determining that the first imaging device information can be acquired. In any of claims 5 to 7,
The information acquisition unit
Control that acquires positional relationship information between the first imaging device and the second imaging device, and determines whether or not the first imaging device information can be acquired based on the positional relationship information apparatus. In any one of Claims 1 thru | or 8.
The information suitability calculation unit
A control device that calculates the information appropriateness level based on a photographing state of a user of the first imaging device. In claim 9,
The information suitability calculation unit
A control device that calculates the information appropriateness degree based on at least one of a face orientation and a face size of a user of the first imaging device. In any one of Claims 1 thru | or 10.
The information suitability calculation unit
The control device, wherein the information suitability is calculated based on at least one of contrast, gradation, brightness, and blur of a captured image of the second imaging device. Of the first imaging device and the second imaging device that can move independently of each other, the control device for the first imaging device,
An image acquisition unit that acquires a captured image captured by the imaging unit;
The second imaging device acquires first imaging device information that is information relating to the first imaging device based on a captured image of the second imaging device, and determines the information appropriateness of the first imaging device information. A receiving unit that receives the information suitability when it is calculated and transmitted;
An information appropriateness evaluation unit that performs an evaluation process of the received information appropriateness;
The control apparatus characterized by including. In claim 12,
A transmission unit that transmits an information transmission request of the information appropriateness,
The receiver is
The control apparatus receives the information suitability as reply information to the information transmission request of the information suitability. In claim 13,
The transmitter is
After receiving the information suitability, send an information transmission request for the first imaging device information,
The receiver is
The control device, wherein the first imaging device information is received as reply information to the information transmission request for the first imaging device information. In claim 13 or 14,
The transmitter is
The control apparatus characterized in that the information transmission request of the information appropriateness level is transmitted by broadcast. In claim 12,
A transmission unit for transmitting an information transmission request for the first imaging device information;
The transmitter is
A control device that determines whether or not to transmit an information transmission request for the first imaging device information based on the evaluation result of the information suitability. In claim 12,
A transmission unit for transmitting an information transmission request for the first imaging device information;
The transmitter is
A control device that determines a transmission destination of an information transmission request for the first imaging device information based on an evaluation result of the information suitability. In claim 17,
The receiver is
Receive multiple information suitability from multiple imaging devices,
The information suitability evaluation unit
Performing the plurality of information suitability evaluation processes;
The transmitter is
When the second imaging device is selected from the plurality of imaging devices based on the evaluation results of the plurality of information appropriateness levels, information on the first imaging device information with respect to the second imaging device A control device that transmits a transmission request. In any of claims 12 to 18,
A control device comprising: a writing unit that writes the received first imaging device information in a storage unit in association with the captured image of the first imaging device.   An image pickup apparatus comprising the control device according to claim 1. An imaging system including a first imaging device and a second imaging device that are movable independently of each other,
The second imaging device includes:
Based on the captured image of the second imaging device, the first imaging device information, which is information related to the first imaging device, is acquired, and the information appropriateness of the acquired first imaging device information is calculated and calculated. Transmitting the information appropriateness level to the first imaging device;
The first imaging device includes:
An imaging system that receives the information appropriateness level and performs an evaluation process on the received information appropriateness level. An image acquisition method using a first imaging device and a second imaging device that are movable independently of each other,
Obtaining a captured image of the second imaging device;
Acquiring first imaging device information, which is information relating to the first imaging device, based on the acquired image of the second imaging device;
Calculating the information appropriateness of the acquired first imaging device information;
The calculated information appropriateness level is transmitted from the second imaging device to the first imaging device,
Perform the information suitability evaluation process,
Obtaining a captured image of the first imaging device;
An image acquisition method, wherein the first imaging device information acquired as a result of the information appropriateness evaluation process is written in a storage unit in association with the acquired captured image of the first imaging device. A program for controlling the second imaging device of the first imaging device and the second imaging device that are movable independently of each other,
An image acquisition unit that acquires a captured image captured by the imaging unit;
An information acquisition unit that acquires first imaging device information that is information related to the first imaging device based on the captured image of the second imaging device;
An information appropriateness calculating unit for calculating the information appropriateness of the acquired first imaging device information;
As a transmission unit that transmits the calculated degree of appropriateness of information to the first imaging device,
A program characterized by causing a computer to function. A program for controlling the first imaging device of the first imaging device and the second imaging device that can move independently from each other,
An image acquisition unit that acquires a captured image captured by the imaging unit;
The second imaging device acquires first imaging device information that is information relating to the first imaging device based on a captured image of the second imaging device, and determines the information appropriateness of the first imaging device information. A receiving unit that receives the information suitability when it is calculated and transmitted;
As an information appropriateness evaluation unit that performs an evaluation process of the received information appropriateness,
A program characterized by causing a computer to function.

Images