JPH11327030A - Camera control device and method, and storage medium - Google Patents

Abstract

(57) [Problem] To control a focus detection area and an exposure detection area of a camera using a natural language. A captured image signal is captured from a camera (S).
1). Check whether there is a voice input from the user (S
2). If there is no voice input (S2), the next image is captured (S1). When there is a voice input (S2), the input voice is recognized and converted into a character string (S3). The recognized character string is morphologically analyzed and divided into words (S4), and words representing states such as adjectives and adjective verbs are extracted from the word group (S5). Next, color information and size information are extracted with reference to the color information table and the size information table (S6). An image area that best matches the color information and the size information is detected from the image signal captured in S1 (S1).
7). The detected position of the target area is output to the camera as control information (S8).

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

The present invention relates to a camera control device and method, and a storage medium.

[0002]

2. Description of the Related Art A camera and a computer are connected by an RS serial cable or the like, and automatic focus adjustment (AF) and automatic exposure (AE) are performed according to a control command from the computer.
Camera control systems for remotely controlling these target areas are well known.

In such a camera control system, software for operating the camera is incorporated in the computer, and a control command is input to the computer using a keyboard and / or a mouse, and transmitted from the computer to the camera. Allows remote control of the camera.

[0004]

In a conventional camera control system, an input device such as a keyboard and a mouse is used for a control input from a user, so that an object of a certain color projected on a photographing screen is focused. Camera operation was troublesome.

[0005] It is an object of the present invention to provide a camera control device and method and a storage medium that can appropriately and easily remotely control a camera.

Another object of the present invention is to provide a camera control device and method capable of remotely controlling a camera by language input, and a storage medium.

[0007]

SUMMARY OF THE INVENTION A camera control device according to the present invention is a camera control device for remotely controlling a camera, comprising: an image signal holding means for holding an image signal output from the camera; Means, a dictionary necessary for natural language analysis, a natural language analysis means for analyzing the natural language sentence input by the natural language input means using the dictionary, and one or more analysis results of the natural language analysis means. A specific linguistic expression extracting means for extracting a linguistic expression belonging to a predetermined category, and an image having characteristics most matching the linguistic expression extracted by the specific linguistic expression extracting means in the image signals held in the image signal holding means An image area detecting means for detecting an area and position information indicating the image area detected by the image area detecting means are supplied to the camera, and a predetermined detection area of the camera is detected. And having an object area output means for moving to the position indicated by the position information.

[0008] A camera control method according to the present invention is a camera control method for remotely controlling a camera, comprising: a natural language input step; and a natural language analysis step for analyzing a natural language sentence input by the natural language input means. Extracting a linguistic expression belonging to one or more predetermined categories from an analysis result of the natural language analyzing step;
An image area detecting step of detecting an image area having a feature that best matches the language expression extracted by the specific language expression extraction unit in the image signal output from the camera;
Supplying a position information indicating the image area detected in the image area detecting step to the camera, and a target area output step of moving a predetermined detection area of the camera to a position indicated by the position information. I do.

In the storage medium according to the present invention, the program / software according to the above-described camera control method is stored so as to be externally readable.

[0010]

An embodiment of the present invention will be described below in detail with reference to the drawings.

FIG. 1 is a block diagram showing a schematic configuration of an embodiment of the present invention. Reference numeral 10 denotes a camera, and reference numeral 20 denotes a camera control device for remotely controlling the camera 10. Camera control device 20
, 22 is a natural language input means, 24 is a natural language analysis means, 26 is a dictionary for natural language analysis, 28 is color information / size information extraction means, 30 is an image area detection means,
Reference numeral 2 denotes an image signal holding unit that temporarily stores an image signal of each screen captured by the camera 10, and reference numeral 34 denotes a target area output unit that outputs the target area to the camera 10 as control information.

In this embodiment, voice input is used for natural language input. Basic characters are written on a keyboard or displayed on a screen, and each character is selected from them by a mouse or the like. May be input.

The camera control device 20 is specifically composed of a computer. FIG. 2 is a schematic block diagram of a hardware configuration of the camera control device 20. CPU 40,
The ROM 42, the RAM 44, and the hard disk device 46 are connected to the bus 48. The dictionary 26 is a hard disk drive 4
6 is stored. The main functions of the natural language analysis unit 24, the color information / size information extraction unit 28, the image area detection unit 30, and the target area output unit 34 are realized by software programs stored in the ROM 42 and the hard disk device 46. .

In the present embodiment, an image photographed by the camera 10 is represented by an L * a * b * color system. L *
The axis represents brightness, and the a * b * plane represents color. Details of this color system are described in, for example, “Computer Image Processing: Applied Practice (3)” edited by Hideyuki Tamura, Soken Shuppan.

FIG. 3 shows an operation flowchart of this embodiment. The captured image signal is taken from the camera 10 and stored in the image signal holding means 32 (S1). It is checked whether there is a voice input from the user (S2). If there is no voice input (S2), the next image is captured (S1).

If there is a voice input (S2), the input voice is recognized and converted into a character string (S3). The natural language analysis means 24 refers to the dictionary 26 and morphologically analyzes the recognized character strings to separate them into words (S4), and extracts words representing states such as adjectives and adjective verbs from the word group (S5). For example, if the input is “red and large”, the morphological analysis indicates that red (adjective) / te (connective particle) / large (adjective) /
(Formal noun), from which the adjectives “red” and “large” are extracted. The utilization form is changed to a standard form such as the termination form.

Next, color information and size information are extracted (S
6). At that time, a table shown in FIG. 4 is referred to. The color information table (a) includes color categories and adjective candidates belonging to each category. Similarly, the size information table (b) includes size categories and adjective candidates belonging to each category. In the case of the above example, the category to which each of the adjectives “red” and “large” belongs is found by referring to the table. It can be seen that “red” belongs to the “red” category of the color information table. Similarly, it can be seen that “large” belongs to the “large” category of the size information table.

In this way, the user can automatically focus (AF)
The feature of the image in the area (target area) to be set as the target area for automatic exposure (AE) is recognized as color category: red, size category: large.

Subsequently, from the image signals input from the camera 10 and held in the image signal holding means 32, an image area which best matches the color information / size information is detected (S7). That is, first, the image signal is divided into several regions. Existing technology (see "Image processing", Kyoritsu Shuppan, etc.) can be used for area division. It is assumed that the image signal is divided into R1, R2,..., Rn by region division.

First, matching based on color information is performed.
If the color category is other than "bright" and "dark", proceed as follows. On the a * b * plane of the L * a * b * color system, FIG.
As shown in (a), a representative point of each color category is given in advance. The average value (a _i *, b _i *) of (a *, b *) of each pixel belonging to the region R _i is obtained, and the Euclidean distance C _i between this and the color category representative point is obtained. In the case of the above example, since the color category extracted from the user's voice input is “red”, the representative points of “red” (a _red *, b _red *) and (a _i *,
b _i *) the distance between the calculated. When the color category is “bright” or “dark”, the value of L * in the L * a * b * color system is used. Similarly, “bright” and “dark” as shown in FIG.
A representative point on each L * axis is given in advance, and a distance l _i between the average value of the L * values of the region R _i and the representative point is obtained.
In the case of the above example, since none of the color categories “bright” or “dark” was extracted from the user's input, l _i = 0.

Next, matching based on the size information is performed. First, a surface area _{S i} of the region _{R i.} Also in this case, as shown in FIG. 5C, a one-dimensional axis S representing the area of the area is considered, and representative points of the size categories “large” and “small” are given in advance on the S axis. deep. In this case, a point on the S-axis corresponding to the entire area of the input image signal may be set as a “large” representative point, and S = 0 may be set as a “small” representative point. Then, a distance m _i of the representative points of the size categories extracted from the point S = S _i and user input on the S-axis. In the above example, the size categories extracted from the user input would determine the distance between the representative point of the "large" So, the point on the S-axis and S = S _i "large".

Each of the distances C _i , l _i , m thus determined
_i is evaluated comprehensively, and how close the region R _i is to the condition of the target region required by the user is evaluated. For example, the following function is used as the evaluation function.

E_i= W_c・ C_i+ W_l・ L_iTen w_m・ M_i  Where w_c, W_l, W_mIs an appropriate weighting factor. this
E_iIs calculated for all regions, and E_iThe value of
Area is regarded as a target area.

Finally, the target area output means 34 outputs the position of the target area to the camera 10 (S8). Camera 1
0 moves the target area of the automatic focus adjustment (AF) and the automatic exposure (AE) to the target area according to this information. Specifically, the camera 10 obtains the coordinates of the center of gravity in the image of the target area, and stores the coordinates of automatic focusing (AF) and automatic exposure (A
The center of the detection area (frame) of E) is moved. Further, the size of the detection area (frame) of the automatic focus adjustment (AF) and the automatic exposure (AE) is changed to the size of the target area.

In the above embodiment, the input from the user is a voice input, but any other input means such as a keyboard input may be used as long as the input is a natural language input.

In the above embodiment, the classifications shown in FIG. 4 are used as the color categories and the size categories. However, any other categories may be defined as long as words corresponding to each category can be prepared in advance. . In the above embodiment, the detection areas for the automatic focus adjustment (AF) and the automatic exposure (AE) are determined based on the characteristics of both the color and the size. You may do so. Of course, the detection area of the automatic focus adjustment (AF) and the detection area of the automatic exposure (AE) do not necessarily need to coincide, and may be determined individually.

In the above embodiment, the L * a * b * color system is used as a method of expressing an image signal. However, as long as the representative point of the defined color category can be set, any other color system such as the RGB color system can be used. A color system may be used.

In the above-described embodiment, the case where each means is realized on the same computer has been described. However, the present invention is not limited to this, and the means are distributed and arranged on computers or data processing devices distributed on a network. May be.

In the above embodiment, the software program for realizing the function of each means is not limited to being stored in the ROM, but may be stored in any other storage medium.
Of course, it may be realized by hardware. It may be realized.

The present invention may be applied to a system composed of a plurality of devices or an apparatus composed of one device. A recording medium storing software program codes for realizing the functions of the above-described embodiments is supplied to a system or an apparatus, and a computer (or CPU or MPU) of the system or apparatus executes the program code stored in the recording medium. Needless to say, this can also be achieved by executing the reading. In this case, the program code itself read from the recording medium implements the functions of the above-described embodiment, and the recording medium on which the program code is recorded constitutes the present invention.

As a recording medium for supplying the program code, for example, a floppy disk, hard disk, optical disk, magneto-optical disk, CD-ROM, C
DR, magnetic tape, nonvolatile memory card and ROM
Etc. can be used.

When the computer executes the readout program code, not only the functions of the above-described embodiment are realized, but also the operating system (OS) running on the computer based on the instructions of the program code. Needless to say, the present invention also includes a case where the system performs some or all of the actual processing and the processing realizes the functions of the above-described embodiments.

Further, after the program code read from the recording medium is written into a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, based on the instructions of the program code, It goes without saying that the CPU included in the function expansion board or the function expansion unit performs part or all of the actual processing, and the processing realizes the functions of the above-described embodiments.

[0034]

As can be easily understood from the above description, according to the present invention, color information and / or size information included in a control input in a natural language is extracted, and a camera that matches the color information and / or the size information is used. Since the image area in the image signal is detected, the automatic focusing (A
F) and / or the target area of automatic exposure (AE) can be controlled.

[Brief description of the drawings]

FIG. 1 is a schematic block diagram of an embodiment of the present invention.

FIG. 2 is a schematic configuration block diagram of a hardware configuration of a camera control device 20 of the present invention.

FIG. 3 is an operation flowchart of the embodiment.

FIG. 4 is a correspondence table between a color category / size category and a word in the embodiment.

FIG. 5 is an example of a coordinate space for matching in the embodiment.

[Explanation of symbols]

 10: Camera 20: Camera control device 22: Natural language input means 24: Natural language analysis means 26: Dictionary 28: Color information / size information extraction means 30: Image area detection means 32: Image signal holding means 34: Target area output Means 40: CPU 42: ROM 44: RAM 46: Hard disk device 48: Bus

────────────────────────────────────────────────── ─── front page continued (51) Int.Cl. ⁶ identifications FI G03B 17/00 G10L 3/00 551Z G06F 17/28 H04N 5/232 B G06T 1/00 G02B 7/11 N G10L 3/00 551 G06F 15/38 Z H04N 5/232 15/62 380

Claims (21)

[Claims] 1. A camera control device for remotely controlling a camera, comprising: an image signal holding unit for holding an image signal output from the camera; a natural language input unit; a dictionary required for natural language analysis; A natural language analyzing means for analyzing the natural language sentence inputted by the natural language input means using the dictionary; and a specific language expression extracting for extracting language expressions belonging to one or more predetermined categories from the analysis result of the natural language analyzing means. Means, an image area detecting means for detecting an image area having a feature most matching the language expression extracted by the specific language expression extracting means in the image signal held by the image signal holding means, A target area that supplies position information indicating an image area detected by the detection unit to the camera and moves a predetermined detection area of the camera to a position indicated by the position information. The camera control apparatus characterized by having a force means. 2. The camera control device according to claim 1, wherein the one or more predetermined categories include a color. 3. The camera control device according to claim 1, wherein the one or more predetermined categories include a size. 4. The target area output means supplies position information and size information indicating the image area detected by the image area detection means to the camera, and sets a predetermined detection area of the camera to the position information of the camera. The camera control device according to claim 1, wherein the camera control device is moved to a position indicated and changed to a size indicated by the size information. 5. The camera control device according to claim 1, wherein the predetermined detection area of the camera is at least one of a focus detection area and an exposure detection area. 6. An image area detecting means for dividing the image signal held in the image signal holding means into a plurality of areas, and extracting the specific language expression in each of the plurality of areas by the dividing means. A matching means for detecting a region matching the language expression extracted by the means.
The camera control device according to item 1. 7. The color matching means for detecting, in each of the plurality of areas by the dividing means, an area that matches the color expression extracted by the specific language expression extracting means; In each of the regions, a size matching unit that detects a region that matches the size expression extracted by the specific language expression extracting unit, and a matching result of the color matching unit and the size matching unit are determined by a predetermined evaluation formula. The camera control device according to claim 6, further comprising: an evaluation unit that performs comprehensive evaluation in (1). 8. A camera control method for remotely controlling a camera, comprising: a natural language input step; a natural language analysis step of analyzing a natural language sentence input by the natural language input means; A specific language expression extraction step of extracting a language expression belonging to one or more predetermined categories from the analysis result; and a feature that best matches the language expression extracted by the specific language expression extraction means in the image signal output from the camera. An image area detecting step of detecting an image area having the following. Supplying position information indicating the image area detected in the image area detecting step to the camera, and setting a predetermined detection area of the camera to a position indicated by the position information. A moving target area output step. 9. The camera control method according to claim 8, wherein the one or more predetermined categories include a color. 10. The camera control method according to claim 8, wherein the one or more predetermined categories include a size. 11. The target area output step supplies position information and size information indicating an area detected in the image area detection step to the camera, and indicates a predetermined detection area of the camera by the position information. 9. While moving to the position, changing the size to the size indicated by the size information.
2. The camera control method according to 1. 12. The camera control method according to claim 8, wherein the predetermined detection area of the camera is at least one of a focus detection area and an exposure detection area. 13. The image area detecting step includes: a dividing step of dividing the image signal held in the image signal holding means into a plurality of areas; and extracting the specific language expression in each of the plurality of areas by the dividing step. 9. The camera control device according to claim 8, further comprising: a matching step of detecting a region matching the linguistic expression extracted in the step. 14. The color matching step comprising: detecting, in each of the plurality of areas in the division step, an area matching the color expression extracted in the specific language expression extraction step; A size matching step of detecting a region that matches the size expression extracted in the specific language expression extraction step in each of the plurality of regions according to, a matching result of the color matching step and the size matching step 14. The camera control method according to claim 13, further comprising: an evaluation step of comprehensively evaluating by a predetermined evaluation formula. 15. A storage medium for externally and readablely storing program software of a camera control method for remotely controlling a camera, wherein the camera control method includes a natural language input step, and a natural language input means. A natural language analysis step of analyzing the extracted natural language sentence, a specific language expression extraction step of extracting language expressions belonging to one or more predetermined categories from the analysis result of the natural language analysis step, and an image signal output from the camera An image area detecting step of detecting an image area having a feature that best matches the linguistic expression extracted by the specific linguistic expression extracting means; and positional information indicating the image area detected in the image area detecting step. A target area output switch that supplies the image to the camera and moves a predetermined detection area of the camera to the position indicated by the position information. Storage medium characterized by having a-up. 16. The storage medium according to claim 15, wherein the one or more predetermined categories include colors. 17. The storage medium according to claim 15, wherein the one or more predetermined categories include a size. 18. The target area outputting step supplies position information and size information indicating an area detected in the image area detecting step to the camera, and indicates a predetermined detection area of the camera by the position information. 2. The method according to claim 1, further comprising: moving to a position and changing the size to the size indicated by the size information.
6. The storage medium according to 5. 19. The storage medium according to claim 15, wherein the predetermined detection area of the camera is at least one of a focus detection area and an exposure detection area. 20. The image area detecting step, comprising: a dividing step of dividing the image signal held by the image signal holding means into a plurality of areas; and extracting the specific language expression in each of the plurality of areas by the dividing step. 16. The storage medium according to claim 15, further comprising: a matching step of detecting a region matching the linguistic expression extracted in the step. 21. A color matching step for detecting, in each of the plurality of regions in the division step, a region that matches the color expression extracted in the specific language expression extraction step, A size matching step of detecting a region that matches the size expression extracted in the specific language expression extraction step in each of the plurality of regions according to, a matching result of the color matching step and the size matching step 21. The storage medium according to claim 20, further comprising: an evaluation step of comprehensively evaluating by a predetermined evaluation formula.