JP2012124554A - Imaging apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an imaging device capable of measuring a distance object by image processing by a simple method and obtaining information on the position and name of the object.
A long-distance ranging calculation unit that performs image processing on a pair of images obtained by photographing one subject from different locations, and calculates a distance to the subject from parallax between the images of the subject detected from the two images; A long-distance ranging control unit that displays posture information from the posture measuring unit and auxiliary information using the azimuth information from the azimuth measuring unit and transmits a photographed image to the long-distance ranging calculation unit;
[Selection] Figure 1

Description

  The present invention relates to an imaging apparatus having a positioning function and capable of displaying and recording not only the position of the apparatus main body but also the position and name of a recorded subject, and a control method for the imaging apparatus.

  2. Description of the Related Art Conventionally, an imaging apparatus has a positioning function, and has a function of easily confirming a shooting location later by adding position information of the apparatus body, that is, position information of the shooting location to an image. In addition, a technique for displaying and recording not only the shooting location but also the position information and name of the subject has been proposed. For example, in Patent Document 1, a camera position is measured by GPS or the like, a subject is measured by a phase difference sensor or the like, and a direction is measured by an azimuth sensor or the like, and the subject is based on the acquired camera position, distance to the subject, and direction. A technique for calculating the position of is disclosed. Further, the database is searched using the calculated subject position as a key, a name corresponding to the zoom value is extracted, the extracted name is superimposed and displayed on the through image, and the subject information associated with the photographed image and the photographed image is obtained when photographing is performed. A technique for recording is disclosed.

  On the other hand, there has been proposed a technique for measuring the distance to a subject by performing image processing on images taken by a plurality of cameras instead of a distance measuring means such as a phase difference sensor. For example, Patent Document 2 discloses a technique for capturing an object scene outside a vehicle from a stereo camera mounted in a vehicle interior, and processing the captured image to measure the distance from the vehicle to the object. Yes.

JP 2001-169164 A Japanese Patent Laid-Open No. 05-114099

  However, the distance measuring means such as the phase difference sensor and the optical pulse distance measuring device disclosed in Patent Document 1 is suitable for measuring an object at a relatively close distance with high accuracy from the distance measuring principle. , Not suitable for distance measurement of long-distance subjects. For example, the phase difference sensor measures the distance between two images that pass through different locations on the camera's photographic lens, using the amount of displacement. Therefore, the distance between the two images decreases as the subject moves further, and the accuracy of distance measurement is reduced. Falls. In addition, since the optical pulse ranging device measures the distance by the time until the projection light is reflected by the subject and returns, the projection light does not reach the subject at a long distance and cannot measure the distance. On the other hand, it can be improved to some extent by using a larger sensor or a device capable of more powerful projection, but it is unrealistic because the outer dimensions of the camera increase and the cost increases. Further, from the viewpoint of AF (Auto Focus) application, even with the same distance measurement error, the farther the subject, the smaller the influence on the defocusing is, and the subject beyond a certain distance can be treated as infinity. Ranging accuracy for distance objects is not required. Therefore, there is a problem that the distance measuring means for AF cannot be used as the distance measuring means for obtaining information such as the position and name of the subject only for a far object.

  On the other hand, the distance measuring method disclosed in Patent Document 2 is suitable for distance measurement in principle if a horizontal distance difference between two images can be ensured, but each imaging optical axis is parallel and a fixed distance. It is necessary to use a plurality of images taken with the above distances. Also in this patent document, there are two stereo cameras whose optical axes are adjusted and fixed in advance, and there is a problem that it is difficult to apply in a small imaging device that can be taken out.

  SUMMARY OF THE INVENTION An object of the present invention is to provide an imaging apparatus capable of measuring a distance object by image processing by a simple method and obtaining information on the position and name of the object.

In order to achieve the above object, the configuration of the imaging apparatus according to the present invention is as follows.
A GPS unit for acquiring current position coordinates;
A near-distance ranging unit that measures the distance to a near-distance subject;
A long-distance ranging unit that measures the distance to a long-distance subject;
An azimuth measuring unit for measuring the shooting direction;
A posture measurement unit that measures the posture in the horizontal and elevation directions during shooting,
A subject distance determination unit that selects either the distance to the subject measured by the short-range distance measuring unit or the distance to the subject measured by the long-range distance measuring unit;
A subject position calculation unit for calculating subject position coordinates;
In an imaging device comprising
The subject distance determination unit selects the distance measured by the short distance measurement unit when the distance to the subject measured by the short distance measurement unit is equal to or less than a preset threshold value, If it is larger, select the distance measured by the long-distance ranging unit,
The subject position calculation unit calculates the subject position using the distance information to the subject selected by the subject distance determination unit, the current position coordinates calculated by the GPS unit, and the shooting direction information measured by the direction measurement unit. It is characterized by doing.

  According to the present invention, it is possible to provide an imaging apparatus capable of measuring a distance object by image processing and obtaining information on the position and name of the object by a simple method.

The block diagram which shows the structure of the to-be-photographed object information acquisition part of this embodiment. Flow chart showing the flow of subject information acquisition processing of this embodiment The block diagram which shows the whole structure of the digital camera to which this embodiment is applied Conceptual diagram of posture detection of digital camera by acceleration sensor of this embodiment Conceptual diagram of subject position calculation after ranging according to this embodiment Illustration of electronic azimuth sensor

  Embodiments in which the present invention is applied to a digital camera will be described below with reference to the drawings.

  First, the overall configuration of the digital camera will be described with reference to FIG. FIG. 3 is a block diagram illustrating the overall configuration of the imaging apparatus. The operation unit 1 of the imaging apparatus includes a power button, a playback button, a display switching button, a mode dial, a zoom switch, a release switch (not shown) and the like, and detects various operations by the operator. When the operator turns on the power button of the operation unit 1, the system power supply circuit of the power supply unit 4 is activated. Thereafter, the CPU 5 is activated to detect the mode setting state of the image pickup apparatus and the like accompanying the mode dial operation by the operator.

  If the detected mode is the photographing mode, the CPU 5 controls the power supply circuit to activate the imaging power supply and the backlight power supply, and initializes the analog front end (hereinafter referred to as AFE) 6, the mechanical driver 7, and the liquid crystal display driver 8. I do. The mechanical driver 7 drives an optical system 11 including a lens group such as a shutter, a diaphragm, a focus, and a zoom. The AFE 6 drives the CCD 12. The liquid crystal display driver 8 drives the liquid crystal panel 9. As a result, the state shifts to the photographing enabled state.

  The CPU 5 controls the entire image pickup apparatus and processes an image obtained from the CCD 12. As control of the entire imaging apparatus, for example, detection of button operation of the operation unit 1 and switching control of the imaging drive mode / display mode corresponding thereto, writing / reading processing to the recording medium 13, display data to the liquid crystal display driver 8 Transmission, ON / OFF control of the power supply unit 4 and the like. In addition to the above, the CPU 5 also performs lighting control of various LEDs, charge / light emission control of the strobe 14, and the like.

  When the photographer presses the release button halfway in a state where photographing is possible, the phase difference sensor 21 as a short-distance ranging unit measures the distance to the subject according to the AF (Auto Focus) setting at that time. The AF setting includes, for example, a multi-pattern AF that divides the screen and performs distance measurement in each area, a center single point AF that performs distance measurement at a fixed point in the center, and the like. At the same time, the CPU 5 determines the shutter speed, aperture value, and ISO sensitivity according to the shooting mode (Av priority, Tv priority, auto, etc.) at that time, and shifts to the shooting standby state. In this state, when the photographer fully presses the release button, an image is captured from the CCD 12 under the determined photographing condition, is subjected to image processing, and is recorded on the medium 13. At this time, strobe charging / flash control is also performed according to the settings and shooting conditions.

  Next, the configuration of the subject information acquisition unit 16 of the present embodiment will be described with reference to FIG. The subject information acquisition unit 16 includes a GPS antenna 22 and a GPS processing CPU 23 as a GPS unit, an orientation sensor 24 as an orientation measurement unit, an acceleration sensor 25 as an orientation measurement unit, a subject information acquisition CPU 26, and subject information as subject information recording means. It consists of a recording ROM 27. The subject information acquisition CPU 26 also has functions as a subject distance determination unit 30, a long-distance ranging control unit 31, a long-distance ranging calculation unit 32, a subject position calculation unit 33, and a subject name search unit.

In this embodiment, a stereo matching image processing technique using two images is used as the long-distance ranging unit. The image processing technique itself is the same as the conventional technique, and uses two images having the same orientation, that is, the optical axes are parallel to each other, and the same posture, that is, not rotating in the horizontal direction. This is a method of finding the same object as the subject shown in one image from the other image by pattern matching, and obtaining the distance to the subject by obtaining the difference (parallax) between these horizontal coordinates. The subject information acquisition CPU 26 stores the orientation information obtained from the orientation sensor 24 and the acceleration sensor 25 and the orientation information of the camera on the liquid crystal panel 9 so that the optical axis orientation between the two images and the orientation state in the horizontal direction do not relatively shift. A pair of images is acquired by controlling the imaging unit via the CPU 5 or the like while displaying and assisting the user's shooting. Further, image calculation processing is performed on the acquired image, and a distance R _FAR to the subject is calculated. Further, the GPS antenna 22 and the GPS processing CPU 23 are controlled to acquire the position information of the camera, and the position coordinates of the subject are calculated using the direction and the R _FAR obtained from the direction sensor 24. Using the calculated position information as a key, a name information database for the position coordinates recorded in the object information recording ROM 27 in the main body is searched to acquire the name of the object and display it on the liquid crystal panel 9.

A specific flow of processing up to subject information acquisition will be described with reference to FIG. First, the subject information acquisition CPU 26 controls the GPS antenna 22 and the GPS processing CPU 23 to acquire camera position information P _C1 (x ₁ , y ₁ , z ₁ ) (S201), and transitions to a subject information acquisition standby state. (S202). If the user in this state by pressing the half-release button to capture in the AF frame to an object to be acquired object information, the object information acquisition CPU26 first obtained from the azimuth sensor 24 orientation D ₁ of the camera optical axis direction (S203) . Orientation D _1, as shown in FIG. 5 (b), the north as a reference (0 °), is output at an angle clockwise. It is 90 ° for the east and 225 ° for the southwest.

Next, the subject information acquisition CPU 26 acquires information from the acceleration sensor 25 and detects the elevation angle of the camera (S204). The acceleration sensor 25 has three detection axes orthogonal to each other, and is installed so that the horizontal direction of the camera is the x axis, the optical axis direction of the lens is the y axis, and the vertical direction of the camera is the z axis as shown in FIG. ing. When held horizontally at the normal position, gravitational acceleration G is applied only to the z-axis, so the acceleration of each axis is Ax = Ay = 0 and Az = −G as shown in FIG. In the normal position, when it is held upward by an angle θ, G is applied to the y-axis and the z-axis, and Ax = 0, Ay = −Gsinθ, Az = −Gcosθ as shown in FIG. If the camera is stationary, the acceleration applied to the camera is only G. Therefore, the elevation angle θ of the camera can be calculated by θ = tan ⁻¹ (Ay / Az).

Acquires the orientation _{D 1,} obtains a distance _{R AF} from the phase difference sensor 21 to the object through the CPU 5, conventionally CPU 5 drives the optical system 11 via a mechanical driver 7, adjust the focus (S205) . The subject distance determination unit 30 of the subject information acquisition CPU 26 determines whether or not R _AF is equal to or _smaller than a predetermined long-distance ranging threshold R _th (S206). When R _AF ≦ R _th , the subject information acquisition CPU 26 subject position calculation unit 33 of the inner performs object position _{P O} calculated using _{R AF} as follows (S207). The position information calculated by the GPS processing CPU 23 is three-dimensional latitude, longitude, and altitude. However, since position information search for obtaining subject information is possible only by latitude and longitude, calculation on a two-dimensional plane is performed. Do. As shown in FIG. 5 (a), the elevation angle θ obtained in _S204, the horizontal component of _{R AF} is determined as _R AF cos [theta]. As shown in FIG. 5B, Po (x ₂ , y ₂ ) is separated from P _C1 (x ₁ , y ₁ ) by the distance R _AF cos θ in the direction D ₁ , so Po (x ₂ , y ₂ ) = (x ₁ + R _AF cos θ sin D ₁ , y ₁ + R _AF cos θ cos D ₁ ).

Next, the subject name retrieval unit in the subject information acquisition CPU 26 retrieves the closest coordinates from the position coordinates recorded in the subject information recording ROM 27 using the calculated subject position _PO as a key, and corresponding name information. Is acquired (S208). Further, the acquired subject name is superimposed and displayed on the live image on the liquid crystal panel 9 via the CPU 5 (S209). When the user fully presses the release button in this state (S210), the image information captured from the CCD 12 via the AFE 6 is subjected to conventional image processing (S211), and the camera position information P _C1 and the subject position are detected. The information _Po and subject name are added to the image and recorded on the medium 13 (S212).

On the other hand, if R _AF > R _th in S206, a screen for allowing the user to select whether or not to perform long-distance ranging for acquiring subject information is displayed on the liquid crystal panel 9 (S213). If the user selects “No”, the release full-press standby state is entered (S214), the image information obtained from the image pickup unit is subjected to conventional image processing (S215), and only the camera position information _PC1 is obtained. Is added to the image and recorded on the medium 13 (S216). On the other hand, when the user selects “Yes” in S212, the long-distance ranging control unit 31 in the subject information acquisition CPU 26 performs long-distance ranging in the following steps.

First, as in S204, the camera posture information calculated by acquiring the acceleration information of each axis from the acceleration sensor 25 is superimposed on the live image as the electronic level 40, and is in a horizontal state, that is, Ax = Ay = 0, Az. = G guide is performed so as to shoot in the state of -G (S217). Here, a release full press standby state is entered (S218), and an image (referred to as image A) taken by the user is temporarily stored in a temporary memory inside the CPU 5 (S219). Next, a message requesting the user to move in the horizontal direction and shoot again is displayed on the liquid crystal panel 9 (S220). At the same time, as shown in FIG. 6 (a), the electronic azimuth difference 41 indicating the deviation angle between the orientation D ₁ of the current heading D and the image A, obtained by calculating the acceleration information from the acceleration sensor 25 The electronic level 40 indicating the current camera posture information is displayed as a guide on the live image (S221).

Electronic azimuth difference 41 is circular simulating a compass as shown in FIG. 6 (a), displays a line of D ₁ to the reference position of the 12 o'clock direction of the watch, with respect to D ₁ the current direction D The line is displayed as a relative orientation. When the relative azimuth is below a certain level, the two lines are displayed in an overlapping manner, so that the photographer can know that the azimuth is the same. In addition, the electronic level 40 notifies the camera posture by moving the level bar 43 to the left and right according to the horizontal posture of the camera. For example, as shown in FIG. 6B, the level bar 43 moves to the left when the right side is viewed from the camera's normal position, and when the left side is viewed from the camera's normal position as shown in FIG. 6C, the level bar 43 is moved. Moves to the right. Since the level bar 43 and the horizontal reference line 44 overlap, the photographer can know that the camera is in a horizontal state.

Since the release half-press standby state is set here (S222), the user moves in accordance with the guide, and then composes the subject so that the subject falls within the angle of view with the lines of the electronic level 40 and the electronic azimuth difference 41 overlapping. Decide. If the release button is half-pressed here, the GPS antenna 22 and the GPS processing CPU 23 are controlled again to acquire the camera position P _C2 (x ₂ , y ₂ , z ₂ ) (S223). Further, to get the orientation _{D 2} at this time (S224), the photographic subject information obtaining CPU26 calculates the deviation angle [Delta] D = _{D 2} -D ₁ with orientation _{D 1} of the _{D 2} and previously acquired image, which is the minimum misorientation determines whether or not the D _th below (S225). When ΔD> D _th , since the accuracy of subject position calculation is not ensured, a message prompting the user to remeasure is displayed (S226), and the process returns to S222. When ΔD ≦ _Dth , a message indicating that the subject position can be calculated is displayed to the user, and the user is prompted to shoot (S227).

Here, a release full press standby state is entered (S228), and an image (referred to as image B) taken by the user in this state is stored in a different area of the temporary memory inside the CPU 5 as in the case of image A (S229). Further, from the PC ₁ and the PC ₂ , the imaging position difference ΔPc = ((x ₁ −x ₂ ) ² + (y ₁ −y ₂ ) ² ) ^1/2 as shown in FIG. 5C is calculated. (S230). In this calculation, only the GPS x-coordinate and y-coordinate are used, and z-coordinate information is not used.

Next, the long-distance ranging calculation unit 32 in the subject information acquisition CPU 26 calculates the subject distance R _FAR from the temporarily stored images A and B and the photographing position difference ΔPc using the above-described stereo matching image processing technique. (S231). At this time, as shown in FIG. 6 (b), the captured images A and B are sequentially displayed on the liquid crystal panel 9, and the user selects the area A42 where the subject whose subject information is desired to be acquired is selected. I do.

Next, the subject position calculation unit 33 in the subject information acquisition CPU 26 calculates the subject position P _OF using _RFAR as in S207 (S232). That is, P _OF (x ₂ , y ₂ ) = (x ₁ + R _FAR cos θ sin D ₁ , y ₁ + R _FAR cos θ cos D ₁ ) can be calculated.

Then, the object name search unit of the photographic subject information obtaining the CPU26 is to the calculated object position _{P OF} a key searching subject information recording ROM 27 (S233), the camera position information _{P C1} to the image A, the position of the subject The camera position information P _C2 , the subject position information P _OF and the subject name are added to the information P _OF and subject name, and the image B is recorded on the medium 13 (S234).

  In any method, when a plurality of candidate names correspond to the search result of the subject name, all the candidates are displayed and selected by the user, and the selected subject name is added to the image.

  The present invention has been described in detail on the basis of preferred embodiments thereof, but the present invention is not limited to these specific embodiments, and various forms within the scope of the present invention are also included in the present invention. included. A part of the above-described embodiments may be appropriately combined. For example, although an acceleration sensor is used as the posture measurement unit in the present embodiment, posture information may be acquired by performing calculation on a gravity sensor or an angular velocity sensor.

  Further, although the phase difference sensor is used as the short-range distance measuring unit, an optical pulse distance measuring device may be used. In this embodiment, the case where the camera has a database of name information for position coordinates is described. However, the database may be on a recording device such as a PC or on a network. In this case, an identifier indicating that a pair of captured images can be measured at a long distance is recorded in association with the image, so that a long distance measurement is performed by post-processing on the PC, and then the PC or A subject name can be acquired by searching a database on the network.

1 Operation unit 5 CPU
9 Liquid crystal panel 21 Phase difference sensor 22 GPS antenna 23 GPS processing CPU
24 Direction sensor 25 Acceleration sensor 26 Subject information acquisition CPU
27 Subject information recording ROM
30 Subject Distance Determination Unit 31 Long Distance Measurement Control Unit 32 Long Distance Distance Calculation Unit 33 Subject Position Calculation Unit 34 Subject Name Search Unit

Claims (9)

A GPS unit for acquiring current position coordinates;
A near-distance ranging unit that measures the distance to a near-distance subject;
A long-distance ranging unit that measures the distance to a long-distance subject;
An azimuth measuring unit for measuring the shooting direction;
A posture measurement unit that measures the posture in the horizontal and elevation directions during shooting,
A subject distance determination unit that selects either the distance to the subject measured by the short-range distance measuring unit or the distance to the subject measured by the long-range distance measuring unit;
A subject position calculation unit for calculating subject position coordinates;
In an imaging device comprising
The subject distance determination unit selects the distance measured by the short distance measurement unit when the distance to the subject measured by the short distance measurement unit is equal to or less than a preset threshold value, If it is larger, select the distance measured by the long-distance ranging unit,
The subject position calculation unit calculates the subject position using the distance information to the subject selected by the subject distance determination unit, the current position coordinates calculated by the GPS unit, and the shooting direction information measured by the direction measurement unit. An imaging apparatus characterized by: The imaging apparatus according to claim 1, further comprising an image recording unit that records the subject position information calculated by the subject position calculation unit on a recording medium together with a captured image. Using the current position coordinates calculated by the GPS unit, the distance information to the subject selected by the subject distance determination unit, the shooting direction information measured by the direction measurement unit, and the posture information measured by the posture measurement unit A subject position calculator for calculating the position coordinates of the subject;
Subject information recording means for recording position coordinates and position names associated with the position coordinates;
A subject for which a candidate for a subject name is obtained by searching the position coordinates recorded by the subject information recording means and the position name database associated with the position coordinates based on the subject position coordinates calculated by the subject position calculation unit A name search unit;
A display unit for displaying subject name candidates acquired by the subject name search unit;
A current position coordinate calculated by the GPS unit, subject position information calculated by the subject position calculation unit, and a recording unit that records the subject name acquired by the subject name search unit on a recording medium in association with a captured image;
The imaging apparatus according to claim 1, further comprising: The long-distance ranging unit is
A long-distance ranging calculation unit that performs image processing on a pair of images obtained by shooting one subject from different locations, and calculates a distance to the subject from the parallax between the images detected from the two images;
A long-distance ranging control unit for displaying auxiliary information using the posture information from the posture measuring unit and the azimuth information from the azimuth measuring unit, and transmitting a captured image to the long-distance ranging calculation unit;
The imaging apparatus according to claim 1, wherein the imaging apparatus includes: The auxiliary information at the time of shooting the first of a pair of images to be shot is:
The imaging apparatus according to claim 4, wherein the imaging apparatus is attitude information from the attitude measurement unit. The auxiliary information at the time of shooting the second of the pair of images to be shot is
Attitude information from the attitude measurement unit;
The difference between the azimuth measured by the azimuth measuring unit in the first shooting of the pair of images to be shot and the azimuth measured by the azimuth measuring unit in the second shooting of the pair of images to be shot. Direction difference information to be displayed,
The imaging device according to claim 4, wherein the imaging device is an imaging device. The long-distance ranging calculation unit is
The photographed image is displayed and the photographer selects a region where the subject whose subject information is to be acquired is photographed, and the region information is added and transmitted to the long-distance ranging calculation unit.
The imaging apparatus according to claim 4, wherein the long-distance ranging calculation unit calculates a subject distance by performing image processing only on the region. The image recording unit
8. The pair of images is recorded on a recording medium in association with the two images, an identifier indicating that the long-distance ranging unit can calculate long-distance ranging. The imaging device according to any one of the above. 9. The object position calculation unit according to claim 1, wherein the object position calculation unit calculates the object position using only latitude information and longitude information among the current position coordinate information acquired from the GPS unit. The imaging device according to item.