JP2009164985A - Imaging apparatus and imaging method, display control apparatus and display control method, program, and recording medium - Google Patents

Abstract

An object of the present invention is to easily perform focusing while checking the color.
An image pickup optical system and an image pickup device pick up an image, a luminance signal generation unit generates a luminance signal Y of the picked-up image, and an image composition unit includes a viewfinder unit. Of the display area of the captured image to be displayed, the range of the rectangular area in the video data is converted to the luminance signal Y, and the synthesized image in which the rectangular area of the video data is converted to the luminance signal Y 27. The present invention can be applied to, for example, a video camera apparatus including an electronic viewfinder.
[Selection] Figure 2

Description

  The present invention relates to an imaging device and an imaging method, a display control device and a display control method, a program, and a recording medium, and more particularly, for example, an imaging device that enables easy focusing while confirming color. The present invention relates to an imaging method, a display control device, a display control method, a program, and a recording medium.

  Electronic viewfinders that are mainly mounted on commercial video camera devices generally have monochrome display using the CRT (Cathode Ray Tube) method. This is because human visual characteristics are more sensitive to changes in light and dark than changes in hue, so that the monochrome display is easy for the user to focus and to confirm the gradation. .

  However, in the video camera device described above, color information of the captured image cannot be obtained. In such a video camera device, when checking the color, it is necessary to separately output video data of a captured image to a monitor device that is arranged outside the camera and capable of color display.

  On the other hand, cheaper video camera devices generally use a liquid crystal display device capable of color display as an electronic viewfinder. When such a video camera device has a manual focus control function, there is an imaging device that can switch the display of the electronic viewfinder between monochrome display and color display (for example, see Non-Patent Document 1).

  Thus, the user can check the color by changing the display of the electronic viewfinder to color display, and in particular, when using the manual focus control function, the display of the electronic viewfinder is changed to monochrome display. By doing so, focusing can be performed easily.

Sony Marketing Inc., [online], October 11, 2004, [searched November 28, 2007], Internet, <URL: http://www.sony.jp/CorporateCruise/Press/200411/11- 1110 />

  However, in the imaging device of Non-Patent Document 1, color display and monochrome display cannot be performed simultaneously on the electronic viewfinder, and the user cannot easily perform focusing while checking the color. Also, it is unrealistic to switch between color display and monochrome display during moving image capturing.

  The present invention has been made in view of such a situation, and makes it possible to easily perform focusing while checking the color.

  An imaging apparatus or a program according to a first aspect of the present invention is an imaging apparatus that displays a captured image, the imaging unit that captures the image, and the luminance signal of the image captured by the imaging unit. A luminance signal generating means for generating, and a composite image generating means for converting image data of a predetermined area of the display area of the image into the luminance signal and generating a composite image of the image data and the luminance signal And a display unit for displaying the composite image, or a program that causes a computer to function as the imaging device.

  The imaging apparatus further includes signal generation means for generating a signal indicating the timing at which the predetermined area of the display area of the image is displayed on the display means, and the image synthesis means Accordingly, the image data for the predetermined region can be converted into the luminance signal to generate the composite image.

  The imaging apparatus includes an extraction unit that extracts at least one high-frequency component in a horizontal direction or a vertical direction in the image captured by the imaging unit, and a horizontal direction or a vertical direction extracted by the extraction unit. A parameter representing the magnitude of the high-frequency component for each of one or a plurality of lines in the predetermined area of the image display area in at least one of the horizontal direction and the vertical direction based on at least one of the high-frequency components Based on the parameter and the calculation means for calculating, an indicator indicating the size of the parameter is generated, and the combined image and the indicator are combined in correspondence with at least one of the horizontal direction and the vertical direction. And an indicator compositing means.

  The indicator composition means can generate the indicator as a line graph.

  The extraction means can extract at least one high-frequency component in the horizontal direction or the vertical direction using the luminance signal generated by the luminance signal generation means.

  The imaging apparatus can further include setting means for setting the predetermined area.

  An imaging method according to a first aspect of the present invention is an imaging method of an imaging device that displays a captured image, captures the image, generates a luminance signal of the image captured by the imaging unit, Converting image data of a predetermined area of the display area of the image into the luminance signal, generating a composite image of the image data and the luminance signal, and displaying the composite image.

  A display control device or a program according to a second aspect of the present invention is a display control device that controls display of the image of a display device that displays an image captured by an imaging device, and acquires the image. Luminance signal generation means for generating a luminance signal of the image, image data for a predetermined area of the display area of the image is converted into the luminance signal, and a composite image of the image data and the luminance signal is obtained. A program that causes a computer to function as a display control device or a display control device that includes a composite image generation unit that generates and a display control unit that controls display of the composite image.

  The display control device further includes signal generation means for generating a signal indicating a timing at which the predetermined area of the display area of the image is displayed on the display apparatus, and the image composition means includes the signal Accordingly, the image data for the predetermined area can be converted into the luminance signal to generate the composite image.

  The display control device includes an extraction unit that extracts at least one high-frequency component in a horizontal direction or a vertical direction in the image captured by the imaging device, and a horizontal direction or a vertical direction extracted by the extraction unit. A parameter that represents the magnitude of the high-frequency component for each of one or more lines in the predetermined area of the image display area in at least one of the horizontal direction and the vertical direction based on at least one high-frequency component in the direction Based on the parameter, and generating an indicator indicating the magnitude of the parameter, and corresponding the position of at least one of the horizontal direction and the vertical direction, the composite image and the indicator Indicator combining means for combining may be further provided.

  The display control device may further include setting means for setting the predetermined area.

  A display control method according to a second aspect of the present invention is a display control method of a display control device that controls display of the image of a display device that displays an image captured by an imaging device, and acquires the image. Generating a luminance signal of the image, converting image data for a predetermined area of the display area of the image into the luminance signal, generating a composite image of the image data and the luminance signal, Controlling the display of the composite image.

  In the first aspect of the present invention, an image is picked up, a luminance signal of the image picked up by the image pickup means is generated, and image data for a predetermined region of the display region of the image is converted into a luminance signal. Then, a composite image of the image data and the luminance signal is generated, and the composite image is displayed.

  In the second aspect of the present invention, an image is acquired, a luminance signal of the image is generated, image data for a predetermined area of the display area of the image is converted into a luminance signal, and the image data and the luminance signal are converted. A composite image is generated and display of the composite image is controlled.

  The program can be recorded on a recording medium.

  According to the first aspect of the present invention, an image can be taken, and in particular, focusing can be easily performed while checking the color.

  According to the second aspect of the present invention, it is possible to display an image picked up by the image pickup apparatus, and in particular, it is possible to easily perform focusing while checking the color.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 is a block diagram showing a configuration example of an embodiment of an imaging apparatus to which the present invention is applied.

  In FIG. 1, the imaging apparatus 11 includes an imaging optical system 21, an imaging element 22, a camera signal processing unit 23, a codec unit 24, a recording unit 25, a finder display image conversion unit 26, a viewfinder unit 27, and an operation input unit 28. Composed.

  The imaging optical system 21 includes a lens, a mechanical shutter, and the like.

  The imaging element 22 receives light incident through the imaging optical system 21. The image sensor 22 supplies the camera signal processing unit 23 with image data digitally converted via an AD (Analog / Digital) converter (not shown).

  Here, the AD converter may be mounted in the image sensor 22, particularly when the image sensor 22 is a CMOS (Complementary Metal Oxide Semiconductor) type solid-state image sensor. When the is a solid-state image sensor other than the CMOS solid-state image sensor, it may be arranged outside the image sensor 22.

  Further, in FIG. 1, the image pickup device 22 of the image pickup apparatus 11 is illustrated as a single plate color type solid-state image pickup device having a color filter that transmits different wavelength ranges for each pixel on the light receiving surface. Needless to say, the imaging system of the imaging device 11 may have a three-plate configuration in which light separated by a wavelength via a prism optical system or the like is received by three different imaging elements.

  Note that the imaging data supplied from the imaging element 22 to the camera signal processing unit 23 is so-called RAW data.

  The camera signal processing unit 23 performs processing such as white balance correction processing, gain adjustment, demosaic processing, linear matrix processing, and gamma correction processing on the RAW data supplied from the image sensor 22, and the processing result is obtained. The video data is supplied to the codec unit 24 and the finder display image conversion unit 26.

  The codec unit 24 compresses the video data supplied from the camera signal processing unit 23 by video signal encoding processing such as MPEG (Moving Picture Experts Group) 2, H.264, or JPEG (Joint Photographic Experts Group) 2000. Then, an encoded stream is generated and supplied to the recording unit 25.

  The recording unit 25 is configured by a recording medium such as an optical disc, a hard disk, or a semiconductor memory, for example, and records the encoded stream supplied from the codec unit 24 or a removable recording medium (optical disc, magneto-optical disc). Or an encoded stream supplied from the codec unit 24 is recorded on a mounted removable recording medium.

  The viewfinder display image conversion unit 26 generates a luminance signal from the video data supplied from the camera signal processing unit 23, and in order to make it easier for the user to grasp the in-focus state, Video data and luminance generated by converting the range of an arbitrary rectangular area into a luminance signal, performing luminance signal conversion processing, and converting the range of an arbitrary rectangular area of the video data into a luminance signal A composite image with the signal is supplied to the viewfinder unit 27, and the display is controlled.

  The video data supplied from the camera signal processing unit 23 to the viewfinder display image conversion unit 26 is displayed on the viewfinder unit 27 as a captured image for user confirmation.

  Details of the finder display image conversion unit 26 will be described later with reference to FIG.

  The viewfinder unit 27 presents the composite image supplied from the viewfinder display image conversion unit 26 to the user as a captured image in which an arbitrary rectangular area is displayed in monochrome.

  The operation input unit 28 includes, for example, various operation buttons, a dial, and the like. The operation input unit 28 is operated by a user to input an instruction to the imaging device 11 and supplies information indicating the operation content to each block of the imaging device 11. .

  In FIG. 1, the codec unit 24 and the recording unit 25 are shown as components of the imaging device 11, but the imaging device 11 does not have these processing units and outputs (supplies) from the camera signal processing unit 23. The video data may be output to an external device or network as it is.

  In FIG. 1, the video data supplied to the codec unit 24 and the video data supplied to the finder display image conversion unit 26 are the same. It is also possible to supply video data whose resolution has been converted according to the resolution of the video.

  Further, in FIG. 1, the viewfinder unit 27 is shown as a component of the imaging device 11, but the imaging device 11 does not have the viewfinder unit 27 and supplies a composite image to a connected external display device. Then, it may be in a form that can be displayed.

  The imaging device 11 includes an imaging optical system 21, an imaging element 22, a camera signal processing unit 23, a codec unit 24, a recording unit 25, and an operation input unit 28, and a finder display image conversion unit 26 and a viewfinder unit 27. May be configured as a separate device.

  At this time, the finder display image conversion unit 26 captures the captured image in order to make it easier for the user to grasp the in-focus state of the video data that is the same as the video data supplied to the codec unit 24 or whose resolution has been converted. Generated by converting the range of an arbitrary rectangular area of the video data for the display area into a luminance signal, and converting the range of the arbitrary rectangular area of the video data into a luminance signal The composite image is supplied to the viewfinder unit 27 and its display is controlled.

  Furthermore, the finder display image conversion unit 26 and the viewfinder unit 27 may be configured as separate devices.

  At this time, the display control device corresponding to the finder display image conversion unit 26 makes it easy for the user to grasp the in-focus state of the video data that is the same as the video data supplied to the codec unit 24 or whose resolution has been converted. In order to achieve this, a process for converting the range of an arbitrary rectangular area of the video data for the display area of the captured image into a luminance signal is performed, and the range of the arbitrary rectangular area of the video data is converted to a luminance signal. The composite image generated by this is supplied to an external display device (display device corresponding to the viewfinder unit 27) (not shown), and the display is controlled.

  FIG. 2 is a block diagram showing a configuration of the finder display image conversion unit 26 of FIG. The finder display image conversion unit 26 includes a luminance signal generation unit 41, a switching pulse generation unit 42, and an image synthesis unit 43.

  Video data (hereinafter also referred to as input video data) supplied from the camera signal processing unit 23 to the finder display image conversion unit 26 is supplied to the luminance signal generation unit 41 and also to the image composition unit 43.

  The luminance signal generation unit 41 performs conversion processing as shown in Expression (1) on the input video data from the camera signal processing unit 23 to generate a luminance signal Y, and supplies the luminance signal Y to the image synthesis unit 43.

      Y = 0.257R + 0.504G + 0.098B (1)

  In Expression (1), R is an R signal of input video data, G is a G signal of input video data, and B is a B signal of input video data.

  Here, the conversion process to the luminance signal Y is not limited to the expression (1), and for example, a process of simply adding the R signal, the G signal, and the B signal may be used.

  The switching pulse generation unit 42 stores information representing a rectangular area that is predetermined in the captured image displayed on the viewfinder unit 27 or determined by the user using the operation input unit 28. Based on the information stored in the rectangular area memory unit 42a, the pulse signal ME indicating the timing displayed on the viewfinder unit 27 is generated and supplied to the image synthesis unit 43. .

  More specifically, the rectangular area memory unit 42a stores pixel positions of pixels representing the rectangular area, which are input by operating the operation input unit 28. The switching pulse generation unit 42 is a period during which pixels in the rectangular area are scanned in scanning of the captured image displayed on the viewfinder unit 27 based on the pixel position of the rectangular area stored in the rectangular area memory unit 42a. Enable (generate) the pulse signal ME. Further, the switching pulse generator 42 invalidates (does not generate) the pulse signal ME during a period in which pixels in a region other than the rectangular region are scanned.

  Details of the pulse signal ME generated by the switching pulse generator 42 will be described later with reference to FIG.

  Based on the pulse signal ME supplied from the switching pulse generator 42, the image synthesizer 43 converts the range of the rectangular area in the video data for the captured image displayed on the viewfinder 27 to the luminance signal generator 41. Is converted into the luminance signal Y from the video signal to generate a composite image of the video data and the luminance signal, which is supplied to the viewfinder unit 27 and its display is controlled.

  The image composition unit 43 is configured to include a rectangular area determination unit 43a and a buffer unit 43b.

  The rectangular area determination unit 43a determines whether the pixel position of the target pixel that is a pixel of interest in scanning of the captured image is within the rectangular area, according to the pulse signal ME supplied from the switching pulse generation unit 42. The video data for the target pixel is converted into a luminance signal Y according to the determination result.

  More specifically, when the pulse signal ME from the switching pulse generation unit 42 is valid, the rectangular area determination unit 43a determines that the pixel position of the target pixel is within the rectangular area, and The video data is converted into a luminance signal Y from the luminance signal generation unit 41 and supplied to the buffer unit 43b.

  In addition, when the pulse signal ME from the switching pulse generation unit 42 is invalid, the rectangular area determination unit 43a determines that the pixel position of the target pixel is not within the rectangular area, and the video data regarding the target pixel is used as it is. This is supplied to the buffer unit 43b.

  The buffer unit 43b temporarily stores the luminance signal Y or video data for each pixel in the captured image from the rectangular area determination unit 43a. When the determination processing in the rectangular area determination unit 43a is completed for all the pixels in the captured image, the image composition unit 43 uses the luminance signal Y and the video data as information on all the pixels stored in the buffer unit 43b. Is supplied to the viewfinder 27 as a composite image.

  In this way, in the synthesized image supplied from the image synthesis unit 43 to the viewfinder unit 27, a part of the rectangular area in the screen of the viewfinder unit 27 is converted into the luminance signal Y according to the pulse signal ME. It is displayed as a captured image.

  The image composition unit 43 may be configured without the buffer unit 43b. In this case, the image composition unit 43 supplies the luminance signal Y or video data from the rectangular area determination unit 43a to the viewfinder unit 27 as it is.

  FIG. 3 is a time chart showing the timing of the pulse signal ME generated by the switching pulse generator 42.

  Here, it is assumed that the captured image displayed on the viewfinder unit 27 is a so-called progressive image that is scanned pixel by pixel from left to right and from top to bottom in a line (horizontal line). .

  In FIG. 3, the waveform shown at the top is the vertical synchronization signal VD synchronized with the input video data supplied from the control unit (not shown) to the switching pulse generation unit 42, and the second and fifth waveforms from the top. Is a horizontal synchronization signal HD that is supplied from a control unit (not shown) to the switching pulse generation unit 42 and is synchronized with the input video data.

  In FIG. 3, the third and sixth waveforms from the top are data enable signals DE representing the effective pixel region actually displayed on the viewfinder section 27, and are shown at the fourth and bottom from the top. The waveform is a pulse signal ME generated by the switching pulse generator 42.

  In FIG. 3, the horizontal synchronization signal HD shown fifth from the top represents one pulse of the horizontal synchronization signal HD shown second from the top, and the data enable signal DE shown sixth from the top is 3 from the top. This represents one pulse of the data enable signal DE shown in the second, and the pulse signal ME shown at the bottom represents one pulse of the pulse signal ME shown fourth from the top.

  When the vertical synchronizing signal VD, the horizontal synchronizing signal HD, and the data enable signal DE are supplied to the switching pulse generating unit 42 in FIG. 2, the switching pulse generating unit 42 determines the timing of these signals and the rectangular area memory unit 42a. In the scanning of the captured image displayed on the viewfinder unit 27 from the pixel position of the rectangular area stored in the frame, the pulse signal ME is enabled during the period during which the pixels of the rectangular area are scanned. The pulse signal ME is invalidated during the period when the pixels in the area are scanned.

  That is, the switching pulse generation unit 42 has the vertical synchronization signal VD (the top waveform in FIG. 3) at the high level and the data enable signal DE (upper side in FIG. 3) for the scanning of the pixels in the vertical direction. The pulse signal ME is set to the HIGH level during the period in which the pixels in the rectangular area are scanned in the vertical direction in the period in which the third waveform from (1) to HIGH).

  In the example of FIG. 3, the switching pulse generator 42 performs the period monov after the period sy (LOW level) has elapsed after the data enable signal DE (third waveform from the top in FIG. 3) has become HIGH level. During this period, the pulse signal ME is set to the HIGH level.

  Then, the rectangular area determination unit 43a applies to the video data supplied from the camera signal processing unit 23 during the period in which the pulse signal ME is invalid (LOW level) including the period sy in the scanning of the pixels in the vertical direction. In the period monov in which the pulse signal ME is valid (HIGH level), the video data supplied from the camera signal processing unit 23 is converted into the luminance signal Y supplied from the luminance signal generating unit 41.

  In addition, the switching pulse generation unit 42 has the horizontal synchronization signal HD (the uppermost waveform in FIG. 3) at the high level and the data enable signal DE (upper side in FIG. 3) with respect to the horizontal pixel scanning. The pulse signal ME is set to the high level during the period in which the pixels in the rectangular area are scanned in the horizontal direction during the period in which the sixth waveform from the first to the sixth waveform) is at the high level.

  In the example of FIG. 3, the switching pulse generator 42 performs the period monoh after the period sx (LOW level) has elapsed after the data enable signal DE (third waveform from the top in FIG. 6) has become HIGH level. During this period, the pulse signal ME is set to the HIGH level.

  Then, the rectangular area determination unit 43a applies to the video data supplied from the camera signal processing unit 23 during a period in which the pulse signal ME is invalid (LOW level) including the period sx in the horizontal pixel scanning. In the period monoh when the pulse signal ME is valid (HIGH level), the video data supplied from the camera signal processing unit 23 is converted into the luminance signal Y supplied from the luminance signal generating unit 41.

  FIG. 4 shows an example of a captured image displayed on the viewfinder unit 27.

  As shown in FIG. 4, the viewfinder unit 27 displays a captured image in which the rectangular area A determined by the pixel position of the rectangular area of the rectangular area memory unit 42 a is converted into the luminance signal Y. This rectangular area A is an area in which a subject that the user wants to focus is shown, and is hereinafter also referred to as a focusing area.

  In FIG. 4, for example, if the pixel position at the upper left corner of the effective pixel area displayed on the viewfinder 27 is the origin (0, 0), the horizontal right direction is the x direction and the vertical downward direction is the y direction, the focusing area Is the pixel position (sx, sy) determined by the pixel position in the x direction scanned in the period sx and the pixel position in the y direction scanned in the period sy from the origin (0, 0). The rectangular area A has two sides, the length scanned in the period monoh in the horizontal right direction (x direction) and the length scanned in the period monov in the vertical downward direction (y direction).

  The focusing area can be set at an arbitrary position, and the user can change the pixel position of the pixel representing the rectangular area stored in the rectangular area memory unit 42a using the operation input unit 28. However, the position and size may be changed dynamically.

  In this embodiment, the focusing area is a rectangular area. However, the switching pulse generator 42 generates a pulse signal ME that is effective (HIGH level) for an area of an arbitrary shape, so that the circular or It may be a hexagonal region.

  Furthermore, in this embodiment, the number of focusing areas is one. However, the switching pulse generator 42 generates a pulse signal ME that is effective (HIGH level) for a plurality of regions, so that a plurality of focusing areas are generated. You may make it display.

  Next, imaging processing executed by the imaging device 11 will be described with reference to the flowchart of FIG. In addition, although each process of step S12 thru | or step S18 can be performed in parallel with the process of step S14 and step S15, and the process of step S16 and step S17 as a process of the flame | frame with the imaged moving image. Basically, they are executed sequentially according to the described steps. However, these are processes executed on the entire moving image composed of a plurality of frames that are sequentially picked up, and therefore are executed in parallel in the image pickup apparatus 11.

  In step S <b> 11, the imaging optical system 21 and the imaging element 22 start imaging based on an operation input from the user by the operation input unit 28. That is, the image sensor 22 receives light incident through the imaging optical system 21 and supplies RAW data as image data digitally converted by the AD converter to the camera signal processing unit 23. The process proceeds to step S12.

  In step S <b> 12, the camera signal processing unit 23 performs processing such as white balance correction, gain adjustment, demosaic processing, linear matrix processing, and gamma correction processing on the RAW data supplied from the image sensor 22. The video data obtained as a result is supplied to the codec unit 24 and the finder display image conversion unit 26, and the process proceeds to step S13.

  In step S13, the operation input unit 28 determines whether or not the start of recording has been instructed based on the operation input from the user.

  If it is determined in step S13 that the start of recording is not instructed, the process proceeds to step S16.

  On the other hand, if it is determined in step S13 that the start of recording has been commanded, the process proceeds to step S14.

  In step S14, the codec unit 24 compresses the video data supplied from the camera signal processing unit 23 by a video signal encoding process such as MPEG2, H.264, or JPEG2000, and generates an encoded stream. This is supplied to the recording unit 25. Thereafter, the process proceeds to step S15.

  In step S15, the recording unit 25 records the encoded stream supplied from the codec unit 24, and the process proceeds to step S16.

  In step S <b> 16, the finder display image conversion unit 26 executes a luminance signal conversion process described with reference to FIG. 6, and supplies a composite image obtained as a result of the luminance signal conversion process to the viewfinder unit 27. The process proceeds to step S17.

  In step S <b> 17, the viewfinder unit 27 converts the range of an arbitrary rectangular area A in the video data into the luminance signal Y in order to make it easier for the user to grasp the in-focus state as described with reference to FIG. 4, for example. The converted captured image is displayed, and the process proceeds to step S18.

  In step S <b> 18, the operation input unit 28 determines whether the end of imaging has been commanded based on the operation input from the user.

  If it is determined in step S18 that the end of imaging has not been commanded, the process returns to step S12, and the subsequent processes are repeated.

  On the other hand, if it is determined in step S18 that the end of imaging has been commanded, the process ends.

  By such processing, the rectangular area A in which the subject to be focused is displayed is displayed with the luminance signal Y (monochrome display) in the viewfinder unit 27, and other areas are displayed in color. For a subject to be focused, the difference between light and dark is easily recognized according to human visual characteristics, and the user can easily perform focusing while checking the color.

  Next, the luminance signal conversion process, which is an example of the process executed in step S16 of FIG. 5, will be described with reference to the flowchart of FIG. The luminance signal conversion process is executed by the finder display image conversion unit 26.

  In step S <b> 21, the luminance signal generation unit 41 performs a conversion process, for example, as shown in Equation (1) above on the video data from the camera signal processing unit 23 to generate a luminance signal. The process proceeds to step S22.

  In step S22, the rectangular area determination unit 43a sets the pixel position (x, y) of the pixel of interest in scanning of the captured image displayed on the viewfinder unit 27 as the origin (0, 0) described in FIG. Advances to step S23.

  In step S23, the switching pulse generation unit 42 generates a pulse signal from the timing of the vertical synchronization signal VD, the horizontal synchronization signal HD, and the data enable signal DE and the pixel position of the rectangular area stored in the rectangular area memory unit 42a. Generation of ME is started, and the process proceeds to step S24.

  In step S24, the rectangular area determination unit 43a determines whether the pixel position (x, y) of the pixel of interest in scanning of the captured image is within the rectangular area based on the pulse signal ME supplied from the switching pulse generation unit 42. Determine whether.

  If it is determined in step S24 that the pixel position (x, y) of the pixel of interest in scanning of the captured image is within the rectangular area, that is, if the pulse signal ME from the switching pulse generator 42 is valid, The area determination unit 43a converts the video data for the target pixel into the luminance signal Y from the luminance signal generation unit 41, supplies the converted signal to the buffer unit 43b, and the process proceeds to step S26.

  On the other hand, when it is determined in step S24 that the pixel position (x, y) of the pixel of interest in the scanning of the captured image is not within the rectangular region, that is, when the pulse signal ME from the switching pulse generation unit 42 is invalid, The rectangular area determination unit 43a supplies the video data for the target pixel as it is to the buffer unit 43b, and the process proceeds to step S26.

  In step S <b> 26, the rectangular area determination unit 43 a determines whether the pixel position (x, y) of the target pixel is within the rectangular area for all the pixels in the x direction in the effective pixel area (that is, one horizontal line). Determine if it is over.

  If it is determined in step S26 that the process has not been completed for all pixels in the x direction, the process proceeds to step S27.

  In step S27, the rectangular area determination unit 43a increments x by 1, the process returns to step S24, and the subsequent processes are repeated. That is, the pixel position (x, y) of the target pixel moves by one in the x direction, and the process for the target pixel is repeated.

  On the other hand, when it is determined in step S26 that the processing for all the pixels in the x direction has been completed, that is, the processing in steps S24 to S27 is repeated, and the processing for the pixels for one line in the horizontal direction is completed. When it is determined that the rectangular area has been determined, the rectangular area determination unit 43a sets x to 0 in step S28, and the process proceeds to step S29.

  In step S29, the rectangular area determination unit 43a determines whether the pixel position (x, y) of the target pixel is within the rectangular area for all pixels in the effective pixel area in the y direction (that is, all horizontal lines). It is determined whether or not.

  If it is determined in step S29 that the process has not been completed for all pixels in the y direction, the process proceeds to step S30.

  In step S30, the rectangular area determination unit 43a increments y by 1, the process returns to step S24, and the subsequent processes are repeated. That is, the pixel position (x, y) of the target pixel becomes the leftmost pixel position of the horizontal line moved by one in the y direction, and the process for the pixels on the horizontal line is repeated.

  On the other hand, when it is determined in step S29 that the processing for all the pixels in the y direction has been completed, that is, the processing in steps S24 to S30 is repeated to determine whether or not all the pixels are in the rectangular area. When it is determined that the determination is made, the image composition unit 43 synthesizes an image composed of all the pixels in the captured image (effective pixel region) stored in the buffer unit 43b with the luminance signal Y and the video data. The image is supplied to the viewfinder unit 27, and the process returns to step S16 in FIG. 5 and proceeds to step S17.

  Through such processing, the range of the rectangular area A in which the subject to be focused is displayed in the video data supplied from the camera signal processing unit 23 can be converted into the luminance signal Y.

  By the way, in a professional video camera device, during manual focus control, various auxiliary information may be displayed on an electronic viewfinder mounted on the video camera device so that the user can easily focus. Has been made.

  For example, conventionally, a spatial high-frequency component corresponding to the contour of a captured image is extracted from the captured image, and the contour component is increased and then mixed with the captured image and displayed on the viewfinder. Is used (for example, see Japanese Patent Laid-Open No. 2000-350224).

  In addition, the result of applying the high-pass filter to a specific area in the center of the captured image is integrated, and the resulting one-dimensional parameter is displayed on the viewfinder to express the degree of focusing. There is a technique that can be used (see, for example, JP-A-3-11884 or JP-A-3-179976).

  Since the image displayed when using the conventional peaking circuit described in Japanese Patent Laid-Open No. 2000-350224 described above is an image in which the contour of the subject in the image is emphasized, the subject to be actually focused There is a feature that it is easy to carry out focusing by specifying.

  However, when a wide-angle lens or the like is used, the depth of field becomes deep and the subject image on the screen becomes small. Therefore, the technique described in Japanese Patent Laid-Open No. 2000-350224 described above is optimal. It is difficult to obtain a correct focus position.

  Further, in the technique described in Japanese Patent Laid-Open No. 2000-350224 described above, only the contour portion of the subject image is displayed in an emphasized manner. It was not possible to judge whether there was.

  On the other hand, in the auxiliary information display method described in Japanese Patent Application Laid-Open No. 3-11884 or Japanese Patent Application Laid-Open No. 3-179976, a parameter indicating the size of a high-frequency component in an image is displayed on the viewfinder as a bar-shaped indicator, for example. Therefore, when the focus position of the lens is moved, it is easy to grasp how much the focusing condition increases or decreases.

  However, in the technique described in Japanese Patent Laid-Open No. 3-11884 or Japanese Patent Laid-Open No. 3-179976, the parameter is set as a result of integrating all high-frequency components in a specific area of the screen. It is difficult for the user to accurately grasp whether the subject is in focus, and to calculate a parameter by applying a high-pass filter to the region where the desired subject that the user is trying to focus on is captured. .

  Therefore, in the imaging device 11, the user can easily grasp the subject in focus in the captured image, and can easily grasp the peak of the focusing state with respect to the predetermined subject. .

  FIG. 7 is a block diagram illustrating another functional configuration example of the imaging device 11. In FIG. 7, parts corresponding to those in FIG. 1 are denoted by the same reference numerals, and description thereof is omitted as appropriate. That is, the imaging apparatus 11 in FIG. 7 basically has the same configuration as that of the imaging apparatus 11 in FIG. 1 except that it includes a focus assist processing unit 51 in place of the finder display image conversion unit 26. .

  The camera signal processing unit 23 performs processing such as white balance correction processing, gain adjustment, demosaic processing, linear matrix processing, and gamma correction processing on the RAW data supplied from the image sensor 22, and the processing result is obtained. The video data is supplied to the codec unit 24 and the focus assist processing unit 51.

  The focus assist processing unit 51 synthesizes a focus indicator, which is auxiliary information for making it easier for the user to grasp the in-focus state, with the video data supplied from the camera signal processing unit 23, and the focus indicator is synthesized. The obtained video data is supplied to the viewfinder unit 27 and the display thereof is controlled.

  Details of the focus assist processing unit 51 will be described later with reference to FIGS.

  The viewfinder unit 27 displays a captured image with a focus indicator added based on the video data supplied from the focus assist processing unit 51. That is, the viewfinder unit 27 presents auxiliary information for focusing together with the captured image to the user.

  FIG. 8 is a block diagram showing a configuration of a focus assist processing unit 51-1 which is a first embodiment of the focus assist processing unit 51 of FIG. The focus assist processing unit 51-1 includes a luminance signal generation unit 71, a horizontal differential filter processing unit 72, an addition unit 73, and an indicator synthesis unit 74.

  The input video data supplied from the camera signal processing unit 23 to the focus assist processing unit 51-1 is supplied to the luminance signal generation unit 71 and is also supplied to the indicator synthesis unit 74.

  Similar to the luminance signal generation unit 41 in the finder display image conversion unit 26 of FIG. 2, the luminance signal generation unit 71 applies the input video data from the camera signal processing unit 23 to the above-described equation (1). Conversion processing is performed to generate a luminance signal Y, which is supplied to the horizontal differential filter processing unit 72.

  The horizontal differential filter processing unit 72 extracts a high-frequency component in the horizontal direction from the luminance signal Y supplied from the luminance signal generation unit 71 and supplies it to the addition unit 73.

  Specifically, the horizontal differential filter processing unit 72 performs, for example, a horizontal differential filter having a coefficient as shown in FIG. 9 on the luminance signal Y from the luminance signal generation unit 71, and obtains the obtained differential data. It supplies to the addition part 73.

  Since the differential filter shown in FIG. 9 performs differential processing in the horizontal space direction on the image, the horizontal differential filter processing unit 72 outputs a large value when an edge exists in the vertical direction.

  Further, the horizontal differential filter processing unit 72 may be configured to process several lines at once instead of processing for each horizontal line.

  In this case, a filter other than the horizontal differential filter shown in FIG. 9 may be used as long as it can extract a high-frequency component in the horizontal direction.

  The adding unit 73 adds the differential data for one horizontal line of the differential data supplied from the horizontal differential filter processing unit 72, and calculates the differential addition value for each horizontal line of the high frequency component corresponding to that horizontal line. This is supplied to the indicator composition unit 74 as a parameter representing the size.

  Moreover, the addition part 73 may be the structure which processes several lines collectively, without performing an addition process for every horizontal line.

  Further, for example, as will be described later with reference to FIG. 13, the adding unit 73 limits the focusing area which is a spatially focused region, and adds only the differential data in the focusing area among the horizontal lines. You may make it do.

  If a parameter representing the magnitude of the high-frequency component for one horizontal line can be obtained, the horizontal differential filter processing unit 72 and the adding unit 73 extract the high-frequency component in the horizontal direction and perform the linear addition process instead of performing other processing. It is good also as a structure which performs the process of. For example, the harmonic component may be obtained by applying FFT (Fast Fourier Transform) to a predetermined region in the image.

  The indicator synthesis unit 74 is auxiliary information serving as an index for the user to focus on a desired subject based on a parameter representing the magnitude of the high-frequency component for each horizontal line supplied from the addition unit 73. A certain focus indicator is generated, synthesized on the video data supplied from the camera signal processing unit 23, supplied to the viewfinder unit 27, and its display is controlled.

  The video data combined with the focus indicator by the indicator combining unit 74 is displayed on the viewfinder unit 27 as a captured image with the focus indicator added.

  Next, an example of the focus indicator combined by the indicator combining unit 74 will be described with reference to FIGS.

  First, a first display example of the focus indicator will be described with reference to FIGS. 10 and 11.

  In the case of FIG. 10 and FIG. 11, the indicator composition unit 74 generates a focus indicator indicating a parameter for each horizontal line as a line graph indicated by a in the figure, and synthesizes it on the video data. 10 and 11, the closer the position of the focus indicator line is to the right end of the screen, the smaller the corresponding horizontal line parameter, that is, the corresponding horizontal line has less harmonic components. The horizontal line corresponding to the mountain portion of the line graph shows that there are many high frequency regions.

  In the case of the example in FIG. 10, the focus is on the woman in the center, and a mountain of focus indicators appears on a horizontal line in which a face portion with a lot of harmonic components is reflected in the image of the woman.

  In the example of FIG. 11, since the focus is on the background, the portion corresponding to the horizontal line with many background portions, not the horizontal line in which the face portion with many harmonic components is reflected in the female image. A pile of focus indicators appears. That is, in the image, the woman is in a state of previous blur and is not resolved.

  When actually capturing a moving image (or still image) with manual focus instead of autofocus, the user determines the focus (focuses) while viewing the image displayed on the viewfinder unit 27. That is, FIG. 10 and FIG. 11 correspond to display contents of the viewfinder 27 for a moment while the user moves the focus position in order to set the focus by manual focus instead of autofocus.

  Since the spatial frequency of the foreground (in this case, a woman) or the background depends on the subject itself, the magnitude of the indicator mountain does not necessarily match the focus of the subject. Actually, when the focus position of the lens is operated and the focus position is moved from the near side to the back side, the displayed captured image and indicator are changed from the state shown in FIG. 10 to the state shown in FIG.

  Specifically, when attention is paid to a line in which a woman's face, which is an area to be focused, is focused, the woman's face is reflected as the focus position gradually moves backward from the state before the woman's position. The peak of the indicator near the line becomes higher, and in the state of FIG. 10, the peak of the indicator near the line in which the female face is reflected becomes the maximum, and the peak becomes smaller toward FIG.

  On the other hand, when attention is paid to the line in which the background is reflected, in the state of FIG. 10, the peak of the indicator in the corresponding part is small, and the peak of the indicator becomes large toward FIG. 11, and the focus position of the lens is moved further backward. In this case, the peak of the indicator line in the background portion becomes the maximum value in the state of FIG. 11, and then gradually decreases.

  Thus, the display of the focus indicator, which is auxiliary information serving as an index for focusing, makes it easy for the user to know the change in focus when the focus position of the lens is operated. In other words, the user can easily recognize whether or not the desired subject is in focus by checking the indicator while moving the focus position of the lens back and forth. Even when a moving image is captured for a subject moving in the screen, checking the focus indicator makes it easy to check the temporal spatial frequency change in the screen. It is possible to easily perform operations such as focus feeding according to the movement of the camera.

  In the examples of FIGS. 10 and 11, the inside of the line graph is filled with an arbitrary color so that the line graph is easy to understand. However, the video data (captured image) of the inner part of the line graph is hidden. In order not to stutter, only the line segment of the line graph may be displayed, and the video data inside the line graph may be displayed as it is, or an appropriate transmittance is set for the inner part of the line graph. It is also possible to perform composite display with an arbitrary color. The display position of the line graph as the focus indicator is not limited to the right edge of the screen, and may be any position such as the left edge of the screen.

  Next, a second display example of the focus indicator will be described with reference to FIG.

  In the case of FIG. 12, the indicator composition unit 74 generates a focus indicator indicating a parameter for each horizontal line as a bar graph as indicated by a ′ in the figure, and synthesizes it on the video data. In the example of FIG. 12, the bar graph is displayed every several lines. However, it may be displayed for all lines, or an average value of parameters of a plurality of lines is calculated, and the average value is expressed as a plurality of lines. You may make it display for every. In this case, the shorter the bar graph of the focus indicator, the smaller the parameter, that is, the horizontal line with less harmonic components, indicating that the longest horizontal line is the horizontal line with the highest frequency range. Is shown.

  In the case of the example of FIG. 12, the focus is on the woman in the center, and the bar graph on the horizontal line corresponding to the portion of the image of the woman in which the face portion with many harmonic components is reflected is long.

  As shown in FIG. 12, even when the focus indicator is displayed as a bar graph, as in the case of FIGS. 10 and 11, the focus indicator is a temporal high-frequency component of each horizontal line in the displayed captured image. Express the transition of.

  Therefore, the user can easily know the change in focus when the lens focus position is operated by checking the indicator while moving the lens focus position back and forth, and can focus on a desired subject. It becomes possible. Even when a moving image is captured for a subject moving in the screen, checking the focus indicator makes it easy to check the temporal spatial frequency change in the screen. Work such as focus feed that matches the movement of the camera becomes easy.

  The display position of the bar graph as the focus indicator is not limited to the right edge of the screen, and may be any position such as the left edge of the screen.

  In the screen, an area to be focused, that is, a focusing area may be limited.

  For example, as shown in FIG. 13, two lines indicated by b-1 and b-2 in the figure are displayed vertically on the screen, and the inside of these lines is set as a focusing area. The addition unit 73 performs addition processing only on pixels within a region sandwiched between two lines indicated by b-1 and b-2 in the drawing. As a result, the display of the focus indicator is affected only by the spatial frequency in the focusing area, so that more accurate focusing can be achieved even in a complicated scene where a plurality of objects exist on one horizontal line. It is possible to display a focus indicator that assists.

  Further, regarding the setting of the focusing area for displaying the focus indicator, the user may be able to freely set it, or it may be determined in advance in the imaging device 11. Further, the focusing area may be a part of the screen cut out in a circle or a polygon, or a plurality of focusing areas may be provided.

  Further, it goes without saying that the user can select whether or not to limit the focusing area.

  In the above description, in the focus assist processing unit 51-1, the horizontal differential filter processing unit 72 extracts a high-frequency component in the horizontal direction from the luminance signal Y supplied from the luminance signal generation unit 71, and an addition unit. 73, the differential data for one horizontal line is added, and the differential addition value for each horizontal line is calculated as a parameter representing the magnitude of the high-frequency component corresponding to that horizontal line.

  On the other hand, for example, a harmonic component in the vertical direction instead of the horizontal direction is extracted, a parameter representing the size of the high frequency component corresponding to one vertical line is calculated, and the magnitude of the high frequency component for each vertical line is calculated. It is also possible to generate an indicator indicating the size of the parameter indicating the length, combine it on the video data supplied from the camera signal processing unit 23, and output the displayed data to the viewfinder unit 27 for display.

  When displaying an indicator indicating the magnitude of the parameter representing the magnitude of the high-frequency component for each vertical line, the focus assist processing unit 51-1 stores, for example, one vertical line for storing the addition result of the differential data of the vertical component. In addition to a horizontal differential filter processing unit 72, a vertical differential filter processing unit that performs vertical differential processing is provided, and vertical differential filter processing is performed for each vertical line instead of the addition unit 73. An addition unit that executes a process of adding the differential data from the unit to the value of the addition memory in the corresponding vertical line, and instead of the indicator synthesis unit 74, an indicator that indicates the parameter size for each vertical line Generated and synthesized on the video data supplied from the camera signal processing unit 23 to the viewfinder unit 27. It shall include an indicator composition unit for feeding.

  With reference to FIG. 14, as a third display example of the focus indicator, a case will be described in which an indicator indicating the parameter size for each vertical line is displayed as a line graph.

  In the case of FIG. 14, the indicator composition unit 74 synthesizes the focus indicator indicating the parameter for each vertical line on the video data as a line graph indicated by c in the figure. In this case, the closer the position of the focus indicator line is to the lower end of the screen, the smaller the parameter, that is, the vertical line with less harmonic components, and the mountain portion of the line graph is a vertical line with more high frequency components. It shows that it corresponds to the line.

  In the example of FIG. 14, the focus is on the woman in the center, and a mountain of focus indicators appears at the position of the vertical line where the image of the woman is shown.

  Also in the case of FIG. 14, the focus indicator expresses temporal high-frequency component transition of each vertical line in the displayed captured image.

  Therefore, the user can easily know the change in focus when the lens focus position is operated by checking the indicator while moving the lens focus position back and forth, and can focus on a desired subject. It becomes possible. Even when a moving image is captured for a subject moving in the screen, checking the focus indicator makes it easy to check the temporal spatial frequency change in the screen. Work such as focus feed that matches the movement of the camera becomes easy.

  Further, the display position of the line graph as the focus indicator is not limited to the lower side of the screen, and may be any position such as the upper side of the screen.

  Further, the focus indicator expressing the high-frequency component in the vertical direction may be represented not only by a line graph but also by a bar graph.

  Also, as a fourth display example of the focus indicator, as shown in FIG. 15, the focus indicator indicated by a in the figure showing the parameter size for each horizontal line and the parameter indicator for each vertical line. A focus indicator indicated by c in the drawing showing the size may be combined with the video data and displayed.

  In the case of FIG. 15, the indicator composition unit 74 uses a focus indicator indicating a parameter for each horizontal line as a line graph indicated by a in the figure, and a focus indicator indicating a parameter for each vertical line as a line graph indicated by c in the figure. Are respectively synthesized on the video data.

  The display examples of the focus indicator have been described above with reference to FIGS. 10 to 15. However, these do not limit the function of the present invention, and the focus indicator corresponds to the size of the spatial high-frequency component of the image. Needless to say, it may be displayed in any format as long as the position in the image space can be understood.

  Next, FIG. 16 is a block diagram showing a configuration of a focus assist processing unit 51-2 which is a second embodiment of the focus assist processing unit 51 of FIG. In FIG. 16, the same reference numerals are given to portions corresponding to those in FIG. 8, and description thereof will be omitted as appropriate. That is, the focus assist processing unit 51-2 in FIG. 16 is newly provided with a comparison unit 91 and a peak memory unit 92, and basically includes an indicator synthesis unit 93 instead of the indicator synthesis unit 74. This has the same configuration as the focus assist processing unit 51-1 in FIG.

  The comparison unit 91 compares the differential addition value of each horizontal line supplied from the addition unit 73 with the maximum value of the differential data of each past horizontal line stored in advance in the peak memory unit 92 for each line. . Then, the comparison unit 91 extracts, in each line, a large value of the past parameter maximum value and the parameter obtained in this frame, and the differential addition value of each horizontal line supplied from the addition unit 73. At the same time, it is supplied to the indicator composition unit 93.

  In addition, when the obtained parameter has a larger value than the past parameter, the comparison unit 91 overwrites the information in the peak memory unit 92 with the differential addition value which is a larger value, and is stored in the peak memory unit 92. The maximum value of differential data for each past horizontal line is updated.

  The peak memory unit 92 is a memory that holds the maximum value of the differential addition value of each horizontal line based on the comparison result by the comparison unit 91. Hereinafter, the maximum value of the differential data of each past horizontal line stored in the peak memory unit 92 is also referred to as a peak hold value.

  The indicator combining unit 93 combines the focus indicator similar to the above case and the line indicating the peak hold value with the video data, outputs the combined video data to the viewfinder unit 27, and controls the display thereof.

  FIG. 17 shows the viewfinder unit 27 when the indicator combining unit 93 generates a focus indicator indicating parameters for each horizontal line indicated by a in the figure and a line indicating the peak hold value indicated by d in the figure. Is a display example.

  The focus indicator indicating the parameter for each horizontal line indicated by “a” in the figure is the same focus indicator as described with reference to FIGS. 10 and 11. In the example of FIG. 17, the result of comparison processing of several lines is shown, but the comparison processing may be performed for each line. The peak hold display method may be any method as long as the maximum value in the past can be expressed.

  When there are many harmonic components in each horizontal line, the pixel of the horizontal line contains many edge elements. In other words, the user checks the indicator while moving the lens focus position back and forth so that the peak of the indicator matches the peak hold value in the line where the peak of the indicator seems to be high when the desired subject is in focus. By adjusting to, it becomes possible to easily focus on a desired subject.

  Further, when the object to be imaged is changed, the past maximum differential addition value is meaningless. Therefore, if the maximum value of the differential addition value of each horizontal line recorded in the peak memory unit 92 is not reset all the time, the meaning of the peak hold value is lost.

  Therefore, it is assumed that the maximum value of the differential addition value of each horizontal line recorded in the peak memory unit 92 is reset every predetermined time or in response to a user operation input. In other words, the maximum differential data until the next reset timing is held in the peak memory unit 92.

  In this way, by displaying the line corresponding to the peak hold value shown by d in FIG. 17 together with the focus indicator, when the user focuses on a desired subject while moving the focus position of the lens back and forth, the focus is adjusted. It is easy to determine whether or not the current focus position is the optimum value of the focus position with respect to a desired subject based on whether or not the indicator of the region to be matched matches or approaches the peak. According to this method, it is possible to reliably focus on a desired subject even under conditions where the depth of field is shallow and focusing is difficult.

  In the case described with reference to FIGS. 16 and 17, the description is limited to only the process in the horizontal direction, but the process in the vertical direction is performed in the same manner as described with reference to FIG. 14. A line corresponding to the vertical focus indicator and the peak hold value may be generated and displayed.

  Similarly to the case described with reference to FIG. 15, it is possible to perform processing in the horizontal direction and the vertical direction at the same time to generate and display a line corresponding to the focus indicator and the peak hold value two-dimensionally. Any configuration may be used.

  Next, FIG. 18 is a block diagram showing a configuration of a focus assist processing unit 51-3 which is a third embodiment of the focus assist processing unit 51 of FIG. In FIG. 18, parts corresponding to those in FIG. 8 are denoted by the same reference numerals, and description thereof is omitted as appropriate. That is, the focus assist processing unit 51-3 in FIG. 18 further includes a switching pulse generation unit 42 and an image synthesis unit 43 of the viewfinder display image conversion unit 26 in FIG. 2 in addition to the focus assist processing unit 51-1 in FIG. Except that the indicator composition unit 111 is provided in place of the indicator composition unit 74, the configuration is basically the same as that of the focus assist processing unit 51-1 in FIG.

  The input video data supplied from the camera signal processing unit 23 to the focus assist processing unit 51-3 is supplied to the luminance signal generation unit 71 and also to the image synthesis unit 43.

  The luminance signal generation unit 71 performs conversion processing as shown in the above-described formula (1) on the input video data from the camera signal processing unit 23 to generate the luminance signal Y, and the image synthesis unit 43 and the horizontal The differential filter processing unit 72 is supplied.

  Based on the pulse signal ME supplied from the switching pulse generation unit 42, the image synthesis unit 43 converts the range of the rectangular area A in the video data for the captured image displayed on the viewfinder unit 27 into the luminance signal generation unit. 71 is converted into a luminance signal Y from 71 and supplied to the indicator composition unit 111.

  The indicator composition unit 111 synthesizes the focus indicator similar to that described above with the video data from the image composition unit 43 in which the range of the rectangular area A in the video data is converted into the luminance signal Y, and the view finder. This is supplied to the unit 27 and its display is controlled.

  Next, a fourth display example of the focus indicator will be described with reference to FIG.

  FIG. 19 shows a composite image obtained by converting the range of the rectangular area A in the video data of the captured image displayed on the viewfinder 27 by the image composition unit 43 into the luminance signal Y. It is a display example of the viewfinder unit 27 when a focus indicator indicating a parameter for each horizontal line indicated by a in the figure is synthesized.

  In the figure, a rectangular area A is the same focusing area as described with reference to FIG. Further, the focus indicator indicating the parameter for each horizontal line indicated by a in the figure is the same focus indicator as described with reference to FIGS. 10 and 11.

  In the example of FIG. 19, the addition unit 73 performs addition processing only on pixels in the focusing area that is the rectangular area A.

  As a result, the focus indicator is not displayed for a horizontal line that does not intersect with the rectangular area A, and the display of the focus indicator is affected only by the spatial frequency in the focusing area. In complex scenes where objects are present, a focus indicator can be displayed to assist in achieving more accurate focus adjustment, and the target to be focused is displayed as a luminance signal (monochrome display) Therefore, it becomes easy for the user to determine the contrast, and further focusing becomes easier.

  In the case described with reference to FIGS. 18 and 19, the description is limited to the processing in only the horizontal direction. However, as in the case described with reference to FIG. 15, the processing in the horizontal direction and the vertical direction is performed. The focus indicator may be generated two-dimensionally and displayed.

  Further, similarly to the case described with reference to FIG. 17, a line corresponding to the horizontal focus indicator and the peak hold value may be generated and displayed.

  Furthermore, it may be configured such that lines corresponding to the focus indicator and the peak hold value can be generated and displayed two-dimensionally after performing the processing in the horizontal direction and the vertical direction simultaneously.

  As described above, in FIG. 7, the configuration of the imaging device 11 that includes the imaging element 22 and the camera signal processing unit 23 and that can control the focus position based on the user's operation input will be described. The processing in the imaging device 11 has been described. For example, one device having a function corresponding to the focus assist processing unit 51 and the viewfinder unit 27 described with reference to FIG. 8, FIG. 16, or FIG. You may comprise as.

  Specifically, based on a user's operation input, a captured image captured by an imaging device whose focus position can be controlled is acquired from an external device, and a focus indicator or the like as described above is created and the captured image is It is also possible to constitute a display device having a function that can be displayed on the viewfinder by performing a composition process.

  Furthermore, a device having a function corresponding to the focus assist processing unit 51 described with reference to FIG. 8, FIG. 16, or FIG. 18 may be configured as one device.

  Specifically, based on a user's operation input, a captured image captured by an imaging device whose focus position can be controlled is acquired from an external device, and a focus indicator or the like as described above is created and the captured image is It is also possible to configure a display control device having a function of performing a synthesis process and outputting and displaying it on another external display device.

  Next, imaging processing executed by the imaging device 11 of FIG. 7 will be described with reference to the flowchart of FIG. In addition, although each process of step S32 thru | or step S38 can be performed in parallel with the process of step S34 and step S35, and the process of step S36 and step S37 as a process of the frame with the imaged moving image. Basically, they are executed sequentially according to the described steps. However, these are processes executed on the entire moving image composed of a plurality of frames that are sequentially picked up, and therefore are executed in parallel in the image pickup apparatus 11.

  In steps S31 to S35, processing similar to that in steps S11 to S15 in FIG. 5 is executed. That is, the imaging optical system 21 and the imaging element 22 in the imaging apparatus 11 of FIG. 7 start imaging, and the camera signal processing unit 23 performs white balance correction and gain adjustment on the RAW data supplied from the imaging element 22. , Demosaic processing, linear matrix processing, and gamma correction processing are performed, the codec unit 24 performs compression processing, and the recording unit 25 records the encoded stream.

  In step S36, the focus assist processing unit 51 executes the focus indicator generation process described with reference to FIG. 21, FIG. 22, or FIG. 23, and the process proceeds to step S37.

  In step S <b> 37, the viewfinder unit 27, for example, a focus indicator that is auxiliary information for making it easier for the user to grasp the in-focus state as described with reference to FIGS. 10 to 15, 17, or 19. Alternatively, the captured image to which the focus indicator and the line indicating the peak hold value are added is displayed, and the process proceeds to step S38.

  In step S38, the same process as in step S18 of FIG. 5 is executed. That is, the operation input unit 28 determines whether or not the end of imaging has been commanded based on the operation input from the user. If it is determined that the end of imaging has not been commanded, the process proceeds to step S32. Return, and the subsequent processing is repeated. If it is determined that the end of imaging has been commanded, the process ends.

  Through such processing, a moving image is captured and recorded, and a focus indicator (or a line indicating the focus indicator and peak hold value) is displayed together with the recorded moving image.

  Therefore, the user can easily focus on a desired object by referring to the focus indicator, for example, by operating the lens ring. Needless to say, the lens may be moved electronically so that the focus can be automatically corrected.

  Next, a focus indicator generation process 1 that is a first example of the process executed in step S36 of FIG. 20 will be described with reference to the flowchart of FIG. The focus indicator generation process 1 is executed by the focus assist processing unit 51-1.

  In step S <b> 41, the luminance signal generation unit 71 performs, for example, the conversion process shown in the above-described formula (1) on the input video data supplied from the camera signal processing unit 23 to generate the luminance signal Y. To the horizontal differential filter processing unit 72. Thereafter, the process proceeds to step S42.

  In step S42, the horizontal differential filter processing unit 72 applies a horizontal differential filter having a coefficient as shown in FIG. 9 to the luminance signal Y from the luminance signal generation unit 71 to obtain differential data, etc. The high-frequency component in the horizontal direction of the signal is extracted and supplied to the adding unit 73, and the process proceeds to step S43.

  In step S43, the addition unit 73 adds a differential data for one horizontal line of the differential data supplied from the horizontal differential filter processing unit 72, thereby representing a parameter representing the magnitude of the high-frequency component for one horizontal line. Is supplied to the indicator composition unit 74, and the process proceeds to step S44.

  In step S44, the indicator combining unit 74 generates an indicator indicating the size of the parameter for each line based on the parameter indicating the size of the high-frequency component for each horizontal line supplied from the adding unit 73, and generates a camera signal. The video data is synthesized on the video data supplied from the processing unit 23. The indicator combining unit 74 supplies the video data combined with the indicator to the viewfinder unit 27 and controls the display thereof, and the process returns to step S36 in FIG. 20 and proceeds to step S37.

  Through such processing, for example, a focus indicator as described with reference to FIGS. 10 to 12 is synthesized on the video data supplied from the camera signal processing unit 23, and the focus indicator is added to the viewfinder unit 27. The captured image can be displayed.

  Next, with reference to the flowchart of FIG. 22, the focus indicator generation process 2 which is the 2nd example of the process performed in FIG.20 S36 is demonstrated. The focus indicator generation process 2 is executed by the focus assist processing unit 51-2.

  In steps S71 to S73, the same processes as in steps S41 to S43 in FIG. 21 are executed. That is, the luminance signal generation unit 71 generates a luminance signal, the horizontal differential filter processing unit 72 extracts a high-frequency component in the horizontal direction of the luminance signal, and the addition unit 73 determines the magnitude of the high-frequency component for one horizontal line. Find the parameters to represent.

  In step S74, the comparison unit 91 receives the differential addition value of each horizontal line from the addition unit 73, and calculates the past parameter maximum value of each past horizontal line stored in the peak memory unit 92 in advance. After reading, the process proceeds to step S75.

  In step S <b> 75, the comparison unit 91 determines whether or not the parameter obtained by the addition unit 73 has a value larger than a past parameter stored in advance in the peak memory unit 92.

  If it is determined in step S75 that there is a value larger than the past parameter, the process proceeds to step S76, and the comparison unit 91 overwrites the information in the peak memory unit 92 with the differential addition value that was a large value. The maximum value of the differential data of each past horizontal line recorded in the peak memory unit 92 is updated.

  If it is determined in step S75 that there is no value larger than the past parameter, or after the process of step S76 is completed, the process proceeds to step S77, and the indicator composition unit 93 determines the differential addition value of each horizontal line. And generating a focus indicator indicating the size of the parameter for each line, and setting the peak hold value based on the maximum differential data of each past horizontal line recorded in the peak memory unit 92. Generate the line shown. Then, the indicator composition unit 93 synthesizes the line indicating the focus indicator and the peak hold value on the video data supplied from the camera signal processing unit 23, supplies the synthesized video data to the viewfinder unit 27, and controls the display thereof. The processing returns to step S36 in FIG. 20 and proceeds to step S37.

  Through such processing, for example, a line indicating a focus indicator and a peak hold value as described with reference to FIG. 17 is synthesized on the video data supplied from the camera signal processing unit 23, and the viewfinder unit 27 A captured image to which a line indicating a focus indicator and a peak hold value is added can be displayed.

  As described above, the maximum value of the parameter recorded in the peak memory unit 92 may be reset when a certain time has elapsed, or may be reset by a user operation input.

  Next, a focus indicator generation process 3 that is a third example of the process executed in step S36 of FIG. 20 will be described with reference to the flowchart of FIG. The focus indicator generation process 3 is executed by the focus assist processing unit 51-3.

  In step S101, processing similar to the luminance signal conversion processing described in FIG. 6 is executed. That is, the luminance signal generation unit 71 generates the luminance signal Y, the switching pulse generation unit 42 starts generating the pulse signal ME, and the image synthesis unit 43 sends the signal to the viewfinder unit 27 according to the pulse signal ME. By converting the range of the rectangular area A in the video data of the captured image to be displayed into the luminance signal Y from the luminance signal generation unit 71, a composite image of the video data and the luminance signal Y is generated.

  In step S102 and step S103, processing similar to that in step S42 and step S43 in FIG. 21 is executed. That is, the horizontal differential filter processing unit 72 extracts a high-frequency component in the horizontal direction of the luminance signal, and the adding unit 73 obtains a parameter representing the size of the high-frequency component for one horizontal line intersecting the rectangular area A.

  In step S104, the indicator composition unit 111 generates an indicator indicating the size of the parameter for each line, based on the parameter indicating the size of the high-frequency component for each horizontal line intersecting the rectangular area A from the addition unit 73. Then, the image is synthesized on the synthesized image supplied from the image synthesizing unit 43. The indicator compositing unit 111 supplies the composite image with the combined indicator to the viewfinder unit 27 and controls the display thereof, and the process returns to step S36 in FIG. 20 and proceeds to step S37.

  By such processing, the range of the rectangular area A in the video data of the captured image displayed on the viewfinder 27 in the video data supplied from the camera signal processing unit 23 is converted into the luminance signal Y. For example, since the focus indicator as described with reference to FIG. 19 is synthesized on the synthesized image generated in this manner, the area where the subject to be focused is displayed is displayed in monochrome on the viewfinder unit 27. A composite image to which a focus indicator is added can be displayed.

  In the processing described with reference to the flowcharts of FIGS. 20 to 23, a case is described in which a parameter indicating the magnitude of the harmonic component is obtained for each horizontal line, and an indicator is generated and displayed. In addition, as described above, a parameter indicating the magnitude of the harmonic component may be obtained for each of a plurality of lines, and an indicator corresponding thereto may be displayed.

  Further, as described with reference to FIGS. 14 and 15, a parameter indicating the magnitude of the harmonic component in the vertical direction instead of the horizontal direction may be obtained, and an indicator corresponding to the parameter may be displayed.

  As described above, the imaging device 11 captures an image, generates the luminance signal Y of the captured image, and among the display areas of the captured image displayed on the viewfinder unit 27, the rectangular area A in the video data. Is converted into the luminance signal Y, and the composite image in which the range of the rectangular area A in the video data is converted into the luminance signal Y is displayed on the viewfinder unit 27. In the viewfinder unit 27, The rectangular area A on which the subject to be focused is displayed is displayed with the luminance signal Y (monochrome display), and the other areas are displayed in color. Especially for the subject to be focused on, according to the human visual characteristics It becomes easier to recognize the difference between light and dark, and the user can easily perform focusing while checking the color.

  The series of processes described above can be executed by hardware or can be executed by software. The software is a computer in which the program constituting the software is incorporated in dedicated hardware, or various functions can be executed by installing various programs, for example, a general-purpose personal computer For example, it is installed from a recording medium. In this case, the processing described above is executed by a personal computer 500 as shown in FIG.

  In FIG. 24, a CPU (Central Processing Unit) 501 performs various processes according to a program stored in a ROM (Read Only Memory) 502 or a program loaded from a storage unit 508 to a RAM (Random Access Memory) 503. Execute. The RAM 503 also appropriately stores data necessary for the CPU 501 to execute various processes.

  The CPU 501, ROM 502, and RAM 503 are connected to each other via an internal bus 504. An input / output interface 505 is also connected to the internal bus 504.

  The input / output interface 505 includes an input unit 506 including a keyboard and a mouse, a display including CRT and LCD, an output unit 507 including a speaker, a storage unit 508 including a hard disk, a modem, a terminal adapter, and the like. A communicator 509 is connected. A communication unit 509 performs communication processing via various networks including a telephone line and CATV.

  A drive 510 is also connected to the input / output interface 505 as necessary, and a removable medium 521 composed of a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, or the like is appropriately mounted, and a computer program read therefrom is It is installed in the storage unit 508 as necessary.

  When a series of processing is executed by software, a program constituting the software is installed from a network or a recording medium.

  As shown in FIG. 24, this recording medium is not only composed of a package medium consisting of a removable medium 521 on which a program is recorded, which is distributed to provide a program to the user, separately from the computer. These are configured by a hard disk including a ROM 502 storing a program and a storage unit 508 provided to the user in a state of being pre-installed in the apparatus main body.

  In the present specification, the step of describing the program recorded on the recording medium is not limited to the processing performed in chronological order according to the described order, but is not necessarily performed in chronological order. It also includes processes that are executed individually.

  Further, in this specification, the system represents the entire apparatus constituted by a plurality of apparatuses.

  The embodiment of the present invention is not limited to the above-described embodiment, and various modifications can be made without departing from the gist of the present invention.

It is a block diagram which shows the structure of one Embodiment of the imaging device to which this invention is applied. It is a block diagram which shows the structural example of the finder display image conversion part of FIG. It is a timing chart explaining the process of a switching pulse generation part. It is a figure explaining a focusing area. 3 is a flowchart for describing imaging processing by the imaging apparatus of FIG. 1. It is a flowchart explaining a luminance signal conversion process. It is a block diagram which shows the other structure of an imaging device. It is a block diagram which shows the 1st structural example of the focus assist process part of FIG. It is a figure explaining a horizontal differential filter. It is a figure explaining a focus indicator. It is a figure explaining a focus indicator. It is a figure explaining the other example of a focus indicator. It is a figure explaining a focusing area. It is a figure explaining the other example of a focus indicator. It is a figure explaining the other example of a focus indicator. It is a block diagram which shows the 2nd structural example of the focus assistance process part of FIG. It is a figure explaining the line which shows a peak hold value. It is a block diagram which shows the 3rd structural example of the focus assist process part of FIG. It is a figure explaining a focusing area and a focus indicator. It is a flowchart explaining the imaging process by the imaging device of FIG. It is a flowchart explaining the focus indicator production | generation process 1. FIG. It is a flowchart explaining the focus indicator production | generation process 2. FIG. It is a flowchart explaining the focus indicator production | generation process 3. FIG. It is a block diagram which shows the structure of a personal computer.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 11 Imaging device, 21 Imaging optical system, 22 Imaging device, 23 Camera signal processing part, 24 Codec part, 25 Recording part, 26 Finder display image conversion part, 27 View finder part, 28 Operation input part, 41 Luminance signal generation part, 42 switching pulse generation unit, 43 image synthesis unit, 51 focus assist processing unit, 71 luminance signal generation unit, 72 horizontal differential filter processing unit, 73 addition unit, 74 indicator synthesis unit, 91 comparison unit, 92 peak memory unit, 93 indicator Combining section, 111 indicator combining section

Claims (16)

In an imaging device that displays a captured image,
Imaging means for capturing the image;
Luminance signal generation means for generating a luminance signal of the image captured by the imaging means;
Image synthesizing means for converting image data for a predetermined area of the display area of the image into the luminance signal and generating a synthesized image of the image data and the luminance signal;
An imaging apparatus comprising: display means for displaying the composite image. Signal generation means for generating a signal indicating a timing at which the predetermined area of the display area of the image is displayed on the display means;
The imaging apparatus according to claim 1, wherein the image synthesizing unit generates the synthesized image by converting the image data of the predetermined region into the luminance signal according to the signal. Extraction means for extracting at least one of the horizontal direction and the vertical direction in the image picked up by the image pickup means;
Based on at least one of the high-frequency components in the horizontal direction or the vertical direction extracted by the extraction unit, 1 or 2 in the predetermined area of the display area of the image in at least one of the horizontal direction and the vertical direction Calculating means for calculating a parameter representing the magnitude of the high-frequency component for each of the plurality of lines;
Based on the parameter, an indicator that indicates the size of the parameter is generated, and an indicator compositing unit that combines the composite image and the indicator in correspondence with at least one position in the horizontal direction or the vertical direction. The imaging apparatus according to claim 1 further provided. The imaging apparatus according to claim 3, wherein the indicator composition unit generates the indicator as a line graph. The imaging apparatus according to claim 3, wherein the extraction unit extracts at least one high-frequency component in a horizontal direction or a vertical direction using the luminance signal generated by the luminance signal generation unit. The imaging apparatus according to claim 1, further comprising setting means for setting the predetermined area. In an imaging method of an imaging apparatus that displays a captured image,
Taking the image,
Generating a luminance signal of the image imaged by the imaging means;
Converting image data of a predetermined area of the display area of the image into the luminance signal, and generating a composite image of the image data and the luminance signal;
An imaging method including a step of displaying the composite image. In a program that causes a computer to function as an imaging device that displays a captured image,
Imaging means for capturing the image;
Luminance signal generation means for generating a luminance signal of the image captured by the imaging means;
Composite image generation means for converting image data of a predetermined area of the display area of the image into the luminance signal and generating a composite image of the image data and the luminance signal;
A program that causes a computer to function as display means for displaying the composite image.   A recording medium on which the program according to claim 8 is recorded. In a display control device that controls display of the image of a display device that displays an image captured by the imaging device,
Luminance signal generation means for acquiring the image and generating a luminance signal of the image;
Image synthesizing means for converting image data for a predetermined area of the display area of the image into the luminance signal and generating a synthesized image of the image data and the luminance signal;
A display control unit that controls display of the composite image. Signal generating means for generating a signal indicating a timing at which the predetermined area of the display area of the image is displayed on the display device;
The display control apparatus according to claim 10, wherein the image synthesis unit converts the image data for the predetermined region into the luminance signal in accordance with the signal, and generates the synthesized image. Extracting means for extracting at least one of the horizontal direction and the vertical direction in the image captured by the imaging device;
Based on at least one of the high-frequency components in the horizontal direction or the vertical direction extracted by the extraction unit, 1 or 2 in the predetermined area of the display area of the image in at least one of the horizontal direction and the vertical direction Calculating means for calculating a parameter representing the magnitude of the high-frequency component for each of the plurality of lines;
Based on the parameter, an indicator that indicates the size of the parameter is generated, and an indicator compositing unit that combines the composite image and the indicator in correspondence with at least one position in the horizontal direction or the vertical direction. The display control device according to claim 10. The display control apparatus according to claim 10, further comprising setting means for setting the predetermined area. In a display control method of a display control device that controls display of the image of a display device that displays an image captured by an imaging device,
Acquiring the image, generating a luminance signal of the image,
Converting image data of a predetermined area of the display area of the image into the luminance signal, and generating a composite image of the image data and the luminance signal;
A display control method including a step of controlling display of the composite image. In a program that causes a computer to function as a display control device that controls display of the image of a display device that displays an image captured by an imaging device,
Luminance signal generation means for acquiring the image and generating a luminance signal of the image;
Composite image generation means for converting image data of a predetermined area of the display area of the image into the luminance signal and generating a composite image of the image data and the luminance signal;
A program that causes a computer to function as display control means for controlling display of the composite image.   A recording medium on which the program according to claim 15 is recorded.

Images