JP2003309763A - Animation imaging system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an animation imaging system which generates a high- definition image with respect to an optional photographic scene by properly switching a gradation conversion curve for converting the gradation width of the image obtained in time series at a low cost corresponding to scene change. <P>SOLUTION: A detection part 115 detects the presence and absence of scene change from a group of photographed images. When the scene change is detected, a calculation part 115 calculates the gradation conversion curve, and a recording part 117 records the calculated gradation conversion curve. A conversion curve composing part 118 composes the new gradation conversion curve calculated by the calculation part 115 and a past gradation conversion curve recorded by the recording part 117. A gradation conversion part 112 converts the gradation conversion characteristic of each image by using the composed gradation conversion curves. Thus, when switching a gradation curve corresponding to the scene change, unnatural feeling is reduced and the high-definition image can be obtained. <P>COPYRIGHT: (C)2004,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a moving image pickup system in which the grayscale width of an image pickup system for obtaining images in time series is wider than the grayscale width of an output system, and it is suitable to switch the grayscale conversion curve due to a scene change. The present invention relates to a moving image capturing system that obtains a high quality output image by performing the above.

[0002]

2. Description of the Related Art In a current digital video camera, in order to prevent image deterioration due to digit cancellation in digital signal processing, the gradation width (normally 8 bits) of the final output image is compared with the image in the input and processing systems. The gradation width is set wider (about 10 to 12 bits). In this case, it is necessary to perform gradation conversion so as to match the gradation width of the output system.

Conventionally, conversion is performed by a fixed gamma curve adapted to a standard scene or an applicable gradation conversion curve based on a histogram or the like.

Further, there has been proposed a method of synthesizing a plurality of images having different exposure amounts to generate a wide dynamic range image having a wider gradation range. Also in this case, it is necessary to perform gradation conversion on the obtained wide dynamic range image so as to match the gradation width of the output system.

Since such gradation conversion characteristics depend on the scene to be photographed, it is necessary to switch appropriately based on the change in the scene. However, if the gradation conversion characteristics are switched immediately after a scene change, the obtained image changes greatly and a feeling of strangeness occurs.
For this reason, for example, Japanese Patent Application Laid-Open No. 11-164190 discloses a method of gradually changing the gradation conversion characteristics by multiplying a tracking speed, which is an interpolation coefficient based on time. Further, Japanese Patent Application Laid-Open No. 2000-307896 discloses an example in which a plurality of gradation conversion characteristics are prepared and hysteresis control is performed when selecting the gradation conversion characteristics to mitigate a sudden change.

[0006]

However, in the method of gradually switching the gradation conversion characteristics by multiplying the interpolation coefficient, the entire gradation conversion curve is multiplied by the interpolation coefficient when the gradation conversion characteristics are switched. . Therefore, the number of data (1024 to 4096 points) for the gradation width (10 to 12 bits) of the image
However, there is a problem that the hardware scale becomes large because it is necessary to correct the above in real time. That is, it is not possible to cope with the change of the gradation conversion characteristic which is comfortable at a low cost.

Further, in the method of preparing a plurality of gradation conversion characteristics and performing the hysteresis control at the time of selection, there is no guarantee that the prepared gradation conversion characteristics match the current scene. That is, it is not possible to deal with obtaining a high-quality image for an arbitrary shooting scene.

Focusing on the above problems, the present invention can appropriately switch a gradation conversion curve for converting gradation widths of images obtained in time series at low cost according to a scene change, and arbitrary photographing can be performed. An object of the present invention is to provide a moving image capturing system capable of generating a high quality image for a scene.

[0009]

According to a first aspect of the present invention, an image group of Mbit gradation widths continuous in time series from an imaging system is converted into an Nbit gradation width of an output system (M and N are natural numbers M ≧ N). In the moving image pickup system that outputs the same, the detecting unit that detects a scene change from the image group, the calculating unit that calculates the gradation conversion curve from the image in which the scene change is detected by the detecting unit, and the gradation conversion. The recording means for recording the curve, the conversion curve synthesizing means for synthesizing the new gradation conversion curve calculated by the calculating means and the past gradation conversion curve recorded in the recording means, and the synthesizing means for synthesizing Gradation conversion means for converting the image group having the Mbit gradation width into the Nbit gradation width by using the gradation conversion curve. The detecting unit is the detecting unit 115 shown in FIG. 1, and the calculating unit is the calculating unit 116.
However, the recording unit corresponds to the recording unit 117, the conversion curve combining unit corresponds to the conversion curve combining unit 118, and the gradation converting unit corresponds to the gradation converting unit 112.

A preferred application example of the first invention is shown in FIG.
The presence or absence of a scene change is detected from the image group photographed by the detection unit 115 shown in FIG. 1, and the gradation conversion curve is calculated by the calculation unit 115 when the scene change is detected, and the calculated gradation conversion curve is calculated. Is recorded in the recording unit 117, the new gradation conversion curve calculated in the calculation unit 115 and the past gradation conversion curve recorded in the recording unit 117 are combined in the conversion curve combining unit 118, and combined. This is a moving image capturing system in which the gradation conversion unit 112 converts the gradation conversion characteristics of each image by using the gradation conversion curve.

According to the first aspect of the invention, when a scene change is detected, a new gradation conversion curve is calculated, and the gradation conversion characteristic is combined with the past gradation conversion curve to use the gradation conversion characteristic. Conversion of. As a result, when switching the gradation conversion curve in response to a scene change, a sense of discomfort is reduced and a high-quality image is obtained.

According to a second invention, in the moving image pickup system of the first invention, further, at least two or more frame- or field-unit images obtained by photographing the same subject under different exposure conditions are combined to obtain an Mbit gradation width. Image synthesizing means for generating a group of images. The image synthesizing means corresponds to the synthesizing unit 110 shown in FIG.

A preferred application example of the second invention is shown in FIG.
Images captured at different exposure amounts by the CCD 103 shown in are stored in the image buffer (for long-exposure image) 105 and the image buffer (for short-exposure image) 106, and these images are combined by the combining unit 110. An image having a wide dynamic range is obtained by synthesizing, a gradation conversion curve is calculated by the calculation unit 116 when a scene change is detected by the detection unit 115, and the calculated gradation conversion curve is recorded by the recording unit 117. The new gradation conversion curve calculated by the calculation unit 116 and the past gradation conversion curve recorded by the recording unit 117 are combined by the conversion curve combining unit 118, and the combined gradation Tone conversion section using the tone conversion curve
110 is an imaging system that converts the gradation conversion characteristics of each image.

In the second aspect of the invention as described above, images of different exposures are combined to generate one image with a wide dynamic range, and when a scene change is detected for this image, new gradation conversion is performed. A curve is calculated, and the gradation conversion characteristic is converted using the gradation conversion curve combined with the past gradation conversion curve.
As a result, since gradation conversion is performed from an image having a gradation width that can be handled by the image pickup system, a higher-quality image with less black crushing and whiteout can be obtained. Further, when switching the gradation conversion curve in response to a scene change, a feeling of strangeness is reduced and a high-quality image can be obtained.

According to a third aspect of the present invention, in the moving image pickup system according to the first or second aspect, the detecting means includes the number of images taken per unit time from the image pickup system, the image size,
Time setting means for setting a time interval for selecting an image from the image group based on at least one of an exposure condition, a focusing condition, a white balance condition, a zoom position, and a camera position. The time setting means corresponds to the time setting unit 200 shown in FIG.

A preferred application example of the third invention is shown in FIG.
The photometric evaluation unit 107, the focus detection unit 108, the external I /
The shooting condition is estimated based on the information from the F unit 121, and
2 is a moving image capturing system in which a time interval for selecting an image is adaptively adjusted by the time setting unit 200 in the detection unit 115 shown in FIG.

In the third aspect, the time interval for selecting an image is automatically changed according to the photographing condition. As a result, the image is selected at a timing according to the shooting situation, and the detectability of the scene change is improved.

In a fourth aspect of the present invention, the detecting means in the moving image capturing system of the third aspect further comprises a reducing means for reducing the selected image to a predetermined size, and an average brightness level is calculated from the reduced image. Brightness calculating means and judging means for judging the presence or absence of a scene change based on the time series change of the average brightness level. The reducing means corresponds to the reducing section 202 shown in FIG. 2, the luminance calculating means corresponds to the luminance calculating section 204 shown in FIG. 2, and the determining means corresponds to the determining section 206 shown in FIG.

A preferred application example of the fourth invention is shown in FIG.
The image read by the image reading unit 201 from the signal processing unit 111 shown in FIG. 2 is reduced to a predetermined size by the reduction unit 202 in the detection unit 115 shown in FIG. This is an imaging system that detects a scene change by calculating a brightness value and determining a change in the brightness value in the determination unit 206.

In the fourth aspect of the invention as described above, the selected image is reduced to a predetermined size, and the scene change is detected from the time-series change in the luminance value of the reduced image. As a result, the subsequent processing is reduced because the selected image is reduced, and a low-cost system can be realized. Further, since the minute change of the image is absorbed at the reduction stage, the characteristic of the calculated gradation conversion curve becomes stable and the obtained image becomes easy to see. Furthermore, since the scene change can be detected from the entire image instead of the specific region, the scene change can be detected regardless of the position of the subject.

In a fifth aspect of the present invention, the detecting means in the moving image capturing system of the third aspect further comprises a reducing means for reducing the selected image to a predetermined size, and a time series of the reduced image. It has a motion vector calculating means for calculating a motion vector from the two images and a judging means for judging the presence or absence of a scene change based on the motion vector amount. The reduction means corresponds to the reduction section 202 shown in FIG. 3, the motion vector calculation means corresponds to the motion vector calculation section 210 shown in FIG. 3, and the determination means corresponds to the determination section 206 shown in FIG.

A preferred application example of the fifth invention is shown in FIG.
The image read by the image reading unit 201 from the signal processing unit 111 shown in FIG. 3 is reduced to a predetermined size by the reduction unit 202 in the detection unit 115 shown in FIG.
Is a moving image capturing system that detects a scene change and a main subject position by calculating a motion vector in the image and determining a change in the motion vector in the determination unit 206.

In the fifth aspect, the selected image is reduced to a predetermined size, the scene change is detected from the time-series change of the motion vector of the reduced image, and the main change is based on the magnitude of the motion. Find the subject position. As a result, the subsequent processing is reduced because the selected image is reduced, and a low-cost system can be realized. In addition, it is possible to perform gradation conversion with emphasis on the main subject, and a high-quality image can be obtained.

According to a sixth aspect of the invention, in the moving image pickup system of the first or second aspect of the invention, the calculating means includes a separating means for separating the image into a luminance signal and a color difference signal, and an appropriate exposure area based on the luminance signal level. An extracting means for extracting a characteristic amount, and a characteristic amount calculating means for calculating a characteristic amount with respect to the proper exposure area,
It has a histogram creating means for creating a histogram on the basis of the characteristic amount, and a gradation conversion curve calculating means for calculating a gradation conversion curve on the basis of the histogram. The separating means is the brightness separating unit 300 shown in FIG.
However, the proper exposure extraction unit 301 shown in FIG.
The feature amount calculating means is the edge extracting unit 302 shown in FIG.
The histogram creation unit corresponds to the histogram creation unit 303 shown in FIG. 4, and the gradation conversion curve calculation unit corresponds to the gradation conversion curve calculation unit 304 shown in FIG.

A preferred application example of the sixth invention is shown in FIG.
For the image transferred from the detection unit 115 shown in FIG. 4, the brightness separation unit 300 in the calculation unit 116 shown in FIG. 4 separates the brightness signal from the image, and the proper exposure extraction unit 301 uses the brightness signal based on the brightness signal. The proper exposure area is extracted, the edge component of the luminance signal is extracted by the edge extraction unit 302, the histogram of the edge portion is calculated by the histogram creation unit 303, and the gradation is calculated from the cumulative histogram by the gradation conversion curve calculation unit 304. It is a moving image capturing system that obtains a conversion curve.

In the sixth aspect, the edge component is extracted from the luminance signal, and the gradation conversion curve is obtained from the cumulative histogram of the edge portion. As a result, a high quality image can be obtained by removing the flat background portion and calculating the gradation conversion curve with emphasis on the main subject.

In a seventh aspect of the present invention, the conversion curve synthesizing means in the moving image pickup system of the first or second aspect of the invention records the new gradation conversion curve calculated by the calculating means and the past in the recording means. Setting means for setting a plurality of representative points on the gradation conversion curve, and a composite value is calculated from the new gradation conversion curve and the past gradation conversion curve based on a predetermined weighting coefficient for the representative point. It has a multiplication and addition means and an interpolation means for interpolating between the representative points processed by the multiplication and addition means. The setting means corresponds to the representative point extraction unit 402 shown in FIG. 5, the multiplication and addition means corresponds to the multiplication and addition unit 405 shown in FIG. 5, and the interpolation means corresponds to the interpolation unit 408 shown in FIG.

A preferred application example of the seventh invention is shown in FIG.
For the two gradation conversion curves fetched by the gradation conversion curve buffers 400 and 401 from the calculation unit 116 and recording unit 117 shown in FIG. At 402, the value at a predetermined representative point is extracted,
This is a moving image pickup system in which the multiplication / addition unit 405 synthesizes the representative points of the two gradation conversion curves and the interpolation unit 408 interpolates between the representative points to generate the gradation conversion curve.

In the seventh invention, several representative points are extracted from the new gradation conversion curve calculated after the scene change and the past gradation conversion curve used before the scene change,
A tone conversion curve that combines the two is generated. This allows
The old and new gradation conversion curves can be combined with a small amount of calculation, and high-speed and low-cost processing can be realized.

According to an eighth aspect of the present invention, the conversion curve synthesizing means in the moving image pickup system according to the seventh aspect of the present invention further uses the new gradation conversion curve from the past gradation conversion curve based on the photographing condition from the imaging system. And a weight control means for controlling the weight coefficient based on the transition time of the transition time setting means. The transition time setting unit corresponds to the transition time setting unit 406 shown in FIG. 5, and the weight control unit corresponds to the weight control unit 407 shown in FIG.

A preferred application example of the eighth invention is shown in FIG.
On the basis of the shooting condition transferred from the control unit 120 shown in FIG. 5, the transition time setting unit 406 in the conversion curve synthesizing unit 118 shown in FIG. The moving image capturing system adjusts the weighting coefficient for the old and new gradation conversion curves in the unit 407 based on the transition time.

In the eighth invention, the transition time for switching the old and new gradation conversion curves is calculated based on the photographing conditions,
The weighting coefficient is adjusted so that the gradation conversion curve in the past is switched to the new gradation conversion curve within this time. As a result, the gradation conversion curve is switched at a transition time suitable for the shooting conditions, and thus switching can be performed with less discomfort.

In a ninth aspect of the present invention, the conversion curve synthesizing means in the moving image pickup system of the seventh aspect further comprises any one of the gradation conversion curve interpolated by the interpolation means and the novel gradation conversion curve. And a switching means for selecting. The switching means corresponds to the switching unit 409 shown in FIG.

A preferred application example of the ninth invention is shown in FIG.
The switching unit 409 switches the gradation conversion curve synthesized by the conversion curve combining unit 118 from the two gradation conversion curves from the calculation unit 116 and the recording unit 117 or the new gradation conversion curve transferred from the calculation unit 116. It is a moving image capturing system that is switched by.

According to the ninth aspect, the gradation conversion curve is switched according to a scene change, and the combined gradation conversion curve is used within a predetermined transition time, and then the gradation conversion curve is switched to a new gradation conversion curve. It is possible. As a result, regarding switching of the gradation conversion curve due to a scene change, after a predetermined transition, the processing unit regarding composition can be bypassed, and low power consumption can be achieved.

In a tenth aspect, the conversion curve synthesizing means in the moving image pickup system according to the seventh aspect further calculates a difference value between the new gradation conversion curve and the past gradation conversion curve with respect to the representative point. Difference calculating means and adjusting means for adjusting the position or the number of representative points based on the difference value. The difference calculating means corresponds to the difference calculating section 403 shown in FIG. 5, and the adjusting means corresponds to the representative point adjusting section 404 shown in FIG.

A preferred application example of the tenth aspect of the invention is that the difference calculating unit 403 sets the difference between the two gradation conversion curves from the calculating unit 116 and the recording unit 117 shown in FIG. This is a moving image capturing system in which a value is calculated and the representative point adjusting unit 404 adjusts the representative point based on the difference value.

In the tenth aspect of the invention, based on the difference value between the old and new tone conversion curves, the representative point is roughly adjusted if the difference value is large and the representative point is densely small. As a result, since the representative points are densely arranged in the gradation region where the difference between the old and new gradation conversion curves is large, a more accurate gradation conversion curve can be synthesized and a high-quality image can be obtained.

An eleventh aspect of the present invention is the moving image pickup system according to the first or second aspect, further comprising a standard gradation conversion curve recording means for recording a standard gradation conversion curve, and an initialization status when the power is turned on. It has initialization detection means for detecting, and transfer means for transferring the standard gradation conversion curve to the gradation conversion means based on the initialization detection means. The standard gradation curve recording means corresponds to the standard gradation curve ROM 119 shown in FIG. 1, the initialization detection means corresponds to the control unit 120 shown in FIG. 1, and the transfer means corresponds to the switching unit 409 shown in FIG. .

A preferred application example of the eleventh invention is a photometric evaluation section 107, a focusing point detection section 108 and an external I shown in FIG.
When the control unit 120 determines the initialization status based on the information from the / F unit 121, and the initialization status is determined, the switching unit 409 shown in FIG. This is a moving image capturing system that reads the following gradation conversion curve.

In the eleventh aspect, the standard gradation conversion curve recorded in advance is read in when it is determined that the initialization status has been reached. As a result, even in the initialization situation where the gradation conversion curve is not calculated when the power is turned on,
Enables image output.

[0042]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described with reference to the drawings. (Configuration) FIG. 1 is a configuration diagram of a moving image capturing system according to an embodiment of the present invention.

Lens system 100, diaphragm 101, low-pass filter
102, the image taken through the single-plate CCD 103 is A
The signal is converted into a digital signal by the / D converter 104. A / D
The signal from the conversion unit 104 includes an image buffer (for long-exposure image) 105 and an image buffer (for short-exposure image) 106.
Is transferred to the combining unit 110 via. Image buffer 105
Is also connected to the photometric evaluation unit 107 and the focus detection unit 108. The photometric evaluation unit 107 is connected to the diaphragm 101 and the CCD 103, and the focus detection unit 108 is connected to the AF motor 109. Synthesis part 110
Signals from the signal processing unit 111, the gradation conversion unit 112, the D / A
It is connected to an output unit 114 such as a monitor or a video recorder via the conversion unit 113.

The signal processor 111 is also connected to the calculator 116 via the detector 115. The calculation unit 116 is a recording unit.
117 and the conversion curve synthesizing unit 118, and the recording unit 117 is also connected to the conversion curve synthesizing unit 118. The conversion curve composition unit 118 is connected to the gradation conversion unit 112, and the standard gradation curve ROM 119 is connected to the conversion curve composition unit 118.

A control unit such as a microcomputer
120 is bidirectionally connected to the synthesis unit 110, the signal processing unit 111, the gradation conversion unit 112, the output unit 114, the detection unit 115, the calculation unit 116, and the conversion curve synthesis unit 118.

Further, an external I / F section 121 having a power switch, a shutter button, and an interface for switching various modes at the time of photographing is also bidirectionally connected to the control section 120. Further, the photometric evaluation unit 107 and the focus detection unit 108 are also connected to the control unit 120.

(Operation) The flow of signals will be described with reference to FIG. The user specifies shooting conditions such as the image size and the number of frames via the external I / F unit 121, and then presses the shutter button to start shooting. Lens system 10
A video signal photographed through 0, the aperture 101, the low-pass filter 102, and the CCD 103 is converted into a digital signal by the A / D converter 104 and transferred to the image buffer 105 (long-time exposure image). In the present embodiment, the gradation width of the digitized signal is set to 12 bits, for example. The video signal in the image buffer 105 is transferred to the photometric evaluation unit 107 and the focus detection unit 108.

The photometric evaluation unit 107 determines the brightness level in the image and controls the electronic shutter speed of the diaphragm 101 and the CCD 103 so that the proper exposure is achieved. Also, the focus detection unit 10
In step 8, the edge strength in the image is detected, and the AF motor 109 is controlled so that the edge strength is maximized to obtain a focused image. The exposure conditions determined by the photometric evaluation unit 107 and the focusing conditions determined by the focus detection unit 108 at the time of shooting are transferred to the control unit 120.

Next, the second image is photographed under an exposure condition that is a predetermined exposure ratio, for example, 1/8 with respect to the exposure condition obtained by the photometric evaluation unit 107, and the A / D conversion unit. 104
At, it is converted into a digital signal and transferred to the image buffer 106 (short-time exposure image).

The synthesizing unit 110 sequentially reads the long-time exposure image and the short-time exposure image on the image buffer 105 and the image buffer 106. First, with respect to a long-time exposure image, a signal in an area equal to or less than a predetermined threshold value (for example, 3890 for 12-bit gradation) is left as an appropriate exposure area. Next, the short-time exposure image corresponding to the area other than the proper exposure area is read, and the exposure ratio is corrected and combined. In the present embodiment, since it is set to be 1/8 for long-time exposure, it is corrected eight times.

The combined signal is transferred to the signal processing unit 111 to generate a signal in a three-plate state in which known interpolation processing, white balance processing, emphasis processing and the like have been performed. Signal processing unit 11
The signal from 1 is transferred to the detection unit 115 at predetermined time intervals under the control of the control unit 120. The detection unit 115 detects the presence or absence of a scene change by calculating predetermined characteristic information from the transferred image and comparing it with the characteristic of the previously selected image.
The result is transferred to the control unit 120.

The control unit 120 controls the calculation unit 116 to detect a new gradation conversion curve when a scene change is detected.

The calculation unit 116 reads the image in which the scene change is detected from the detection unit 115, calculates the gradation conversion curve based on the histogram, and transfers the gradation conversion curve to the recording unit 117 and the conversion curve synthesis unit 118. The recording unit 117 records one gradation conversion curve, and is overwritten when the gradation conversion curve from the calculation unit 116 is transferred. The conversion curve synthesizing unit 118 serves as a gradation conversion curve synthesized from old and new gradation conversion curves under the control of the control unit 120, a new gradation conversion curve from the calculation unit 116, and a standard from the standard gradation curve ROM 119. One of the gradation conversion curves is transferred to the gradation conversion unit 112.

The gradation conversion unit 112 converts the signal from the signal processing unit 111 based on the gradation conversion curve transferred from the conversion curve synthesis unit 118 so as to match the gradation width of the output system. In this embodiment, the gradation width of the output system is set to 8 bits, for example. After this, the D / A converter 113 converts the analog signal,
It is output to the output unit 114 such as a monitor or a video recorder.

FIG. 2 shows an example of the structure of the detection unit 115 for detecting a scene change from the brightness, which includes a time setting unit 200 and
An image reading unit 201, a reduction unit 202, a reduced image buffer 203, a brightness calculation unit 204, a brightness value buffer 205,
The determination unit 206 is included. The control unit 120 is bidirectionally connected to the time setting unit 200, the image reading unit 201, the brightness calculation unit 204, and the determination unit 206. The time setting unit 200 is the image reading unit 2
Connected to 01. The signal from the signal processing unit 111 includes an image reading unit 201, a reduction unit 202, and a reduced image buffer 203.
It is connected to the calculation unit 116 via.

The signal from the reduced image buffer 203 is passed through the brightness calculation section 204 and the brightness value buffer 205, and then the determination section 20.
Connected to 6. From the control unit 120 to the photometric evaluation unit 107,
Information from the focus detection unit 108 and the external I / F unit 121 is transferred to the time setting unit 200.

In the time setting unit 200, information such as the image size and the number of frames obtained from the external I / F unit 121 is sent to the control unit 12.
Then, the time interval for selecting the image from the signal processing unit 111 is determined based on these information. In this control, for example, the time interval is lengthened as the image size or the number of frames increases, so that the load on the subsequent processing system does not increase. Also, the time setting unit 20
0 is the photometry evaluation unit 107, the photometry from the focus detection unit 108, it is determined that a scene change has occurred when the focusing conditions suddenly changed,
The time interval is set to 0 and the image is controlled to be read immediately. Other than this, it is also possible to detect a scene change using information such as white balance, lens zoom position, and camera movement.

The image reading section 201 reads the signal from the signal processing section 111 at a predetermined time interval under the control of the time setting section 200 and transfers the signal to the reducing section 202. The reduction unit 202 reduces the image at a predetermined reduction rate, for example, 1/8, and transfers the image to the reduced image buffer 203. This reduction rate does not have to be fixed, and may be variable. For example, it is possible to obtain the amount of information within a unit time from the reduced image size and the time interval to be selected, and control the reduction rate so that this amount of information is below a certain value. The reduced image buffer 203 is a ring-shaped buffer capable of recording a plurality of reduced images, and is configured to overwrite the old image when the buffer is full. The reduced image in the reduced image buffer 203 is transferred to the calculation unit 116.

On the other hand, the brightness calculation section 204 takes in the reduced image from the reduced image buffer 203 under the control of the control section 120 and calculates the average luminance value of the reduced image. This brightness value is transferred to and stored in the brightness value buffer 205. The brightness value buffer 205 is a ring-shaped buffer capable of recording the brightness value, and is configured such that when the buffer is full, the old brightness value is overwritten.

The judging section 206 monitors the time series change of the brightness value from the brightness value buffer 205, and when the change of a predetermined threshold value or more occurs and the situation continues for a predetermined number of times, a scene change occurs. to decide. The result of this determination is transferred to the control unit 120. Scene changes need not be limited to the brightness values above.

For example, FIG. 3 shows an example of the configuration of the detection unit 115 for detecting a scene change from a motion vector. The time setting unit 200, the image reading unit 201, the reduction unit 202, the reduced image buffer 203, The motion vector calculation unit 210, the motion vector buffer 211, and the determination unit 206 are included. Control unit 12
0 is bidirectionally connected to the time setting unit 200, the image reading unit 201, the motion vector calculation unit 210, and the determination unit 206. The time setting unit 200 is connected to the image reading unit 201.
The signal from the signal processing unit 111 is connected to the calculation unit 116 via the image reading unit 201, the reduction unit 202, and the reduced image buffer 203.

The signal from the reduced image buffer 203 is connected to the determination unit 206 via the motion vector calculation unit 210 and the motion vector buffer 211. Motion vector calculator 2
Reference numeral 10 reads the reduced images that are placed one after another in time series from the reduced image buffer 203, divides the image into predetermined blocks, and performs known motion detection based on matching. As a result, the motion vector is detected for each block, and the result is transferred to the motion vector buffer 211.

The determination unit 206 determines the motion vector buffer 211.
The motion vector information is read from, and it is determined that a scene change has occurred when a motion vector of a predetermined value or more is detected in a predetermined number of blocks or more. The result of this determination is the control unit 12
Transferred to 0.

Further, when the calculating unit 116 calculates the gradation conversion curve based on the histogram, the area corresponding to the block in which the motion vector is detected is multiplied by the weighting coefficient to give the gradation width to the area. Can also be processed.

FIG. 4 shows a calculator 11 for calculating a gradation conversion curve.
6 shows an example of the configuration of 6, which includes a luminance separating section 300, a proper exposure extracting section 301, an edge extracting section 302, a histogram creating section 303, and a conversion curve calculating section 304. The control unit 120
Luminance separation unit 300, proper exposure extraction unit 301, conversion curve calculation unit 30
It is bidirectionally connected with 4. The detection unit 115 is the brightness separation unit 30.
The brightness separating unit 300 is connected to the appropriate exposure extracting unit 301 and the histogram creating unit 303. The proper exposure extraction unit 301 is connected to the edge extraction unit 302 and the histogram creation unit 303, and the edge extraction unit 302 is connected to the histogram creation unit 303. The histogram creation unit 303 is a conversion curve calculation unit 304
It is connected to the recording unit 117 and the conversion curve synthesizing unit 118 via.

When the detection unit 115 detects a scene change, the control unit 120 controls the luminance separation unit 300 to capture the reduced image from the detection unit 115. The brightness separation unit 300 calculates brightness from the reduced image. The luminance signal is supplied to the proper exposure extraction unit 301 as a predetermined threshold value (for example, 12b) for the dark portion and the bright portion.
In the case of it gradation, the dark part is compared with 128 and the bright part is compared with 3968), and a luminance signal equal to or higher than the threshold value of the dark part and equal to or lower than the threshold value of the bright part is transferred to the edge extraction part 302 and the histogram creation part 303 as an appropriate region.

The edge extraction unit 302 performs publicly-known edge detection, extracts pixels having an edge strength equal to or higher than a predetermined threshold value as an edge portion, and uses this information as a histogram creation unit 303.
Transfer to. The histogram creation unit 303 creates a histogram of the edge portion from the brightness signal of the proper exposure area from the brightness separation unit 300, based on the information of the proper exposure area and the information of the edge portion.

The conversion curve calculation unit 304 obtains a gradation conversion curve by accumulating the above histograms, and the conversion curve synthesis unit 11
Transfer to 8. Further, based on the control of the control unit 120, the recording unit 11
Transfer the gradation conversion curve to 7. The recording unit 117 records one gradation conversion curve and is overwritten when the gradation conversion curve is transferred from the calculation unit 116.

FIG. 5 shows an example of the configuration of the conversion curve synthesizing unit 118. The gradation conversion curve buffer 400, the gradation conversion curve buffer 401, the representative point extraction unit 402, and the difference calculation unit 40.
3, a representative point adjustment unit 404, a multiplication and addition unit 405, a transition time setting unit 406, a weight control unit 407, an interpolation unit 408, and a switching unit 409.
Consists of. The control unit 120 is bidirectionally connected to the representative point extraction unit 402, the representative point adjustment unit 404, the multiplication and addition unit 405, the transition time setting unit 406, and the switching unit 409. The calculation unit 116 is connected to the gradation conversion curve buffer 400, and the recording unit 117 is connected to the gradation conversion curve buffer 401. The gradation conversion curve buffer 400 and the gradation conversion curve buffer 401 are connected to the gradation conversion unit 112 via the representative point extraction unit 402, the multiplication / addition unit 405, the interpolation unit 408, and the switching unit 409.

The representative point extracting unit 402 is connected to the representative point adjusting unit 404 via the difference calculating unit 403, and the representative point adjusting unit 404 is connected to the representative point extracting unit 402. Transition time setting unit 406
Is connected to the multiplication / addition unit 405 via the weight control unit 407. The standard gradation curve ROM 119 and the calculation unit 116 include a switching unit.
It is connected to 409. The gradation conversion curve from the calculation unit 116 is transferred to the gradation conversion curve buffer 400, and the gradation conversion curve from the recording unit 117 is transferred to the gradation conversion curve buffer 401 and stored.

Under the control of the control unit 120, the representative point extraction unit 402 extracts, as a representative point, values corresponding to the gradation values set at predetermined intervals, for example, equal intervals, from the gradation conversion curves of the two. This is transferred to the difference calculation unit 403. Difference calculation unit 403
Calculates the absolute value between the corresponding representative points on both gradation conversion curves and transfers it to the representative point adjusting unit 404 as a difference value. The representative point adjusting unit 404 adds a new representative point in the center between the representative points adjacent to the representative point when the difference value is equal to or larger than a predetermined threshold value. On the other hand, if the difference value is less than or equal to the predetermined threshold value, the representative point is deleted. Note that these additions and deletions are not performed for the start point and end point of the gradation conversion curve.

FIG. 6 shows the adjustment of the representative point based on the difference value. Based on the difference values between the old and new gradation conversion curves shown in FIG. 6A, the representative points set at equal intervals as shown in FIG. Is large, that is, the section where the gradation conversion is steep is densely adjusted, and the section where the difference value is small, that is, the section where the gradation conversion is gentle is adjusted roughly.

The representative point adjustment unit 404 notifies the control unit 120 when the above adjustment work is completed. The control unit 120 controls the representative point extracting unit 402 to extract a value corresponding to the adjusted representative point, and transfers the value to the multiplication and addition unit 405. Further, the control unit 120 sends the transition time setting unit 406 to the external I / F unit 12
Transfer information such as image size and number of frames obtained from 1. The transition time setting unit 406 is based on these pieces of information,
Adjust the transition time to switch the old and new gradation conversion curves.
This is, for example, as long as 2-3 seconds when processing a high-quality image with a large frame rate or image size, and about 1 second when processing a low-quality image with a small frame rate or image size. And shorten. In the weight control unit 408, the set transition time is set to the set transition time T (second).
From the current frame rate (number of frames / second) F, the number N (images) of images generated during the transition time is calculated.

N = TF (1) The basic unit of the weighting factor is given by 1 / N. The basic unit of the weight coefficient is transferred to the multiplication / addition unit 405.

When the representative point of the new gradation conversion curve from the calculation section 116 is Vnew and the representative point of the past gradation conversion curve from the recording section 117 is Vold, the multiplication and addition section 405 makes the following (2 ), The representative point V of the gradation conversion curve combined is calculated.

V = i / N Vnew + (1-i / N) Vold (i = 0 to N) (2) where i is the count value of the number of images generated during the transition time, and is expressed by the formula (2). One is added every time all the representative points of the gradation conversion curve synthesized based on are obtained.
The multiplication / addition unit 405 transfers the combined representative point to the interpolation unit 408. The interpolation unit 408 generates the inter-representative-point value by known linear interpolation, and transfers it to the switching unit 409 as a combined gradation conversion curve.

FIG. 7 shows the synthesis of the above gradation conversion curves. 7A shows a synthesized gradation conversion curve that gradually shifts from the past gradation conversion curve shown in FIG. 7A to the new gradation conversion curve shown in FIG. 7E. Some representative points [see FIG. 6 (b)] are extracted from the past gradation conversion curve and
As shown in (b) to (d), the transition is performed sequentially while performing the polygonal line approximation. As shown in FIG. 7, a polygonal line approximation can be performed to calculate an intermediate gradation conversion curve at low cost.

The switching unit 409 is based on the control of the control unit 120.
If a scene change is detected, within the transition time set by the transition time setting unit 406, the gradation conversion curve from the multiplication and addition unit 405,
Otherwise, the gradation conversion curve from the calculation unit 116 is converted to the gradation conversion unit 1
Transfer to 12. When the control unit 120 detects an initialization situation such as power-on, the switching unit 409 transfers the gradation conversion curve from the standard gradation curve ROM 119 to the gradation conversion unit 112 under the control of the control unit 120. .

When the transition time ends and the switching to the new gradation conversion curve is completed, the control unit 120 transfers the gradation conversion curve from the calculation unit 116 to the recording unit 117 for recording. In addition,
Although the interval between the representative points is adjusted based on the difference value in the above, it is possible to omit this if a slight decrease in accuracy is acceptable. In this case, the difference calculation unit 403,
The representative point adjusting unit 404 can be deleted, and a lower cost configuration is possible. Further, the interpolation between the representative points in the interpolation unit 408 is not limited to the linear interpolation, and any interpolation method can be used.

FIG. 8 shows an example of the configuration of the gradation conversion unit 112, which includes a Y / C separation unit 500 and a luminance signal buffer 501.
A color difference signal buffer 502, a luminance correction unit 503, a corrected luminance signal buffer 504, a color difference correction unit 505, and a Y / C synthesis unit.
It consists of 506. The control unit 120 is bidirectionally connected to the Y / C separation unit 500, the brightness correction unit 503, the color difference correction unit 505, and the Y / C synthesis unit 506. The signal processing unit 111 goes to the Y / C separation unit 500,
The Y / C separation unit 500 is connected to the luminance signal buffer 501 and the color difference signal buffer 502. The luminance signal buffer 501 is connected to the luminance correction unit 503 and the color difference correction unit 505, and the luminance correction unit 503 is connected to the Y / C synthesis unit 506 via the corrected luminance signal buffer 504. Further, the conversion curve synthesis unit 118 is connected to the brightness correction unit 503. The color difference signal buffer 502 is a color difference correction unit 505.
It is connected to the Y / C composition unit 506 via. The corrected luminance signal buffer 504 sends the color difference correction unit 505 a Y / C synthesis unit 506.
Are connected to the D / A converter 113.

The signal from the signal processing unit 111 is separated into a luminance signal and a color difference signal by the Y / C separation unit 500 and stored in the luminance signal buffer 501 and the color difference signal buffer 502, respectively. The brightness signal of the brightness signal buffer 501 is transferred to the brightness correction unit 503, and converted to a gradation width of the output system, which is 8 bits in the present embodiment, by a predetermined gradation conversion curve. The gradation conversion curve used by the brightness correction unit 503 is read from the conversion curve synthesis unit 118 under the control of the control unit 120. The luminance signal whose gradation has been converted by the luminance correction unit 503 is transferred to the corrected luminance signal buffer 504.

Next, the color-difference correction unit 505 uses the color-difference signal buffer 5
The color difference signal on 02 is read, and the brightness signal before gradation conversion from the brightness signal buffer 501 and the brightness signal after gradation conversion from the corrected brightness signal buffer 504 are received. A correction coefficient for correcting the color difference signal is calculated from the theoretical limit model in which the luminance signal before and after the conversion and the color may exist to correct the color difference signal. The corrected color difference signal is transferred to the Y / C combining unit 506, combined with the converted brightness signal from the corrected brightness signal buffer 504, and transferred to the D / A conversion unit 113.

With the above configuration, for any image captured in time series, the gradation conversion curve corresponding to the scene change can be switched within the transition time according to the image capturing conditions such as the image size and the number of frames. It is possible to change continuously, and a high-quality image with less discomfort can be obtained. Further, since the image to be processed is reduced and the gradation conversion curve within the transition time is calculated from several representative points, the amount of calculation can be reduced and a low-cost processing system can be realized. In addition, the image obtained is stable because the minute change of the image is absorbed by the reduction processing. Further, by preparing a standard gradation curve, it becomes possible to deal with the initialization situation.

In the above configuration, one wide dynamic range image is composed from two images taken with different exposure amounts (long and short exposure) for the same subject. Need not be limited to. Normal 1
It can also be applied to a single-photographing system. In the case of this single-photographing system, the image buffer 106 and the combining unit 110 can be omitted.

[0085]

As described above, according to the present invention, the gradation conversion curve for converting the gradation width of the images obtained in time series can be appropriately switched at a low cost according to the scene change. The system can be realized. Further, it is possible to realize a moving image capturing system that generates a high quality image for an arbitrary shooting scene.

[Brief description of drawings]

FIG. 1 is a block diagram of a moving image capturing system according to an embodiment of the present invention.

FIG. 2 is a block diagram showing an example of a configuration of a detection unit that detects a scene change from luminance.

FIG. 3 is a block diagram showing a configuration example of a detection unit that detects a scene change from a motion vector.

FIG. 4 is a block diagram showing an example of a configuration of a calculation unit that calculates a gradation conversion curve.

FIG. 5 is a block diagram showing an example of a configuration of a conversion curve synthesizing unit.

FIG. 6 is a diagram illustrating adjustment of a representative point based on a difference value.

FIG. 7 is a diagram for explaining synthesis of gradation conversion curves.

FIG. 8 is a block diagram showing an example of the configuration of a gradation conversion unit.

[Explanation of symbols] 110 ... Composition unit (image composition means) 112 ... Gradation conversion unit (gradation conversion means) 115 ... Detection unit (detection means) 116 ... Calculation unit (calculation means) 117 ... Recording unit (recording means) 118 ... Conversion curve synthesizing unit (transformation curve synthesizing means)

Claims (11)

[Claims] 1. A moving image pickup system for converting an image group of Mbit gradation widths continuous in time series from an image pickup system into Nbit gradation widths (M and N are natural numbers M ≧ N) of an output system and outputting the converted image. Detecting means for detecting a scene change from the image group; calculating means for calculating a gradation conversion curve from the image in which the scene change is detected by the detecting means; recording means for recording the gradation conversion curve; The conversion curve synthesizing means for synthesizing the new gradation conversion curve calculated by the means and the past gradation conversion curve recorded in the recording means, and the gradation conversion curve combined by the synthesizing means are used for the Mbi.
A moving image pickup system, comprising: a gradation conversion unit that converts an image group having a t gradation width into an Nbit gradation width. 2. An image synthesizing unit for synthesizing at least two or more frame or field unit images of the same subject under different exposure conditions to generate an image group having a gradation width of Mbit. The moving image pickup system according to claim 1. 3. The number of images photographed from the image pickup system per unit time,
A time setting means for setting a time interval for selecting an image from the image group based on at least one of image size, exposure condition, focus condition, white balance condition, zoom position, and camera position. Item 2. The moving image capturing system according to item 1 or 2. 4. The reducing means further reduces the selected image to a predetermined size, a brightness calculating means for calculating an average brightness level from the reduced image, and a time series of the average brightness level. 4. The moving image capturing system according to claim 3, further comprising: a determining unit that determines whether or not there is a scene change based on the dynamic change. 5. The detecting means further reduces a size of the selected image to a predetermined size, and a motion vector calculating a motion vector from two images which are temporally before and after the reduced image. The moving image capturing system according to claim 3, further comprising: a calculating unit; and a determining unit that determines whether or not there is a scene change based on the motion vector amount. 6. The calculating means separates the image into a luminance signal and a color difference signal, an extracting means for extracting a proper exposure area based on the luminance signal level, and a feature amount for the proper exposure area. 3. A characteristic amount calculating means, a histogram generating means for generating a histogram based on the characteristic amount, and a gradation conversion curve calculating means for calculating a gradation conversion curve based on the histogram. Alternatively, the moving image capturing system described in 2. 7. The conversion curve synthesizing means includes setting means for setting a plurality of representative points on the new gradation conversion curve calculated by the calculating means and the past gradation conversion curve recorded on the recording means. , A multiplication and addition means for calculating a composite value from the new gradation conversion curve and the past gradation conversion curve based on a predetermined weighting coefficient for the representative point, and the representative point processed by the multiplication and addition means The moving image capturing system according to claim 1 or 2, further comprising: an interpolating unit that interpolates between the two. 8. The conversion curve synthesizing means further comprises a transition time setting means for determining a transition time for switching from the past gradation conversion curve to the new gradation conversion curve based on the photographing condition from the imaging system. The moving image capturing system according to claim 7, further comprising: a weight control unit that controls the weight coefficient based on a transition time of the transition time setting unit. 9. The conversion curve synthesizing means further comprises switching means for selecting one of the gradation conversion curve interpolated by the interpolation means and the new gradation conversion curve. The moving image pickup system according to claim 7. 10. The conversion curve synthesizing means further calculates a difference value between the new gradation conversion curve and the past gradation conversion curve with respect to the representative point, and a representative point based on the difference value. 8. The moving image pickup system according to claim 7, further comprising: an adjusting unit that adjusts the position or the number of. 11. A standard gradation conversion curve recording means for recording a standard gradation conversion curve, an initialization detecting means for detecting an initialization situation when the power is turned on, and the standard based on the initialization detecting means. 3. The moving image pickup system according to claim 1, further comprising: a transfer unit that transfers a gradation conversion curve to the gradation conversion unit.