JP2010263394A - Video signal processing device - Google Patents

Abstract

To perform resolution conversion of various magnifications with a small line memory.
An input unit 11 to which a video signal is input, N line memories 13 having a predetermined number of words, and a line memory 13 are rearranged according to the number of horizontal and vertical pixels of the video signal, and the video signal Control is performed so that video signals for one line are stored in at least K line memories 13 necessary for storing the number of pixels for one line, and at most N / K lines are stored in the line memory 13. A memory control unit; and a video signal processing arithmetic unit that performs predetermined video signal processing using the video signal read from the line memory.
[Selection] Figure 1

Description

  The present invention relates to a video signal processing apparatus used in a digital video camera or the like, and more particularly to a filter calculation process used for resolution conversion processing of a video signal.

  In general, in an apparatus for recording a video signal such as a digital video camera, a user can select a plurality of recording formats by setting a mode.

  For example, in the case of a video camera equipped with an HD (High-Definition) camera, the recording format can be selected from HD or SD (Standard Definition). Recording formats such as 1080i (1920 × 1080 pixels) and 720p (1280 × 720 pixels) are generally used for HD, and 480i (720 × 480 pixels) and 576i (720 × 576 pixels) are generally used for SD. . Further, in the thumbnail display for each shooting scene on the menu screen at the time of reproduction, it is conceivable that a reduced thumbnail image (for example, 320 × 180 pixels) is generated and recorded. As a configuration of the video camera main body, a method in which the imaging unit always obtains a maximum video signal of 1920 × 1080 pixels and performs a video signal processing calculation to convert the resolution to a set recording format can be considered.

  Patent Document 1 discloses an image signal processing apparatus that generates image signals of a plurality of types of formats in a time division manner from a line memory that stores data.

  When performing such resolution conversion, the video signal is subjected to filter processing and pixel thinning is performed. This resolution conversion processing is performed by the video signal processing calculation unit. In a filter circuit that handles input images of multiple sizes, it is common to equip the size of the line memory with the number of lines for vertical taps necessary to achieve the maximum horizontal size and maximum reduction ratio. .

  As an example, the number of taps required for each reduction ratio is given below. The reduction ratio is large, such as 4 taps if the reduction ratio is about 1/2 times, 6 taps if the reduction ratio is about 1/4 times, and 12 taps if the reduction ratio is about 1/8 times. The number of taps required increases according to The reason is to prevent the loss of information and the reduction of the smoothness of the output image when the filter calculation (averaging process) is performed with a small number of pixels even though the thinning amount is large.

  The line memory required in the above example requires 1920 words for the horizontal size and 12 lines for the number of vertical lines.

  The video signal generally handles both a luminance signal and a color signal. Therefore, in order to perform resolution conversion, it is necessary to prepare two circuits of the above configuration for the luminance signal and the color signal. In addition, since the luminance signal is more susceptible to resolution degradation than the color signal, it is preferable to assign a larger number of taps to the luminance signal and apply a vertical filter with good frequency characteristics. For this purpose, a line memory must be added to the luminance signal system to realize a higher-order filter.

  In addition, when a very large reduction ratio such as 1/12 times is realized as in the case of obtaining a thumbnail image, it is possible to adopt a configuration in which a simple filter is first halved and then 1/6 times. . In another case, it is beneficial to adopt the above configuration when it is desired to obtain a half-fold intermediate image and a 1 / 12-fold thumbnail image at the same time. The first resolution conversion process for obtaining an intermediate image is equipped with some line memory for vertical processing. When the generation of the intermediate image is not necessary (when the reduction ratio is not so large or when the intermediate image is not necessary at the same time), the first resolution conversion process is bypassed and only the second resolution conversion process is performed. Should be changed.

JP 2001-268527 A

  As described above, in the conventional video signal processing apparatus, it is necessary to provide the line memory with the number of lines necessary to satisfy the maximum reduction ratio with the number of words corresponding to the maximum number of horizontal pixels of the input video signal. Therefore, when the horizontal size of the input video signal to the vertical filter unit is small and the reduction ratio is large, a large number of lines are required, but many of the horizontal words in the line memory remain unused. On the other hand, when the number of horizontal words is large and the reduction ratio is small, all the horizontal word numbers are used but the number of necessary lines may be small. Thus, when converting to various resolutions, there is a problem that the line memory becomes redundant in order to satisfy all conversion applications. Further, in order to assign a larger number of taps to the luminance signal, it is necessary to add a line memory. Furthermore, when realizing a high magnification or for obtaining an image with a plurality of resolutions, it is necessary to further add a line memory for the added resolution conversion processing.

  The video signal processing apparatus of the present invention rearranges the line memory according to the input unit to which the video signal is input, N line memories having a predetermined number of words, and the number of horizontal and vertical pixels of the video signal. The video signal for one line is stored in at least K line memories necessary for storing the number of pixels for one line of the video signal, and at most N / K lines are stored in the line memory. A memory control unit that controls to store the video signal, and a video signal processing arithmetic unit that performs predetermined video signal processing using the video signal read from the line memory.

  As described above, the present invention can take an optimum configuration according to the horizontal size of the input video signal to the vertical filter unit, so that the required line memory size can be reduced.

  Also, the line memory configuration can be dynamically changed when it is desired to dynamically change the output image size.

  Further, when changing the characteristics of the filter processing applied to the luminance signal and the color signal, the line memory configuration can be changed appropriately, and the line memory capacity is not increased.

  Further, when a plurality of resolution conversions are connected in series to obtain a signal having a plurality of resolutions, the line memory configuration can be appropriately changed, and the line memory capacity is not increased.

FIG. 2 is a block diagram illustrating a configuration of a video signal processing device according to the first embodiment. Conceptual diagram of line memory allocation of memory control means in the first embodiment Explanatory drawing of the writing control method to the line memory of the memory control means in Embodiment 1. Timing diagram for changing tap configuration of memory control means in embodiment 1 FIG. 3 is a block diagram illustrating a configuration of a video signal processing device according to a second embodiment. Conceptual diagram of line memory allocation of memory control means in the second embodiment The figure which shows the usage example of the allocation line memory of the memory control means in Embodiment 2. FIG. 9 is a block diagram illustrating a configuration of a video signal processing device according to a third embodiment.

  Hereinafter, an embodiment of a video signal processing apparatus will be described in detail with reference to the drawings.

(Embodiment 1)
FIG. 1 is a block diagram showing the configuration of the video signal processing apparatus according to the first embodiment. Here, the contents of the video signal processing calculation will be described on the assumption that a vertical reduction process is performed when the input video signal is reduced to a designated resolution.

  In FIG. 1, the video signal processing apparatus includes a horizontal reduction unit 10, an input terminal 11, a vertical reduction unit 12, a line memory 13, a memory control unit 14, a controller 15, and an output terminal 16.

  The operation of the video signal processing apparatus configured as described above will be described below.

  First, the input video signal is reduced in the horizontal direction by the horizontal reduction means 10 so that the number of horizontal pixels of the designated output resolution is obtained. The reduction process is realized by thinning out pixels by applying a band limiting filter corresponding to the reduction ratio. This filter performs a product-sum operation by delaying an input pixel sample by a delay stage corresponding to the number of filter taps. In the case of a horizontal filter, the delay stage may be constituted by a flip-flop, and an SRAM such as a line memory is not necessary, so that the circuit scale is not considered as a problem.

  The video signal reduced to the designated horizontal resolution by the horizontal reduction means 10 is supplied from the input terminal 11 to the vertical reduction means 12.

  The vertical reduction means 12 is a video signal processing calculation unit that performs a filter calculation using the video signal read from the line memory 13 and performs a reduction process. Here, the vertical filter must store data for the number of taps necessary for calculation in a line memory such as SRAM. Since this line memory greatly affects the circuit scale, it is necessary to consider the balance between the number of taps that determine the performance and the circuit scale.

  The line memory 13 includes N line memories having the number of words W. In the figure, M1 to M12 each indicate a single line memory, and N = 12 states.

  The memory control unit 14 controls writing and reading of N memories included in the line memory 13. The memory control means 14 uses K line memories so that the number of horizontal pixels for one line of the video signal input from the input terminal 11 is sufficient to store the input video signal. Data for one line is stored. Then, using all line memories, the maximum N / K lines are stored and used as taps for the vertical filter.

  The vertical reduction unit 12 uses the read data from the line memory 13 output from the memory control unit 14 to perform a filter operation and a thinning process. In the case of the reduction process, since the through (not stored) output of the input signal can be used as a tap, a filter operation of (N / K) +1 tap is possible. Further, since video signals having various numbers of horizontal pixels are input from the input terminal 11 in accordance with the designated reduction ratio, the value of K corresponding to the number of input horizontal pixels may be changed.

  The controller 15 gives the value K to the memory control means 14. As an example, if the number of words is W = 512 words, N = 12, and K = 4, the number of horizontal pixels can be up to 2048, the number of lines that can be stored in the line memory is 3, and the reduction filter has a maximum of 4 taps. It becomes.

  Finally, the video signal reduced by the vertical reduction means 12 is output from the output terminal 16 to the outside.

  Next, detailed operation of the write control of the memory control unit 14 will be described. FIG. 2 is a conceptual diagram of a line memory arrangement corresponding to the number of input horizontal pixels.

  FIG. 2A shows an image of the line memory arrangement when K = 4. Four lines are assigned to one line. Up to 2048 pixels can be supported, but the number of taps is up to 4 taps. FIG. 2B shows a case where K = 2, and two lines are assigned to one line. Up to 1024 pixels can be supported, but the number of taps is up to 7 taps. FIG. 2C shows a case where K = 1, and one line is assigned to one line. With up to 512 pixels, the number of taps can be up to 13 taps.

  The operation timing of the above configuration will be described with reference to FIG. However, for the sake of simplicity, only the operations of the line memories M1 to M4 are extracted and described.

  FIG. 3A is a timing chart in the case of K = 4. The data of 2048 pixels for one line is divided into four regions A1 to A4, the data in the period A1 is stored in the line memory M1, and the data in the period A2. Write timing is generated so that data is stored in the line memory M2, data in the period A3 is stored in the line memory M3, and data in the period A4 is stored in the line memory M4 (write enable control).

  FIG. 3B is a timing chart when K = 2, and 1024 pixel data for two lines are divided into A1 and A2, and A3 and A4. For A1 and A2 obtained by dividing the data for one line, the write timing is generated so that the data for the period A1 is stored in the line memory M1 and the data for the period A2 is stored in the line memory M2. For A3 and A4 obtained by dividing the data of the previous line, the write timing is generated so that the data in the period A3 is stored in the line memory M3 and the data in the period A4 is stored in the line memory M4. In this case, it can be operated as a memory for two lines.

  FIG. 3C is a timing chart in the case of K = 1, and data for the period A1 is stored in the line memory M1 for 512 pixels of data A1 for one line. For the data A2 of 512 pixels before one line, the data for the period A2 is stored in the line memory M2. For the data A3 of 512 pixels before two lines, the data for the period A3 is stored in the line memory M3. For 512 pixels of data A4 three lines before, the data for the period A4 is stored in the line memory M4. In this case, it can be operated as a memory for four lines. Here, the additions of n, n-1, n-2, and n-3 in the input video signal in FIG. 3 mean the line number of the video signal input to the line memory, where n is an input through, n− 1 indicates a signal one line before, n-2 indicates a signal two lines before, and n-3 indicates a signal three lines before. Data read from the line memory is used except for the input through.

  Next, the operation of changing the tap configuration of the vertical filter in the controller 15 will be described with reference to FIG. In FIG. 4, the vertical synchronizing signal is a signal indicating one field or one frame period (1 V period) of the input video signal. FIG. 4 shows that the tap configuration can be changed every 1V period. The controller 15 changes the memory control of the memory control means 14 by changing the value of K at the start timing of the 1V period. One circuit can be executed when two kinds of reduced images are obtained from the original signal having the same angle of view. For example, even if it is a moving image, it is possible to obtain images of two different resolutions at the same time by supplying images at the same time in the 2V period. In this way, the controller 15 can dynamically change the tap configuration.

(Embodiment 2)
FIG. 5 is a block diagram showing the configuration of the video signal processing apparatus according to the second embodiment. Here, the contents of the video signal processing calculation will be described on the assumption that a vertical reduction process is performed when the input video signal is reduced to a designated resolution.

  In FIG. 5, the video signal processing apparatus includes horizontal reduction means 20 and 30, input terminals 21 and 31, vertical reduction means 22 and 32, line memory 23, memory control means 24, controller 25, and output terminal. 26 and 33.

  The operation of the video signal processing apparatus configured as described above will be described below.

  As a general example, consider a case where a video signal is input after being separated into a luminance signal and a color signal. The resolution conversion of the video signal must be performed for each of the luminance signal and the color signal. The configuration shown in FIG. 5 uses the configuration shown in FIG. 1 for two systems for luminance signals and color signals. However, two lines are not arranged as they are, but the line memory 23 is shared.

  First, the input luminance signal is subjected to horizontal reduction processing by the horizontal reduction means 20 so that the number of horizontal pixels of the designated output resolution is obtained. The luminance signal reduced to the designated horizontal resolution by the horizontal reduction means 20 is supplied from the input terminal 21 to the vertical reduction means 22.

  The vertical reduction means 22 is a video signal processing calculation unit, which performs a filter calculation using the luminance signal data read from the line memory 23 and performs a reduction process.

  The line memory 23 includes N line memories having the number of words W. In the figure, M1 to M12 and N1 to N12 each indicate a single line memory, and N = 24 states.

  On the other hand, the input color signal is subjected to horizontal reduction processing by the horizontal reduction means 30 so as to have the number of horizontal pixels of the designated output resolution. The color signal reduced to the designated horizontal resolution by the horizontal reduction means 30 is supplied from the input terminal 31 to the vertical reduction means 32.

  The vertical reduction means 32 is a video signal processing calculation unit that performs a filter calculation using the color signal data read from the line memory 23 and performs a reduction process.

  The memory control unit 24 controls writing and reading of the N memories included in the line memory 23. The memory control means 24 uses K line memories so that the number of words is sufficient to store the number of horizontal pixels for one line of luminance signals and color signals input from the input terminals 21 and 31. The data for one line of the input luminance signal and color signal is stored. At this time, the number of lines that can be stored is a maximum of N / K lines including the luminance signal and the color signal. Of these, L lines are assigned to the luminance signal and a maximum N / K-L lines are assigned to the color signal. That is, M = L × K line memories are allocated to the luminance signal, and a maximum of NM line memories are allocated to the color signal. Thus, the luminance signal data is stored in the line memory assigned to the luminance signal, the color signal data is stored in the line memory assigned to the color signal, these are read out, and the luminance signal is vertically reduced. 22 and the color signal vertical reduction means 32.

  The controller 25 controls the value of M (of course, the value of K is also related) to adjust the distribution of the line memory to the luminance signal and the color signal.

  Finally, the luminance signal reduced by the vertical reduction means 22 is output from the output terminal 26 to the outside. The color signal reduced by the vertical reduction means 32 is output from the output terminal 33 to the outside.

  Next, detailed operation of the write control of the memory control means 24 will be described. FIG. 6 is a diagram for explaining how the line memory is distributed to the luminance signal and the color signal. FIG. 6A shows a case where 12 line memories of N = 24 are used for luminance signals and 12 for color signals. This is referred to as mode A. Next, FIG. 6B shows a case where N = 24 line memories are used for the luminance signal and 20 for the color signal. This is referred to as mode B.

  FIG. 7 shows an example of using the line memory for the luminance signal and the color signal when distributed in this way. FIG. 7A shows an example of the line memory configuration of the luminance signal and the color signal in the mode A, and shows a case where up to 2048 horizontal pixels are handled. Up to four tap vertical filter operations can be performed using a memory for three lines for luminance signals. For color signals, a memory for 3 lines is used, and a vertical filter operation with a maximum of 4 taps is possible. FIG. 7B shows an example of the line memory configuration of the luminance signal and the color signal in the mode B, and shows a case where up to 2048 horizontal pixels are handled. Up to 6 tap vertical filter operations can be performed using a memory for 5 lines for luminance signals. For color signals, only one line of memory is allocated, and vertical filter operation with a maximum of 2 taps is possible.

  In general, the luminance signal is more conspicuous than the color signal in terms of resolution degradation due to filtering. Therefore, it is desirable to use a high-order filter with good frequency characteristics for the luminance signal. For example, if the reduction ratio is about 1 to 1/2 times, the color signal may be thinned out by 2-tap linear interpolation, and the visual effect is not noticeable even if the frequency characteristic is not good. The purpose is to distribute a line memory used for color signals to luminance signals so that a high-order filter having good frequency characteristics can be used. However, if the signal band of the color signal is high and deterioration is conspicuous by thinning out by 2-tap linear interpolation, the configuration of mode A can be taken.

  In the second embodiment, as in the first embodiment, the controller 25 can change the tap configuration in units of 1 V period indicated by the vertical synchronization signal of the video signal. It can be changed according to the respective signal bands of the luminance signal and color signal of the input video signal.

(Embodiment 3)
FIG. 8 is a block diagram showing the configuration of the video signal processing apparatus according to the third embodiment. When the input video signal is reduced to a designated resolution, a configuration in which a video signal having a designated output resolution is obtained while generating a video signal having an intermediate resolution by using a two-stage reduction unit.

  In FIG. 8, the video signal processing apparatus includes a first reduction means 40, a second reduction means 41, a line memory 44, a memory control means 45, a controller 46, and an output terminal 47. The second reduction unit 41 includes a horizontal reduction unit 42 and a vertical reduction unit 43.

  The operation of the video signal processing apparatus configured as described above will be described below.

  First, the input video signal is subjected to first resolution conversion by the first reduction means 40. Although not shown, horizontal and vertical reduction processing (filter processing and thinning processing) is performed. The vertical reduction process requires a line memory as described above. The output data from the first reduction means 40 is an intermediate resolution video signal. Output to the outside if necessary.

  This intermediate resolution video signal is input to the second reduction means 41 and converted to the designated output resolution. The second reduction means 41 includes a horizontal reduction means 42 and a vertical reduction means 43.

  First, the horizontal reduction means 42 performs a horizontal reduction process so that the number of horizontal pixels of the designated output resolution is obtained. The video signal reduced to the designated horizontal resolution by the horizontal reduction means 42 is supplied to the vertical reduction means 43. The vertical reduction means 43 performs a filter operation using the video signal read from the line memory 44 and performs a reduction process.

  The line memory 44 includes N line memories having the number of words W. In the figure, M1 to M12 each indicate a single line memory, and N = 12 states.

  The memory control unit 45 controls writing and reading of N memories included in the line memory 44. The memory control means 45 allocates S line memories to the first reduction means 40, and the first reduction means 40 is used for conversion to the intermediate resolution. Next, the remaining maximum N−S line memories are allocated to the second reduction unit 41, and the second reduction unit 41 is used for conversion to the output resolution.

  The controller 46 controls the value of S (of course, the value of K is also related) and adjusts the distribution of the line memory to the first reduction means 40 and the second reduction means 41.

  Finally, the luminance signal reduced by the vertical reduction means 22 is output from the output terminal 47 to the outside.

  In this configuration, when the generation of the intermediate image is unnecessary (when the reduction ratio is not so large or when the intermediate image is not necessary at the same time), the first reduction unit 40 is bypassed and the second reduction unit is bypassed. Only 41 processing can be performed. At this time, the memory control unit 45 operates to distribute all the line memory resources allocated to the first reduction unit 40 to the second reduction unit 41 in accordance with an instruction from the controller 46.

  Also in the third embodiment, as in the first and second embodiments, the controller 46 can change the tap configuration in units of 1V period indicated by the vertical synchronization signal of the video signal.

  In the above description, the case where the resolution conversion process is performed on the video signal has been described. However, the present invention can be applied to any video signal process involving a vertical filter operation.

  As described above, the video signal processing apparatus of the present invention can realize resolution conversion at various magnifications by changing the configuration of the line memory according to the number of horizontal pixels of the input video signal. There is no need to provide redundant memory.

  In addition, since the line memory resource of the color signal can be allocated to the luminance signal according to the respective signal bands of the luminance signal and the color signal, the image quality can be improved without increasing the circuit scale.

  Further, the present invention can be applied to a case where a high magnification is realized or an application for obtaining an image having a plurality of resolutions without requiring an additional line memory resource.

  Therefore, the present invention is extremely useful for various video signal processing apparatuses and software used for digital video cameras, video recorders, and the like.

10, 20, 30, 42 Horizontal reduction means 11, 21, 31 Input terminal 12, 22, 32, 43 Vertical reduction means 13, 23, 44 Line memory 14, 24, 45 Memory control means 15, 25, 46 Controller 16, 26, 33, 47 Output terminal 40 First reduction means 41 Second reduction means

Claims (7)

An input unit for inputting a video signal;
N line memories having a predetermined number of words;
The line memory is rearranged according to the number of horizontal and vertical pixels of the video signal, and the video signal for one line is stored in at least K line memories necessary for storing the number of pixels for one line of the video signal. A memory control unit that controls to store at most N / K lines in the line memory;
A video signal processing arithmetic unit that performs predetermined video signal processing using the video signal read from the line memory;
A video signal processing apparatus comprising: The memory control unit
The video signal processing apparatus according to claim 1, wherein the value of K is changed in accordance with a timing when the number of horizontal and vertical pixels of the video signal changes. An input unit to which a luminance signal and a color signal of a video signal are input;
A plurality of line memories having a predetermined number of words;
A memory controller that distributes the line memory and stores the luminance signal and the color signal in the distributed line memory;
A video signal processing operation unit for performing video signal processing for luminance signals using the luminance signals read from the line memory, and performing video signal processing for color signals using the color signals read from the line memory;
A video signal processing apparatus comprising: The memory control unit
4. The video signal processing apparatus according to claim 3, wherein distribution of line memories for the luminance signal and the color signal is changed according to a change in the content of the video signal processing for the luminance signal and the color signal. The video signal processing arithmetic unit is
The video signal processing apparatus according to claim 1, wherein a filter operation using the line memory is performed. The video signal processing arithmetic unit is
5. The video signal processing apparatus according to claim 1, wherein a filter operation for converting a resolution of the video signal is performed using the line memory. An input unit for inputting a video signal;
A plurality of line memories having a predetermined number of words;
A first resolution converter that obtains an intermediate image of a first resolution from the video signal;
A second resolution converter that obtains an output image of a second resolution from the intermediate image;
A memory control unit that distributes and allocates the line memory to the first resolution conversion unit and the second resolution conversion unit;
A video signal processing apparatus comprising:

Images