JP2009141445A - Sampling rate conversion device and sampling rate conversion method - Google Patents

Abstract

【Task】
Sampling rate conversion is suitably performed by small-scale signal processing.
[Solution]
A first arithmetic unit (odd tap symmetric filter) that inputs digital signal data and outputs a convolution calculation result of the tap data whose coefficient values are symmetrical about the first symmetry axis and the input data; A second arithmetic unit (even-numbered tap symmetric filter) that outputs a convolution operation result of the tap coefficient having a coefficient value symmetrical about the second symmetry axis and the input data, and a pixel position of the output data The output of the sampling rate is generated by mixing the outputs of the first and second arithmetic units according to the mixing ratio determined from the relationship between the pixel positions of the input data.
[Selection] Figure 1

Description

  The present invention relates to a technique for converting a sampling rate by digital signal processing.

  With the diversification of information media such as broadcasting, communication, and storage, there is an increasing need for technology that enlarges and reduces the size of an image by digital signal processing, and performs deformation such as rotation.

  Therefore, as described in Patent Document 1, a technique (hereinafter referred to as a conventional technique) for converting a sampling rate by a polyphase filter and interpolating a sample signal (hereinafter referred to as a pixel) has been proposed.

JP2004-214827

  This conventional technology combines an upsampler that increases the number of pixels in the input image by a factor of U (where U is a positive integer) and a downsampler that thins out the number of pixels to 1 / D (where D is a positive integer). By converting the total number of pixels to U / D times, pixels at positions that did not exist in the original input image are generated, and an enlarged or reduced output image is obtained.

  FIG. 2 shows the operating principle of this prior art. This figure shows the number of pixels as an example based on (a) input signals composed of digitized (sampled) pixels (201-1), (201-2), (201-3), and (201-4). (D) shows the processing contents when the output signal is obtained by increasing the value by 8/3 times. Here, the pixels (201-1), (201-2), (201-3), and (201-4) are referred to as actual pixels in the sense that they actually exist.

  In the prior art, when increasing the number of pixels by 8/3 times, as shown in FIG. 1 (b), pixels (202) (hereinafter referred to as zeros) whose signal intensity is 0 are equally spaced by an upsampler described later. And increase the number of pixels by 8 times. Subsequently, as shown in FIG. 7C, after generating the interpolation pixel (203) by the interpolation filter described later, the output pixel (204-1) (204-1) ( 204-2) (204-3) ... and (d) output signal is obtained. Further, the interval between the output pixels (204-1) (204-2) (204-3)... Is equal to the interval between the actual pixels (201-1) (201-2) (201-3). If rearranged, the image size is enlarged by 8/3 times.

  In the above, the case where 8/3 times enlargement processing is performed with U = 8 and D = 3 has been described as an example, but the same applies to the case of reduction, for example, if U = 4 and D = 9, 4/9 times reduction processing can be performed.

  FIG. 3 shows a configuration for realizing the above-described conventional operation. FIG. 2A shows a signal processing configuration of the operation principle itself described in FIG. With respect to the input real pixel (201), the upsampler (301) inserts zero points (202) at equal intervals to increase the number of pixels U times, and the interpolation filter (302) sets the interpolation pixel (203). The output pixel (204) is obtained by thinning out the pixels to 1 / D by the down sampler (303). In this configuration, (c) in the signal after the interpolation filter, the number of pixels in the middle of processing is very large, and the multiplication for the zero point (202) whose signal intensity is 0 is useless. Alternatively, the processing is generally replaced with the configuration shown in FIG.

  FIG. 3 (b) shows that the signal strength is reduced by decomposing the tap coefficients of the interpolation filter (302) described above into partial filters (304-1) (304-2) (304-3) (304-U). In this configuration, multiplication is not performed on the zero point (202) of 0. Thus, the control unit (306) controls the switching unit (305) to select one of the outputs of the partial filters, and an output signal is obtained.

  In FIG. 3C, only one partial filter (307) is provided, and the partial filter (304-U) is generated by the tap coefficient generators (308-1) (308-2) (308-3) (308-U). 1) Generate the same tap coefficient as (304-2) (304-3) ... (304-U). Next, the control unit (310) controls the switch (309) to select one of the tap coefficients and set it to the partial filter (307), and the configuration shown in FIG. Get the same output signal. The tap coefficient generator (308) is generally constituted by a memory for storing tap coefficients.

  FIG. 4 shows the interpolation filter (302), partial filters ((304-1) (304-2) (304-3)... (304-U), and tap coefficient generator (308-1)) shown in FIG. (308-2) (308-3) ... (308-U) shows an example of each operation. (A) is an example of tap coefficients when the number of taps of the interpolation filter (302) is 25. Yes, it is an example of 8/3 times enlargement processing, that is, assuming U = 8, D = 3, where (* 1/256) at the end means that all tap coefficients are multiplied by 1/256. From the left, the tap coefficients of the interpolation filter (302) are -1/256, -3/256, -6/256, etc. These tap coefficients are converted into the signal after up-sampling in FIG. By performing the convolution, the signal after the interpolation filter in FIG. 2C is obtained.

  FIG. 7B shows the tap coefficients of the interpolation filter (302) shown in FIG. 4A converted into the tap coefficients of the partial filters (304) and (307). That is, the center tap of the interpolation filter (302) shown in FIG. 4 (a) is placed at the pixel position of the output pixel (204-1) (204-2) (204-3)... Shown in FIG. After the position is matched, the tap coefficient at the position corresponding to the zero point (202) is removed to make a new tap coefficient, and the relationship between the output pixel number and the input pixel number at that time is also shown. Yes. The control units (306) and (310) shown in FIGS. 3 (b) and 3 (c) output the results of the partial filters (304) and (308) in which the tap coefficients shown in FIG. 4 (b) are set. It is necessary to control the switches (305) and (309).

  The prior art shown in FIGS. 3 and 4 can convert the sampling rate to U / D times, that is, the number of pixels can be converted to U / D times, but has the following problems.

  The first problem is that as the value of U increases, the number of partial filters (304) or tap coefficient generators (308) shown in FIG.

  For example, when the image is enlarged to 256/173 times, since U = 256, 256 partial filters (304) or tap coefficient generators (308) are required. In addition, when the average number of taps per partial filter (304) is 3, the number of taps of the interpolation filter (302) that is the source is about 750 taps (= 3 * 256).

  For example, assuming that 1 byte (= 8 bits) of memory is required to store one tap coefficient, the amount of memory for storing all the tap coefficients is about 750 bytes. In order to obtain a sharp cut-off characteristic of the interpolation filter in order to obtain high image quality, it is necessary to greatly increase the number of taps of each partial filter (304).

  For example, if the average number of taps per partial filter (304) is 10, the number of taps of the interpolation filter (302) that is the base is about 2500 taps (= 10 * 256), so the tap coefficient is stored. The memory for will be about 2500 bytes. Furthermore, in order to correspond to a plurality of magnifications (U / D), it is necessary to prepare a plurality of sets of tap coefficients of each partial filter (304).

  For example, when U = 256 is fixed and 10 types of 256 / D times D are prepared to correspond to 10 types of magnification, the number of taps is 10 times the number of taps described above (= about 25000 taps) A large amount of memory (= about 25000 bytes) is required to store

  The second problem is that it is difficult to continuously change the magnification (U / D). For example, in a television receiver, as shown in FIG. 15, an image with an aspect ratio of 16: 9 is (a) an input image, and only the center portion of 12: 9 (= 4: 3) is extracted in the horizontal direction. There is a mode to enlarge and display as a 16: 9 (b) output image. At this time, a mode is often used in which the central portion of the screen is not enlarged so much that it is greatly enlarged as it approaches the edge of the screen. At this time, it is necessary to continuously change the enlargement ratio (U / D) according to the horizontal position of the output pixel. However, in the configuration of the prior art shown in FIG. 3, the partial filter (304) or tap coefficient is generated. It is necessary to continuously change the number of containers (308) according to the value of U, which makes mounting difficult. In addition, if the candidate for the enlargement ratio (U / D) to be used is determined in advance, the maximum value of U is determined in advance, and the corresponding partial filter (304) or tap coefficient generator (308) is prepared, the implementation will be Although it is possible, in this case, it is difficult to immediately cope with an arbitrary enlargement ratio (U / D).

  The third problem is that the operations of the control units (306) and (310) become complicated, and the processing amount of hardware and software for realizing this control becomes large. In the example of the television receiver described above, the value of U changes according to the horizontal position of the output pixel. Therefore, the number of partial filters changes for each pixel, and the control method of the control units (306) and (310) that control the switches (305) and (309) needs to be changed in a complicated manner according to the horizontal position. In addition, when handling an image signal with a very large number of pixels per unit time, such as HDTV (High Definition TV), the characteristics of the partial filter (304) or tap coefficient generator (308) described above can be reduced for a short time. It is necessary to change in this, and the implementation becomes difficult. For example, when the number of pixels per screen (frame) is horizontal 1920 pixels × vertical 1080 pixels, and the frame rate is 60 fps (frame per second), the processing time per pixel is 8 nanoseconds (= 1 / (1920 * 1080 * 60) seconds), and it is necessary to switch the partial filter (304) or the tap coefficient generator (308) during this period, which makes implementation difficult especially when the number of taps is large.

  The present invention has been made in view of the above-described problems of the prior art, and an object thereof is to provide a technique for suitably converting a sampling rate.

  In order to achieve the above object, an embodiment of the present invention may be configured as described in the claims, for example.

  According to the present invention, it is possible to convert the sampling rate more suitably.

  Embodiments of the present invention will be described below with reference to the drawings.

Moreover, in each drawing, the component to which the same code | symbol was attached | subjected shall have the same function.

  In addition, the expression “pixel” in each description and each drawing of this specification also includes the meaning of a sample signal or a sample point, such as a digitized audio signal sequence or a continuous data sequence obtained from a sensor or the like. Including meaning.

  Embodiments of the present invention will be described below with reference to the drawings.

  FIG. 1 shows a sampling rate conversion apparatus according to Embodiment 1 of the present invention, and its features will be described. The sampling rate conversion apparatus according to this embodiment is applied to an image display apparatus such as a television receiver. In the following description of the present embodiment, an image display device will be described as an example of the sampling rate conversion device.

  In FIG. 1, the sampling rate conversion apparatus according to the present embodiment includes an input unit (1) to which a frame sequence of a moving image such as a television broadcast signal is input, and a frame input from the input unit (1). A sampling rate conversion unit (2) for converting the number of pixels in a one-dimensional direction, and a display unit (3) for displaying an image based on the frame in which the number of pixels is converted by the sampling rate conversion unit (2) I have. As the display unit (3), for example, a plasma display panel, a liquid crystal display panel, or an electron / field emission display panel is used. Details of the sampling rate conversion unit (2) will be described below.

  In FIG. 1, a signal input from the input unit (1) is input to a filter A (101) and a filter B (102). Here, the filter A (101) is a one-dimensional filter in which the number of taps is an odd number and the tap coefficients are symmetrical in the left-right (or up-and-down) direction. The filter B (102) is a one-dimensional filter having an even number of taps and symmetrical tap coefficients. Here, the outputs of the filters A (101) and B (102) are input to the multipliers (105) and (106), respectively.

  On the other hand, based on the information indicating the position of the pixel to be interpolated generated by the control unit (103) (interpolation position information (108)), the coefficient generation unit (104) is a coefficient (1-k) that is a mixture ratio And coefficient k. Multipliers (105) and (106) multiply the coefficients (1-k) and the coefficients k, respectively, by the outputs of the filters A (101) and B (106). Lastly, the adder (107) adds the outputs of the multiplier (105) and the multiplier (106) to obtain an output signal of the sampling rate converter (2). That is, the tap coefficients of the filter A (101) and the filter B (102) are fixed, and the output of each filter is weighted and added (mixed) according to the position of the pixel to be interpolated. Thereby, arbitrary sampling rate conversion is realizable. Detailed operation of each part will be described later.

  FIGS. 5A and 5B show examples of filter A (101) (odd tap symmetric filter) and filter B (even tap symmetric filter), respectively. In each figure, the horizontal axis indicates the pixel position, and the vertical axis indicates the value of the pixel at the corresponding position and the magnitude of the tap coefficient value for performing the convolution operation, and indicates the impulse response of each filter. FIG. 5A shows a 5-tap filter, which is composed of symmetrical tap coefficients (501-a) with the symmetry center (502-a) as the axis of symmetry. In other words, the impulse response is left-right symmetric and the axis of symmetry is located at (502-a). FIG. 5B shows a 4-tap filter, which is a filter composed of symmetrical tap coefficients (501-b) with the symmetry center (502-b) as the axis of symmetry. That is, the impulse response is a filter that is bilaterally symmetric and whose axis of symmetry is located at (501-b). The filter used in the present invention is not limited to the number of taps and tap coefficients shown in FIG.

  FIG. 6 shows the operation of the coefficient generation unit (104). In the figure, the horizontal axis indicates the pixel position, and shows the operation when generating the output pixel (602) located between the real pixel (601-1) and the real pixel (601-2). At this time, the position (coordinates) of the real pixel (601-1) is x1, the position of the real pixel (601-2) adjacent to the right of the real pixel (601-1) is (x1 + 1), and the output pixel (602 ) Is x2, and the relationship x1 ≦ x2 <(x1 + 1) is satisfied.

  Here, as shown in FIG. 5A, the symmetrical center of the filter A (101) is located at (502-a) or (605), and the real pixel (601-1) or real pixel (601- It matches the pixel position of 2). Further, the center of symmetry of the filter B (102) is located at (502-b), which is the pixel position just in the middle between the real pixel (601-1) and the real pixel (601-2). In the coefficient generation unit (104), when the interval between the symmetric center position of the filter A and the symmetric center position of the filter B is 1.0 (reference), the symmetric center of the output pixel (602) and the filter A (502- The interval a) is output as a coefficient k (603), and the interval between the output pixel (602) and the symmetric center (502-b) of the filter B is output as a coefficient (1-k) (604). At this time, since a relationship of x2-x1: (x1 + 1-x1) /2=k:1.0 is satisfied, k = 2.0 * (x2-x1) may be set.

  Similarly, when the position of the output pixel (602) is closer to the actual pixel (601-2) than the actual pixel (601-1), the symmetrical center (605) of the filter A closer to the output pixel (602). Is output as a coefficient k, and an interval between the output pixel (602) and the center of symmetry of the filter B (502-b) is output as a coefficient (1-k). Using the coefficients k and (1-k) generated in this manner, the multipliers (105) and (106) shown in FIG. 1 are operated.

  FIG. 17 shows an example of a flowchart of this coefficient generation method. The flowchart of FIG. 17 starts from Step (1701), and in Step (1702), the position x2 of the output pixel (602), and the center position of the real pixel (601-1) and the real pixel (601-2) ( Compare x1 + 0.5). Here, in the case of (x2 ≦ x1 + 0.5), that is, the position x2 of the output pixel (602) is on the left side of the center position (x1 + 0.5) of the real pixel (601-1) and the real pixel (601-2). In this case, the process proceeds to step (1703), k = 2.0 * (x2-x1) is output, and the process ends at step (1705). On the other hand, in the case of (x2> x1 + 0.5), that is, the position x2 of the output pixel (602) is on the right side of the center position (x1 + 0.5) between the real pixel (601-1) and the real pixel (601-2). In this case, the process proceeds to step (1704), k = 2.0 * (x1 + 1−x2) is output, and the process ends in step (1705). The coefficient (1-k) may be obtained by subtracting the above-described coefficient k from 1. In the case where the position x2 of the output pixel (602) is the position exactly in the middle between the real pixel (601-1) and the real pixel (601-2) (that is, when x2 = x1 + 0.5), step (1703) In either case (1704), k becomes the same value (1.0).

  As an example, FIG. 7 shows an operation when the coefficient generation method shown in FIGS. 6 and 17 is applied to sampling rate conversion of 8/3 times. FIG. 7A shows the positions of the actual pixels (201-1), (201-2), (201-3) and (201-4) in the input signal, and FIG. 7 (b) shows the output pixels in the output signal. The positions of (204-4), (204-5), (204-6), and (204-7) are shown. At this time, the center of symmetry of the filter A coincides with the positions of the real pixels (201-2) (201-3) and the like, so that the positions of (702-a) and (703-a) and the like are located. On the other hand, the center of symmetry of the filter B is a position such as (702-b) because the center of the real pixel (201-2) (201-3) is just the center. Therefore, when the coefficient generation method shown in FIGS. 6 and 17 is applied, as shown in FIG. 6B, when calculating the output pixel (204-4), k = 0.25 (701-4), the output When calculating pixel (204-5), k = 1.0 (701-5), when calculating output pixel (204-6), k = 0.25 (701-6), output pixel (204-7) K = 0.5 (701-7) when calculating.

  Next, in FIGS. 9 (a) and 9 (b), when the sampling rate conversion of the present invention is applied to enlargement processing, that is, when U> D and U> D, a filter A (101) and a filter B (102 ) Shows an example of each frequency characteristic. In both (a) and (b) of the figure, the horizontal axis indicates the frequency, and the vertical axis indicates the gain, from DC (= frequency = 0) to Nyquist frequency (901) (= 1/2 of the sampling frequency). The gain of each filter is shown. Here, by making the frequency characteristic (902) of the filter A (101) in the same figure (a) and the frequency characteristic (904) of the filter B (102) in the same figure (b) substantially the same, The frequency characteristic resulting from the weighted addition of the output according to the value of the coefficient k is substantially the same as the frequency characteristics (902) and (904) regardless of the value of the coefficient k. That is, only the position (phase) of the output pixel can be changed by the value of the coefficient k while keeping the frequency characteristics substantially the same regardless of the position of the output pixel.

  In the even-tap symmetric filter (filter B), the gain is always 0 at the Nyquist frequency (901), but in the odd-tap symmetric filter (filter A), the gain is not necessarily 0 even at the Nyquist frequency (901). Design with flexibility. At this time, if the gain of the filter A at the Nyquist frequency (901) is set to a value close to 0, the characteristics of the filter B are substantially the same, and the occurrence of chessboard distortion as shown in Patent Document 1 described above is generated. Can be prevented. On the other hand, if the gain of the filter A at the Nyquist frequency (901) is a value close to 1, it is possible to pass even high-frequency components, so the output after sampling rate conversion can be a high-definition image become. Accordingly, since both are in a trade-off relationship, it is preferable to design the frequency characteristics of the filter A and the filter B so as to be optimal while confirming the image quality of the output image.

  10 (a) and 10 (b), when the sampling rate conversion of the present invention is applied to the reduction process, that is, each of the filters A (101) and B (102) when U <D at U / D times. An example of a frequency characteristic is shown. In both figures (a) and (b), the horizontal axis indicates the frequency, and the vertical axis indicates the gain, from DC (= frequency = 0) to Nyquist frequency (1001) (= 1/2 of the sampling frequency). The gain of each filter is shown. Here, by making the frequency characteristic (1002) of the filter A (101) in the same figure (a) and the frequency characteristic (1004) of the filter B (102) in the same figure (b) substantially the same, The frequency characteristic resulting from the weighted addition of the output by the value of the coefficient k is substantially the same as the frequency characteristics (1002) and (1004) regardless of the value of the coefficient k. That is, it is possible to change only the position (phase) of the output pixel by the value of the coefficient k while keeping the frequency characteristics substantially the same regardless of the position of the output pixel. The difference between (a) filter A and (b) filter B used in the reduction process is that the cutoff frequency (1003) (1005) indicating the upper limit of the passband is the Uy of Nyquist frequency. It is to set it near the frequency of / D times.

  Here, if the cutoff frequency (1003) (1005) is set lower (closer to 0) than the U / D times the Nyquist frequency, the high frequency component is attenuated and the output after sampling rate conversion is The image becomes blurred. On the other hand, if the cutoff frequency (1003) (1005) is set higher than the frequency that is U / D times the Nyquist frequency (= closer to the Nyquist frequency), components that exceed the U / D times the Nyquist frequency This causes aliasing and causes moiré. Accordingly, since both are in a trade-off relationship, it is preferable to design the frequency characteristics of the filter A and the filter B so as to be optimal while confirming the image quality of the output image.

  In FIG. 8, as an example, when the sampling rate is expanded to 8/3, that is, when U = 8, D = 3, the output pixel number, the value of coefficient k, the tap coefficients of filter A and filter B, and Input pixel numbers are shown together. Here, the output pixel number in the figure is the last number of (d) output pixels (204-1), (204-2), (204-3)... In FIG. By setting the integer n shown in FIG. 8 to 0, 1, 2,..., One cycle is formed for each U (= 8) output pixels. The value of the coefficient k in the figure is obtained from the positional relationship between the actual pixel and the output pixel based on the method shown in FIGS. 6 and 17, and the value of the coefficient k shown in FIG. It has been extended to. The tap coefficients of the filter A and the filter B in the figure are an example in which a characteristic close to the frequency characteristic shown in FIGS. 9A and 9B is specifically realized. The input pixel number in the figure is the last number of (a) input signals (201-1), (201-2), (201-3)... In FIG. By setting the integer n shown in FIG. 8 to 0, 1, 2,..., One period is formed for every D (= 3) real pixels.

  Here, the five numbers in parentheses {} shown in the “A” column in the input pixel number indicate the number of the pixel that performs the convolution operation with each tap coefficient of the filter A composed of five taps. Also, the four numbers in parentheses {} shown in the “B” column in the input pixel number indicate the number of the pixel that performs the convolution operation with each tap coefficient of the filter B composed of four taps. This input pixel number is located so that the position of the real pixel closest to the position of the output pixel determined from the magnification rate (U / D) of the sampling rate is the center of symmetry of the filter A and sandwiches the output pixel 2 The center of one real pixel is determined to be the center of symmetry of the filter B. The present invention is not limited to the number of taps and tap coefficients of filter A and filter B shown in FIG. For example, if the number of taps is further increased to make the cut-off characteristic steep, the filter can pass up to a higher frequency component, and a high-definition output image can be obtained. Also, in the case of other magnifications, that is, other U and D values, each value may be determined by the same method as described above.

  An image signal processing method for realizing a process equivalent to the sampling rate conversion apparatus shown in FIG. 1 by a control unit cooperating with software will be described with reference to FIGS.

  First, a sampling rate conversion apparatus for realizing the image signal processing method according to the present embodiment will be described with reference to FIG. The sampling rate conversion apparatus shown in FIG. 14 stores, for example, an input unit (1) to which an image signal such as a television broadcast signal is input, and software for processing a signal input from the input unit (1). A control unit (10) that performs image signal processing on a signal input from the input unit (1) in cooperation with the unit (11) and software stored in the storage unit (11), and an input unit (1) Buffer # 1 (21) for temporarily storing data inputted from the buffer, buffer # 2 (22) for temporarily storing data in the sampling rate conversion processing by the control unit (10), buffer # 3 (23) and a buffer # 4 (24) for temporarily storing the signal after the sampling rate conversion processing output from the control unit (10) to the output unit (3).

  Here, buffer # 1 (21), buffer # 2 (22), buffer # 3 (23), buffer # 4 (24) for temporarily storing data, and storage unit (11) for storing software are respectively However, each memory address may be divided and used by using one or a plurality of memory chips.

  In addition, in the figure, for the image signal input from the input unit (1), the control unit (10) performs a sampling rate conversion process in cooperation with software stored in the storage unit (11), and the display unit ( Output to 3). Details of the sampling rate conversion processing will be described with reference to FIG.

  FIG. 20 shows an example of a flowchart of the sampling rate conversion method according to the present embodiment. The flowchart of FIG. 20 starts from step (2001), and the input signal (pixel) is written to buffer # 1 (21) in step (2002). At this time, the input signal (pixel) may be written to the buffer for each frame constituting the input image, or the input signal (pixel) may be written to the buffer for each block obtained by dividing the input image into small regions. Subsequently, in step (2003), filter A (101) is applied to the signal in buffer # 1 (21), and the output is written in buffer # 2 (22). On the other hand, in step (2004), filter B (102) is applied to the signal in buffer # 1 (21), and the output is written in buffer # 3 (23). Here, as the characteristics of the filter A (101) and the filter B (102), the characteristics described with reference to FIG. 5, FIG. 8, FIG. 9, and FIG. .

  In step (2005), a coefficient k is generated for each output signal (pixel) based on the positional relationship between the output signal (pixel) and the input signal (pixel). As the coefficient generation method at this time, since the methods shown in FIGS. 6, 17, and 7 can be used as they are, illustration and description thereof are omitted. In the following step (2006), the signals of buffer # 2 (22) and buffer # 3 (23) are weighted and added to buffer # 4 (24) according to the value of coefficient k obtained as described above. Output and end in step (2007). As shown in FIG. 1, the weighted addition method at this time is obtained by multiplying the output of the filter A (101) (= buffer # 2 (22)) by the coefficient (1-k) and the filter B (102). The output (= buffer # 3 (23)) is multiplied by a coefficient k, and both are added to generate an output signal.

  Since steps (2003), (2004), and (2005) are independent from each other, the steps may be processed in order, or the steps may be processed simultaneously in parallel. With the processing steps described above, the sampling rate conversion method according to this embodiment can be implemented.

  FIG. 15 shows an application example in which an image is enlarged in the horizontal direction while continuously changing the magnification (U / D) using the sampling rate conversion apparatus or the sampling rate conversion method according to the present embodiment. In this figure, an image with an aspect ratio of 16: 9 is used as the input image (a), and only the center 12: 9 (= 4: 3) is extracted and enlarged in the horizontal direction. The operation of displaying as an output image is shown. In particular, the figure shows an operation in which the central portion of the screen does not expand so much and greatly expands as it approaches the edge of the screen. At this time, the enlargement ratio (U / D) continuously changes in accordance with the horizontal position of the output pixel.

  Here, it is assumed that the relationship between the pixel positions of the input image and the output image is as shown in FIG. That is, as an example, the enlargement ratio is 1.7 times (U = 17, D = 10) at the left edge of the screen, 1.0 times (U = 1, D = 1) at the center of the screen, and 1.7 times (U = 17 and D = 10). Also, if (b) the number of pixels in the horizontal direction of the output image is, for example, 1920 pixels, (b) the pixel position (pixel number) on the frame of the output image is 1 at the left end, 960 at the center, and 1920 at the right end . Similarly, if the original horizontal pixel count of the input image is also 1920 pixels, only the 12: 9 (= 4: 3) portion in the center of the screen is extracted, so (b) corresponds to the left edge of the output image (a) The pixel position of the input image is 241, (b) corresponding to the center of the output image (a) the pixel position of the input image is 960, (b) the pixel position of the input image corresponding to the right end of the output image is 1680 Become. At this time, since the enlargement ratio is 1.7 at the left end of the screen, (b) the interval of one pixel on the frame of the output image corresponds to (a) 1.7 pixels on the frame of the input image. Accordingly, (b) the pixel position of the input image corresponding to (b) the pixel position of the output image = 2 is 242.7. Similarly, (b) all the pixel positions of the input image corresponding to the pixel positions of the output image are obtained, and the results are stored in a table in advance.

  In the sampling rate converter (2) shown in FIG. 1, the control unit (103) generates the interpolation position information (108) according to the contents of the table, and the coefficient generation unit (104) is based on the flowchart shown in FIG. When the coefficient k is generated and the outputs of the filters A (101) and B (102) are weighted and added using the multipliers (105) and (106) and the adder (107), as shown in FIG. (b) An output image can be easily obtained from an input image.

  As described above, the sampling rate conversion apparatus and the sampling rate conversion method according to the first embodiment can realize arbitrary sampling rate conversion only by weighted addition of outputs of two filters having fixed characteristics. This has the effect of reducing the hardware scale and the amount of software processing.

  In addition, even when the sampling rate magnification (U / D) changes, if the magnification is U, ie, U> D at U / D times, each advantage of filter A (101) and filter B (102) Since the sampling rate conversion process can be realized simply by changing the operation of the control unit (103) and the coefficient generation unit (104) according to the interpolation position information (108) while Also, there is an effect that it becomes easy to continuously change the magnification (U / D).

  Further, since it is not necessary to change the characteristics of the filter A (101) and the filter B (102) for each output pixel, there is an effect that the operations of the control units (103) and (10) can be simplified as compared with the prior art.

  Next, the configuration and operation of the second embodiment according to the sampling rate conversion apparatus of the present invention will be described with reference to FIGS. The present embodiment is a configuration example of a sampling rate conversion apparatus in which the above-described sampling rate conversion in the one-dimensional direction is expanded to two dimensions in the horizontal direction and the vertical direction.

  In FIG. 12, the sampling rate conversion apparatus according to the present embodiment includes an input unit (1) to which a frame sequence of a moving image such as a television broadcast signal is input, and a frame input from the input unit (1). A sampling rate converter (4) for converting the number of pixels in a two-dimensional direction, and a display unit (3) for displaying an image based on the frame in which the number of pixels is converted by the sampling rate converter (4). I have. As the display unit (3), for example, a plasma display panel, a liquid crystal display panel, or an electron / field emission display panel is used. Details of the sampling rate conversion unit (4) will be described below.

  In FIG. 12, the signal input from the input unit (1) is subjected to two-dimensional sampling rate conversion by the vertical processing unit (1216) and the horizontal processing unit (1217).

  First, in the vertical processing unit (1216), signals are input to the vertical filter V-A (1201) and the vertical filter V-B (1202). Here, the vertical filter V-A (1201) is a one-dimensional filter having an odd number of taps and symmetrical tap coefficients. The vertical filter V-B (1202) is a one-dimensional filter having an even number of taps and symmetrical tap coefficients. Here, the outputs of the vertical filter V-A (1201) and the vertical filter V-B (1202) are input to the multipliers (1205) (1206), respectively.

  On the other hand, based on the information indicating the vertical position of the pixel to be interpolated generated by the control unit (1203) (vertical interpolation position information (1208)), the coefficient generation unit (1204) includes the coefficient (1-kV) and the coefficient Generate kV. Multipliers (1205) and (1206) multiply the coefficients (1-kV) and the coefficients kV, respectively, by the outputs of the vertical filter V-A (1201) and the vertical filter V-B (1206). Next, the adder (1207) adds the outputs of the multiplier (1205) and the multiplier (1206) to obtain an output signal of the vertical processing unit (1216).

  Subsequently, the horizontal processing unit (1217) inputs a signal to the horizontal filter H-A (1209) and the horizontal filter H-B (1210). Here, the horizontal filter HA (1209) is a one-dimensional filter having an odd number of taps and symmetrical tap coefficients. Further, the horizontal filter H-B (1210) is a one-dimensional filter having an even number of taps and symmetrical tap coefficients. Here, the outputs of the horizontal filter H-A (1209) and the horizontal filter H-B (1210) are input to multipliers (1212) and (1213), respectively.

  On the other hand, based on the information indicating the horizontal position of the pixel to be interpolated generated by the control unit (1203) (horizontal interpolation position information (1215)), the coefficient generation unit (1211) includes the coefficient (1-kH) and the coefficient Generate kH. Multipliers (1212) and (1213) multiply the coefficients (1-kH) and the coefficients kH, respectively, by the outputs of the horizontal filter H-A (1209) and the horizontal filter H-B (1210). Next, the adder (1214) adds the outputs of the multipliers (1212) and (1213) to obtain the output signal of the horizontal processing unit (1216). The output signal becomes the output of the sampling rate converter (4).

  That is, the tap coefficients of the vertical filter VA (1201), the vertical filter VB (1202), the horizontal filter HA (1209), and the horizontal filter HB (1210) are fixed, and each filter has a fixed coefficient depending on the position of the pixel to be interpolated. Arbitrary two-dimensional sampling rate conversion is realized by weighted addition of outputs. Similar results can be obtained even if the processing order of the vertical processing unit (1216) and the horizontal processing unit (1217) is reversed.

  Referring to FIG. 11, real pixels (1101-1) (1101-2) (1101-3) (1101-4), output pixels (1102), positions of symmetrical centers of the filters, and the weighted addition The relationship between the coefficient kH (1105) and the coefficient kV (1106) will be described. First, based on the position (x1, y1) of the real pixel (1101-1), the position of the real pixel (1101-2) adjacent to the right is (x1 + 1, y1), the real pixel (1101-1) The position of the real pixel (1101-3) adjacent to the bottom of (x1, y1 + 1) and the position of the real pixel (1101-4) adjacent to the bottom right of the real pixel (1101-1) (x1 + 1) , y1 + 1), and the position of the output pixel (1102) is (x2, y2). At this time, it is assumed that the relationship of x1 ≦ x2 <(x1 + 1) and y1 ≦ y2 <(y1 + 1) is satisfied.

  Here, four one-dimensional filters, that is, an odd-numbered tap symmetric filter HA in the horizontal direction, an even-numbered tap symmetric filter HB in the horizontal direction, an odd-numbered tap symmetric filter VA in the vertical direction, and an even-numbered tap symmetric filter VB in the vertical direction are prepared. . The center of symmetry of the odd-numbered tap symmetrical filter HA in the horizontal direction is located at (1103-a) or (1107), and the real pixel (1101-1) (1101-3) or real pixel (1101-2) (1101-4 ) Matches the horizontal position. Further, the center of symmetry of the even-tap symmetric filter HB in the horizontal direction is located at (1103-b), and the real pixel (1101-1) and real pixel (1101-2), or real pixel (1101-3) and real The horizontal position is just in the center of the pixel (1101-4). The center of symmetry of the odd-numbered tap symmetric filter VA in the vertical direction is located at (1104-a) or (1108), and the real pixels (1101-1) (1101-2) or real pixels (1101-3) (1101 It matches the horizontal position of -4). The center of symmetry of the even-tap symmetric filter VB in the vertical direction is located at (1104-b), and the real pixel (1101-1) and real pixel (1101-3), or the real pixel (1101-2) and real The vertical position is just in the center of the pixel (1101-4).

  Here, in the coefficient generation unit (1204), the vertical interval between the symmetric center position of the vertical filter VA (1104-a) and the symmetric center position of the vertical filter VB (1104-b) is 1.0 (reference). The interval between the output pixel (1102) and the symmetric center (1104-a) of the vertical filter VA is output as a coefficient kV (1106). At this time, kV = 2.0 * (y2-y1) may be set based on the relationship y2-y1: (y1 + 1-y1) /2=kV:1.0. Similarly, when the vertical position of the output pixel (1102) is closer to the real pixel (1101-3) than the real pixel (1101-1), the center of symmetry of the vertical filter VA closer to the output pixel (1102) ( The interval of 1101-3) is output as a coefficient kV. Further, the vertical interval between the output pixel (1102) and the symmetrical center (502-b) of the vertical filter V-B is output as a coefficient (1-kV). Using the coefficients kV and (1-kV) generated in this way, the multipliers (1205) and (1206) shown in FIG. 12 are operated.

  Similarly, in the coefficient generator (1211), the horizontal interval between the symmetrical center position of the horizontal filter HA (1103-a) and the symmetrical center position of the horizontal filter HB (1103-b) is 1.0 (reference). The interval between the output pixel (1102) and the symmetrical center (1103-a) of the horizontal filter HA is output as a coefficient kH (1105). At this time, kH = 2.0 * (x2-x1) may be set according to the relationship of x2-x1: (x1 + 1-x1) /2=kH:1.0. Similarly, when the horizontal position of the output pixel (1102) is closer to the actual pixel (1101-2) than to the actual pixel (1101-1), the center of symmetry of the horizontal filter HA closer to the output pixel (1102) ( 1101-2) is output as a coefficient kH. Further, the horizontal interval between the symmetrical center (1103-b) of the output pixel (1102) and the horizontal filter H-B is output as a coefficient (1-kH). Using the coefficients kH and (1-kH) generated in this way, the multipliers (1212) and (1213) shown in FIG. 12 are operated.

  FIG. 18 shows an example of a flowchart of this coefficient generation method. The flowchart of FIG. 18 starts from step (1801), and in step (1802), the vertical position y2 of the output pixel (1102) and the vertical center of the real pixel (1101-1) and the real pixel (1101-3). Compare position (y1 + 0.5). Here, in the case of (y2 ≦ y1 + 0.5), that is, the vertical position y2 of the output pixel (1102) is greater than the vertical position (y1 + 0.5) at the center of the real pixel (1101-1) and the real pixel (1101-3) If it is on the upper side, the process proceeds to step (1803), outputs kV = 2.0 * (y2-y1), and proceeds to step (1805). On the other hand, when (y2> y1 + 0.5), that is, the vertical position y2 of the output pixel (1102) is lower than the center position (y1 + 0.5) of the real pixel (1101-1) and the real pixel (1101-3). In the case of the side, the process proceeds to Step (1804), outputs kV = 2.0 * (y1 + 1-y2), and proceeds to Step (1805).

  Subsequently, in step (1805), the horizontal position x2 of the output pixel (1102) is compared with the horizontal position (x1 + 0.5) of the center between the real pixel (1101-1) and the real pixel (1101-2). Here, in the case of (x2 ≦ x1 + 0.5), that is, the horizontal position x2 of the output pixel (1102) is greater than the horizontal position (x1 + 0.5) at the center between the real pixel (1101-1) and the real pixel (1101-2). In the case of the left side, the process proceeds to step (1806), outputs kH = 2.0 * (x2-x1), and ends in step (1808). On the other hand, when (x2> x1 + 0.5), that is, the horizontal position x2 of the output pixel (1102) is on the right side of the center position (x1 + 0.5) of the real pixel (1101-1) and the real pixel (1101-2) In this case, the process proceeds to step (1807), kH = 2.0 * (x1 + 1−x2) is output, and the process ends at step (1808).

  The coefficient (1-kV) or coefficient (1-kH) may be obtained by subtracting the value of the coefficient kV or coefficient kH from 1 respectively.

  Also, if the vertical position y2 of the output pixel (1102) is the vertical position just in the middle between the real pixel (1101-1) and the real pixel (1101-3) (that is, if y2 = y1 + 0.5), the step ( Regardless of which of (1803) and (1804), kV becomes the same value (1.0).

  Also, if the horizontal position x2 of the output pixel (1102) is the horizontal position just in the middle between the real pixel (1101-1) and the real pixel (1101-2) (i.e., x2 = x1 + 0.5), the step ( Regardless of which of (1806) and (1807), kH becomes the same value (1.0).

  The frequency characteristics of the vertical filter VA (1201) and the vertical filter VB (1202) shown in FIG. 12 are simply replaced with the vertical frequency characteristics of the frequency characteristics of the filters A and B described with reference to FIGS. Therefore, illustration and description are omitted. Similarly, the frequency characteristics of the horizontal filter HA (1209) and the horizontal filter HB (1210) shown in FIG. 12 replace the frequency characteristics of the filters A and B described with reference to FIGS. 9 and 10 with horizontal frequency characteristics. Therefore, illustration and description are omitted.

  For specific examples of the output pixel number, coefficient kV and coefficient kH, the tap coefficient of each filter, and the input pixel number in the two-dimensional sampling rate conversion, obtain each value in the one-dimensional sampling rate conversion described with reference to FIG. Since this method only needs to be applied to the horizontal direction and the vertical direction, illustration and description are omitted.

  Further, it is clear that the horizontal magnification and the vertical magnification (that is, the values of U and D in each direction) can be set independently, and as described with reference to FIGS. 9 and 10, A suitable filter frequency characteristic may be used according to the magnification in each direction.

  An image signal processing method for realizing processing equivalent to the sampling rate conversion apparatus shown in FIG. 12 by a control unit cooperating with software will be described with reference to FIGS.

  First, a sampling rate conversion apparatus for realizing the image signal processing method according to the present embodiment will be described with reference to FIG. The sampling rate converter shown in FIG. 19 stores, for example, an input unit (1) to which an image signal such as a television broadcast signal is input, and software for processing a signal input from the input unit (1). A control unit (10) that performs image signal processing on a signal input from the input unit (1) in cooperation with the software stored in the storage unit (11), and the input unit (1) Buffer # 1 (31) for temporarily storing input data, buffer # 2 (32) for temporarily storing data in the sampling rate conversion process by the control unit (10), buffer # 3 (33), buffer # 4 (34), buffer # 5 (35), buffer # 6 (36), and the signal after sampling rate conversion processing output from the control unit (10) to the output unit (3) Buffer # 7 (37) for temporary storage.

  Here, buffer # 1 (31), buffer # 2 (32), buffer # 3 (33), buffer # 4 (34), buffer # 5 (35), buffer # 6 (36 ), The buffer # 7 (37) and the storage unit (11) for storing software may be configured using individual memory chips, or each data address using one or a plurality of memory chips. May be configured to be used in a divided manner.

  In the figure, for the image signal input from the input unit (1), the control unit (10) performs a sampling rate conversion process in cooperation with software stored in the storage unit (11), and the display unit (3) Output to. Details of the sampling rate conversion processing will be described with reference to FIG.

  FIG. 21 shows an example of a flowchart of the sampling rate conversion method according to the present embodiment. The flowchart of FIG. 21 starts from step (2101), and in step (2102), an input signal (pixel) is written to buffer # 1 (31). At this time, the input signal (pixel) may be written to the buffer for each frame constituting the input image, or the input signal (pixel) may be written to the buffer for each block obtained by dividing the input image into small regions.

  Subsequently, in step (2103), the vertical filter V-A (1201) is applied to the signal in the buffer # 1 (31), and the output is written in the buffer # 2 (32). On the other hand, in step (2104), the vertical filter V-B (1202) is applied to the signal in the buffer # 1 (31), and the output is written in the buffer # 3 (33). Here, the characteristics of the vertical filter VA (1201) and the vertical filter VB (1202) are used as they are by replacing the characteristics described with reference to FIGS. 5, 8, 9, and 10 with the characteristics in the vertical direction. Therefore, illustration and description are omitted. In step (2105), a coefficient kV is generated for each output signal (pixel) based on the vertical positional relationship between the output signal (pixel) and the input signal (pixel). As the coefficient generation method at this time, the methods shown in FIGS. 11 and 18 can be used as they are, and thus illustration and description thereof are omitted.

  In the following step (2106), each signal of buffer # 2 (32) and buffer # 3 (33) is weighted and added to buffer # 4 (34) according to the value of coefficient kV obtained as described above. Write. As shown in FIG. 12, the weighted addition method at this time is obtained by multiplying the output (= buffer # 2 (32)) of the vertical filter VA (1201) by a coefficient (1-kV), and obtaining the vertical filter VB (1202 ) (= Buffer # 3 (33)) is multiplied by coefficient k, and both are added to obtain a signal to be written to buffer # 4 (34).

  Subsequently, in step (2107), the horizontal filter HA (1209) is applied to the signal in the buffer # 4 (34), and the output is written in the buffer # 5 (35). On the other hand, in step (2108), the horizontal filter H-B (1210) is applied to the signal in the buffer # 4 (34), and the output is written in the buffer # 6 (36). Here, the characteristics of the horizontal filter HA (1209) and the horizontal filter HB (1210) are used as they are by replacing the characteristics described with reference to FIGS. 5, 8, 9, and 10 with the characteristics in the horizontal direction. Therefore, illustration and description are omitted. In step (2109), a coefficient kH is generated for each output signal (pixel) based on the horizontal positional relationship between the output signal (pixel) and the input signal (pixel). As the coefficient generation method at this time, the methods shown in FIGS. 11 and 18 can be used as they are, and thus illustration and description thereof are omitted.

  In the following step (2110), the signals of buffer # 5 (35) and buffer # 6 (36) are weighted and added to buffer # 7 (37) according to the value of coefficient kH obtained as described above. Write and end at step (2111).

  As shown in FIG. 12, the weighted addition method at this time is obtained by multiplying the output (= buffer # 5 (35)) of the horizontal filter HA (1209) by a coefficient (1-kH), thereby obtaining the horizontal filter HB (1210 ) (= Buffer # 6 (36)) is multiplied by a coefficient kH, and both are added. A signal obtained by the weighted addition is used as an output signal.

  Steps (2103), (2104), (2105), and steps (2107), (2108), and (2109) are independent of each other. You may process in parallel simultaneously. Also, the processing order of the steps (2103) (2104) (2105) (2106) for performing the processing in the vertical direction and the steps (2107) (2108) (2109) (2110) for performing the processing in the horizontal direction may be changed. Similar results can be obtained. With the processing steps described above, the sampling rate conversion method according to this embodiment can be implemented.

  As described above, the sampling rate conversion apparatus and the sampling rate conversion method according to the second embodiment use a vertical filter and a horizontal filter having fixed characteristics, and perform a weighted addition of the respective outputs to obtain any two-dimensional arbitrary Since sampling rate conversion can be realized, the hardware scale and the amount of software processing can be reduced compared to the prior art.

  Even if the magnification (U / D) of the sampling rate is changed, in the case of enlargement, that is, in the case of U> D at U / D times, filter VA (1201), filter VB (1202), filter HA ( 1209), the operation of the control unit (1203) and coefficient generation unit (1204) (1211) is changed according to the interpolation position information (1208) (1215), with the characteristics of the filter HB (1210) fixed. Since a two-dimensional sampling rate conversion process can be realized, there is an effect that it is easier to continuously change the magnification (U / D) than in the conventional technique.

  In addition, since it is not necessary to change the characteristics of the filter VA (1201), the filter VB (1202), the filter HA (1209), and the filter HB (1210) for each output pixel, the control unit (1203) ( There is an effect that the operation of 10) can be simplified.

  Next, the configuration and operation of the third embodiment according to the sampling rate conversion apparatus of the present invention will be described with reference to FIG. The present embodiment is a configuration example of a sampling rate conversion device that is modified to cope with complicated deformation such as image rotation, for example, by modifying the configuration of the two-dimensional sampling rate conversion described with reference to FIG. .

  In FIG. 13, the sampling rate conversion apparatus according to the present embodiment includes an input unit (1) to which a frame sequence of a moving image such as a television broadcast signal is input, and a frame input from the input unit (1). A sampling rate conversion unit (7) for converting the number of pixels in a two-dimensional direction, and a display unit (3) for displaying an image based on the frame in which the number of pixels is converted by the sampling rate conversion unit (7) I have. As the display unit (3), for example, a plasma display panel, a liquid crystal display panel, or an electron / field emission display panel is used. Details of the sampling rate conversion unit (7) will be described below.

  In FIG. 13, the signal input from the input unit (1) is input to the vertical filter V-A (1301-a) and the vertical filter V-B (1301-b). Here, the vertical filter V-A (1301-a) is a one-dimensional filter having an odd number of taps and a vertically symmetrical tap coefficient. The vertical filter V-B (1301-b) is a one-dimensional filter having an even number of taps and symmetrical tap coefficients. Subsequently, the output signal of the vertical filter V-A (1301-a) is input to the horizontal filter H-A (1302-a) and the horizontal filter H-B (1302-b). Here, the horizontal filter HA (1302-a) is a one-dimensional filter having an odd number of taps and symmetrical tap coefficients. Further, the horizontal filter H-B (1302-b) is a one-dimensional filter having an even number of taps and symmetrical tap coefficients.

  The outputs of the horizontal filter H-A (1302-a) and the horizontal filter H-B (1302-b) are input to the multipliers (1303) (1304) via the buffers (1313) (1314), respectively. Here, based on the information (interpolation position information (1310)) indicating the horizontal position and vertical position of the pixel to be interpolated generated by the control unit (1309), the coefficient generation unit (1311) calculates the coefficient k1 and the coefficient k2. Generate. Multipliers (1303) and (1304) multiply the coefficients k1 and k2 by the outputs of the buffers (1313) and (1314), and input them to the adder (1308).

  On the other hand, the output signal of the vertical filter V-B (1301-b) is input to the horizontal filter H-A (1305-a) and the horizontal filter H-B (1305-b). Here, the horizontal filter HA (1305-a) is a one-dimensional filter with an odd number of taps and symmetrical tap coefficients, and may have the same characteristics as the horizontal filter HA (1302-a). The horizontal filter H-B (1305-b) is a one-dimensional filter having an even number of taps and symmetrical tap coefficients, and has the same characteristics as the horizontal filter H-B (1302-b).

  The outputs of the horizontal filter H-A (1305-a) and the horizontal filter H-B (1305-b) are input to the multipliers (1306) (1307) via the buffers (1315) (1316), respectively. Here, based on the information (interpolation position information (1310)) indicating the horizontal position and vertical position of the pixel to be interpolated generated by the control unit (1309), the coefficient generation unit (1311) calculates the coefficient k3 and the coefficient k4. Generate. The multipliers (1306) and (1307) multiply the coefficients k3 and k4 by the outputs of the buffers (1315) and (1316), and input the result to the adder (1308).

  Next, an adder (1308) adds the four input signals to obtain an output of the sampling rate converter (7). The vertical filter VA (1301-a) and horizontal filter HA (1302-a), vertical filter VA (1301-a) and horizontal filter HB (1302-b), vertical filter VB (1301-b) and horizontal filter Each of the HB (1305-a), the vertical filter VB (1301-b), and the horizontal filter HB (1305-b) may be preliminarily subjected to a convolution operation to form one two-dimensional filter. In this case, the tap coefficient and impulse response of each two-dimensional filter are equal to the result of convolving the tap coefficient and impulse response of the original one-dimensional filter, respectively.

  Here, the coefficient k1, the coefficient k2, the coefficient k3, and the coefficient k4 are expressed as k1 = (1-kV) * (1-kH), k2 = (1-kV) * kH as shown in the coefficient generation procedure (1312). , K3 = kV * (1-kH) and k4 = kV * kH, an operation equivalent to that of the sampling rate converter shown in FIG. 12 can be realized. Furthermore, by using the buffer (1313) (1314) (1315) (1316) and the coefficient k1, the coefficient k2, the coefficient k3, and the coefficient k4, the horizontal position and the vertical position of the pixel to be interpolated are complicatedly converted for each pixel. Such deformation, for example, rotation can be dealt with.

  Here, the buffers (1313), (1314), (1315), and (1316) shown in FIG. 13 are used as buffers in units of frames, and can be controlled so that another data is not written to the same address before the data is read. To do. That is, the data write operation to the buffer (1313) (1314) (1315) (1316) and the data read operation from the buffer (1313) (1314) (1315) (1316) are independently performed in frame units. Configure to run. For example, the above control can be realized by preparing two equivalent frame buffers and using a general double buffer configuration in which two processes of data reading and data writing are made parallel by switching them appropriately.

  FIG. 16 shows the operation of the application example in which (a) the input image is rotated and (b) the output image is obtained by using the sampling rate conversion apparatus of FIG. In the figure, (a) the pixel position (coordinates) on the frame of the input image is (x1, x2), (b) the pixel position (coordinates) on the frame of the output image is (x2, y2), and 2 Assume that two pixel positions correspond to each other. As shown in (a) and (b), when the pixel position (x0, y0) is the rotation center and the rotation angle θ is rotated, the pixel position (x1, y1) of the input image and the pixel of the output image The relational expression shown in FIG. 3C is established between the positions (x2, y2). When this relational expression is transformed, it becomes as shown in FIG.

  Here, when (b) the pixel position (x2, y2) on the frame of the output image is scanned in raster order from the upper left to the lower right, for example, from the relational expression (d) of FIG. A corresponding pixel position (x1, y1) on the frame is obtained. At this time, by replacing the pixel position (x2, y2) of the output pixel (1102) shown in FIG. 11 with the pixel position (x1, y1) obtained by the relational expression of FIG. Using the flowchart shown in FIG. 16, the coefficient kV and the coefficient kH corresponding to the pixel position (x2, y2) on the frame of the output image (b) in FIG. 16 can be obtained.

  From the coefficient kV and the coefficient kH thus obtained, the coefficient k1 = (1-kV) * (1-kH), coefficient k2 = (1-kV) according to the coefficient generation procedure (1312) shown in FIG. * kH, coefficient k3 = kV * (1-kH), coefficient k4 = kV * kH are calculated, and each buffer (1313) is used with multipliers (1303) (1304) (1306) (1307) and adders (1308). ) (1314), (1315), and (1316) are weighted and added, so that (a) the input image in FIG. 16 can be rotated and (b) the output image can be easily obtained.

  Note that this rotation does not enlarge or reduce the image, but applies the same sampling rate conversion to the deformation of the image that generates the output pixel at the position where the original actual pixel does not exist. Can do.

  Note that the buffers (1313), (1314), (1315), and (1316) shown in FIG. 13 are not limited to the buffers in units of frames as described above. For example, in units of blocks in which one frame is divided into small areas. You may comprise so that data can be stored.

  An image signal processing method for realizing a process equivalent to the sampling rate conversion apparatus shown in FIG. 13 by a control unit cooperating with software will be described with reference to FIGS.

  First, a sampling rate conversion apparatus for realizing the image signal processing method according to the present embodiment will be described with reference to FIG. The sampling rate conversion apparatus shown in FIG. 22 stores, for example, an input unit (1) to which an image signal such as a television broadcast signal is input, and software for processing a signal input from the input unit (1). A control unit (10) that performs image signal processing on a signal input from the input unit (1) in cooperation with the software stored in the storage unit (11), and the input unit (1) Buffer # 1 (41) for temporarily storing input data, buffer # 2 (42) for temporarily storing data in the sampling rate conversion process by the control unit (10), buffer # 3 (43), buffer # 4 (44), buffer # 5 (45), and temporarily store the signal after sampling rate conversion processing output from the control unit (10) to the output unit (3) Buffer # 6 (46). Here, buffer # 1 (41), buffer # 2 (42), buffer # 3 (43), buffer # 4 (44), buffer # 5 (45), buffer # 6 (46) that temporarily store data ) And the storage unit (11) for storing the software may each be configured using individual memory chips, or may be configured using one or a plurality of memory chips and dividing each data address It may be.

  In the figure, for the image signal input from the input unit (1), the control unit (10) performs a sampling rate conversion process in cooperation with software stored in the storage unit (11), and the display unit (3) Output to. Details of the sampling rate conversion processing will be described with reference to FIG.

  FIG. 23 shows an example of a flowchart of the sampling rate conversion method according to the present embodiment. The flowchart of FIG. 23 starts from step (2301), and in step (2302), the input signal (pixel) is written to buffer # 1 (41). At this time, the input signal (pixel) may be written to the buffer for each frame constituting the input image, or the input signal (pixel) may be written to the buffer for each block obtained by dividing the input image into small regions. Subsequently, in step (2303), the vertical filter VA (1301-a) and the horizontal filter HA (1302-a) are applied to the signal of the buffer # 1 (41), and the output is supplied to the buffer # 2 (42). Write. In step (2304), vertical filter VA (1301-a) and horizontal filter HB (1302-b) are applied to the signal in buffer # 1 (41), and the output is written to buffer # 3 (43). . In step (2305), vertical filter VB (1301-b) and horizontal filter HA (1305-a) are applied to the signal of buffer # 1 (41), and the output is written to buffer # 4 (44). . In step (2306), vertical filter VB (1301-b) and horizontal filter HB (1305-b) are applied to the signal in buffer # 1 (41), and the output is written to buffer # 5 (45). .

  Here, the horizontal filters H-A (1302-a) and H-A (1305-a), and the horizontal filters H-B (1305-b) and H-B (1305-b) have the same characteristics. Further, it is clear that the same effect can be obtained even if the processing order of the respective vertical filters and horizontal filters in steps (2302), (2303), (2304), and (2305) is reversed. The characteristics of each filter can be used as they are by replacing the characteristics described with reference to FIGS. 5, 8, 9, and 10 with the characteristics in the vertical direction or the horizontal direction. To do.

  In step (2307), the coefficient k1, the coefficient k2, the coefficient k3, and the coefficient k4 are generated for each output signal (pixel) based on the positional relationship between the output signal (pixel) and the input signal (pixel). As the coefficient generation method at this time, since the method shown in FIGS. 6, 17, 7, and 13 (1312) can be used as it is, the illustration and description thereof are omitted.

  In the following step (2308), buffer # 2 (42), buffer # 3 (43), buffer # 4 (44) according to the values of coefficient k1, coefficient k2, coefficient k3, coefficient k4 obtained as described above. ), Each signal in buffer # 5 (45) is weighted and added to buffer # 6 (46), and the process ends in step (2309).

  As shown in FIG. 13, the weighted addition method at this time is obtained by multiplying the output of the buffer (1313) (= buffer # 2 (42)) by a coefficient k1, and then buffer (1314) (= buffer # 3 (43 )) Output multiplied by coefficient k2, the output of buffer (1315) (= buffer # 3 (43)) multiplied by coefficient k3, and the output of buffer (1316) (= buffer # 4 (44)) Multiply k4 and add all the results. A signal calculated by this weighted addition is used as an output signal.

  Steps (2303), (2304), (2305), (2306), and (2307) are independent of each other, so each step may be processed in order, or each step may be processed simultaneously in parallel. . With the processing steps described above, the sampling rate conversion method according to this embodiment can be implemented.

Further, the present invention is not limited to the use for image rotation. For example, the corresponding pixel position between the image shown in FIG. 16A and the image shown in FIG. It can also be used for the purpose of estimating by a position estimation technique and generating an image close to (b) based on the image (a). That is, for example, in the case of a frame in which the same figure (a) and the same figure (b) are continuous, the position estimation technique corresponds to a motion vector search in pixel units, Thus, the operation for generating an image close to FIG. 5B is an operation for motion compensation using the result of the motion vector search. Here, as the position estimation technique, for example, a method as described in Reference Document 1 or Reference Document 2 may be used as it is, and therefore illustration and description thereof are omitted.
[Reference 1] Shigeru Ando “Velocity vector distribution measurement system using spatio-temporal differential calculation of images”, Transactions of the Society of Instrument and Control Engineers, pp. 1330-1336, Vol. 22, No. 12, 1986
[Reference 2] Hiroyuki Kobayashi et al. “Phase-Only Correlation of Images Based on DCT Transform”, IEICE Technical Report ITS2005-92, IE2005-299 (2006-02), pp.73-78
As described above, in addition to the effects of the sampling rate conversion apparatus and the sampling rate conversion method according to the second embodiment, the sampling rate conversion apparatus and the sampling rate conversion method according to the third embodiment, for example, rotate images and It is possible to cope with complicated deformation such as motion compensation using the position estimation.

  Each embodiment of the present invention can be similarly applied to, for example, a DVD player, a magnetic disk player, or a semiconductor memory player in addition to the image display device described in the above embodiments. For example, the present invention can also be applied to a portable image display terminal (for example, a mobile phone) for receiving 1-segment broadcasting.

  As the image frame, an image frame of a signal other than the television broadcast signal may be used. Further, for example, a streaming image transmitted via the Internet or an image frame of an image reproduced from a DVD player or an HDD player may be used.

  Further, in each of the above-described embodiments, the case where the sampling rate conversion is performed on an image by frame basis has been described as an example. However, the sampling rate conversion target is not necessarily the entire frame. For example, a part of the frame of the input image or input video may be set as the sampling rate conversion target. That is, if the sampling rate conversion according to the embodiment of the present invention described above is performed on an input video frame, an enlarged image, a reduced image, a rotated image, or a deformed image of the input image or a part of the input video is obtained. Can do. This can be applied to, for example, an enlarged display, a reduced display, a rotation display, and a deformation display of a part of an image.

  In the above description, the case of performing two-dimensional sampling rate conversion consisting of the horizontal direction and the vertical direction has been described, but the present invention is not limited to this, for example, the horizontal direction and the time direction (frame direction). It is also possible to apply to two-dimensional sampling rate conversion consisting of or two-dimensional sampling rate conversion consisting of a vertical direction and a time direction (frame direction).

  In addition, the expression “pixel” in each description and each drawing in this specification includes a meaning of a sample signal or a sample point, and is a digitized audio signal sequence, a continuous data sequence obtained from a sensor, or the like. The meaning of is also included. Therefore, the above-described one-dimensional sampling rate conversion apparatus or sampling rate conversion method can be applied as it is as a sampling rate conversion apparatus or sampling rate conversion method for audio signals or data strings.

It is explanatory drawing of Example 1 which concerns on this invention. It is a figure explaining an example of operation | movement of general sampling rate conversion. It is a figure explaining the structure of the prior art. It is a figure explaining operation | movement of a prior art. It is explanatory drawing of Example 1 which concerns on this invention. It is explanatory drawing of Example 1 which concerns on this invention. It is explanatory drawing of Example 1 which concerns on this invention. It is explanatory drawing of Example 1 which concerns on this invention. It is explanatory drawing of Example 1 which concerns on this invention. It is explanatory drawing of Example 1 which concerns on this invention. It is explanatory drawing of Example 2 which concerns on this invention. It is explanatory drawing of Example 2 which concerns on this invention. It is explanatory drawing of Example 3 which concerns on this invention. It is explanatory drawing of Example 1 which concerns on this invention. It is explanatory drawing of an example of the process of Example 1 which concerns on this invention. It is explanatory drawing of an example of the process of Example 3 which concerns on this invention. It is explanatory drawing of Example 1 which concerns on this invention. It is explanatory drawing of Example 1 which concerns on this invention. It is explanatory drawing of Example 2 which concerns on this invention. It is explanatory drawing of Example 1 which concerns on this invention. It is explanatory drawing of Example 2 which concerns on this invention. It is explanatory drawing of Example 3 which concerns on this invention. It is explanatory drawing of Example 3 which concerns on this invention.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Input part; 2,4,7 ... Sampling rate conversion part; 3 ... Display part; 10,103,306,310,1203,1309 ... Control part; 11 ... Storage part; 21,22,23,24,31,32,33,34 , 35, 36, 37, 41, 42, 43, 44, 45, 46, 1313, 1314, 1315, 1316 ... buffer; 101, 102, 1201, 1202, 1209, 1210, 1301, 1302, 1305 ... filter; 104, 1204 , 1211, 1311 ... coefficient generation unit; 105, 106, 1205, 1206, 1212, 1213, 1303, 1304, 1306, 1307 ... multipliers; 107, 1207, 1214, 1308 ... adders; 108, 1208, 1215, 1310 ... interpolation Position information; 201, 601, 1101 ... Real pixel; 202 ... Zero point; 203 ... Interpolation pixel; 204, 602, 1102 ... Output pixel; 301 ... Upsampler; 302 ... Interpolation filter; 303 ... Downsampler; 304,307 ... Partial filter; 308: Tap coefficient generator; 501: Tap coefficient; 502,605,702,703,1103,1104,1107,1108 ... Center of symmetry; 603,604,701,1105,1106 ... Coefficient; 901, 1001 ... Nyquist frequency; 902,904,1002,1004 ... Frequency characteristics , 1003, 1005 ... Cutoff frequency; 1216 ... Vertical processing unit; 1217 ... Horizontal processing unit 1312 ... coefficient generation procedure.

Claims (12)

In a device that converts the sampling rate of a digital signal,
An input unit to which data of the digital signal is input;
A first calculation unit that outputs a convolution calculation result of a tap coefficient whose coefficient value is symmetrical about the first axis of symmetry and the input data;
A second computing unit that outputs a convolution computation result of the tap coefficient whose coefficient value is symmetrical about the second symmetry axis and the input data;
A mixing unit for mixing the output of the first calculation unit and the output of the second calculation unit;
An output unit for outputting data from the mixing unit;
A sampling rate conversion device comprising:   The position of the first symmetry axis coincides with the pixel position of the input data, and the position of the second symmetry axis coincides with the center of each pixel position of the two adjacent input data. The sampling rate converter according to claim 1.   The mixing unit outputs the first calculation unit and the second calculation based on a predetermined mixing ratio determined from a relationship between a pixel position of the input data and a pixel position of data that can be output from the output unit. 2. The sampling rate conversion apparatus according to claim 1, wherein the outputs of the units are mixed. In an apparatus for converting a sampling rate of a digital signal having a two-dimensional sampling point composed of a first direction and a second direction,
An input unit to which data of the digital signal is input;
A tap coefficient whose coefficient value is symmetric about the first symmetry axis in the first direction; a tap coefficient whose coefficient value is symmetric about the third symmetry axis in the second direction; and the input A first calculation unit that outputs a result of convolution calculation with data;
A tap coefficient whose coefficient value is symmetric about the first symmetry axis in the first direction; a tap coefficient whose coefficient value is symmetric about the fourth symmetry axis in the second direction; and the input A second calculation unit that outputs a result of convolution calculation with data;
A tap coefficient whose coefficient value is symmetric about the second axis of symmetry in the first direction; a tap coefficient whose coefficient value is symmetric about the third axis of symmetry in the second direction; and the input A third calculation unit that outputs a result of the convolution calculation with data;
A tap coefficient whose coefficient value is symmetrical about the second symmetry axis in the first direction; a tap coefficient whose coefficient value is symmetrical about the fourth symmetry axis in the second direction; and the input A fourth operation unit that outputs a result of the convolution operation with the data;
A mixing unit that mixes outputs of the first to fourth calculation units;
An output unit for outputting data from the mixing unit;
A sampling rate conversion device comprising:   The position of the first symmetry axis coincides with the pixel position in the first direction of the input data, and the position of the second symmetry axis corresponds to each of the two adjacent input data in the first direction. The position of the third symmetry axis coincides with the center of the pixel position, the position of the third symmetry axis coincides with the pixel position in the second direction of the input data, and the position of the fourth symmetry axis corresponds to the two adjacent inputs. 5. The sampling rate conversion apparatus according to claim 4, wherein the sampling rate conversion apparatus matches the center of each pixel position in the second direction of data.   The mixing unit mixes outputs of the first to fourth calculation units based on a predetermined mixing ratio determined from a relationship between a pixel position of output data and a pixel position of input data. The sampling rate conversion device according to claim 4. In a method for converting a sampling rate of a digital signal,
An input step in which data of the digital signal is input;
A first calculation step of outputting a convolution calculation result of the tap coefficient having a coefficient value symmetrical about the first symmetry axis and the input data;
A second calculation step of outputting a convolution calculation result of the tap coefficient having a coefficient value symmetrical about the second axis of symmetry and the input data;
A mixing step of mixing the output of the first calculation step and the output of the second calculation step;
An output step for outputting data from the mixing step;
A sampling rate conversion method comprising:   The position of the first symmetry axis coincides with the pixel position of the input data, and the position of the second symmetry axis coincides with the center of each pixel position of the two adjacent input data. The sampling rate conversion method according to claim 7.   In the mixing step, the output of the first calculation step and the second calculation are based on a predetermined mixing ratio determined from the relationship between the pixel position of the input data and the pixel position of data that can be output in the output step. 8. The sampling rate conversion method according to claim 7, wherein outputs of the steps are mixed. In a method for converting a sampling rate of a digital signal having a two-dimensional sampling point composed of a first direction and a second direction,
An input step in which data of the digital signal is input;
A tap coefficient whose coefficient value is symmetric about the first symmetry axis in the first direction; a tap coefficient whose coefficient value is symmetric about the third symmetry axis in the second direction; and the input A first calculation step for outputting a convolution calculation result with data;
A tap coefficient whose coefficient value is symmetric about the first symmetry axis in the first direction; a tap coefficient whose coefficient value is symmetric about the fourth symmetry axis in the second direction; and the input A second calculation step for outputting a convolution calculation result with the data;
A tap coefficient whose coefficient value is symmetric about the second axis of symmetry in the first direction; a tap coefficient whose coefficient value is symmetric about the third axis of symmetry in the second direction; and the input A third calculation step for outputting a convolution calculation result with the data;
A tap coefficient whose coefficient value is symmetrical about the second symmetry axis in the first direction; a tap coefficient whose coefficient value is symmetrical about the fourth symmetry axis in the second direction; and the input A fourth calculation step for outputting a convolution calculation result with the data;
A mixing step of mixing respective outputs of the first to fourth calculation steps;
An output step for outputting data from the mixing step;
A sampling rate conversion method comprising:   The position of the first symmetry axis coincides with the pixel position in the first direction of the input data, and the position of the second symmetry axis corresponds to each of the two adjacent input data in the first direction. The position of the third symmetry axis coincides with the center of the pixel position, the position of the third symmetry axis coincides with the pixel position in the second direction of the input data, and the position of the fourth symmetry axis corresponds to the two adjacent inputs. The sampling rate conversion method according to claim 10, wherein the sampling rate conversion method matches the center of each pixel position in the second direction of data.   The mixing step mixes the outputs of the first to fourth calculation steps based on a predetermined mixing ratio determined from the relationship between the pixel position of the output data and the pixel position of the input data. The sampling rate conversion method according to claim 10.

Images