Usually, interlacing has been widely used in many artifacts (artifacts) that minimizing causes because of interlacing scan to changing (IPC) device line by line, that is, glimmering such as the vertical resolution deterioration in the systems such as NTSC, PAL, SECAM, scan line flicker and wide territory.
Recently, because high definition TV (HDTV) system adopted for the multiple standards of signal format and often need change between the standard input/output signal of different-format, therefore, interlacing has become more important to conversion equipment line by line.
In early days the stage development interlacing of getting up to transfer algorithm line by line for using such as systems such as NTSC, PAL, SECAM.The various algorithms that proposed are excluded the interpolation of the row beyond interlacing scan usually based on playback.
The list of references [1] that " is used for reducing the comparison of the interpolater of the medium filtering of flicker and vertical edge control " by D.I.C.Hentschei being entitled as of delivering at No. 3 279-289 page or leaf of IEEE journal consumer electronics (consumer electronics) the 35th volume in August, 1989 disclose these artifacts of minimizing that proposed algorithm, repeat with the interpolating method of vertical filtering with based on the method for vertical edge information according to simple row.
Go up the interpolating method that discloses in the list of references of delivering by D.Bagni, R.Lancini, S.Landi and S.Tubaro [2] that is entitled as " the HD-TV space-time is to up conversion " according to edge direction at 1994 " HDTV international symposium proceedings ".
Non-linear high speed interpolation method based on weighted median filter is disclosed in the list of references [3] that " uses the scan rate conversion of weighted median filtering " in being entitled as of delivering by J.Juhola, A.Nieminen, J.Salo and Y.Neuvo of IEEE ISCAS-89 proceedings 433-436 page or leaf of in May, 1989 U.S. Portland.
Algorithm based on FIR intermediate value compound filter is disclosed in the list of references [4] that is entitled as " non-linear quincunx interpolation filtering " delivered by A.Lehtonen and M.Renfors of " SPIE face-to-face communication and image processing proceedings " 132-135 page or leaf of Switzerland Lausanne in October nineteen ninety.
Go up the algorithm that discloses in the list of references of delivering by H.Blume, L.Schwoerer and K.Zygis [5] that is entitled as " based on the subband that uses complementary median filter " based on complementary median filter to up conversion at 1994 " HDTV international symposium proceedings ".
Be located at the L.Chiariglione Ed. of North Holland, Elsevier Science Press discloses in the list of references [6] that is entitled as " interlacing that EDTV uses is to conversion line by line " of T.Doyle that " HDTV signal processing " 421-430 page or leaf in 1988 is delivered based on the algorithm according to the medium filtering of direction.
The list of references of delivering by P.Frenchen at No. 3 237-240 page or leaf of IEEE journal consumer electronics in 1986 the 32nd volume [7] that is entitled as " two integrated progressive conversion devices "; Algorithm based on vertical-time median filter is disclosed with the list of references [8] that is entitled as " television image medium filtering " that " IEEE technical papers digest " 186-187 page or leaf of 1986 is delivered by T.Doyle and P.Frencken.
In addition, the list of references of delivering by people such as N.Suzuki at No. 3 266-271 page or leaf of IEEE journal consumer electronics the 33rd volume in August, 1989 [9] that is entitled as " the improvement resultant motion signal that in the IDTV receiver, is used for improving the Motion Adaptive pre-scan converted "; By the list of references [10] that is entitled as " having the Motion Adaptive pre-scan converted device that two-dimensional silhouette strengthens " that C.P.Markhauser delivers the Motion Adaptive scheme is disclosed with No. 2 110-114 page or leaf of IEEE journal consumer electronics the 36th volume in May nineteen ninety.
Above interlacing can roughly be divided into the three-dimensional interpolating method of spatial interpolation method, temporal interpolation method and spatial interpolation and temporal interpolation combination to conversion method line by line.
In three-dimensional interpolating method because wrong temporal interpolation can cause the deterioration in image quality such as artifact is torn (tearing-artifact), the motion in the detected image and according to the motion that is detected suitably the time of implementation interpolation be very important.This is owing to owing to time sampling speed in the actual image signal causes the limited fact of maximum time frequency that can determine in theory less than Nyquist (Nyquist) speed.
Yet, just as proposed by the present invention: the reliability that can increase movable information by the spatial information that uses sampling with movable information.
Below, the interlacing of using motion and space correlation is described with reference to the drawings to conversion equipment and method and preferred embodiment thereof line by line.
Fig. 1 be according to the present invention interlacing to the block diagram of an embodiment of conversion equipment line by line.
This interlacing comprises a spatial interpolator 110, temporal interpolation device 120, correlator 130, a selector 140 to conversion equipment line by line.
The interlaced image signal Vin that spatial interpolator 110 uses pre-defined algorithm to come spatial interpolation to import.
Temporal interpolation device 120 uses pre-defined algorithm to come the interlaced image signal Vin of this input of temporal interpolation.
Correlator 130 detects relevant (hereinafter being referred to as " vertical being correlated with of time ") DT between the relevant DM of motion, relevant (hereinafter being referred to as " vertical direction the is relevant ") DV of vertical direction and temporal interpolation and the vertical interpolation from the interlaced image signal imported.
Selector 140 will move relevant DM, vertically the vertical relevant DT of relevant DV with the time respectively with predetermined constant TM, T1 and T2 relatively, and according to comparative result select by the signal Is of spatial interpolator 110 outputs or by the signal It of temporal interpolation device 120 outputs so that it is exported as interpolated signal Vout.
The operation of this device shown in Figure 1 is described now.
Among Fig. 1, interlaced image signal Vin is applied to spatial interpolator 110, temporal interpolation device 120 and correlator 130.
Simultaneously, owing to the present invention relates to switch the progressive video signal (hereinafter being referred to as " spatial interpolation signal ") and the temporally interpolated progressive video signal (hereinafter being referred to as " temporal interpolation signal ") of spatial interpolation and export selected signal, therefore have nothing to do with in spatial interpolator 110 and temporal interpolation device 120, using any spatial interpolation algorithm and temporal interpolation algorithm respectively according to movable information and spatial information.
Correlator 130 is imported interpolated signal Vin and is detected the relevant DM of motion, the vertical DT of being correlated with the time of the relevant DV of vertical direction from interlaced image signal Vin.After this, correlator 130 output signal DM, DV and DT.
Selector 140 will move relevant DM, vertically the vertical relevant DT of relevant DV with the time respectively with predetermined constant TM, T1 and T2 comparison, and select, and export selected signal Vout by the signal Is of spatial interpolator 110 outputs or the signal It that exports by temporal interpolation device 120.
Fig. 2 is the more detailed block diagram of correlator shown in Figure 1.
Sampling delayer 201,203,204 and 206, row delayer 202 and 205 constitute the first sampling detector, and detect sampling w1 to w5.Field memory 210, sampling delayer 211 and 213, row delayer 212 constitute the second sampling detector, and detect sampling x1 and x2.In addition, field memory 220, sampling delayer 221,223,224 and 226, row delayer 222 and 225 constitute the 3rd sampling detector, and detect sampling v1 to v5.
Motion correlator 230 receives sampling v1 and v5 and w1 and w5, and the relevant DM with output movement of calculating.Subtracter 241 and absolute value circuit 242 receive sampling v1 to v3 and w1 to w5, and calculate vertical relevant DV with output.In addition, adder 251 and 252, subtracter 253 and absolute value circuit 254 receive sampling x1, x2, v3 and w3, and the vertical relevant DT with output time of calculating.
Geometrical relationship shown in Fig. 3 key diagram 2 between the sampling.
Among Fig. 3, sampling " x " expression will be by the sampling of interpolation (being called as " current interpolation sampling ") recovery.The sampling that has the par position in sampling " x1 " the expression previous row with current interpolation sampling.In addition, the sampling that has the par position in sampling " x2 " the expression next line with current interpolation sampling.
The sampling that has same position with current interpolation sampling in sampling " v3 " the expression previous field.Sampling " v2 " and " v4 " is the previous and back sampling of sampling v3.In addition, and sampling " v1,, represent to have with sampling v3 in the previous row sampling of par position.The sampling that has the par position in sampling " v5 " the expression next line with sampling v3.
Equally, the sampling that has same position with current interpolation sampling is represented in next in sampling " w3 ".Sampling " w2 " and " w4 " is the previous and back sampling of sampling w3.In addition, the sampling that has the par position in sampling " w1 " the expression previous row with sampling w3.The sampling that has the par position in sampling " w5 " the expression next line with sampling w3.
Equally, be odd field if work as the front court, because this input signal is the interlace signal of input, previous field and next then are even fields.In addition, be even field if work as the front court, previous field and next then are odd fields.
With reference now to Fig. 3, the operation of device shown in Figure 2 is described.
In Fig. 2, interlaced image signal Vin is imported into field memory 210, sampling delayer 201 and row delayer 202.
Also export the picture signal that is delayed a field duration for one of the picture signal Vin that field memory 210 storages are imported.Field memory 220 is from the picture signal of these delays of field memory 210 input, store of picture signal of this delay, and output compared the picture signal that is delayed two field duration with the original input signal Vin that is imported. Field memory 210 and 220 is made of first-in first-out (FIFO) memory.
Sampling delayer 201 input interlaced image signal Vin with 1 sampling period of this signal delay, and export sampling w5 shown in Figure 3 next.The delayer 202 of going receives this input signal Vin, with 1 horizontal line cycle of this signal delay, and exports sampling w4 when sampling delayer 201 output sampling w5.Sampling delayer 203 will be gone 1 sampling period of signal delay of delayer 202 outputs, and export sampled signal w3.Sampling delayer 204 will be taken a sample 1 sampling period of signal delay of delayer 203 outputs, and export sampled signal w2.Row delayer 205 will be gone 1 horizontal line cycle of signal delay that delayer 202 is exported.Sampling delayer 206 will be gone 1 sampling period of signal delay of delayer 205 outputs, and export sampled signal w1.
Simultaneously, sampling delayer 211 further will be from 1 sampling period of the signal delay that is delayed 1 field duration of field memory 210, and output sampling x2.Row delayer 212 is with 1 horizontal line cycle of signal delay of field memory 210 outputs.Sampling delayer 213 will be gone 1 sampling period of signal delay of delayer 212 outputs, and export sampled signal x1.
Sampling delayer 221 further will be from 1 sampling period of the signal delay that is delayed 2 field duration of field memory 210, and output sampling v5.Row delayer 222 is with 1 horizontal line cycle of signal delay of field memory 220 outputs, and output sampling v4.Sampling delayer 223 will be gone 1 sampling period of signal delay of delayer 222 outputs, and export sampled signal v3.Sampling delayer 224 will be taken a sample 1 sampling period of signal delay of delayer 223 outputs, and export sampled signal v2.Row delayer 225 will be gone 1 horizontal line cycle of signal delay that delayer 222 is exported.Sampling delayer 226 will be gone 1 sampling period of signal delay of delayer 225 outputs, and export sampled signal v1.
Motion correlator 230 receives the sampling v1 to v5 of previous field and the sampling w1 to w5 of next, calculates and the relevant DM of output movement.At this moment, calculate the relevant DM of motion of motion correlator 230 outputs by formula (1).
Wherein, ai is predefined coefficient.
The relevant DM of motion is used for estimating whether the adjacent of the position of the sampling that will be recovered by interpolation produces the measurement result (x of Fig. 3 is the geometry mid points of x1 and x2) of motion.
Subtracter 241 deducts the sampled data x2 by 211 outputs of sampling delayer from the sampled data x1 by 213 outputs of sampling delayer, and the result is subtracted each other in output.After this, absolute value circuit 242 calculates and subtracts each other result's absolute value and export this absolute value as vertical relevant DV.
Therefore, the vertical relevant DV of available formula (2) expression.
DV=|x1-x2| …(2)
Adder 251 is added to sampled data w3 with sampled data v3.Adder 252 is added to sampled data x2 with sampled data x1.
Subtracter 253 deducts the signal by adder 252 outputs from the signal by adder 251 outputs, and the result is subtracted each other in output.After this, absolute value circuit 254 calculates and subtracts each other result's absolute value and export this absolute value as vertically relevant DT of time.
Therefore, the vertically relevant DT of available formula (3) express time.
DT=|(v3+w3)-(x1+x2)| …(3)
The relevant DM of motion of motion correlator 230 outputs, the vertical relevant DT with the time of absolute value circuit 254 outputs of vertical relevant DV of absolute value circuit 242 outputs are input to selector shown in Figure 1 140.
The operation of selector 140 is described with reference to flow process shown in Figure 4.
Among Fig. 4, selector 140 will move relevant DM and constant TM comparison (step S101).If the relevant DM of motion produces constant of the motion TM greater than showing, selector 140 selections and output are from the spatial interpolation signal (step S102) of spatial interpolator 110.
If the relevant DM of motion is not more than constant of motion TM in step S101, selector 140 will vertically be correlated with DV and constant T1 comparison (step S103).If vertically relevant DV is greater than constant T1, selector 140 selections and output are from the temporal interpolation signal It (step S104) of temporal interpolation device 120.
At this, the DV that when the relevant DM of motion is not more than constant TM, will vertically be correlated with reason that constant T1 compares be because can by the artifact of the temporal interpolation generation of mistake by the size of the vertical DV of being correlated with visually impression different.As vertical relevant DV during,, visually be not easy to perceive the artifact that the temporal interpolation because of mistake produces because picture signal is relevant littler in vertical direction greater than constant T1.Therefore, when DM is not more than TM and DV greater than T1, by selector 140 select and output from the signal It (step S104) of temporal interpolation device 120.
Yet, when DM less than TM and DV during less than T1, the artifact that the easier temporal interpolation of visually awaring because of mistake produces.Therefore, selector 140 according to show that vertically relevant DT of time relevant between temporal interpolation and the vertical interpolation selects and output from the signal Is of spatial interpolator 110 or from the signal It of temporal interpolation device 120.
That is, when the relevant DM of motion was not more than constant TM and vertically relevant DV and is not more than constant T1, selector 140 compared (step S105) with vertically relevant DT of time with constant T2.If vertically relevant DT of time is greater than constant T2, because relevant less between temporal interpolation and the vertical interpolation, signal Is from spatial interpolator 110 is selected and exported to selector 140, so that reduce the influence (step S106) of artifact.Simultaneously, if vertically relevant DT of time is not more than constant T2, because being correlated with between temporal interpolation and the vertical interpolation is bigger, selector 140 selections and output are from the output signal It (step S107) of temporal interpolation device 120.
As mentioned above, interlacing according to the present invention is used motion and space correlation interpolation interlaced image signal to conversion equipment and method line by line with coming time or space, and strengthens the reliability of relevant movable information and reduce artifact effectively.