HK1019129B - Apparatus for sampling and displaying an auxiliary image with a main image - Google Patents

Description

Device for sampling and displaying auxiliary image and main image

The present invention relates to a picture-in-picture system for improving horizontal sharpness using quincunx sampling.

Current image display systems are capable of displaying small auxiliary images while simultaneously displaying a larger main image. Such smaller images may be displayed within the boundaries of the larger image, in which case such systems are referred to as picture-in-picture (PIP) systems, or the smaller image may be outside the main image (e.g., outside the left or right boundaries of the main image), in which case such systems are referred to as picture-outside-picture (POP) systems. The main image and auxiliary image may be from the same image source, such as a frozen PIP image frame of the main image, or from separate sources, such as in a system where one tuner is tuned to one video signal that will be displayed as the main image and the other tuner is tuned to another video signal independent of the first tuner that will be displayed as a PIP image.

The working principle of the PIP or POP system is as follows: compressed data representing the auxiliary image is stored in the presence of the auxiliary video signal and the main image signal is then replaced with the compressed data specifying the location in the main image at which the auxiliary image is to be displayed. The system must provide sufficient storage space to store the auxiliary image data from the time it appears in the video signal to the time it is displayed in the main image. The storage space provided by the presently known systems is sufficient to store one frame or one field of auxiliary video data. Since memory is expensive, it is desirable to minimize the amount of memory required. To reduce the amount of memory required, currently known PIP and POP systems subsample the auxiliary video signal and store only one field of subsampled auxiliary video data. A display method complementary to the subsampling method is used to generate a display image signal for inserting PIP or POP images.

Currently known sub-sampling techniques include repeating "take one sample, lose N samples" directly for each line of the auxiliary video signal. This approach is unsatisfactory in that the horizontal resolution of the PIP or POP image is reduced, thereby reducing the subjective quality of the PIP or POP image displayed. A satisfactory sub-sampling method is to increase the horizontal resolution of the PIP or POP image without increasing the capacity of the memory for storing PIP or POP image data to be displayed with the main image in the future.

In accordance with the principles of the present invention, an apparatus for displaying a combined image of an auxiliary image and a main image includes a source of a main image signal and a source of samples representing the auxiliary image signal. A quincunx subsampler is connected to the secondary image sample point source. A signal combiner is coupled to the main image signal source and the quincunx subsampler. A sample combiner combines the main image signal and the signal representing the quincunx subsampled samples to produce a signal representing a combined image of the main image and the auxiliary image.

In the drawings:

figure 1 shows a representation of a main image and a picture-in-picture (PIP) image and a display image embodying the invention,

fig. 2 presents in part a block diagram and in part a logic diagram a portion of a PIP system embodying the present invention;

fig. 3 illustrates quincunx sampling with a more detailed PIP display;

fig. 4 is a waveform diagram of a main video signal and a PIP memory access address;

fig. 5 and 6 illustrate in more detail certain parts of the PIP picture system to explain the problem and solution of quincunx sampling for PIP pictures;

fig. 7 presents, partly in block diagram form and partly in logic diagram form, an encoder for generating PIP data for storage in a field memory;

fig. 8 shows, partly in block diagram form and partly in logic diagram form, a decoder for generating PIP data for insertion into a main picture;

the illustrated embodiment will be described in terms of a picture-in-picture (PIP) system that produces the image shown in fig. 1. The principles of the present invention are applicable to other multi-picture systems such as a picture-outside-picture (POP) in which a small image is placed outside (e.g., to the left or right) of a main image.

Fig. 1 shows a PIP display image including a main image and an auxiliary image and embodying the present invention. In fig. 1, a main image 2 is shown as it might be displayed on a display device such as a television receiver or monitor. The main image 2 may be generated by known signal processing circuitry including a tuner responsive to receiving a television signal from an antenna or cable. Figure 1 also shows another image 4, which is inserted in the lower left corner of the main image 2. The auxiliary image 4 is in the illustrated embodiment called a picture-in-picture (PIP) image. The PIP picture is also generated by known signal processing circuitry, most likely including another tuner responsive to receiving another television signal from an antenna or cable. Alternatively, the second tuner may be implemented by a Video Cassette Recorder (VCR). The signals representing the PIP image 4 and the main image 2 will be combined in a known manner and the combined signal is fed to a display device which will display the image shown in fig. 1.

Fig. 2 shows a portion of a PIP system embodying the present invention, partly in block diagram form and partly in logic diagram form. In fig. 2, the output of a source 102 representing a main video image signal is connected to a sync component processor 103 and a video component processor 104, respectively. The output of the synchronization component processor 103 is connected to a master timing signal generator 106. An output of the video component processor 104 is coupled to a first data input of a multiplexer 108. The output of the multiplexer 108 produces a combined video signal and is connected to a display device (not shown) in a known manner. Respective outputs of the main timing signal generator 106 are connected to a clock input of the video component processor 104 and a control input of the multiplexer 108.

A source 110 of PIP video image signals is connected to an input of the PIP ADC 112. The output of the PIPADC 112 is connected to the input of a PIP video processor 113 and a PIP timing generator 114, respectively. An output terminal of the PIP video processor 113 is connected to a data input terminal of the quincunx subsampler 116. An output of the subsampler 116 is coupled to a data input of the field memory 120. A data output of the field memory 120 is connected to a data input of an insertion image display signal generator 124. An output of the display generator 124 is coupled to a second data input of the multiplexer 108.

Respective output terminals of the PIP timing signal generator 114 are coupled to corresponding input terminals of the subsampler 116 and the write address generator 118. An output terminal of the write address generator 118 is connected to a write address input terminal of the subsampler 116 and the field memory 120, respectively. Respective output terminals of the main timing signal generator 106 are connected to corresponding input terminals of the read address generator 122 and the display generator 124. The output of the read address generator 122 is connected to the respective read address inputs of the subsampler 116 and the field memory 120.

Referring to fig. 1 and 2, a main video signal from a main video source 102 includes a video component and a synchronization component. The synchronization component processor 103 extracts and processes the synchronization component signal. The synchronization signal will be provided to a timing generator 106, which generates the AND 4f_sc(color subcarrier frequency) clock signal. This timing signal is coupled to the main video processor 104. The video processor 104 processes the main video signal in a known manner. For example, in a preferred embodiment, the main video processor 104 includes a luma/chroma separator (e.g., a comb filter) to generate separate luma and chroma signals. The main video processor 104 may also include a demodulator for the chrominance signals and generate separate I and Q or U and V signals, and may also include a matrix of color signals and generate R, G and B chrominance component signals. Also in the preferred embodiment, the main video processor 104 includes an analog-to-digital converter and may perform a portion of its processing in its digital circuitry, including comb filtering. Alternatively, it may not perform video processing at all. In this case, the main video signal will pass from the main video source 102 through to the multiplexer 108 without any change.

The main timing signal generator 106 also generates a signal indicating when the display device scans the portion of the display image where the PIP insert image is located. This signal will be provided to the control input of the multiplexer 108. Multiplexer 108 will be controlled to connect main video processor 103 to its output when a main picture is to be displayed and multiplexer 108 will be controlled to connect display generator 124 to its output when a PIP picture is to be displayed.

At the same time, the PIP ADC112 will be at 4f_scTo process the PIP video signal to obtain samples representing the PIP video signal. These samples are processed by a PIP timing signal generator 114 and a PIP video signal processor 113. Specifically, the PIP timing signal generator 114 will identify, extract and process the PIP synchronization component. PIP video processor 113 contains digital circuitry for processing PIP video signal samples from PIP ADC 112. Similar to the main video processor 104 described above, in a preferred embodiment, the video processing includes circuitry, such as a comb filter, for separating the luminance and chrominance components from the PIP video signal. The PIP video processor 113 may also include a demodulator that separates the I and Q or U and V chrominance components. As an alternative, it is also possible that the PIP video processor 113 does not further process the samples of the PIP video signal, in which case the output of the PIP ADC112 would be connected directly to the input of the quincunx subsampler 116.

Quincunx subsampler 116 subsamples the sequence of PIP image samples from PIP video processor 113 in a manner described in detail below, based on the timing signal from PIP timing generator 114 and the read and write addresses of the field memory. In general, the sequence of PIP samples for each field will be sub-sampled independently in the following manner. In the vertical direction, three vertically aligned samples will be filtered to produce a single PIP subsampled sample. In a preferred embodiment, the three vertically aligned samples are averaged. In the horizontal direction, the filtered sequence is subsampled by a ratio of 6: 1, i.e. by keeping one sample and discarding 5 samples, as will be explained in more detail below. The timing of the horizontal subsampling is controlled in a manner described in more detail below to provide quincunx sampling. It is also possible to quincunx sample only one component of the PIP image sample stream. For example, in a preferred embodiment, only the sample stream of the luminance component is quincunx subsampled.

The field memory 120 will store a field of subsampled PIP samples from the subsampler 116. Under the control of the main timing signal generator 106, when the PIP picture 4 is displayed, the display generator 124 extracts the previously stored subsampled samples from the field memory 120. Display generator 124 performs the inverse function of the subsampling performed by quincunx subsampler 116 to produce a sequence of samples representing the inserted auxiliary image. In addition, if the main video signal 104 is processed in the analog domain, the display generator 124 will include a digital-to-analog converter. Thus, the output signal of the display generator 124 corresponds to the output signal of the main video processor 104. Namely: if the output signals of the main video processor 104 are analog luminance and chrominance signals, respectively (as in the preferred embodiment), then the output signals of the display generator 124 will also be analog luminance and chrominance signals, respectively.

During the display of the PIP insert picture 4 the multiplexer 108 is controlled to supply decoded samples of the decoder 124 to its output. When displaying main image 2, the main video sample points from the main ADC104 will pass through the multiplexer 108. In the preferred embodiment, the main image signal and the PIP image signal include analog luminance and chrominance signals, respectively, and multiplexer 108 includes two analog signal switches, one for switching between the main video signal and the analog luminance component of the PIP video signal and the other for switching between the main video signal and the analog chrominance component of the PIP video signal. If the main video signal is demodulated into I and Q or U and V signals or further converted into R, G and B chrominance component signals, three multiplexers are provided, one for each component.

As described above, if horizontal subsampling is performed, the horizontal resolution of the image represented by the subsampled samples will be lower. Quincunx sampling, described below, is one method to overcome the reduced horizontal resolution of the PIP image after sub-sampling.

Fig. 3 illustrates the received PIP picture 4 in more detail, explaining quincunx sampling. As described above, in a preferred embodiment, the luminance and chrominance components of the PIP video signal are separated into respective streams of samples. The upper part of fig. 3 shows the sampling pattern of a part of a sequence of samples of a frame of a PIP picture. Each row in fig. 3 represents the results of three vertically adjacent rows in each field after vertical filtering (e.g., averaging). Each line of vertically filtered samples (hereinafter simply referred to as a line) is represented by a horizontal line made up of "x" or "+" (hereinafter simply referred to as a sample). Each such sample is generated at a period of the PIP 4fsc timing signal (PIP CLK), as shown in the lower portion of the received PIP display image, and includes a portion representing the luminance component of the sample and a portion representing the chrominance component of the sample. Each "x" represents a sample taken during horizontal subsampling, and each "-" represents a skipped sample. Since the PIP picture is interlaced, adjacent vertical lines that are filtered are transmitted in successive fields.

In the top row of fig. 3, every six samples have an "x" to be sampled, and the leftmost sample is the first sample to be sampled. Five samples "+" will be skipped before sampling the next sample "×". This is repeated for the remainder of the row. The samples of the third and fifth rows of the vertically filtered rows shown in the figure will be sampled at the same horizontal position as in the first row. Due to the interlaced scanning, the lines all belong to the same field. So that every line in a field will be sampled in the same way. This sampling pattern is denoted as sampling pattern SP 1.

The second line of fig. 3 is located in the next field, and the first sampled spot is the fourth spot "x". Then skip five samples "+" before taking the next sample. This is repeated for the remainder of the row. The samples in the fourth row will be decimated at the same horizontal position as the second row. The position of the sampled point in this field is located in the middle of the horizontal position of the sample point on the adjacent line in the previous field. This pattern, denoted as sample pattern SP2, may be obtained by delaying the sub-sample timing signal that generated sample pattern SP1 by three 4fsc PIP clock cycles.

The application of sampling patterns SP1 and SP2 results in a sampling pattern known as quincunx sampling and increases the subjective horizontal resolution of the PIP image by providing samples from more horizontal positions in the PIP image. The samples marked with an "x" drawn in the upper part of fig. 3 will be stored in memory 120 (of fig. 2).

The lower part of fig. 3 illustrates a method for displaying PIP insertion samples previously stored in the memory 120 (of fig. 2). The lower part of fig. 3 shows a part when the PIP picture 4 is displayed in the combined picture shown in fig. 1. Each sample point shown in the lower portion of FIG. 3 is obtained at the rate of the 4fsc master timing signal (MAIN CLK). In general, while each spot indicated by "x" is displayed, the display is immediately repeated next for the spot indicated by "o" immediately to the right of the corresponding "x".

The leftmost sample point "x" in the uppermost line of the PIP picture 4 shown will be displayed. This sample will be repeated at the next 4fsc master clock time "∘". Next, the next "X" spot "saved before is displayed and immediately repeated at" O ". This process is repeated for the remaining samples of the PIP picture line. This would be repeated for the third and fifth rows (and all odd rows) of the PIP picture. These lines are in the same field as the first line. Thus, for the PIP video signal sampling process described above, each line in a field is displayed in the same manner. This mode is designated display mode DP1 and corresponds to sampling mode SP1 described above.

The leftmost sample in the second row is "O", which is an immediate repetition of the previous sample to its left (not shown). The second sample point "x" in the second row is next displayed and immediately repeated at the next 4fsc master clock time "o". The next previously saved "x" sample will then be displayed and repeated at "o". This will be repeated for the PIP picture line. This would be repeated for the fourth row (and all even rows) of the PIP image. This pattern is designated display pattern DP2 and corresponds to sampling pattern SP2 described above. Display mode DP2 may be implemented by delaying the samples obtained in display mode DP1 by one 4fsc master clock cycle.

Each sample includes a luminance component portion and a chrominance component portion representing the sample. In a preferred embodiment, these portions of the samples will be independently converted to analog form and produce analog luminance and chrominance signals, respectively, corresponding to the analog luminance and chrominance signals produced by the main video processor 104. As we can see, the first displayed picture samples "x" representing the previously saved PIPs are arranged in the form of a quincunx display, in the same way as the sampling shown in the upper part of fig. 3. In this way, the subjective horizontal resolution of the PIP picture will be increased without increasing the storage space required by the field memory 120 (of fig. 2).

However, using quincunx sampling for PIP pictures can present a problem. This problem will be more easily understood with reference to fig. 4 and 5. Fig. 4 is a waveform diagram of a main video signal and a PIP memory access address. In fig. 4, the signal from the main video source 102 (of fig. 2) is shown as the uppermost waveform in the form of successive frames. In the case of a standard interlaced video signal, there are two fields in each frame, the lines of which are interlaced together in a known manner and are commonly referred to as the odd and even fields. Each field of the main video signal is represented by a rectangle. It is indicated in fig. 4 by the reference numerals 1 and 2, respectively, for the rectangles representing the main video signal fields. Two complete fields of the main video signal are shown in fig. 4, respectively. The left edge of each rectangle represents the temporal position of the vertical sync pulse associated with that field. The field numbers 1 and 2 in fig. 4 do not intentionally correspond to the odd and even fields of a video frame.

The signal from the PIP video signal source 110 is shown as a second waveform. This video signal also comprises successive frames, each frame comprising two fields of interlaced scanning, represented by rectangles denoted by the numbers 1 and 2, respectively. The left edge of each rectangle represents the temporal position of the vertical sync pulse associated with that field. The PIP video signal is not temporally aligned with the main video signal as indicated by the difference in the vertical sync pulse time positions of the main video signal and the PIP video signal.

Referring to fig. 2, since in the preferred embodiment the samples of the PIP video signal, which include a luminance portion and a chrominance portion, are generated by the PIP video processor 113, they are first filtered in the vertical direction and then subsampled in the horizontal direction at times controlled by the subsampler 116 in a manner described in more detail below, the subsampled samples being stored in locations in the memory 120 controlled by the write address generator 118. In response to the PIP vertical sync pulse, the write address generator resets the write address of the field memory 120 holding the subsampled PIP inserted image samples to the start position (or to the start position of the internal buffer of the field memory 120). Such buffers are typically filled from a low address to a high address, so the initial address is the minimum address.

The 3 rd waveform in fig. 4 represents the write address generated by write address generator 118. At the beginning of PIP signal field 2 (second waveform) write address generator 118 will be controlled to generate a PIP write address that is the minimum address. When the subsampler 116 (of fig. 2) generates subsampled samples, they are stored in the field memory 120 with an incremented write address generated by the write address generator 118. This is indicated in fig. 4 by the write address signals being incremented. When field 2 ends, the subsampled samples have been written into all write buffers and the address signal has reached its maximum value. The vertical sync pulse of the next field (field 1) will then reset the write address generator 118 to the beginning of the buffer (i.e., the minimum address) again and the process will repeat itself.

When the subsampled PIP samples are written to the field memory 120 in the manner described above under the control of the subsampler 116 and the write address generator 118, the display generator 124 is monitoring the scan location of the main video signal. In the first part 6 (of fig. 1) of the main picture there is no PIP picture to be displayed. The subsampled samples previously stored in the field memory 120 in the last portion 8 of the main image (say the bottom third of the combined image in the illustrated embodiment) will be read by the display generator 124 from the addresses controlled by the read address generator 122. The display generator 124 will process these samples in a manner described in more detail below to produce the sample form shown at the bottom of fig. 3. In a preferred embodiment, these samples are further processed to form respective analog luminance and chrominance signals. These signals will replace the luminance and chrominance signals of the corresponding main video at multiplexer 108 during the display of the PIP picture.

Similar to write address generator 118, the vertical sync pulse of the main video signal will control read address generator 122 to generate a start address pointing to field memory 120 (or a start address of a buffer within field memory 120) holding the PIP subsampled samples. During insertion of the PIP picture into the lower portion 8 of the combined picture, the read address generator 122 controls the field memory 120 to read PIP samples from the field memory 120 in the same order in which the subsampler 116 writes the samples into the field memory 120. Thus, when PIP image samples are inserted into the combined image, the reading of the samples proceeds gradually from the minimum position to the maximum position.

The read address generated by the read address generator 122 is represented by a fourth waveform. When the vertical sync pulse of the main video image arrives, the read address generator 122 is controlled to generate the start address of the field memory (or a buffer in the field memory). The address does not change during the first part 6 of the combined image. When the PIP samples are read from the field memory 120, the read address will increase to the maximum address at the end of the main video field 2. The read address generator 122 will be reset at the beginning of the next main video field 1 and the process is repeated.

Fig. 5 illustrates the problem that arises when quincunx sampling is employed in a PIP system. Fig. 5 shows part of the content of the field memory 120 at time TS1, which occurs in field 1 of the PIP picture shown in fig. 4. Referring to fig. 5, the upper left part of fig. 5 shows a part of the previous field (field 2) of the PIP picture and shows its sampling pattern SP2 (using the same reference numerals as in fig. 3). The dashed lines represent the lines of field 1 and field 2 is scanned from top to bottom in the normal manner, as indicated by the arrow to the left of field 2. Immediately below field 2, a portion of the next successive field (field 1) is shown, which is sampled in the manner of SP 1. The dashed lines indicate the lines of field 2. Field 1 is also scanned from top to bottom as indicated by the arrow to the left of field 1. To the right of fields 1 and 2 is the state of the portion of memory 120 corresponding to the field 1 and field 2 portions at time TS 1. As shown in fig. 5, memory 120 will be written from top to bottom as indicated by the arrow to the right of memory 120. Those skilled in the art will recognize that only subsampled samples "×" are written to memory and intervening samples "+" are not written. The graph of the memory block 120 shown in fig. 5 only shows the way in which the subsampled samples stored in the portion of the field memory 120 shown are obtained.

At the end of PIP picture field 2, field memory 120 is completely filled with subsampled samples "x" taken from field 2 using sampling pattern SP 2. At the beginning of field 1, the samples of field 2 in memory 120 are overwritten from the top to the bottom of the memory as shown in FIG. 5 by the samples of field 1 using sampling pattern SP 1. At time TS1, the bottom of memory 120 contains samples from frame 2, as indicated by the arrow for frame 2 at the bottom of memory 120; while the top portion includes samples of frame 1 as indicated by the arrow from frame 1 to the top of memory 120.

Referring to fig. 4, a portion of the PIP write address waveform of PIP video signal field 1 and the dotted line on top of the PIP read address waveform of main video signal field 2 are superimposed. At time TS1, the PIP read address is the same as the PIP write address. Referring to fig. 5, immediately prior to time TS1, 202 lines of PIP picture field 1 are subsampled from the PIP video signal and written to memory 120 at the address provided by write address generator 118 (of fig. 2). In addition, the 202 lines that were just written to memory 120 will be read from the same location in memory 120 and displayed at the appropriate location in PIP inset image 4 (of fig. 1). Immediately after time TS1, the samples of the next line 204 that make up PIP picture 4 (of fig. 1) are read from the address provided by read address generator 124. However, this line has not been received from field 1 of the PIP video signal at this time, and has not been subsampled. In contrast, the lines 204 shown on the PIP picture 4 are generated by subsampling the resulting samples in the previous field 2.

The displayed lines 202 (and the previous lines) of the PIP picture 4 are sampled from the PIP picture field 1; and line 204 (and subsequent lines) is sampled from the previous field 2 so that it is 1/60 seconds earlier than field 1 (according to the us NTSC standard). This stitching creates a so-called time seam TS, a well-known phenomenon in PIP systems using field memories. In addition to temporal seams, in the system shown in FIG. 5, samples from the low end of the field memory 120 (e.g., from the current field 1) are taken using SP1 sampling, while samples from the high end of the field memory 120 (e.g., from the previous field 2) are taken using SP2 sampling. The difference in horizontal position of the samples obtained for fields 1 and 2 using the different sampling regimes SP1 and SP2 (emphasized by the sampling regime shown in memory block 120 of fig. 5) will produce a significant discontinuity in the temporal seam position of the PIP image 4 displayed, hereinafter referred to as a spatial seam in this application. Thus, variations in the sampling pattern over the temporal seam location cause significant degradation of the PIP image over the temporal seam.

Fig. 6 corresponds to fig. 5, which shows a solution to the problem of spatial seams at temporal seam TS1 and the perceived degradation of the PIP picture. Sampling is performed at the beginning of field 2 using sampling pattern SP2 (shown in fig. 3), which continues until time TS2, shown at the bottom of fig. 4. At time TS2, the sub-sampling pattern for field 2 is changed from sampling pattern SP2 to sampling pattern SP1, and the remainder of field 2 is also sampled using sampling pattern SP 1. Sampling continues at the beginning of field 1 in sampling mode SP1 until time TS 1. At time TS1, the sampling scheme is changed to sampling scheme SP2 again. This process is repeated for all successive fields.

The result of controlling the sampling pattern in this manner appears as a state in fig. 6 in the memory 120 at time TS 1. At time TS1, the bottom of memory 120 still holds the partial samples taken at the bottom of field 2 (e.g., below the line representing time TS2 in field 2). These samples have been sampled in the manner described above using sampling SP 1. The top of memory 120 holds samples taken at the top of field 1 (e.g., above the line representing time TS1 in field 1). These samples are also sampled in sampling mode SP 1. Thus, when these samples are read out from the memory 120 to generate the PIP picture 4, the sampling pattern is uniform from the top to the bottom of the PIP picture. So that no spatial seams will be perceptible at the location of temporal seams in the main video field 1. In the same way, although not shown in the figure, all samples of the PIP image 4 of the next field (of fig. 1) to be displayed will be sampled in a sampling manner SP 2. There will likewise be no perceptible spatial seam at the temporal seam location of the field.

Fig. 7 shows a sub-sampler 116, partly in block diagram form and partly in logic diagram form, which produces sub-sampled PIP samples for storage in the field memory 120 (of fig. 2). In fig. 7, a read address signal from read address generator 122 is coupled to a first input of comparator 405, and a write address signal from write address generator 118 is coupled to a second input of comparator 405. The output of comparator 405 is connected to the set input S of S-R flip-flop 410. The non-inverted output Q of the S-R flip-flop 410 is connected to the input of an inverter 420. The output of the inverter is connected to a first input of the exclusive or gate 430. The output of exclusive or gate 430 is connected to a first input of and gate 435. The output of the and gate 435 is connected to a control input of the multiplexer 450. The PIP vertical sync pulse from the PIP timing signal generator 114 is coupled to the reset input terminal R of the S-R flip-flop 410. A signal representing the current PIP video signal field type (described below) is coupled to a second input of the xor gate 430. The active low signal indicating that a frame FREEZE operation is to be performed is connected to a second input of the and gate 435.

The PIP horizontal sync reset signal from the PIP timing signal generator 114 is coupled to a first data input of the multiplexer 450 and to an input of the delay circuit 440 for three clock cycles of the 4fsc PIP timing signal. In a preferred embodiment, the PIP horizontal sync reset signal is a pulse having a single PIP 4fsc period width that appears in the middle of the PIP horizontal sync signal. Alternatively, such a pulse signal may also occur at the beginning or end of the PIP horizontal sync signal, or anywhere within the horizontal sync signal.

An output of the delay circuit 440 is coupled to a second data input of the multiplexer 450. The output of the multiplexer 450 is connected to the reset input of the divide-by-6 circuit 460. The timing signal output of divide by 6 circuit 460 provides the PIP sub-sample timing signal and is coupled to a corresponding input of sub-sampler 470. The 4fsc PIP timing signal from PIP timing signal generator 114 is coupled to a timing signal input of divide by 6 circuit 460. A stream of PIP video samples from the PIP video processor 113 is connected to a data input of a subsampler 470. The output of the subsampler 470 provides the subsampled data and is connected to the field memory 120.

Referring to fig. 3, it can be seen that the sampling patterns SP2 and SP1 are identical, but are delayed by 3 cycles of the 4fsc PIP timing signal. In operation, the S-R flip-flop 410 will be reset by the PIP vertical sync pulse at the beginning of each PIP field. Thus at the beginning of each field, the signal at the Q output of the S-R flip-flop 410 will be a logic "0" signal. Comparator 405 monitors the read address of the PIP and the write address of the PIP. The comparator generates a logic "1" signal when they are the same (e.g., at time TS), and otherwise generates a logic "0" signal. This logic "1" signal from comparator 405 sets S-R flip-flop 410 at time TS, producing a logic "1" signal at the Q output. This signal is inverted by inverter 420 to produce a signal that is a logic "1" before TS time and a logic "0" after TS time in a field.

The PIP field type signal is a two state signal indicating the type of PIP field currently being received. Referring to fig. 4, a PIP field is either field type 1 or field type 2. As described above, the odd and even fields and field types 1 and 2 need not correspond intentionally. In the illustrated embodiment, a logical "0" represents a field type 1 and a logical "1" represents a field type 2. The xor gate 430 is used to generate a signal indicative of the correct sampling pattern (e.g., SP1 or SP2) that will be used to sub-sample the PIP video signal. In the illustrated embodiment, sampling mode SP1 will be used when the output of XOR gate 430 is a logic "1" signal, and sampling mode SP2 will be used when it is a logic "0" signal.

Referring again to the above description of fig. 5 and 6, for field type 1, sampling pattern SP1 will be used before time TS1 and sampling pattern SP2 will be used after time TS 1. For field type 2, sampling mode SP2 will be used before time TS2 and sampling mode SP1 will be used after time TS 2. If the PIP field type signal is a logic "0" indicating a field 1 and the beam TS (BEFORE TS) signal is a logic "1" preceding, for example, TS1, then the output of xor gate 430 will be a logic "1" signal indicating sampling pattern SP 1. If the BEFORE TS signal changes to a logic "0" at time TS1, the output of XOR gate 430 changes to a logic "0" signal representing sample mode SP 2. If the PIP field type signal is a logic "1" representing field 2 and the beam TS signal is a logic "1" preceding, for example, TS2, then the output of xor gate 430 will be a logic "0" signal representing sample pattern SP 2. If the BEFORE TS signal changes to a logic "0" at time TS2, the output of XOR gate 430 changes to a logic "1" signal representing sample mode SP 1.

The sampling mode select signal SP 1/' SP2 from the xor gate 430 will be controlled by the FREEZE signal at the and gate 435. As mentioned above, the FREEZE signal is used to indicate that the PIP frame is to be frozen. The writing of data to the field memory 120 (of fig. 2) is suspended during the freeze frame operation, while the read operation continues unchanged. Since no new samples are written to the field memory 120 under such conditions, the same samples are constantly read from the field memory 120 and used to generate the inserted PIP picture. The effect is to obtain a fixed or frozen inset PIP picture 4 on the display device. However, if the write operation is paused at the end of the quincunx sampled PIP field, then the portion of the field memory 120 written before time TS will include samples that are sampled in one sampling manner, while the portion of the field memory 120 written after time TS will include samples that are sampled in another sampling manner.

The spatial seam at the temporal seam location described above will be avoided by the method of controlling the sampling pattern just described during the display of a full motion PIP picture. However, this same global sampling pattern still results in spatial seams in the PIP picture at the locations of the temporal seams when the PIP picture is frozen. To prevent the display of a spatial seam at a temporal seam location when displaying a frozen PIP picture, quincunx subsampling is paused for at least two fields before the writing of PIP subsampled samples to the field memory 120 is paused. The subsampling of the PIP interpolated picture will be replaced by a rectangular subsampling. While this reduces the subjective horizontal resolution, it eliminates spatial seams at temporal seam locations due to frozen quincunx sampled signals.

When the FREEZE signal is a logic "1" signal indicating that freezing is not imminent, the SP 1/SP 2 signal will pass through the and gate 435 to the control input of the multiplexer 450. When the FREEZE signal is a logic "0" signal indicating an impending FREEZE operation, the SP 1/SP 2 signals will be blocked and the output of the and gate 435 will be a logic "0" signal indicating that the subsequent field will be sampled SP 2. This suspends the quincunx sampling mode using both sampling modes SP1 and SP 2. Instead, the auxiliary image will be rectangular subsampled at a series of horizontal positions defined by sampling pattern SP 2. This process is continued for 2 fields under the control of further known circuitry (not shown). This further circuitry then suspends the write operation to the field memory 120. When the FREEZE is removed, the FREEZE signal is controlled to become a logic "1" signal, and therefore quincunx sampling will be initiated again.

If the signal from and gate 435 is a logic "1" signal indicating that sampling mode SP1 is used, multiplexer 450 will be controlled to connect the horizontal sync reset pulse of the PIP directly from PIP timing signal generator 114 to the reset input of divide-by-6 circuit 460. If the signal from and gate 435 is a logic "0" signal indicating that sampling mode SP2 is used, multiplexer 450 will be controlled to connect the PIP horizontal sync reset pulse three 4fsc PIP timing signal period delay circuit 440 to the reset input of divide by 6 circuit 460.

The divide-by-6 counter generates a sub-sampling pulse every 6 th cycle of the 4fsc timing signal from the time the reset pulse is received from the multiplexer 450. If the input of divide by 6 circuit 460 receives a non-delayed PIP horizontal sync reset pulse from multiplexer 450, then sampling will occur at various times in sampling mode SP1 (of fig. 3). If the input of divide by 6 circuit 460 receives a PIP horizontal sync reset pulse from multiplexer 450 delayed by 3 clock cycles of PIP 4fsc, then sampling will occur at various times in sampling mode SP 2. A sub-sampler 470 sub-samples the stream of PIP video samples from the PIP video processor 113 according to the sampled signal from the divide-by-6 circuit 460. These subsampled samples are fed to field memory 120 (of fig. 2).

Fig. 8 shows, partly in block diagram form and partly in logic diagram form, a PIP insert image display generator 124 for generating PIP data for insertion into a main image. In fig. 8, subsampled samples from field memory 120 (of fig. 2) are supplied to an input of sample acquisition circuit 479. An output of the sample acquisition circuit 479 is coupled to an input of a master timing signal 4fsc clock cycle delay circuit 480 and to a first data input of a multiplexer 490. An output of the delay circuit 480 is connected to a second data input of the multiplexer 490. The output of the multiplexer 490 is connected to the multiplexer 108 (of fig. 2). The main field type signal is connected to a first input of the and gate 485 and the free signal (of fig. 7) is connected to a second input of the and gate 485. The output of and gate 485 is connected to the control input of multiplexer 490.

In operation, the sampling point acquisition circuit 479 reads out sampling points from the field memory 120 at positions specified by the read address signal from the read address generator 122. As previously mentioned, in a preferred embodiment, the samples have portions representing the luminance components of the samples, together with portions representing the chrominance components of the samples. Sample acquisition circuit 479 takes samples at the rate of 4fsc periods of the PIP timing signal and provides these samples to its output. Sample acquisition circuit 479 will then hold the sample at its output to repeat the next 4fsc period of the PIP timing signal. The next sample will then be fetched from the field memory 120. This process will be repeated for all samples on each row in the field memory 120.

Referring to fig. 3, during operation, the main field type signal will provide an indication of the display type (DP1 and DP2) which corresponds to the sampling mode used to sub-sample the PIP video data currently being read out of the field memory 120 (SP 1 or SP2, respectively), similar to the PIP field type signal described in fig. 7. That is, if the data currently being read out from the field memory 120 is sampled in the sampling pattern SP1, the data is displayed in the display pattern DP1, and if the data is sampled in the sampling pattern SP2, the data is displayed in the display pattern DP 2. Referring to fig. 7, the FREEZE signal is active low and is active when a FREEZE operation is to be performed. Based on this signal, quincunx sampling will be disabled. This signal is used to control the arrival of the main field type signal at the control input of the multiplexer 490.

In the illustrated embodiment, the sub-sampled samples will be displayed in display mode DP1 if the main field type signal is a logic "1" and in display mode DP2 if it is a logic "0". If enabled by the FREEZE signal, the main field signal will control the multiplexer 490. When the main field signal is a logic "1" signal indicating display mode DP1, multiplexer 490 will be controlled to connect the sampled PIP data directly from sample acquisition circuit 479 to multiplexer 108. This provides an undelayed subsampled sample and results in display DP1 shown in fig. 3. When the main field signal is a logic "0" signal indicating display mode DP2, multiplexer 490 is controlled to connect the delayed sampled PIP data directly from a main 4fsc timing signal period delay circuit 480 to multiplexer 108. The delay introduced by delay circuit 480 will provide display mode DP2 as shown in fig. 3. When the FREEZE signal indicates that the FREEZE function is to be implemented, the output of and gate 485 will be a logic "0" representing display mode DP2, which corresponds to sampling mode SP2 selected by the FREEZE signal in fig. 7.

The PIP sampling system described above and shown in the drawings provides PIP images with improved horizontal resolution without the need for additional samples or larger field memories. In addition, the system eliminates spatial seams that may exist in such systems that occur at temporal seam locations. Finally, the system also provides a method that can be used to provide a frame freeze function without introducing spatial seams at temporal seam locations of the frozen PIP pictures.

Claims (17)

1. A combined image device for displaying a main image and an auxiliary image, comprising:

a main image signal source;

a source of samples representing the auxiliary image signal;

a quincunx subsampler connected to the auxiliary image sample point source for compressing said auxiliary signal in quincunx by vertically filtering and subsampling said vertically filtered signal in a horizontal direction to produce a quincunx version; and

a signal combiner is coupled to the quincunx subsampler and the main image signal source and combines the main image signal and the compressed and quincunx sampled signal to produce a signal representing a combined image of the main image and the auxiliary image.

2. The apparatus of claim 1 wherein the quincunx subsampler subsamples the auxiliary image samples selectively in a sampling pattern from the following two sampling patterns, the first sampling pattern being samples at a first plurality of horizontal locations; and the second sampling mode is to sample the sample points at a second series of horizontal positions that are intermediate to the first series of horizontal positions.

3. The apparatus of claim 1, wherein the auxiliary image sample source comprises:

an auxiliary image signal source including a video component and a sync component;

an auxiliary analog-to-digital converter responsive to the video component of the auxiliary image signal and producing samples representative of the auxiliary video component;

an auxiliary timing signal generator responsive to the synchronous component of the auxiliary image signal generates an auxiliary sample timing signal.

4. The apparatus of claim 3 wherein the quincunx subsampler is further responsive to a timing signal from the auxiliary timing signal generator;

5. the apparatus of claim 4, wherein:

the auxiliary timing signal generator further generates an auxiliary horizontal synchronization reset signal and an auxiliary field type signal, the auxiliary field type signal being in a first state when the auxiliary video is an odd field and being in a second state when the auxiliary video is an even field;

plum blossom subsampler includes:

a sub-sampler coupled to the auxiliary sample source for sub-sampling representative samples of the auxiliary video component in response to the sub-sampling timing signal at times determined by the sub-sampling timing signal;

a timing signal divider generating a sub-sampling timing signal in response to the auxiliary sampling point timing signal and the reset signal; and

a reset signal generator responsive to the auxiliary horizontal synchronization reset signal and the auxiliary field type signal to generate a reset signal for generating a first sampling pattern in an even auxiliary field and a reset signal for generating a second sampling pattern in an odd auxiliary field.

6. The apparatus of claim 5, wherein:

the timing signal divider generates a sub-sampling timing signal for each of a predetermined number of auxiliary sample timing signals from when a reset signal is received; and

the reset signal generator generates a reset signal in response to the auxiliary field type signal in the first state, which is generated simultaneously with the auxiliary horizontal synchronization reset signal, and a reset signal in response to the auxiliary field type signal in the second state, which is a delayed auxiliary horizontal synchronization reset signal having a half of a predetermined number of the auxiliary sample timing signals.

7. The apparatus of claim 6, wherein the reset signal generator comprises:

a delay circuit delaying the auxiliary horizontal synchronization reset signal in response to the auxiliary horizontal synchronization reset signal, the delay length being half of a predetermined number of the auxiliary sampling point timing signals;

a multiplexer having a first data input responsive to the auxiliary horizontal synchronization reset signal, a second data input coupled to the delay circuit, a control input responsive to the auxiliary field type signal, and an output for generating the reset signal.

8. The apparatus of claim 5, wherein:

subsampling in a first sampling manner at respective times of horizontal lines corresponding to respective horizontal positions, and subsampling in a second sampling manner at respective times of horizontal lines corresponding to such respective horizontal positions, said respective horizontal positions being intermediate respective horizontal positions in the first sampling manner.

9. The apparatus of claim 1, further comprising a primary timing signal generator coupled to the primary image signal source, and the sample combiner comprising:

an insertion picture display generator connected to the quincunx subsampler and generating a signal representing an insertion picture in response to the timing signal from the master clock generator; and

a multiplexer having a first data input coupled to the insertion image display generator, a second data input coupled to the source of the main image signal, and an output for producing a signal representing the combined image.

10. The apparatus of claim 9, wherein:

the main timing signal generator further generates a selection signal which is in a first state when the signal from the insertion image display generator forms the combined image and which is in a second state when the main image signal forms the combined image; and

the multiplexer also includes a control input responsive to the select signal and connects the insertion image signal to its output when the select signal is in the first state and connects the main image signal to its output when the select signal is in the second state.

11. The apparatus of claim 10, wherein:

the main field type signal generated by the main timing signal generator is in a first state at an even main field and in a second state at an odd main field;

the insertion picture display generator generates the insertion picture signal in a first display mode when the main field type signal is in a first state, and in a second display mode when the main field type signal is in a second state.

12. The apparatus of claim 11, wherein:

the main time sequence signal generator generates a main sampling point time sequence signal in the form of continuous time sequence signal pulses; and is

The insertion image display generator includes:

a sampling point acquisition circuit connected to the quincunx subsampler which generates interpolated image display sampling points comprising successive pairs of sampling points, a first one of the pairs being a sampling point obtained by quincunx subsampling an auxiliary image sampling point with a main timing signal pulse, a second one of the pairs being a repetition of the first one of the pairs at the position of the next main timing signal pulse;

a circuit generates pairs of successive samples synchronized with a timing signal of a main sample when a main field type signal is in a first state, and pairs of successive samples synchronized with the timing signal of the main sample delayed by one period of a clock of the main sample when the main field type signal is in a second state.

13. The apparatus of claim 12, wherein the insertion image display generator comprises:

a circuit for delaying a period of a main sampling point timing signal, which is connected to the sampling point acquisition circuit;

a multiplexer having a first data input coupled to the sample acquisition circuit, a second data input coupled to the delay circuit, and a control input responsive to the main field type signal and an output for producing interpolated image samples.

14. The apparatus of claim 1, wherein the signal combiner comprises:

an insertion image display generator connected to the quincunx subsampler; and

a multiplexer having a first data input coupled to the insertion image display generator, a second data input coupled to the source of the main image signal, and an output for generating the combined image representative signal.

15. The apparatus of claim 1, further comprising a memory coupled between the quincunx subsampler and the signal combiner for storing a field of quincunx subsampled samples.

16. The apparatus of claim 15, wherein the memory is responsive to write address signals, and further comprising:

an auxiliary image video signal source including a sync component;

an auxiliary image timing signal generator responsive to the synchronous component of the auxiliary image signal for generating an auxiliary sample timing signal; and

a write address generator responsive to the auxiliary sample timing signals for generating write address signals for the memory.

17. The apparatus of claim 15, wherein the memory is to respond to read address signals, and further comprising:

a source of a main video signal including a synchronization component;

a main image timing signal generator responsive to the main image signal synchronization component for generating a main sample timing signal;

a read address generator responsive to the master sample timing signal for generating read address signals for the memory.