DE29909708U1 - PC plug-in card - Google Patents

Description

BOEHMERT & BOEHMERT

LAW FIRM

Boehnxit & Boehmert · Nicmannswcg 133 · D-24105 Kiel

German Patent and Trademark Office Zweibrückenstr.

80297 Munich

DR.-ING. KARL BOEHMERT, PA (1899-1973) DIPL.-ING. ALBERT BOEHMERT, PA (190:-1 »3) WILHELM ;. H. STAHLBERG, RA, Bnn DR.-1NG. WALTER HOORMANN, PA·, Brcma DIiL-PHYS. DR. HEINZGODDAR.PA·,Munich DR.-ING. ROLAND LIESEGANG, PA·, Munich WOLF-DIETER KUNTZE, RA, Brems«, ABauBe DIPL-PHVS. ROBERT MÜNZHUBER, Pa (1933.1992) DR. LUDWIG KOUKER, RA, Brcran DR. (CHEM.) ANDREAS WINKLER, PA·, Bremen MICHAELA HUTH-DIERIG, RA, Munich DIPL.-PHYS. DR. MARION TÖNHARDT, PA*. DOueldorf DR. ANDREAS EBERT-WEIDENFELLER, RA, itara DIPU-ING. EVA LIESEGANG, PA·, Munich

PROF. DR. WILHELM NORDEMANN, RA, amdcnbur, DR. AXEL NORDEMANN, RA. Bcrtü. DR. JAN BERND NORDEMANN, LLM, RA, Berti« DIPL.-PHYS. EDUARD BAUMANN, PA·, Hshenkirfieii DR.-ING. GERALD KLOPSCH PA·, DOsKldorf DIPL.-ING. HANS W. GROENING, PA·, Manchen DIPL.-ING. SIEGFRIED SCHIRMER, PA·. Bielefeld DIPL-ING. DR. JAN TÖNNIES, PA, RA, Kiel DIPL.-PHYS. CHRISTIAN BfEHL, PA·, Kiel DIPL.-PHYS. DR. DOROTHEE WEBER-BRULS, PA·, Fmkim DR.-ING. MATTHIAS PHILIPP, PA·, Breram DIPL.-PHYS. DR. STEFAN SCHOHE, PA·. Some MARTIN WIRTZ, LAWYER, Bi*™ DR. DETMAR SCHAFER. RA, Bremen DIPL-CHEM. DR. ROLAND WEISS, PA, Dmseldorf DIPL.-PHYS. DR.-ING. UWE MANASSE, PA. Bremen DR. CHRISTIAN CZYCHOWSKL RA. Condition DR. CARL-RICHARD HAARMANN, LAWYER, Munich DIPL.-BIOL. DR. ARMIN K. BOHMANN, PA. Some« DIPL.-PHYS. DR. THOMAS L BITTNER, PA, Berlin DR. VOLKER SCHMITZ, LAWYER, Munich DR. FRIEDRICH NICOLAS HEISE, RA. P,

Your letter PA -PnJentanWBtfPatenl Attorney
RA - Rßeto*tnwnlf Attorney «t Law
* - European Patent Attorney
ADb EogaUuco ibt Represent^ before the Eor
Profeuioiu] RepremUikn «1 tb· Conmuar optiicfacn Mnteunu, AJicuM In ZussoTCKnateti mil/in coopenÜDa with
in TmtanrtotSc. Alien«, DIPL.-CHEM. DR. HANS ULRICH MAY. PA·, Manchen Your sign Your letter of Our sign Kiel, Your ref. Our ref. New registration D5063 June 2, 1999

De La Rue Cash Systems GmbH Starkenburgstr. 11-13, 64546 Mörfelden-Walldorf

PC plug-in card

The invention relates to a PC plug-in card for digitizing the analog color video signals of a plurality of video cameras for further processing of the data with a computer.

The effective storage of video data on a personal computer (PC) requires, according to the current state of technology, the use of a PC plug-in card, essentially consisting of a video decoder for converting the analog video data.

1966

Niemannsweg 133 · D-24105 Kiel · Telephone+49-431-84075 · Fax+49-431-84077

MUNICH - BREMEN - BERLIN - FRANKFURT - DÜSSELDORF - POTSDAM - BRANDENBURG - HÖHENKIRCHEN - KIEL - BIELEFELD - ALICANTE

http://www.boehmert.de e-mail: postmaster@boehmert.de

OUlirUVltK 1 <x tHJiitiM.cis. b · · · · ·

stream into a digital one and a chipset for compressing the individual images, e.g. according to the standardized JPEG process.

Such commercially available cards are generally used for video editing applications, i.e. there is usually only one analog video source connected and no switching between different, unsynchronized video sources takes place. This means that on the one hand only a single video decoder is used, and on the other hand this video decoder synchronizes itself to the color subcarrier of the video source using a PLL circuit.

Nevertheless, such PC plug-in cards for video data compression have several video inputs, whereby one of the inputs can be selected using an analog video multiplexer.

However, if a commercially available PC plug-in card for video data compression is switched to another camera input during operation using the analog video multiplexer, up to 500ms ('switching time') can pass until the PLL circuit has locked onto the color subcarrier of the new video source and a stable digital video data stream is available at the output of the decoder.

Commercially available PC plug-in cards are therefore not suitable for applications where more than one video source is connected, where the video sources are not synchronized with each other and where it is necessary to switch between the cameras in rapid succession, for example to record only one or two images from this source at a time.

However, the object of the invention is to achieve precisely this. This is achieved with an insert card according to the main claim.

For applications where fast switching between different, non-synchronized video sources is a priority, the switching time can be drastically reduced by the measures described below:

BOEHMERT & ÖOÖHMERi i Vl '"I*"\

1. Use of video decoders that do not use a PLL circuit, but extract the color information from an oversampled raw data stream using methods from the field of digital signal processing. These usually only require a few milliseconds to synchronize to the new color subcarrier. Since the sampling frequency is greater than the number of valid pixels, the oversampling at the output of the video decoder creates a digital video data stream with temporal gaps in the pixel reference signal.

2. Parallel use of two independent video decoders, where each video decoder is equipped with its own analog video multiplexer so that each camera signal can be connected to each decoder input. A digital multiplexer is then used to select which data stream is to be processed further. This makes it possible to always use the other decoder to synchronize with the next video source to be processed while the data from one video source is being processed by one decoder.

3. However, the use of the video decoders specified under 1. leads to the new problem that, due to the time gaps in the pixel reference signal, the digital video data stream cannot be further processed by commercially available compression chipsets or video encoders, as these require a data stream without time gaps in the pixel reference signal with a correspondingly lower pixel clock. It is therefore necessary to synthesize a digital video data stream without time gaps in the pixel reference signal from the digital video data stream supplied by the video decoders with time gaps in the pixel reference signal, which also works with the pixel frequencies according to PAL CCIR (13.5 MHz) or PAL Square Pixel (14.75 MHz).

For this purpose, two quartz oscillators generate four times these frequencies, i.e. 54 MHz and 59 MHz. The required pixel frequencies are derived from this using a so-called synchronous timing generator. A memory with FiFo behavior is used to temporarily store digital video data. The FiFo memory is controlled by the synchronous timing generator in such a way that image-relevant digital information is passed on from the input to the output without omissions. Times in which image-irrelevant information is present at the input, however, are used to synchronize the fill level of the FiFo memory. This can compensate for a vertical frequency of the video source that may deviate from the PAL standard. The size of the FiFo memory determines the maximum permissible deviation.

A particularly advantageous option is a plug-in card for PCs for digitizing the analog color video signals of a number of unsynchronized video cameras with the possibility of quickly switching between different camera inputs and the aim of

BOEHMERT&

-A-

processing of the digital data by the PC, with two independent analog video multiplexers, with the help of which the analog FBAS or Y/C video signal of any camera can be switched to the input of one of two self-contained video decoders, whose outputs, each containing the digital brightness, color and timing information, can be selected via a digital multiplexer in such a way that the information from one of the decoders is further processed by a subsequent logic, while the other decoder can already synchronize to a new video source.

The card has a circuitry link for generating a synchronous digital video data stream without time gaps in the pixel reference signal from a digital video data stream supplied by one or more video decoders with time gaps in the pixel reference signal, consisting of a memory with FiFo behavior for intermediate storage of digital video data, two quartz oscillators with frequencies of 54 MHz and 59 MHz and a synchronous timing generator for generating a digital video data stream with pixel frequencies according to PAL CCIR (13.5 MHz) or PAL Square Pixel (14.75 MHz) from the frequencies of the two quartz oscillators and for controlling the FiFo memory in such a way that image-relevant digital information is forwarded from the input to the output without omissions and times in which image-irrelevant information is present at the input are used to synchronize the fill level of the FiFo memory and thus a vertical frequency of the video source that may deviate from the PAL standard can be compensated.

Further advantages and features of the invention are described below.

Fig. 1 shows the basic structure of the image processing components of a PC plug-in card according to the invention and the interaction,

Fig. 2 the internal structure of a video decoder Bt829,

Fig. 3 the output timing of the video decoder,

Fig. 4 the output timing of the Bt856 video encoder and

Fig. 5 the VSYNC phase (picture synchronization) of a PAL video signal.

The inventive PC plug-in cards (for ISA and PCI bus) for digitizing the analog color video signals of a plurality of video cameras for further processing of the data

BOEHMERT & pOEHMEJR*r ::···. .

with a computer, have two video decoders, each with an automatic gain control, which transmit timing information to a downstream multiplexer in addition to brightness and color information, wherein a downstream multiplexer for receiving the data from one or the other video decoder is connected upstream of a FIFO memory for decoupling the asynchronous timing and generating a synchronous timing, wherein control information is applied in parallel to a synchronous timing generator, and wherein a video encoder is also provided for receiving data from the synchronous clock generator and, via a compression chip set, from the FIFO memory. The synchronous timing generator for PAL video sources is advantageously clocked with 54 MHz and 59 MHz quartz oscillators.

The individual components of the DLR MiCo ISA Light and DLR MiCo PCI PC plug-in cards and the interaction of the individual functional groups are explained below. All numbers refer to Fig. 1 in the appendix.

(1) Inputs for analog video signals: 8 or 16 analog video signals can be applied to the DLR MiCo ISA Light and DLR MiCo PCI. To illustrate the principle, only 8 video signal inputs have been drawn.

These signals provided by the video cameras are specified according to CCIR 601 as follows (only the essential parameters):

&bull; European television standard: 625 lines, interlaced scanning, 50 Hz refresh rate, PAL colour coding.

&bull; US television standard: 625 lines, interlaced scanning, 60 Hz refresh rate, NTSC color coding.

Each individual video input is provided with a switchable terminating resistor of 75 Ω (not shown here).

(2) Analog video multiplexers 2a and 2b: These multiplexers have the task of selecting one of the video signals and making it available at the output for one of the video decoders 3a and 3b. Both multiplexers can be controlled independently of each other by the control, management and readout logic, so that each video input signal can be switched to each decoder input.

(3) Video decoders 3a and 3b: The video decoders are essentially used to digitize the analog video signals and to separate the color component. Furthermore, the level of the video signal is adjusted by an automatic gain control (AGC) so that video signal levels that are too weak or too strong are "normalized" accordingly and do not lead to differences in brightness in the video images.

At the output of the video decoder, 8-bit brightness information, 8-bit color information and timing information (e.g. vertical retrace VSYNC, horizontal retrace HSYNC, horizontal pixel reference signal HREF) are available for each decoded pixel. Both multiplexers can be controlled independently of each other by the control, management and readout logic.

BOEHMERT&β&Ogr;&Egr;&EEgr;&Mgr;&Egr;&Kgr;&Eacgr; :: I'"I

When selecting video decoders, components are used that can synchronize to the frequency of the color subcarrier within a very short time after switching the analog video signal. This is usually done using methods from the field of digital signal processing.

Although the method described here works independently of the video decoder used, as long as it meets the above criteria, the type Bt829 used will be explained in more detail. Fig. 2 in the appendix shows the internal structure of the Bt829.

The Bt829 supports both FBAS and SVHS video sources (Y/C) and the video formats NTSC, PAL and SECAM. One of three FBAS or two FBAS and one SVHS sources can be selected by the internal 3-to-1 multiplexer. The analog video signals are digitized by two analog/digital converters; a double oversampling is carried out at four times the frequency of the color subcarrier Fsc.

The clock is generated by the Bt829 by connecting two quartz crystals with different frequencies. A frequency of 35.47 MHz is required to decode a PAL video signal, while an NTSC video signal requires a frequency of 28.64 MHz. This corresponds exactly to eight times the frequency of the color subcarrier Fsc of the respective video format. This makes it much easier to implement the separation of the color component from the video signal.

To synchronize to an analog video signal, the Bt829 uses the so-called Ultralock technology, which always generates the required number of pixels in a line, even with small fluctuations in the timing of the analog signal. Since the Ultralock technology is based entirely on digital signal processing methods, it enables much faster and more precise synchronization to video sources than corresponding analog technologies, which are based, for example, on a PLL that is coordinated with the color subcarrier.

The Bt829 provides a continuous pixel data stream at the output. The data rate is four times the frequency of the color subcarrier Fsc, which also corresponds to the rate of the analog/digital conversion. The Bt829 also provides the necessary clock signals, namely CLKxI and CLKx2, which operate at 4x Fsc and 8x Fsc respectively; as well as QCLK, which can be configured to only identify the valid pixels.

In addition to decoding the video signal according to CCIR 601, the Bt829 can also perform geometrically corrected decoding if the image is to be displayed on a PC monitor, a so-called square pixel sampling.

This results in the following sizes for a field:

&bull; CCIR 601, PAL: 720 (H) &khgr; 288 (V)

&bull; Square Pixel, PAL: 768 (H) &khgr; 288 (V)

&bull; CCIR 601, NTSC: 720 (H) &khgr; 288 (V)

&bull; Square Pixel, NTSC: 640 (H) &khgr; 288 (V)

The output timing of the Bt829 is shown in Fig. 3 in the appendix. It should be noted that the Bt829 delivers significantly more pixels per line due to oversampling

BOEHMERT & BÖEÜKiERl^: .* _# : * "!

than would be required based on the selected mode. It is only possible to determine from the QCLK signal which pixels can be further processed and which cannot.

Both video decoders can be controlled independently of each other by the control, management and readout logic via the I ² C bus (not shown).

(4) Digital video multiplexer: since only the digital data stream of one of the two video decoders 3a and 3b can be further processed, the digital video multiplexer is used for the corresponding selection. It is controlled accordingly by the control, management and readout logic.

(5) FIFO memory: The FIFO memory is used to temporarily store digital video data and associated timing information. It is used to decouple the asynchronous timing at the output of the digital video multiplexer and to generate synchronous timing for the compression chipset 8 and the video encoder 9. The FIFO memory is controlled by the synchronous timing generator.

(6) Synchronous timing generator: The synchronous timing generator uses the FIFO memory to generate a video data stream with synchronous timing from the asynchronous timing provided by the video decoders.

Synchronous video timing is required by most components, including the compression chipset 8 and the video encoder 9. For the example video encoder Bt856, a corresponding timing example is given in Fig. 4 in the appendix.

The differences between both timing variants are:

&bull; Every pixel is valid, i.e. exactly as many pixels are delivered as are expected after a CCIR or square pixel scan. A subsequent distinction between valid and invalid pixels using a qualification signal (QCLK) is not necessary.

&bull; In contrast to the video decoder, CLKx2 operates at a lower frequency of 27 MHz for CCIR and 29.5 MHz for square pixel sampling.

&bull; CLKxI also operates at a lower frequency of 13.5 MHz for CCIR and 14.75 MHz for square pixel sampling. Each CLKxI clock corresponds to a valid pixel; CLKxI also has no 'gaps' like the QCLK signal of the video decoder.

The synchronous timing generator generates corresponding pixel signal clocks CLKx2 and CLKxI by dividing the output signal from the external quartz oscillators 7. Depending on whether sampling is carried out according to CCIR or square pixel, the 54 MHz or 59 MHz quartz oscillator is used.

In order to generate synchronous timing from the asynchronous timing of the video decoder, the image data together with the horizontal and vertical reference signals VSYNC and HREF are written into the FiFo memory with each valid QCLK and read out again with the synthesized CLKxI.

The writing of the asynchronous data into the FiFo memory is dependent on the timing of the video source connected to the video decoder, but the reading out of the FiFo memory is done with a fixed quartz frequency. This results in

BOEHMERT & BOEtflvtERi: I !**: · "··· *··

the need for synchronization between FiFo input and FiFo output in such a way that neither image nor timing information is lost. Fig. 5 in the appendix shows the VSYNC phase (image synchronization) of a PAL video signal for explanation.

The synchronization between FiFo input and FiFo output takes place during the lines at the end of an image before a new image synchronization - indicated by VSYNC - begins. These lines do not contain any image information and are not evaluated by the compression chipset 8 and the video encoder 9.

During these lines, the synchronization algorithm attempts to fill the memory exactly halfway with image information by evaluating the 'half full flag' provided by the FiFo memory. If the FiFo memory is too full, the writing of further pixels is suppressed; otherwise, more pixels are artificially written in than are actually marked as valid by QCLK.

When VSYNC starts, the synchronization algorithm described stops and only valid pixels are written to or read from the FiFo memory. However, this means that if the timing of the video source deviates significantly from the norm, the FiFo memory may overflow or run out of power, which can lead to image and timing information being lost and the image may no longer be correctly compressed by the compression chipset 8. In practice, however, this effect usually only occurs with old video cameras whose timing is not determined by a quartz oscillator, but by a free-running RC oscillator.

In order to avoid disturbing the subsequent functional groups - especially the compression chipset 8 and the video encoder 9 - with two VSYNC pulses following one another in quick succession after a switchover of the digital video multiplexer and thus causing malfunctions, the synchronous timing generator can suppress the first VSYNC pulse after a switchover.

The synchronous timing generator is controlled by the control, management and readout logic. This determines, among other things, the quartz oscillator to be used and the image lines to be used to synchronize the FIFO memory depending on the format of the video source (PAL or NTSC).

(7) 54 MHz or 59 MHz crystal oscillators: These crystal oscillators generate the frequencies required for the synchronous timing generator 6. The frequencies specified are only suitable for operation with PAL video sources. Other frequencies must be used for NTSC video sources.

(8) Compression chipset: The image information supplied by the synchronous timing generator is compressed by the compression chipset according to a corresponding standard (JPEG, MPEG, Wavelet or others) and is then available to the PC for further processing. The compression chipset is controlled by the control, management and readout logic.

Chipsets from the manufacturer ZORAN are used for JPEG compression on the PC plug-in cards DLR MiCo ISA Light and DLR MiCo PCI. Despite different bus interfaces (ISA and PCI), both chipsets have the common feature that the size of the compressed image (target size) can be set in advance. In order to achieve this target size, the chipsets use an iteration algorithm: Based on the results ^{of a} compressed image, the parameters for the next image are calculated accordingly.

BOEHMERT & BDEHRtERTi

The assumption here is that the next image differs only slightly from the previous one.

This means that when switching the video source, the wrong parameters from the old image are used for the new image due to the change in image content, which results in significantly different target sizes than were actually set. In this case, this image is discarded and the next image must be waited for, which is then compressed with the parameters that have now been correctly determined iteratively.

This discarding of the first compressed image by the compression chipset due to an incorrect image size is referred to below as 'SoftwareSkip' and is included in the calculation of the maximum theoretical switching speed.

(9) Video encoder: The video encoder converts the image information supplied by the synchronous timing generator back into an analog video signal 10. The Bt856 can be used here, for example. In practice, however, the video encoder is only needed to commission a card, since the image at its output also switches when the signals at the input of the analog multiplexer are switched. The video encoder can be controlled by the control, management and readout logic using the I ² C bus (not shown).

(10) Analog video signal output. Carries the analog video signal generated by the video encoder 9.

(11) Control, management and readout logic: Used for communication between the PC via the PC bus 12 and the individual function groups on the DLR MiCo ISA Light and DLR MiCo PCI. Depending on the bus used (ISA or PCI), the control, management and readout logic has a different structure and other specific properties.

(12) PC bus: Used to connect the control, management and readout logic to the PC bus, the power supply of the card and the exchange of data and control information. Depending on the card, this is an ISA or PCI bus.

Interaction of the individual functional groups:

'Video double decoder': The video double decoder enables faster synchronization to the new video signal when switching between analog video sources. While one decoder decodes the current video source and supplies the necessary signals for further processing by the 'synchronous timing generator', the other decoder synchronizes itself to the next video source to be processed.

This functionality can be illustrated by an example: One image from each camera is to be processed.

function Analogue Multiple. Video Encoder VE No. 2 Digital Multiplexer Process camera 1 number 1 No. 2 number 1 K2 VEl (Kl) Switch Kl K2 Kl K2 VEl -» VE2 Camera 2 process K1->K3 K2 Sync.K3 K2 VE2 (K2) ^Turn on K3 K2 K3 Sync. K4 VE2 -» VE1 K3 K2->K4 K3

t ·

BOEHMERT & BDEHRIERTi

- 10 -

Camera 3 process K3 K4 K3 &Kgr;4 VEl (&Kgr;3) Switch K3->K5 K4 Sync.K5 &Kgr;4 VEl -» VE2 Camera 4 process K5 K4 K5 &Kgr;4 VE2 (&Kgr;4) Switch K5 K4^K1 K5 Sync. Kl VE2 -> VE1 Camera 5 processing K5 Kl K5 Kl VEl (&Kgr;5) Switch K5->K2 Kl Sync.K2 Kl VEl -> VE2 Process camera 1 K2 Kl K2 Kl VE2 (Cl) Switch K2 K1^K3 &Kgr;2 Sync.K3 VE2 -» VE1

The prerequisite for this mode of operation is that the video decoders can synchronize to the new video signal quickly enough. This is only possible when using video decoders that work on the basis of digital signal processing algorithms and are offered by several manufacturers (Harris, Brooktree). The disadvantage of these components, however, is that they generate an asynchronous digital video timing at the output, which cannot be processed in this form by the standard compression chipsets and video decoders.

'Synchronous timing generator'. The asynchronous digital video timing provided by the video decoders is converted by the 'synchronous timing generator' by controlling a FIFO accordingly into a synchronous video timing that is again compatible with the subsequent components. The only disadvantage of the 'synchronous timing generator' results from the finite size of the FIFO memory. Therefore, quartz-stable video sources must be used, although these represent the current state of the art.

Claims (1)

PC plug-in card for digitizing the analog color video signals of a plurality of video cameras for further processing of the data with a computer, characterized by 1. two independent units each consisting of an analog multiplexer and a downstream video decoder, 2. a downstream digital multiplexer designed to deliver asynchronous brightness, colour and timing information from one of the two units to a FiFo memory and in parallel to a synchronous timing generator, 3. two clock generators are provided to control the synchronous timing generator, 4. a compression chipset that is connected to receive parts of the data stream from the FiFo and remaining parts from the synchronous timing generator, 5. a bus module that is used to pass parameters to the PC plug-in card's ports and to read the compression chipset.