WO2000074391A1 - Interchangeable ccd camera with integral blemish compensation - Google Patents

Abstract

A CCD-type video camera (2) is provided with an electronically programmable non-volatile memory (42) which stores the location of imperfect pixel ('blemish') that can be accessed by a blemish compensator located remotely from the camera (2) in a separate camera control unit (4). This arrangement allows different CCD-type video cameras to be interchangeable, regardless of their CCD blemish content, with different camera control units containing blemish correction circuitry (22).

Description

INTERCHANGEABLE CCD CAMERA WITH INTEGRAL BLEMISH COMPENSATION

This application claims the benefit of U.S. Provisional application Serial No. 60/137,741 , filed June 2, 1999 (Attorney Docket No. OKTA-17 Prov.).

This invention relates to compensating for CCD blemishes in video cameras and more particularly to cameras that are designed to be used interchangeably with camera control units, notably but not exclusively video cameras that are incorporated in endoscopes for use with medical imaging systems.

Background And Prior Art

Video cameras with detectors based on a Charge Coupled Device (CCD) are available in many formats and are used advantageously in many disciplines, including broadcast and home video, medical imaging, industrial process control, and quality assurance. CCD cameras have several desirable qualities that make them more suitable than other image sensors in many applications; these advantages include relatively low noise, high dynamic range, high linearity, and good light sensitivity. Manufacturers constantly compete to produce devices with improved specifications, continually stretching the limits of processes, materials, and techniques to produce better devices. Cost competition is also a factor, and drives manufacturers to reduce manufacturing cycle times and material costs in efforts to maximize margins. These reductions, in turn, can affect image quality and product performance. For example, to reduce semiconductor processing time usually means less time is spent cleaning the semiconductor wafer during chip fabrication. This can result in contamination that causes performance errors, ranging from entirely non-functional chips to chips with reduced image quality. Process improvements can improve quality, but in volume production, a economically acceptable defect level is quickly reached, and it becomes cost effective to reject a certain percentage of chips or accept a certain level of defect.

Further, even under the best circumstances, variations in the properties of the semiconductor wafer and minor uncontrolled variations in the processing and printing of the various semiconductor layers onto the chip affects operating characteristics. For example, local variations in the size and electrical characteristics of individual CCD pixels affects current leakage and illumination response. This can result in pixel to pixel variations in both the baseline signal level with no illumination (dark current) and the output level with illumination applied (responsivity). Additionally, CCD defects often result in completely nonfunctional cells that show up as constant black or white pixels in the video image.

Blemish compensation for video cameras is well known, as evidenced by the following U. S. Patents: 4,193,093, issued to Richard C. St. Clair; 4,454,541 , issued to Robert A. Duschl; 4,701 ,784, issued to Hiroki Matsuoka et al.; and 5,144,446, issued to Fumihiko Sudo. Usually the blemish-compensating function is a built-in capability of the associated camera control unit (CCU). The blemish compensator portion of the CCU determines the location of pixels that fail to meet a specific performance criteria and the addresses of those "bad pixels" or blemishes are stored in digital memory in the CCU. This selection and storage may be a manual process in which the user selects individual pixels with a cursor and then presses a command button on the CCU to cause the latter to store the address of the selected pixel in the digital memory of the CCU. Subsequently when the camera is used to acquire video, a pixel address register in the CCU keeps track of each current pixel and the blemish compensator portion of the CCU causes another "better" pixel to be substituted for the "bad pixel" in the video image output of the CCU. The substituted pixels may be selected from the video stream, for example, by selecting the adjacent earlier pixel, or may be created by some other more complex process, such as by vertical and horizontal interpolation. More complex compensation schemes are also common. In the CCD blemish identification and compensation process the identification of "bad pixels" is determined by preventing light from passing into the camera to the CCD array, and measuring the signals for each pixel of the CCD.

Heretofore CCUs have also been adapted to provide an automated blemish compensation capability in which the system automatically recognizes pixels that need to be replaced and stores the addresses of those pixels in the CCU digital memory. Subsequently when the camera is used to acquire a video stream, the automatic blemish compensation process substitutes acceptable pixels for the those whose addresses are stored in the digital memory.

Figure 1 illustrates a prior art video system comprising a camera 2 and a CCU 4. The camera comprises an optical system 6 and a CCD array 10. The latter produces a video signal output when it is illuminated with light via optical system 6. This video signal output is fed by an internal transmission bus 12 to an output connector 14 which forms part of the camera. The CCU 4 comprises a controller 20 with local non-volatile digital memory 46, a blemish compensator 22 with associated volatile digital memory 48, a video signal processor 24, a pixel clock/pixel address register 50, and a push-button interface and mode selector 28 for directing operation of controller 20. The compensator 22 may be a discrete stage of the CCU, as shown. Alternatively, it may be incorporated as part of the circuitry of video processor 24. As a general matter, the blemish compensator draws pixel detail information from the volatile memory 48 to perform the appropriate compensation, for example, pixel substitution.

The CCU 4 also comprises an input connector 30 whereby the CCU may be coupled to camera output connector 14 via a suitable cable 32. Connector 30 is connected to the CCU data bus 36 which provides data access for controller 20. Connector 30 is further connected to a video bus 54 which applies the video signal to blemish compensator 22. The pixel clock/pixel address register 50 clocks the blemish compensator 22 in time with the video signal received via bus 54. The video signal stream is applied as shown at 38 to video processor 24 via blemish compensator 22. The video processor 24 produces a video output signal which appears on an output bus 40 which may be connected to a suitable display device, e.g., a TV monitor, and/or a video recorder and/or a video transmitter.

The controller 20, blemish compensator 22, and memories 46 and 48 operate differently depending on the mode of operation. All four elements are typically comprised of distributed portions of several integrated circuits mounted on a motherboard of the CCU. There are two modes of operation -- an address storage mode and an operational mode. The address storage mode and the operational mode are either commanded by the user via the push-button interface 28, or are executed by a fully automatic system commanded by controller 20. When commanded by the user via push-button interface 28, the selection and storage of the addresses of "bad pixels" is accomplished using a cursor as described above.

In the address storage mode, i.e., when the camera's CCD is being evaluated, the controller 20, blemish compensator 22 and memory 48 cooperate to store the address of pixels to be replaced in the local memory 46 of the controller. In this mode, passage of light to the CCD 10 is intentionally blocked by a suitable light blocking means 8, thereby making it possible to identify the pixels to be replaced. Typically these are identified, with the incoming light blocked, as pixels that have signals higher than the surrounding pixels. During this address storage process, the controller 20 supervises address recording and transfer to and from memories 46 and 48, while pixel clock/pixel address register 50 provides timing signals to the CCD and supplies the address of the current pixel to blemish compensator 22. The pixel clock of pixel clock/register 50 provides a vertical field reset pulse to the pixel address register of clock/register 50 once for each video field, while the pixel address register counts each pixel in the two interlaced frames that typically comprise a video frame. Thus each pixel in a frame is uniquely identified with an address. Each pixel is evaluated by compensator 22 to determine if it is higher in signal level than any element on a list of previously stored pixels, and if it is higher, its address and value are recorded in a properly ordered place on the list. The list is temporarily stored in the volatile memory 48 of the blemish compensator. When the pixel evaluation process is completed for the entire video frame, a list of addresses to be replaced is transferred to the non-volatile memory 46 of the controller.

Subsequently, in the camera operational mode, i.e., when the camera is acquiring video information for relaying to a video transmitter, a video display, or a video recorder, the controller 20, compensator 22 and memory 48 cooperate to substitute replacement pixels for those whose address is stored in memory. More specifically, when the camera is acquiring video information for relaying to a video display, transmitter or recorder, the controller 20 uploads to volatile memory 48 the address information stored in non-volatile memory 46 for use by the compensator to substitute replacement pixels for those whose address is stored on the list. In this mode, the compensator tracks the value of the pixel address register (which supplies the address of the current pixel) and replaces any pixels with an address matching an address in memory 48 with another pixel of acceptable value. Typically the system is programmed to substitute an adjacent pixel for a "bad pixel" identified in the list.

From the foregoing description of the prior art, it is clear that the "bad pixel" data is never transferred to the camera portion, but instead remains in the local controller memory. Accordingly if a camera has been subjected to pixel evaluation for existence of blemishes using a CCU with a blemish compensating capacity as described above, another camera cannot be used in its place with the same CCU unless the CCU blemish compensator is again operated to evaluate the pixels of the new camera. Moreover, assuming that blemish compensation is desired, a camera that has been evaluated by the CCU of Fig. 1 is not interchangeable with another like blemish compensator-equipped CCU unless it is first re-evaluated for blemishes.

This non-interchangeability of cameras with a CCU is especially limiting in the case of video endoscopes. During surgery, it may be necessary to employ two or more video endoscopes when only one CCU may be available. With a compensator-equipped CCU as shown in Fig. 1 , substitution of one video camera endoscope for another may be frustrated by the need to first evaluate the camera for blemishes and record the "bad pixels" in the CCU's digital memory so that compensation may be accomplished when video information is acquired by the endoscope's camera.

Object And Summary Of Invention

The object of this invention is to facilitate use of video cameras having known pixel blemishes with different camera control units.

The present invention comprises incorporating into a CCD-type video camera an electronically programmable non-volatile memory which stores the location of "bad" pixels and is controlled by a device which is located remotely from the camera in a separate CCU. This arrangement assures that video cameras are interchangeable with CCU's regardless of CCD blemish content.

The Drawings

Fig. 1 diagrammatically illustrates a prior art video system employing blemish compensation; and

Fig. 2 diagrammatically illustrates a video camera incorporating the present invention. Description Of Invention

Fig. 2 illustrates the invention. Except as described hereinafter, the system shown in Fig. 2 incorporates the same elements and the same functions as the system of Fig. 1 , and identical elements are identified by the same numerals. In this case, the camera 2 is modified by incorporating therein an electronically programmable non-volatile digital memory 42 which is coupled to camera connector 14 via a suitable bus 44. It should be noted that bus 44 may be combined with other interconnections on the motherboard and need not comprise a separate bus. The CCU 4 of Fig. 2 is like the CCU 4 of Fig. 1 in that it comprises a local digital memory 46 which is accessed via controller 20, and a second volatile blemish compensator digital memory 48. Similarly Fig. 2 illustrates a clock/pixel address register 50 which provides timing signals for the CCD readout and also clocks the blemish compensator 22. A CCU bus 52 applies the camera's video signal output containing the pixel blemish data to controller 20, and also serves to transmit pixel address data from controller 20 to camera memory 42. A second bus 54 applies the video signal to blemish compensator 22 for relay to video processor 24. Although not shown, the CCU 4 of Fig. 2 may include push button interface 28. Preferably, however, the controller 20 is programmed to automatically perform the operations described hereinafter.

Operation of the system of Fig. 2 will now be described. As in the prior art system of Fig. 1 , the controller 20 and blemish compensator 22 operate differently depending on whether the system is conducting the address storage mode or the operational mode. Similarly the controller 20 controls operation of the memories 46 and 48 as well as the blemish compensator 22. As in Fig. 1 , controller 20 controls data transfer to and from memories 46 and 48 and also, in address storage mode, it directs the address recording. In the address storage mode, the controller causes the addresses of "bad pixels" to be relayed from blemish compensator 22 to memories 48 and 46. After all the pixels are compensated, the locations are transferred to memory 42 via buses 52 and 44. In the operational mode, the data in memory 42 is transferred to memories 46 and 48 whenever the unit is energized. The blemish compensation operation is then performed by the blemish compensator 22 using pixel address data downloaded from camera memory 42. The pixel address data is downloaded from camera memory 42 to blemish compensator memory 48 by the controller 20 via the buses 44 and 52.

The address storage mode is utilized during manufacture of the camera head using the CCU 4 shown in Fig. 2 or some other CCU capable of blemish correction. During the address storage mode, passage of light to the CCD 10 is blocked by a suitable light-blocking means 8, thereby making it possible to identify the pixels to be replaced. Typically these are identified, with the incoming light blocked, as pixels that have signals higher than the surrounding pixels. Controller 20 supervises recording and transfer of data to and from memories 46 and 48, while pixel clock/pixel address register 50 provides timing signals for the CCD readout. As in Fig. 1 , during each readout of the CCD, a vertical field pulse is used to reset the pixel address register 50 which preferably is an internal counter. After this reset pulse is received, each pixel is read out in time to a "pixel clock" signal generated by clock/register 50. During each pixel read, the pixel is evaluated by blemish compensator 22 to determine if it is higher in signal level than any element on a list of previously stored pixels. If it is higher, it takes a properly ordered place on the list; if not, it is ignored. The list is temporarily stored in the blemish compensator's volatile memory 48. At the end of the process, the pixel address list in memory 48 is uploaded to memory 46 and then to the electronically programmable non-volatile memory 42 in the camera 2 via bus 44. In accordance with the stated object of this invention, in the operational mode the camera is coupled to a CCU that has a blemish correction capability. That CCU may be the same one used to determine the camera's blemish control during its manufacture or some other CCU having components like those of the CCU 4 shown in Fig. 2 for compensating for "bad" pixels.

When the operational mode is initiated with the camera coupled to a CCU 4 as described, the controller 22 downloads the pixel address data from the camera's memory 42 to the protected local memory 46 of the controller 20. Subsequently that data is loaded into the volatile memory 48 of the blemish compensator 22. Typically this data transfer from camera memory 42 to the local memory 46 of the controller occurs after a forced rest imposed by controller 20, which clears, resets, and reloads all non-protected CCU memory. Alternatively the forced rest imposed by controller 20 may be delayed until data transfer from the camera memory 42 to the protected local memory 46 of controller 20 has occurred. The double memory buffering via memories 46 and 48 permits complete reset and purge of the active video processing circuitry, including blemish compensator 22, video processor 24, memory 48, and register 50; it also permits use of faster volatile memory in the blemish compensation circuit.

Subsequently when the camera is being used to acquire video information, the blemish compensator 22 tracks the value of the pixel register 50 which supplies the address of each pixel as it is read from the CCD. When a pixel address from the pixel register matches an address in volatile memory 48, that pixel is replaced. Preferably, but not necessarily, the system is programmed so that the replacement pixel is selected from an adjacent pixel. Optionally, other modes of selecting pixels, such as averaging may also be used. After replacement, the pixel data is sent on to the video processor 24 which generates the video output from the system at output bus 40. The advantage of this invention is that once blemishes are located, this information is translated to the non-volatile memory in the camera so that it is accessible by any CCU to which the camera is attached.

Claims

WHAT IS CLAIMED IS:

1. A video camera with integral blemish compensation interchangeable with different camera control units each having a pixel blemish compensating circuits, said camera comprising a CCD imaging device for producing an output signal containing image information, a non-volatile digital memory for storing the addresses of defective pixels of the CCD imaging device, and connector means for coupling said output signal and the addresses stored in said non-volatile digital memory to a camera control unit having means for producing an output video signal and means responsive to said addresses for replacing defective pixels with acceptable pixels in each video frame in said output video signal.

2. A method for compensating for the occurrence of defective pixels in the CCD imaging device of a video camera comprising the following steps: (1 ) identifying defective pixels in the CCD imaging device and storing the addresses of said defective pixels in a non-volatile memory that forms part of the camera, and (2) subsequently while using the camera in a recording mode downloading the addresses of said defective pixels from said non-volatile memory to a pixel blemish compensator and using that blemish compensator to substitute acceptable pixels for the defective pixels in each video image frame.

3. A method according to claim 2 wherein said camera is used by coupling it to a camera control unit that includes a controller having a non-volatile local memory for storing the addresses of defective pixels downloaded from said nonvolatile of the camera and transferring said addresses to said blemish compensator.

4. A method according to claim 3 wherein said blemish compensator includes a volatile memory for storing the addresses of defective pixels transferred by said non-volatile local memory of said controller.

5. A method for compensating for pixel imperfections of a CCD imaging device that forms part of a video camera comprising the steps of providing the camera with a non-volatile digital memory, determining the addresses of the pixels of the CCD that have imperfections, and storing a map of those addresses in said nonvolatile digital memory, whereby subsequently when the camera is being used to record images those addresses can be accessed by a pixel blemish compensating means for the purpose of replacing imperfect pixels with acceptable pixels in each video frame output of the camera.

6. A video camera system comprising a video camera that includes a CCD imaging device and a non-volatile digital CCD memory containing the addresses of imperfect pixels of said CCD imaging device, and a camera control unit adapted to be coupled to said camera, said camera control unit comprising means for producing a video signal output in response to the signal output of said imaging device, a blemish compensator for replacing imperfect pixels with acceptable pixels in each video frame embodied in said video signal output, a volatile digital memory for said blemish compensator for storing the addresses of pixels, a local digital memory coupled to said controller for use in storing pixel addresses received from said non-volatile CCD memory and transferring those pixel addresses to said blemish compensator volatile digital memory, and controller for (1) accessing the imperfect pixel addresses stored in said CCD memory and transferring said addresses to said local digital memory, (2) reading said addresses out from said local memory and storing said addresses in said blemish compensator volatile digital memory, and (3) controlling operation of said blemish compensator so as to cause said blemish compensator to replace imperfect pixels with acceptable pixels in each video frame output of the camera.

7. A video camera system according to claim 6 wherein said camera control unit includes a clock for generating a timing signals for the CCD.

8. A video camera system according to claim 7 wherein said camera control unit includes a register for storing the values of pixels and supplying those values to said blemish compensator.

9. A method of compensating for imperfect pixels in video signals produced by a CCD imaging device of a video camera and rendering said camera interchangeable with different camera control units that have blemish compensating means for substituting acceptable pixels for imperfect pixels in individual video frames, said method comprising the following steps: evaluating the signal output of said CCD imaging device and determining the addresses of imperfect pixels, and storing the addresses of said imperfect pixels in a nonvolatile digital memory that forms part of the video camera.