WO2002018913A2

WO2002018913A2 - Systems and methods for providing illumination in machine vision systems

Info

Publication number: WO2002018913A2
Application number: PCT/US2001/027324
Authority: WO
Inventors: Kevin Dowling
Original assignee: Color Kinetics Inc
Current assignee: Signify North America Corp
Priority date: 2000-09-01
Filing date: 2001-08-31
Publication date: 2002-03-07
Anticipated expiration: 2003-03-01
Also published as: WO2002018913A3; AU2001288659A1

Abstract

One embodiment is a lighting system associated with a machine vision system. The machine vision system (102) may direct lighting control commands to the lighting system (114) to change the illumination conditions provided to an object (112). A vision system (110) may also be provided and associated with the machine vision system such that the vision system views and captures an image(s) of the object when lit by the lighting system. The machine vision system may direct the lighting system to change the illumination conditions and then capture the image.

Description

SYSTEMS AND METHODS FOR PROVIDING ILLUMINATION IN MACHINE VISION SYSTEMS

Related Applications This Application claims priority to and incorporates by reference herein the entire disclosure of U.S. Provisional Patent Application Serial Number 60/229,849 "Machine Vision", filed September 1, 2000.

Background of The Invention 1. Field of the Invention

The invention relates to machine vision systems. More particularly, this invention relates to lighting systems for machine vision systems. 2. Description of Related Art

Machine vision systems have been in use for many years. These systems are designed to remove the human element from the manufacturing and inspection process. Many manufacturing and inspection operations are not necessarily difficult but they are boring. It is the boredom of these tasks that causes people to become unfocused during the operation, leading to defects in the product.

Vision systems traditionally have used monochrome or gray scale systems to capture ' the images of objects. These systems typically view the object through a video camera to i detennine particular attributes of the object. The system may be used to determine if a particular attribute is present or if it is properly oriented or located within a predetermined area. These systems may provide feedback during the manufacturing process or they may be ¹ used off-line as inspection devices or manufacturing aides. The feedback system may include a computer controlled feedback loop or it may be provided through human interaction. The scales of gray may be appropriate for detennining some material attributes such as size, shape or color differences; however, when the degrees of differentiation between similarly colored parts is slight, a color system may be required. Thus, color recognition systems have also been incorporated into machine vision systems for more complex tasks. All of these systems rely on a sensory input device, typically a camera, a processor and a lighting device, to control the lighting of the object to be analyzed. The advent of powerful computing systems has allowed the development of the colored system. These color systems require more computations per second because of the complexity of the system. Many of these systems work on high-speed productions lines where any reduction in speed is intolerable, necessitating the need for increased computational power.

Lighting is a vital part of both the gray-scale and color systems. If the lighting does not create the proper contrast or color, the system will slow down, give false indications, or completely fail to operate. Several lighting systems are available for these vision system applications.

Vision systems typically require a light source with full visible-spectrum light emission. The light source produces light and the light is reflected off of an object to be inspected. The sensing system (e.g., camera) collects the reflected light and a processor analyzes the image for color and/or contrast. If the light source does not emit wavelengths compatible with the color of the object, the contrast between the targeted attribute and other attributes will be low. This low contrast can lead to false readings or false indications. To avoid such problems, vision systems generally use light sources that emit light throughout the visible spectrum. These light sources typically do not provide equal amounts of energy throughout the visible spectrum but they do emit some energy throughout most of the spectrum. In the case of an incandescent or halogen source, the red emission is high but the energy level is drastically reduced as the wavelength shortens, so there is very little blue emission. Fluorescent lamps tend to have gaps in the spectral output where there may be little or no emission. Once a light source is selected and the vision system is programmed, it is important that the lighting conditions remain constant. If the lighting conditions change, the reflected light and the resulting image may also change. Periodic calibrations may be provided to compensate for changing conditions within the light source.

There are several methods for analyzing the reflected light. One such method is referred to as the RGB or red, green, blue method. The RGB method involves representing all colors as a mixture of red, green and blue which constitute the primary colors used by video cameras, televisions, and PC monitors. When combined, these colors can create any color within the visible light spectrum. The second method is referred to as the HIS or hue, saturation and intensity method. HIS is based on how humans perceive color so the method is more intuitive than RGB. The method of data interpretation depends on the complexity of the task. Where coarse degrees of color separation are required, such as when identifying a red component from a green component, the RGB method may be chosen. Where the degrees of color are much finer, such as sorting pharmaceutical tablets or inspecting fruit, the HIS method may be used. Many other methods of data interpretation and manipulation could be used or developed.

Lighting is an integral part of all vision systems and it would be useful to have a lighting system incorporated into the vision system. It would also be useful to have the vision system communicate with the lighting system to create various effects.

Summary of the Invention One embodiment of the present invention is directed to a machine vision system comprising a first processor, and a lighting system capable of producing and modulating light, wherein the first processor communicates lighting command signals to the lighting system.

An embodiment of the present invention is directed to a method of lighting an object or surface in a machine vision system. The method comprises acts of providing a machine vision system, providing a lighting system capable of producing and modulating light, and communicating lighting command signals from the machine vision system to the lighting system.

Another embodiment of the present invention is directed to a method of lighting an object or surface in a machine vision system. The method comprises acts of providing a machine vision system that includes an optical system for viewing an object; providing an object; arranging the optical system to view the object; arranging a lighting system to irradiate the object, wherein the lighting system is capable of producing and modulating light; communicating first lighting control commands from the machine vision system to the lighting system; and executing a machine vision function and then having the machine vision system communicate second lighting control commands to the lighting system.

Brief Description of the Drawings The following figures depict certain illustrative embodiments of the invention in which like reference numerals refer to like elements. These depicted embodiments are to be understood as illustrative of the invention and not as limiting in any way.

Figure 1 illustrates a machine vision system according to one embodiment of the present invention; Figure 2 illustrates a lighting system that can be used with the machine system of Figure 1 ; and

Figure 3 illustrates a flow diagram of a lighting process for a machine vision system to one embodiment of the present invention.

Detailed Description of the Illustrative Embodiment(s^') The description below pertains to several illustrative embodiments of the invention. These are described merely for illustrative purposes, as the scope of the invention is not to be limited in any way by the disclosure below. The development of high brightness LEDs and the development of digital lighting technology has created many opportunities for LED lighting. U.S. Patents 6,016,038 and 6,211,626 describe many such lighting devices. U.S. Patents 6,016,038 and 6,211,626 are hereby incorporated by reference herein. These patents describe methods of controlling the light output from a lighting system comprised of different colored LEDs for example. Embodiments of these patents describe the use of pulse width modulation (PWM) and independent control of the LEDs to produce varying color within a color spectrum.

In one embodiment of the present invention, an LED lighting device with color changing capabilities may be used to illuminate objects in a vision system or machine vision system. The device may be an independently controlled device with operator control or the device could be incorporated into the machine vision system. The lighting device may also be networked to other devices in the manufacturing or inspection system. As an example, an upstream process change may take place and a signal could be sent to the lighting device to change its mode of operation. If the lighting device was incorporated into the vision system, the vision system processor may control the operation of the device. A light detection unit could also be employed to constantly or periodically measure the light output from the lighting device. This information could be used to recalibrate the device for overall intensity or particular wavelengths.

A lighting device according to the principles of the present invention may include two or more different colored LEDs or it could have a single colored LED. The intensities of the respective LEDs could be varied and controlled through pulse width modulation or other LED control techniques. This system may be suited for color changing effects but it may also be used in a constant color mode. This system may be used for vision applications that require fast activation times, such as on a high-speed production line. The LED device can be turned off or the intensity reduced during the periods where the product or objects are being transferred into the vision station and be reactivated during the period of desired exposure. The LED system may also be used for providing saturated colors with high energy at wavelengths where other light sources cannot produce. With red, green and blue LEDs, the high energy can be produced in all three of these areas, as compared with halogen lamps that produce very little radiation in the blue and fluorescent lamps that have gaps in one of these areas. Any variation of hue, saturation, and intensity can be made with a combination of LEDs. The LED control signals can also be adjusted to avoid any mismatch between the video capture rate and the lighting. Another unique attribute of the LED device is that it activates so quickly that the lighting can give a strobing effect. This can be used on continuous flow systems where the strobe rate appears to freeze the objects so they can be viewed. The strobing effect is commonly used where high-speed visualization is required, however, an LED system according to the principles of the present invention can provide strobing in different colors. The rate of strobe and the rate of color change may be independent and may be controlled through the vision system or as an independent device. Another example of the usefulness of a system according to one embodiment of the present invention is where the vision system needs to recognize several colors. The lighting effects can change as the system recognizes the attributes. If the object being viewed by the system has a blue and red wires that need to be identified, the system can activate the blue LEDs to create contrast between the blue wire and its surroundings and then perform a similar function with the red LEDs activated for the red wire. When only the blue LEDs are on, the blue wire will reflect the light and the red wire will not, enabling the blue wire to be easily identified. This is merely one simple example of an identification method that can be employed using an LED light source - many other recognition patterns are possible. This example serves to illustrate that the lighting effects can quickly and easily be changed to suit the particular application. Techniques using the principles of the present invention may also reduce the undesirable effects caused by uncontrolled changing lighting conditions. The high contrast provided by such a system may reduce or eliminate the need for calibration of the lighting system.

In one embodiment, a lighting system may also be used in conjunction with other lighting devices or filters to create other desired effects. Moving filters can be used with this sysiem and may be desired when the filters create a desired lighting effect and the speed of the system is not as critical.

Another example of the advantages of using a device as described herein is where ultraviolet light is required to image the object or where both ultraviolet and visible or infrared are requited. The vision system can be configured to receive the ultraviolet, visible and/or infrared radiation to make these lighting effects useful. The system can then activate the LEDs separately or simultaneously to provide various effects. Many objects reflect or fluoresce under ultraviolet light and thus can be easily identified through ultraviolet irradiation. These effects may be most pronounced when radiation is limited to the ultraviolet and in this case the ultmviolet radiation and identification process could take place before any visible identifications where made. Again, these lighting conditions can be changed almost instantaneously to provide fast system response. j Another method of computer vision involves creating three-dimensional shapes through the analysis of images and shadows cast oh the imago. The shading produced when an object is illuminated can be used effectively to recover its shape, reflection parameters, and also the direction and characteristics of the light source illuminating it, A lighting system according to the principles of the present invention may be used in such computer vision systems, The addition of color control may also aid in the reconstruction of the surface texture and color. By controlling the wavelength of radiation to more precise limits, the reflectance properties can be better approximated leading to a superior rendition of the surface properties.

Figure 1 illustrates a machine vision system 100 according to one embodiment of the present invention. In this embodiment,, the machine vision system is constructed of several elements such as a lighting system 114, a vision system (e,g._» camera) 108, and an assembly apparatus 110. These elements may be considered peripherals to the central machine vision system 102. The machine vision system-may include a processor 104 that controls and/or- receives data from the peripherals 108, 1 14, and 1 10, Tlie vision system 108 may be arranged to view an object 1 12, The object 1 12 may be any object, surface, or material and may be the subject of inspection or manufacture by or through the machine vision system. In another embodiment, the machine vision system 102 may not include assembly apparatus

1 10, because the primary objective of the system 102 may be inspection rather than assembly.

RECTIFIED SHEET (RULE 91) ISA/EP In one embodiment, the lighting system 114 may be associated with the processor 104 such that the processor directs commands to the lighting system 114 and the lighting system 114 responds by changing the illumination conditions in accordance with the commands. In one embodiment, the lighting system 114 is an LED lighting system 114 and the LED lighting system 114 may receive control signals from the processor 104. For example, the LED lighting system 114 may include one or more LEDs and the processor may generate and communicate control signals to the LED(s). The control signals may be pulse width modulated signals, pulse amplitude modulated signals, pulse displacement modulated signals or other modulated signals. The control signals may also be analog control signals (e.g., voltage, current, power control signals). In another embodiment, the lighting system 114 may include a second processor 202 (Figure 2) wherein the processor 104 communicates lighting control signals to the second processor 202. The second processor 202 may receive and interpret the lighting control signals and generate lighting element control signals. For example, the processor 104 may communicate lighting control signals indicating a specific value for the hue, saturation and intensity of the light to be generated by the lighting system 114. The second processor 202 may receive the control signals and generate lighting element control signals to control the lighting element(s) of the lighting system 114. In one embodiment, the lighting system 114 may be an LED lighting system 114 wherein the LED lighting system 114 includes a second processor. As shown in Figure 2, the LED lighting system 114 may also include several LEDs of different colors (e.g., red 204, green 208 and blue 210). The second processor may, for example, control the LEDs independently with pulse width modulated signals to change the light emitted from the lighting system 114.

In one embodiment, the lighting system 200 (Figure 2) may be included in a lighting system 114, and includes a processor 202 and one or more LEDs 204, 208, 210. The LEDs in this example are illustrated as red 204, green 208, and blue 210, but the LEDs may be any color suitable for the user's task. The processor 202 may include a data input 212. hi one embodiment, the machine vision processor 104 may communicate lighting control signals to the lighting system processor 202 through the data input 212. The communication may be accomplished through wired or wireless communication techniques. The vision system 108 may be a color vision system or a gray scale (e.g., shade of one color) or other system. Changing the lighting conditions on the object 112 can increase the contrast between sections of the object 112 in either a color system or a gray scale system, and as a result, an intelligent lighting system 114 may be useful in either system.

In one embodiment, the lighting system 114 may be capable of producing varying degrees of color (e.g., hue, saturation and intensity) and the color may include ultraviolet spectra, infrared spectra or other spectra. In one embodiment, the lighting system may be an LED lighting system with three colors (e.g., red 204, green 208 and blue 210) and these individual or groups of LEDs may be varied in intensity. As the light from these LEDs combines, a combined color is formed (additive color mixing) and the varied intensities may cause the combined color to change. For example, the machine vision system may be viewing the object 112 through the vision system 108 wherein the processor 104 is attempting to locate a blue portion of the object 112 (e.g., a wire), so the processor 104 may communicate lighting control signals to the lighting system such that the lighting system produces blue light to increase the contrast between the blue wire and the other portions of the object 112. hi one embodiment, the processor 104 may attempt to find another portion of the object 112 (e.g., a red wire) and the processor 104 may communicate lighting control signals to the lighting system 114 to change the light to red before attempting to locate the red wire to increase the contrast between the red wire and other portions of the object 112. This is just one example of how a machine vision system 102 with a controllable lighting system 114 according to the principles of the present invention can be used to increase the effectiveness of such a system, and it should be understood that such a system may be used in a variety of ways. hi one embodiment, the machine vision system 102 may be associated with more than one lighting system 114. More than one lighting systems 114 may be used to light a same object 112, or the lighting systems 114 may be used to light more than one object 112. For example, an object 112 may be lit with two or more lighting systems 114 to provide irradiation from different angles or for different cameras 108. The machine vision system 102 may direct lighting commands to the first lighting system such that the object is lit and the first camera identifies a portion of the object and then the machine vision system may direct a new command to the first lighting system 114 (e.g., turning it off) followed by lighting commands for the second lighting system 114. In this arrangement, it may be useful to provide addressable processors in the lighting systems 114 such that they can read network commands to simplify the communication system. In another embodiment, it may be desirable to have direct communication from the machine vision system 102 to the lighting systems 1 14,

One of the advantages of providing a lighting system 1 14 that is comprised of LEDs is that the LEDs can react very quickly. The LEDs can produce full output and then return to

5 zero output within frame rates of the vision system. This means an LED lighting system 1 14 can change the lighting effects or illumination conditions provided to the object 112 within the time it takes the machine vision system to recognize the portion of the object 1 12 it is attempting to identify. This provides a machine vision system 102 that can change the illumination conditions for each operation it under takes without slowing the equipment,

I o which can be important in some applications.

Although many of the embodiments herein describe LED lighting systems I \4_t in other embodiments the lighting system may be comprised of illumination sources other than LEDs. For example, the lighting system 114 may include a ^"fluorescent, incandescent, halogen, high intensity discharge or other illumination source, In one embodiment, one of

IS these other illumination sources may be combined with LEDs to form the lighting system 114, For example, the lighting system may be arranged with a halogen illumination source and an LED illumination source such that the machine vision system 102 can direct lighting commands to the lighting system 114 to control the halogen and LED illumination sources. Figure 3 Is a flow diagram of a process 300 according to one embodiment of the 0 present invention. Tn this embodiment, the first act 302 is to irradiate an object, surface or the like with light from a lighting system. The lighting system may be capable of changing the properties of the light produced from the lighting system. A vision system may then be provided to view and capture an image of a portion of the object (act 304). The illumination conditions provided by the lighting system ma be used to alter the ima e as viewed or

25 captured. The image then may be communicated to a machine vision system in act 308. The machine vision system may be arranged to then perform an inspection, manipulation or other fuh-etion (act 3Ϊ0) and then direct new ^'lighting commands to the lighting system to change the illumination conditions provided to the object (act 312). In response to the lighting commands, the lighting system may then change the hue, saturation and/or intensity of the

30 light produced by the lighting system (act 314.) to provide new lighting conditions for the next portion of the object to bo identified. While this process can be implemented in

RECTIFIED SHEET (RULE 91) ISA/EP numerous ways, in an embodiment the process is executed on a processor (e.g., 104 in Figure 1) in the machine vision system.

As used herein, the term "LED" should be understood to include light emitting diodes of all types, light emitting polymers, semiconductor dies that produce light in response to current, organic LEDs, electro-luminescent strips, and other such systems. An "LED" may refer to a single light emitting diode having multiple semiconductor dies that are individually controlled. It should also be understood that the term "LED" does not restrict the package type of the LED. The term "LED" includes packaged LEDs, non-packaged LEDs, surface mount LEDs, chip on board LEDs and LEDs of all other configurations. The term "LED" also includes LEDs packaged or associated with material (e.g., a phosphor) wherein the material may convert energy from the LED to a different wavelength.

The term "illuminate" should be understood to refer to the production of a frequency of radiation by an illumination source. The term "color" should be understood to refer to any frequency of radiation within a spectrum; that is, a "color," as used herein, should be understood to encompass frequencies not only of the visible spectrum, but also frequencies in the infrared and ultraviolet areas of the spectrum, and in other areas of the electromagnetic spectrum.

Having described several embodiments of the invention in detail, various modifications and improvements will readily occur to those skilled in the art. Such modifications and improvements are intended to be within the spirit and scope of the invention. Accordingly, the foregoing description is provided by way of example only, and is not intended to be limiting. The invention is limited only as defined by the following claims and the equivalents thereto.

What is claimed is:

Claims

CLAΓMS

1. A machine vision system, comprising: a first processor; and a lighting system capable of producing and modulating light; wherein the first processor communicates lighting command signals to the lighting system.

2. The machine vision system of claim 1, wherein the lighting system further comprises an LED lighting system.

3. The machine vision system of claim 1, wherein the lighting system modulates light by changing at least one of a hue, saturation and intensity of the light.

4. The machine vision system of claim 2, wherein the LED lighting system further comprises: at least one first LED; and at least one second LED, wherein the at least one second LED produces a different color light than the at least one first LED.

5. The machine vision system of claim 2, wherein the LED lighting system further comprises: a second processor that controls the light output from the LED lighting system in accordance with the lighting command signals received from the first processor.

6. The machine vision system of claim 5, wherein the second processor independently controls the output from at least one first LED and at least one second LED, and wherein the at least one second LED produces a different color than the at least one first LED.

7. The machine vision system of claim 1 , wherein communication between the first processor and the lighting system is accomplished through at least one of wireless and wired transmission.

8. The machine vision system of claim 4, wherein the first processor communicates lighting command signals to at least one of the at least one first LED and the at least one second LED.

9. The machine vision system of claim 8, wherein the lighting command signals are at least one of pulse width modulated, pulse amplitude modulated, and pulse displacement modulated.

10. The machine vision system of claim 8, wherein the lighting command signals are at least one of analog voltage control signals, analog current control signals, and analog power control signals.

11. The machine vision system of claim 5, wherein the LED lighting system further comprises: at least one first LED; at least one second LED; wherein the at least one second LED produces a different color light than the at least one first LED; and wherein the second processor independently controls the at least one first LED and the at least one second LED.

12. The machine vision system of claim 11 , wherein the second processor controls the at least one first LED and the at least one second LED with signals that are at least one of pulse width modulated, pulse amplitude modulated, and pulse displacement modulated.

13. The machine vision system of claim 11, wherein the second processor controls at least one of the at least one first LED and the at least one second LED with signals that are analog voltage control signals, analog current control signals, and analog power control signals.

14. The machine vision system of claim 5, wherein the second processor comprises an addressable processor.

15. The machine vision system of claim 14, wherein the lighting command signals comprise addressed lighting command signals.

16. The machine vision system of claim 1 , wherein the lighting system further comprises an addressable processor, and wherein the lighting command signals are directed to the addressable processor.

17. The machine vision system of claim 16, wherein the lighting command signals comprise addressed lighting command signals.

18. The machine vision system of claim 16, wherein the lighting system further comprises an LED lighting system.

19. The machine vision system of claim 18, wherein the lighting system further comprises at least one of a fluorescent light, compact fluorescent light, incandescent light, halogen light, and high intensity discharge light.

20. A method of lighting an object or surface in a machine vision system, comprising acts of: providing a machine vision system; providing a lighting system capable of producing and modulating light; and communicating lighting command signals from the machine vision system to the lighting system.

21. The method of claim 20, wherein the lighting system comprises an LED lighting system.

22. The method of claim 21, wherein the LED system is capable of modulating at least one of hue, saturation and intensity of the light produced by the LED lighting system.

23. The method of claim 20, wherein communication between the machine vision system and the lighting system is accomplished through at least one of wired and wireless communication.

24. The method of claim 20, wherein the lighting system comprises a networked lighting system.

25. The method of claim 21, wherein the LED system further comprises at least one first LED and at least one second LED, wherein the at least one first LED produces a different color than the at least one second LED.

26. The method of claim 25, wherein the lighting system further comprises a processor that controls at least one of the at least one first LED and the at least one second LED.

27. The method of claim 25, wherein the lighting system further comprises a processor that independently controls the at least one first LED and the at least one second LED.

28. The method of claim 21 , wherein the machine vision system communicates LED control signals to the LED lighting system.

29. The method of claim 22, wherein the LED control signals are at least one of pulse width modulated signals, pulse amplitude modulated signals, and pulse displacement modulated signals.

30. The method of claim 22, wherein the LED control signals are at least one of analog voltage control signals, analog current control signals, and analog power control signals.

31. The method of claim 27, wherein the processor controls the at least one first LED and the at least one second LED with at least one of pulse width modulated signals, pulse amplitude modulated signals, and pulse displacement modulated signals.

32. The method of claim 27, wherein the processor controls the at least one first LED and the at least one second LED with at least one of analog voltage control signals, analog current control signals, and analog power control signals.

33. The method of claim 27, wherein the processor is an addressable processor.

34. The method of claim 21, wherein the lighting system further comprises at least one of a fluorescent light, compact fluorescent light, incandescent light, halogen light, and high intensity discharge light.

35. A method of lighting an object or surface in a machine vision system, comprising acts of: providing a machine vision system that includes an a optical system for viewing an object; providing an object; arranging the optical system to view the object; arranging a lighting system to irradiate the object, wherein the lighting system is capable of producing and modulating light; communicating first lighting control commands from the machine vision system to the lighting system; executing a machine vision function and then having the machine vision system communicate second lighting control commands to the lighting system.

36. The method of claim 35, wherein the machine vision system comprises a color machine vision system.

37. The method of claim 35, wherein the machine vision system comprises a grey scale vision system.

38. The method of claim 35, wherein the lighting system further comprises an LED lighting system.

39. The method of claim 38, wherein the LED lighting system further comprises: at least one first LED; and at least one second LED that produces a different color than the at least one first LED.

40. The method of claim 39, wherein the LED lighting system further comprises a processor that controls the output from at least one of the at least one first LED and the at least one second LED.

41. The method of claim 40, wherein the processor comprises an addressable processor.