DE102008052930A1 - Vision sensor for recording image of part to be tested in quality control area during production processes, has image pickup sensor, where image data read from sensor is stored in image storage that is connected to software processing unit - Google Patents

Abstract

The invention relates to an image-processing sensor and comprises an image sensor (2) generating image data as well as a hardware computer unit (3) and a software computer unit (4) for evaluating the image data. The hardware computer unit (3) is arranged between the image recorder (2) and the software computer unit (4), so that image data from the image recorder (2) into the hardware computer unit (3) and from the hardware computer unit (3) into the software Computer unit (4) are transferable. The hardware computer unit (3) is assigned a local image memory in which image data read by the image recorder (2) into the hardware computer unit (3) is buffered. The local image memory is coupled to the software computer unit (4).

Description

The The invention relates to an image-processing sensor.

such image-processing sensors, that is to say so-called vision sensors, especially in the field of quality control in Manufacturing processes and the like used. Generally is there the operation of a vision sensor such that with this of a picture to be examined is taken and this Image assessed by comparison with a reference image to that effect will, whether certain quality requirements or generally Properties of the part are met.

Vision Sensors the type in question have an essential component Unit for image processing on. Unlike smart cameras Vision sensors are not based on PC architectures, but on proprietary, that is not standardized Architectures.

there exist for image processing in vision systems in general, the options that are computer-controlled Work steps, such as the image preprocessing, the implementation of image processing algorithms and the like, in hardware or Software to perform.

at The first alternative is on powerful hardware computing devices ASIC (application specific integrated circuit), CLPD (complex programmable logic device) or FPGA (field programmable gate array) to be performed Arithmetic operations or general processing steps in image processing implemented in hardware. This has the advantage that the machining operations are fast, with low power consumption and low production costs can be performed. However, because the editing operations are implemented in hardware, these can not or be changed only to a small extent, whereby such Hardware computer units are inflexible, that does not mean can be adapted to specific processes. Therefore generally become simple in such hardware computing devices Arithmetic operations such as image preprocessing or filtering performed by image data generated by the imager.

at The second alternative is the arithmetic operations or Machining processes realized in software, this being Software implemented on suitable software computer units is. Such software computer units can be used by microcontrollers or by DSP (Digital Signal Processors). such Although software computer units have the disadvantage relatively slower Processing speeds and high power consumption and high production costs. It is advantageous, however, that the implementation of the processing steps in software a high Flexibility and a high level of functionality with simpler Operation and low development costs are possible.

In The practice is generally hybrid architectures, that is Vision sensors equipped with software computer units and hardware computer units work, used.

One first example of this is a flow-through architecture, in which a hardware computer unit in the form of a CPLD or FPGA between the imager and a software computer unit is arranged in the form of a microcontroller or DSP. The way of working such a built-up vision sensor is such that with the hardware computer unit the image capture, that is the reading of image data the image sensor, takes place. Additionally takes place in the hardware computer unit an image preprocessing, such as a filtering of the image data. In the software computer unit then the further image processing steps carried out. The disadvantage of this flow-through architecture is that the powerful processing of image data in the hardware computer unit only once during image acquisition can be used.

A alternative hybrid architecture for vision sensors is the so-called co-processor architecture. In this architecture a software computer unit is downstream of the imager and a hardware computing unit is used as a co-processor. In this case, the software computer unit takes over the Image acquisition of the image data from the image recorder. This is eliminated the benefit of the flow-through architecture, in which in the hardware computing unit carried out with the image capture a first image preprocessing becomes. However, the advantage of the co-processor structure is that the hardware computer unit several times for the implementation of different Image processing steps can be used. This is however a high data transfer rate between the software computer unit and the hardware computer unit necessary, what the design effort increased for this architecture.

Of the Invention has for its object to provide a vision sensor, the high processing power with low product cost flexible programmability combined.

to Solution of this problem are the features of claim 1 intended. Advantageous embodiments and expedient Further developments of the invention are in the subclaims described.

Of the Image-processing sensor according to the invention comprises an image data-generating image sensor and a hardware computer unit and a software computer unit for evaluating the image data. The Hardware computer unit is arranged between image recorder and software computer unit, allowing image data from the imager into the hardware computing unit and transferable from the hardware computer unit to the software computer unit are. The hardware computer unit is assigned a local image memory, in which read by the imager in the hardware computer unit Image data is cached. The local image memory is coupled to the software computer unit.

The Performance of the invention Vision sensor is increased with little design effort, in which the benefits of a flow-through architecture and a co-processor architecture united in the hybrid architecture according to the invention are.

There in the case of the vision sensor according to the invention, the hardware computer unit disposed between the imager and the software computer unit is, the hardware computer unit for image acquisition, that is be used to read the image data from the image sensor, in which case it is advantageous that with the image recording in the hardware computer unit an image preprocessing, in particular a filtering of the image data, can be carried out.

One Another essential aspect of the invention is that the with the hardware computer unit read by the image sensor image data not just preprocessed and then sent to the software computer unit be delivered further, but parallel to this in the local Image memory to be cached. In addition, the local image memory, preferably broadband, coupled to the software computer unit.

By this architecture can, if necessary, the image data from the local image memory are read out and then according to a Flow-through architecture first in the hardware computing unit and then edited in the software computer unit. This means, that the hardware computer unit is usable as a co-processor, wherein the image data is multiply differently in the hardware computing unit can be edited. Due to this multiple use of the hardware computer unit On the other hand, the processing speed for image processing can be reduced be increased.

Farther is advantageous that by buffering the image data in the local computer memory associated with the hardware computer unit the frequency of image transport operations and copy operations can be minimized.

The The invention will be explained below with reference to the drawings. Show it:

1 : Schematic representation of an embodiment of the vision sensor according to the invention.

2 : Representation of one with the vision sensor according to 1 generated image with a workpiece carrying a code.

1 shows an embodiment of an image sensor in the form of a vision sensor 1 , The vision sensor 1 has an imager 2 on, which consists of a camera in the form of a CMOS sensor or the like and an upstream optics. This imager 2 is associated with a lighting, not shown, with which a space area in which the image acquisition is to take place, is illuminated.

In the imager 2 generated image data is processed in a hybrid computing architecture that is a hardware computing device 3 and a software computing unit 4 includes. The hardware computer unit 3 is formed in the present case of an FPGA. Alternatively, the hardware computer unit 3 be formed by a CPLD. The software computer unit 4 is formed in this case by a DSP. Alternatively, the software computer unit 4 be formed by a microcontroller.

When Examples are the Texas Instruments DSP TMS 320 DM 642 with 4800 MMACS (Million Multiply Accumulate Cycles Per Second) at 600 MHz Clock frequency and a 64 bit, 133 MHz memory interface and the XILINX FPGA XC3C1600E with a complexity of 1.6 million System gates and 36 integrated multipliers called.

The imager 2 and the hardware computer unit 3 be from the software computer unit 4 controlled by a processor or the like.

How out 1 it can be seen, is the hardware computer unit 3 between the imager 2 and the software computer unit 4 arranged. This will work with the hardware computer unit 3 the image capture performed, that is, the reading of the image data from the image sensor 2 , This is a data connection 6 between imagers 2 and hardware computer unit 3 intended. In the hardware computer unit 3 an image preprocessing is performed. The thus preprocessed image data be connected via another data connection 7 the software computer unit 4 supplied, where further image processing takes place.

In the software computer unit 4 In this case, an output variable is generated, which is transmitted via a communication controller 8th can be output as an interface element. About the communication controller 8th is also a configuration of the vision sensor 1 possible.

How out 1 further apparent is the hardware computer unit 3 a local image store 9 assigned. The local image memory 9 is formed in the present case of an SD-RAM. This is the local image memory 9 to the hardware computer unit 3 via a bidirectional broadband (typically 64 bit, 133 MHz) data connection 10 coupled. When the image is taken in the hardware computer unit 3 not just an image preprocessing of the image data. Rather, the image data is parallel to the local image memory 9 cached.

Also the software computer unit 4 is a local image store 11 assigned in the form of an SD-RAM, whereby also this local image memory 11 via a bidirectional, broadband data connection 12 to the software computer unit 4 is coupled. It has the software computer unit 4 forming DSP an EMIF 13 (external memory interface) as an interface for this coupling on.

From the bidirectional data connection 12 between the software computer unit 4 and the local image memory 11 branches another bidirectional broadband data connection 14 to which the hardware computer unit 3 forming FPGA. Through this data connection 14 becomes a coupling between the software computer unit 4 and the local image memory 9 the hardware computer unit 3 realized, with this coupling via the hardware computer unit 3 he follows.

From the hardware computer unit 3 associated local image memory 9 can, in particular by the software computer unit 4 , the image data are read several times and edited in various ways. Here, the flow-through structure of the architecture according to 1 be used such that the from the local image memory 9 initially read data in the hardware computer unit 3 and then in the software computer unit 4 to be edited. Thus, in this architecture without complex image transport processes, the hardware computer unit 3 used multiple times and thus used as a co-processor.

The functioning of the vision sensor 1 according to 1 will be described below with reference to the embodiment according to 2 explained.

2 shows the image area 15 one with the imager 2 recorded image with a workpiece 16 on which a data matrix code 17 is arranged. When capturing the data matrix codes 17 The problem is that the workpiece in any orientation and position in the image area 15 of the vision sensor 1 can be located, so that according to the position of the data matrix code 17 is unknown. Thus, from the outset, the search area 18 , the so-called ROI (region of interest) for the decoding of the data matrix code 17 not be determined.

This problem is solved by determining the position and orientation of the workpiece in a first step. For this purpose, a so-called Blobized (Binarized large object) analysis is used, for example, in the article "Development of FPGA-based real-time BLOB analysis circuit", J. Trein, A. Schwarzbacher, B. Hoppe, K.-H. Noffz, T. Trenschel in ISSC 2007 is described. In this BLOB analysis, image processing is performed such that objects or object features are separated from a background or other objects. A BLOB or an object is defined by a set of pixels of the image that are interrelated and different from a background. In this case, a binarized image is used, so that, for example, a BLOB is formed by a contiguous raster of dark pixels, which stand out from a background formed by light pixels. By suitable image processing algorithms, the position and orientation is then determined for such a BLOB.

In a second step, a decoding tool is used to decode the data matrix code 17 repositioned by the fact that the position and orientation of the workpiece, the position of the data matrix code 17 is determined and transforms this position into a search area (ROI) within which the data matrix codes 17 can be captured and decoded. The decoding of the data matrix code 17 will then be the output of the vision sensor 1 output.

The distribution of the individual processes required for this decoding to the individual components of the vision sensor 1 according to 1 is explained below.

In the FPGA, that is in the hardware computer unit 3 First, the image input, that is the reading of the image data of an image with the workpiece.

The original image will be in local image memory 9 cached, so that it can be read out of this at a later time again in order to carry out further processing.

Farther the FPGA optionally filters the original image. Subsequently, a binarization is carried out in the FPGA, that is from the original image, which consists of gray values or the same for the individual pixels of the image, A binary image is generated where each pixel is either a Light or dark value is assigned. Optionally, then in the FPGA a morphological processing of the binary image be performed.

Then, as a first step of the BLOB analysis in the FPGA, the search for relevant BLOBs, that is, object structures that are the workpiece with the data matrix code 17 correspond. Finally, the transfer of the BLOBs to the DSP, that is, the software computer unit 4 , for further processing.

there In the DSP, the calculation is based on the transferred BLOBs the BLOB parameter is performed, that is it will determine the position, size and direction of the blobs and thus the position and orientation of the workpiece carried out.

The determination of the new search area (ROI), in which the data matrix code is determined, is likewise carried out in the DSP on the basis of the workpiece parameters 17 of the workpiece.

Now to the data matrix code 17 To be able to decode in the FPGA, the call is made in the local image memory 9 stored original image, that is, there is a reading of the original image from the local image memory 9 in the FPGA. Optionally, filtering of the original image can first be performed in FPGA. Thereafter, the transfer of the original image (which is not binarized now) takes place to the DSP.

The decoding tool implemented in the DSP then decodes the data matrix code 17 within the recalculated search area.

Finally, via the communication controller 8th the output of an output quantity, in the simplest case of good / bad information, that is to say whether the data matrix code 17 has been detected on the workpiece or not, is formed.

1

Vision Sensor

2

imager

3

Hardware computer unit

4

Software computer unit

6

Data Connection

7

Data Connection

8th

Communications controller

9

local image memory

10

Datenvorverbindung

11

local image memory

12

Data Connection

13

EMIF

14

Data Connection

15

image area

16

workpiece

17

Data Matrix Code

18

search area

QUOTES INCLUDE IN THE DESCRIPTION

This list The documents listed by the applicant have been automated generated and is solely for better information recorded by the reader. The list is not part of the German Patent or utility model application. The DPMA takes over no liability for any errors or omissions.

Cited non-patent literature

• "Development of FPGA-based real-time BLOB-analysis circuit", J. Trein, A. Schwarzbacher, B. Hoppe, K.-H. Noffz, T. Trenschel in ISSC 2007 [0033]

Claims (9)

An image-processing sensor with an image sensor that generates an image data and with a hardware computer unit and a software computer unit for evaluating the image data, wherein the hardware computer unit is arranged between the image recorder and the software computer unit, so that image data from the image recorder into the hardware computer unit and from the Hardware computer unit can be transferred to the software computer unit, characterized in that the hardware computer unit ( 3 ) a local image memory ( 9 ) in which the image recorder ( 2 ) in the hardware computer unit ( 3 ) are buffered, and that the local image memory is sent to the software computer unit ( 4 ) is coupled. Sensor according to claim 1, characterized in that the local image memory ( 9 ) through a broadband interface ( 14 ) via the hardware computer unit ( 3 ) to the software computer unit ( 4 ) is coupled. Sensor according to one of claims 1 or 2, characterized in that in the image sensor ( 2 ) generated image data in the hardware computer unit ( 3 ) and then in the software computer unit ( 4 ) and that in parallel the image data is buffered in the local image memory. Sensor according to one of claims 1 to 3, characterized in that the image data from the local image memory can be read and then first in the hardware computer unit ( 3 ) and then in the software computer unit ( 4 ) to be edited. Sensor according to one of claims 1 to 4, characterized in that the software computer unit ( 4 ) is assigned a further local image memory. Sensor according to one of claims 1 to 5, characterized in that the image sensor ( 2 ), the software computer unit ( 4 ) and the hardware computer unit ( 3 ) from a control unit ( 5 ) to be controlled. Sensor according to one of claims 1 to 6, characterized in that the software computer unit ( 4 ) a communication controller ( 8th ) assigned. Sensor according to one of claims 1 to 7, characterized in that the hardware computer unit ( 3 ) is formed by an FPGA (field programmable gate array) or by a CPLD (complex programmable logic device). Sensor according to one of claims 1 to 8, characterized in that the software computer unit ( 4 ) is formed by a DSP (Digital Signal Processor) or by a microcontroller.