Disclosure of Invention
The invention aims to provide a device and a method for acquiring original image data in real time, which solve the problem that an ISP cannot acquire and store the original data from an image sensor in real time.
In order to solve the problems, the invention provides a real-time acquisition device of original image data, which is characterized by comprising an image sensor, a GMSL module, an FPGA and an image signal processing unit, wherein the image sensor is used for obtaining the original image data through exposure, the GMSL module is used for transmitting and copying the original image data to obtain the same two original image data, the two original image data are respectively transmitted to the FPGA and the image signal processing unit, the image signal processing unit is used for configuring exposure parameters of the image sensor according to the original image data, and the FPGA is used for acquiring and storing the original image data in real time.
Optionally, the image sensor transmits the raw image data to the GMSL module through an MIPI interface.
Optionally, the GMSL module comprises a serializer, a deserializer and a controller which are connected with each other,
The serializer is used for receiving the original image data transmitted by the image sensor and serializing the original image data;
The deserializer receives the serialized original image data transmitted by the deserializer, deserializes the original image data, copies the deserialized original image data to obtain two identical original image data, and transmits the two identical original image data to the FPGA and the image signal processing unit respectively, and
The controller is used for configuring the serializer and the deserializer so that the serializer is matched with the image sensor, and also so that the deserializer is respectively matched with the FPGA and the image signal processing unit.
Further, the serializer transmits the serialized raw image data to the deserializer via a GMSL link.
Furthermore, the deserializer transmits two identical original image data to the FPGA and the image signal processing unit through an MIPI interface.
Further, the controller is connected with the serializer and the deserializer through an I2C interface.
Optionally, the image signal processing unit is configured to obtain, according to the raw image data transmitted by the deserializer, a register parameter and an exposure parameter required by the image sensor to maintain an exposure state, configure the register parameter to the image sensor, and automatically control the exposure parameter of the image sensor according to the exposure parameter.
Optionally, the image signal processing unit is connected to the image sensor through an I2C interface.
Optionally, the device further comprises an external storage device, and the FPGA is connected with the external storage device, so that the FPGA is used for collecting the original image data transmitted by the deserializer in real time, and storing the original image data in the external storage device.
The invention also provides a real-time acquisition method of the original image data, which adopts the real-time acquisition device of the original image data and comprises the following steps:
Step S1, an image sensor performs exposure according to initial exposure parameters to obtain original image data, and transmits the original image data to a GMSL module;
s2, the GMSL module transmits and copies the original image data to obtain two identical original image data;
s3, the GMSL module transmits two identical original image data to the FPGA3 and an image signal processing unit respectively;
Step S4, the image signal processing unit obtains new exposure parameters of the image sensor according to the original image data, the image sensor exposes according to the new exposure parameters to obtain the original image data and transmits the original image data to a GMSL module, meanwhile, the FPGA3 collects and stores the original image data in real time, and
And S5, circularly executing the steps S2-S4.
Compared with the prior art, the invention has the following beneficial effects:
The invention provides a real-time acquisition device and a method for original image data, wherein the real-time acquisition device comprises an image sensor, a GMSL module, an FPGA and an image signal processing unit, wherein the image sensor is used for acquiring the original image data through exposure, the GMSL module is used for transmitting and copying the original image data to acquire the same two original image data, the two original image data are respectively transmitted to the FPGA and the image signal processing unit, the image signal processing unit is used for configuring exposure parameters of the image sensor according to the original image data, and the FPGA is used for acquiring and storing the original image data in real time so as to solve the problem that an ISP cannot grasp and store the original data from the image sensor in real time.
Detailed Description
A real-time acquisition apparatus of raw image data and a method thereof according to the present invention will be described in further detail. The present invention will be described in more detail below with reference to the attached drawings, in which preferred embodiments of the present invention are shown, it being understood that one skilled in the art can modify the present invention described herein while still achieving the advantageous effects of the present invention. Accordingly, the following description is to be construed as broadly known to those skilled in the art and not as limiting the invention.
In the interest of clarity, not all features of an actual implementation are described. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. It should be appreciated that in the development of any such actual embodiment, numerous implementation details must be made to achieve the developer's specific goals, such as compliance with system-related or business-related constraints, which will vary from one implementation to another. In addition, it will be appreciated that such a development effort might be complex and time-consuming, but would nevertheless be a routine undertaking for those of ordinary skill in the art.
In order to make the objects and features of the present invention more comprehensible, embodiments accompanied with figures are described in detail below. It is noted that the drawings are in a very simplified form and utilize non-precise ratios, and are intended to facilitate a convenient, clear, description of the embodiments of the invention.
Fig. 1 is a block diagram of a real-time acquisition apparatus for original image data according to the present embodiment. As shown in fig. 1, the embodiment provides a real-time acquisition device of original image data, which comprises an image sensor 1, a GMSL (Gigabit Multimedia SERIAL LINKS, gigabit multimedia serial link) module 2, an FPGA3 and an image signal processing unit 4, wherein the image sensor 1 is used for obtaining original image data through exposure, the GMSL module 2 is used for receiving the original image data provided by the image sensor 1, transmitting and copying the original image data to obtain the same two original image data, and transmitting the two original image data to the FPGA3 and the image signal processing unit 4 respectively, the image signal processing unit 4 configures parameters of the image sensor 1 according to the original image data, so that the original image data obtained by the image sensor 1 can be self-adapted to a current scene, and the FPGA3 is used for acquiring and storing the original image data in real time.
The image sensor 1 is, for example, a COMS image sensor for obtaining raw image data through exposure. The original image data comprises image data information, transmission rate, len number and the like, wherein the data format of the original image data is original image data raw format. In this embodiment, for simple and fast transmission of the raw image data, the image sensor 1 transmits the raw image data to the GMSL module 2 through the MIPI interface.
The GMSL module 2 includes a serializer 21, a deserializer 22, and a controller 23 connected to each other, and the serializer 21 is configured to receive the raw image data transmitted from the image sensor 1, and serialize the raw image data to arrange the raw image data into a packet form, and transmit the packet to the deserializer 22 in a serial form. The image sensor 1 transmits the raw image data to the serializer 21 through the MIPI interface, and the serializer 21 transmits packets to the deserializer 22 through the GMSL link.
The deserializer 22 receives the original image data serialized by the serializer 21, deserializes the original image data again, so that the original image data is restored to the original pattern, and copies the original image data restored to the original pattern to obtain two identical original image data. The deserializer 22 transmits two identical copies of the original image data to the FPGA3 and the image signal processing unit 4, respectively. In detail, the deserializer 22 transmits two identical copies of the original image data to the FPGA3 and the image signal processing unit 4, respectively, through the MIPI interface.
The controller 23 is, for example, an MCU (Microcontroller Unit, a microcontroller), and the controller 23 is configured to configure the serializer 21 and deserializer 22 so that the serializer 21 can be matched with the image sensor 1, so that the deserializer 21 can be respectively matched with the FPGA3 and the image signal processing unit 4, so that the GMSL module 2 can normally access the corresponding image sensor 1. In detail, the controller 23 configures the input mode of the serializer 21 and the len number so that the input mode of the serializer 21 matches the output mode of the image sensor 1, and the len number of the serializer 21 matches the len number of the image sensor 1. The output mode of the deserializer 22 is matched with the input mode of the FPGA3 and is also matched with the input mode of the image signal processing unit 4, the number of the deserializers 22 is matched with the number of the FPGA3 and is also matched with the number of the image signal processing unit 4, and the transmission rate of the deserializers 22 is ensured to be larger than the transmission rate of the image sensor 1, so that the transmission can be normally performed. In this embodiment, the controller 23 connects the serializer 21 and the deserializer 22 through an I2C interface.
The image signal processing unit 4 is configured to obtain register parameters and exposure parameters required for maintaining a normal exposure state of the image sensor 1 according to the raw image data transmitted by the deserializer 22, configure the register parameters to the image sensor 1, and automatically control the exposure parameters of the image sensor 1 according to the exposure parameters, so that the raw image data obtained by the image sensor 1 can adapt to a current scene. In the present embodiment, the image signal processing unit 4 is connected to the image sensor 1 through an I2C interface.
The FPGA3 is connected to an external storage device, for example, a computer, and is configured to collect the raw image data transmitted by the deserializer 22 in real time, and store the raw image data in the external storage device.
The embodiment also provides a real-time acquisition method of the original image data, which comprises the following steps:
Step S1, the image sensor 1 exposes according to initial exposure parameters to obtain original image data, and transmits the original image data to the GMSL module 2;
Step S2, the GMSL module 2 transmits and copies the original image data to obtain two identical original image data;
step S3, the GMSL module 2 respectively transmits two identical original image data to the FPGA3 and the image signal processing unit 4;
Step S4, the image signal processing unit 4 obtains new exposure parameters of the image sensor 1 according to the original image data, the image sensor performs exposure according to the new exposure parameters to obtain the original image data, and transmits the original image data to the GMSL module 2;
And step S5, circularly executing the steps S2-S4 until the exposure of the image sensor 1 is finished.
Step S2 includes the steps of:
First, the image sensor 1 transmits the raw image data to the serializer 21 through an MIPI interface, and the serializer 21 transmits a packet to a deserializer through a GMSL link.
Then, the deserializer 22 receives the original image data serialized by the serializer 21, deserializes the original image data again, so that the original image data is restored to the original pattern, and copies the original image data restored to the original pattern to obtain two identical original image data.
Next, the deserializer 22 sends two identical copies of the original image data to the FPGA3 and the image signal processing unit 4, respectively, through the MIPI interface.
Next, the controller 23 configures the serializer 21 and the deserializer 22 so that the serializer 21 can be matched with the image sensor 1, so that the deserializer 22 can be respectively matched with the FPGA3 and the image signal processing unit 4, so that the GMSL module 2 can normally access the corresponding image sensor 1.
In summary, the invention provides a device and a method for acquiring original image data in real time, which acquire the original image data of an image sensor by using a GMSL link and an FPGA, wherein the FPGA can acquire the original image data in real time, so that the invention can acquire the original image data and simultaneously support continuous acquisition of the original image data of the image sensor to perform image algorithm simulation.
Furthermore, unless specifically stated or indicated otherwise, the description of the terms "first," "second," and the like in the specification merely serve to distinguish between various components, elements, steps, etc. in the specification, and do not necessarily represent a logical or sequential relationship between various components, elements, steps, etc.
It will be appreciated that although the invention has been described above in terms of preferred embodiments, the above embodiments are not intended to limit the invention. Many possible variations and modifications of the disclosed technology can be made by anyone skilled in the art without departing from the scope of the technology, or the technology can be modified to be equivalent. Therefore, any simple modification, equivalent variation and modification of the above embodiments according to the technical substance of the present invention still fall within the scope of the technical solution of the present invention.