[go: up one dir, main page]

CN114666515B - A device and method for real-time acquisition of original image data - Google Patents

A device and method for real-time acquisition of original image data Download PDF

Info

Publication number
CN114666515B
CN114666515B CN202210323674.XA CN202210323674A CN114666515B CN 114666515 B CN114666515 B CN 114666515B CN 202210323674 A CN202210323674 A CN 202210323674A CN 114666515 B CN114666515 B CN 114666515B
Authority
CN
China
Prior art keywords
image data
original image
deserializer
signal processing
processing unit
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202210323674.XA
Other languages
Chinese (zh)
Other versions
CN114666515A (en
Inventor
赵国涛
饶文俊
潘曦明
王磊
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Shanghai Fullhan Microelectronics Co ltd
Original Assignee
Shanghai Fullhan Microelectronics Co ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Shanghai Fullhan Microelectronics Co ltd filed Critical Shanghai Fullhan Microelectronics Co ltd
Priority to CN202210323674.XA priority Critical patent/CN114666515B/en
Publication of CN114666515A publication Critical patent/CN114666515A/en
Application granted granted Critical
Publication of CN114666515B publication Critical patent/CN114666515B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N5/00Details of television systems
    • H04N5/222Studio circuitry; Studio devices; Studio equipment
    • H04N5/262Studio circuits, e.g. for mixing, switching-over, change of character of image, other special effects ; Cameras specially adapted for the electronic generation of special effects
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/50Constructional details
    • H04N23/54Mounting of pick-up tubes, electronic image sensors, deviation or focusing coils
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/70Circuitry for compensating brightness variation in the scene
    • H04N23/741Circuitry for compensating brightness variation in the scene by increasing the dynamic range of the image compared to the dynamic range of the electronic image sensors
    • HELECTRICITY
    • H04ELECTRIC COMMUNICATION TECHNIQUE
    • H04NPICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80Camera processing pipelines; Components thereof

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Studio Devices (AREA)

Abstract

本发明提供一种原始图像数据的实时获取装置及其方法,实时获取装置包括图像传感器、GMSL模组、FPGA和图像信号处理单元,所述图像传感器用于曝光获得原始图像数据;所述GMSL模组用于传输并复制所述原始图像数据,以得到相同的两份所述原始图像数据,并将两份所述原始图像数据分别传输给所述FPGA和图像信号处理单元;所述图像信号处理单元用于根据所述原始图像数据配置所述图像传感器的曝光参数;所述FPGA用于实时采集并保存所述原始图像数据,以解决ISP无法实时抓取并保存来自图像传感器的原始数据的问题。

The present invention provides a device and method for real-time acquisition of raw image data. The real-time acquisition device comprises an image sensor, a GMSL module, an FPGA and an image signal processing unit. The image sensor is used for exposing to obtain raw image data; the GMSL module is used for transmitting and copying the raw image data to obtain two identical copies of the raw image data, and transmitting the two copies of the raw image data to the FPGA and the image signal processing unit respectively; the image signal processing unit is used for configuring the exposure parameters of the image sensor according to the raw image data; the FPGA is used for real-time acquisition and storage of the raw image data to solve the problem that the ISP cannot capture and store the raw data from the image sensor in real time.

Description

Real-time acquisition device and method for original image data
Technical Field
The invention relates to an image signal processing technology, in particular to a device and a method for acquiring original image data in real time.
Background
In the field of ISP (IMAGE SIGNAL Processing) such as security and protection, developers often need to acquire original image data of an image sensor to perform lens shading correction, CCM correction and the like, sometimes also need to locate problems encountered in development through the original image data, and in addition, a chip design company needs to acquire the original image data to perform algorithm simulation, and these all need the original image data of the image sensor.
At present, for reasons of cost and the like, an ISP does not have enough resources to support the function of capturing original image data in real time, or can only capture one frame of original image data and then store the original image data into a computer through a serial port and other protocols, and the efficiency is particularly low and the stability is not good enough.
In order to support capturing raw image data in real time, one approach is for the ISP to open up adequate buffering at design time and support a high-speed transmission interface like USB, which is not acceptable at cost-sensitive ISPs. In addition, the ISP can support capturing a frame of original image data during design and then store the original image data in a serial port mode, so that the ISP cannot capture the original image data in real time, and the application scene is very limited.
In addition, general image sensor suppliers all have FPGA to acquire the original image data of image sensor, but do not support automatic exposure, adapt to the scene very singlely, can not satisfy many cases.
Disclosure of Invention
The invention aims to provide a device and a method for acquiring original image data in real time, which solve the problem that an ISP cannot acquire and store the original data from an image sensor in real time.
In order to solve the problems, the invention provides a real-time acquisition device of original image data, which is characterized by comprising an image sensor, a GMSL module, an FPGA and an image signal processing unit, wherein the image sensor is used for obtaining the original image data through exposure, the GMSL module is used for transmitting and copying the original image data to obtain the same two original image data, the two original image data are respectively transmitted to the FPGA and the image signal processing unit, the image signal processing unit is used for configuring exposure parameters of the image sensor according to the original image data, and the FPGA is used for acquiring and storing the original image data in real time.
Optionally, the image sensor transmits the raw image data to the GMSL module through an MIPI interface.
Optionally, the GMSL module comprises a serializer, a deserializer and a controller which are connected with each other,
The serializer is used for receiving the original image data transmitted by the image sensor and serializing the original image data;
The deserializer receives the serialized original image data transmitted by the deserializer, deserializes the original image data, copies the deserialized original image data to obtain two identical original image data, and transmits the two identical original image data to the FPGA and the image signal processing unit respectively, and
The controller is used for configuring the serializer and the deserializer so that the serializer is matched with the image sensor, and also so that the deserializer is respectively matched with the FPGA and the image signal processing unit.
Further, the serializer transmits the serialized raw image data to the deserializer via a GMSL link.
Furthermore, the deserializer transmits two identical original image data to the FPGA and the image signal processing unit through an MIPI interface.
Further, the controller is connected with the serializer and the deserializer through an I2C interface.
Optionally, the image signal processing unit is configured to obtain, according to the raw image data transmitted by the deserializer, a register parameter and an exposure parameter required by the image sensor to maintain an exposure state, configure the register parameter to the image sensor, and automatically control the exposure parameter of the image sensor according to the exposure parameter.
Optionally, the image signal processing unit is connected to the image sensor through an I2C interface.
Optionally, the device further comprises an external storage device, and the FPGA is connected with the external storage device, so that the FPGA is used for collecting the original image data transmitted by the deserializer in real time, and storing the original image data in the external storage device.
The invention also provides a real-time acquisition method of the original image data, which adopts the real-time acquisition device of the original image data and comprises the following steps:
Step S1, an image sensor performs exposure according to initial exposure parameters to obtain original image data, and transmits the original image data to a GMSL module;
s2, the GMSL module transmits and copies the original image data to obtain two identical original image data;
s3, the GMSL module transmits two identical original image data to the FPGA3 and an image signal processing unit respectively;
Step S4, the image signal processing unit obtains new exposure parameters of the image sensor according to the original image data, the image sensor exposes according to the new exposure parameters to obtain the original image data and transmits the original image data to a GMSL module, meanwhile, the FPGA3 collects and stores the original image data in real time, and
And S5, circularly executing the steps S2-S4.
Compared with the prior art, the invention has the following beneficial effects:
The invention provides a real-time acquisition device and a method for original image data, wherein the real-time acquisition device comprises an image sensor, a GMSL module, an FPGA and an image signal processing unit, wherein the image sensor is used for acquiring the original image data through exposure, the GMSL module is used for transmitting and copying the original image data to acquire the same two original image data, the two original image data are respectively transmitted to the FPGA and the image signal processing unit, the image signal processing unit is used for configuring exposure parameters of the image sensor according to the original image data, and the FPGA is used for acquiring and storing the original image data in real time so as to solve the problem that an ISP cannot grasp and store the original data from the image sensor in real time.
Drawings
Fig. 1 is a block diagram of a real-time acquisition device for original image data according to an embodiment of the present invention.
Detailed Description
A real-time acquisition apparatus of raw image data and a method thereof according to the present invention will be described in further detail. The present invention will be described in more detail below with reference to the attached drawings, in which preferred embodiments of the present invention are shown, it being understood that one skilled in the art can modify the present invention described herein while still achieving the advantageous effects of the present invention. Accordingly, the following description is to be construed as broadly known to those skilled in the art and not as limiting the invention.
In the interest of clarity, not all features of an actual implementation are described. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail. It should be appreciated that in the development of any such actual embodiment, numerous implementation details must be made to achieve the developer's specific goals, such as compliance with system-related or business-related constraints, which will vary from one implementation to another. In addition, it will be appreciated that such a development effort might be complex and time-consuming, but would nevertheless be a routine undertaking for those of ordinary skill in the art.
In order to make the objects and features of the present invention more comprehensible, embodiments accompanied with figures are described in detail below. It is noted that the drawings are in a very simplified form and utilize non-precise ratios, and are intended to facilitate a convenient, clear, description of the embodiments of the invention.
Fig. 1 is a block diagram of a real-time acquisition apparatus for original image data according to the present embodiment. As shown in fig. 1, the embodiment provides a real-time acquisition device of original image data, which comprises an image sensor 1, a GMSL (Gigabit Multimedia SERIAL LINKS, gigabit multimedia serial link) module 2, an FPGA3 and an image signal processing unit 4, wherein the image sensor 1 is used for obtaining original image data through exposure, the GMSL module 2 is used for receiving the original image data provided by the image sensor 1, transmitting and copying the original image data to obtain the same two original image data, and transmitting the two original image data to the FPGA3 and the image signal processing unit 4 respectively, the image signal processing unit 4 configures parameters of the image sensor 1 according to the original image data, so that the original image data obtained by the image sensor 1 can be self-adapted to a current scene, and the FPGA3 is used for acquiring and storing the original image data in real time.
The image sensor 1 is, for example, a COMS image sensor for obtaining raw image data through exposure. The original image data comprises image data information, transmission rate, len number and the like, wherein the data format of the original image data is original image data raw format. In this embodiment, for simple and fast transmission of the raw image data, the image sensor 1 transmits the raw image data to the GMSL module 2 through the MIPI interface.
The GMSL module 2 includes a serializer 21, a deserializer 22, and a controller 23 connected to each other, and the serializer 21 is configured to receive the raw image data transmitted from the image sensor 1, and serialize the raw image data to arrange the raw image data into a packet form, and transmit the packet to the deserializer 22 in a serial form. The image sensor 1 transmits the raw image data to the serializer 21 through the MIPI interface, and the serializer 21 transmits packets to the deserializer 22 through the GMSL link.
The deserializer 22 receives the original image data serialized by the serializer 21, deserializes the original image data again, so that the original image data is restored to the original pattern, and copies the original image data restored to the original pattern to obtain two identical original image data. The deserializer 22 transmits two identical copies of the original image data to the FPGA3 and the image signal processing unit 4, respectively. In detail, the deserializer 22 transmits two identical copies of the original image data to the FPGA3 and the image signal processing unit 4, respectively, through the MIPI interface.
The controller 23 is, for example, an MCU (Microcontroller Unit, a microcontroller), and the controller 23 is configured to configure the serializer 21 and deserializer 22 so that the serializer 21 can be matched with the image sensor 1, so that the deserializer 21 can be respectively matched with the FPGA3 and the image signal processing unit 4, so that the GMSL module 2 can normally access the corresponding image sensor 1. In detail, the controller 23 configures the input mode of the serializer 21 and the len number so that the input mode of the serializer 21 matches the output mode of the image sensor 1, and the len number of the serializer 21 matches the len number of the image sensor 1. The output mode of the deserializer 22 is matched with the input mode of the FPGA3 and is also matched with the input mode of the image signal processing unit 4, the number of the deserializers 22 is matched with the number of the FPGA3 and is also matched with the number of the image signal processing unit 4, and the transmission rate of the deserializers 22 is ensured to be larger than the transmission rate of the image sensor 1, so that the transmission can be normally performed. In this embodiment, the controller 23 connects the serializer 21 and the deserializer 22 through an I2C interface.
The image signal processing unit 4 is configured to obtain register parameters and exposure parameters required for maintaining a normal exposure state of the image sensor 1 according to the raw image data transmitted by the deserializer 22, configure the register parameters to the image sensor 1, and automatically control the exposure parameters of the image sensor 1 according to the exposure parameters, so that the raw image data obtained by the image sensor 1 can adapt to a current scene. In the present embodiment, the image signal processing unit 4 is connected to the image sensor 1 through an I2C interface.
The FPGA3 is connected to an external storage device, for example, a computer, and is configured to collect the raw image data transmitted by the deserializer 22 in real time, and store the raw image data in the external storage device.
The embodiment also provides a real-time acquisition method of the original image data, which comprises the following steps:
Step S1, the image sensor 1 exposes according to initial exposure parameters to obtain original image data, and transmits the original image data to the GMSL module 2;
Step S2, the GMSL module 2 transmits and copies the original image data to obtain two identical original image data;
step S3, the GMSL module 2 respectively transmits two identical original image data to the FPGA3 and the image signal processing unit 4;
Step S4, the image signal processing unit 4 obtains new exposure parameters of the image sensor 1 according to the original image data, the image sensor performs exposure according to the new exposure parameters to obtain the original image data, and transmits the original image data to the GMSL module 2;
And step S5, circularly executing the steps S2-S4 until the exposure of the image sensor 1 is finished.
Step S2 includes the steps of:
First, the image sensor 1 transmits the raw image data to the serializer 21 through an MIPI interface, and the serializer 21 transmits a packet to a deserializer through a GMSL link.
Then, the deserializer 22 receives the original image data serialized by the serializer 21, deserializes the original image data again, so that the original image data is restored to the original pattern, and copies the original image data restored to the original pattern to obtain two identical original image data.
Next, the deserializer 22 sends two identical copies of the original image data to the FPGA3 and the image signal processing unit 4, respectively, through the MIPI interface.
Next, the controller 23 configures the serializer 21 and the deserializer 22 so that the serializer 21 can be matched with the image sensor 1, so that the deserializer 22 can be respectively matched with the FPGA3 and the image signal processing unit 4, so that the GMSL module 2 can normally access the corresponding image sensor 1.
In summary, the invention provides a device and a method for acquiring original image data in real time, which acquire the original image data of an image sensor by using a GMSL link and an FPGA, wherein the FPGA can acquire the original image data in real time, so that the invention can acquire the original image data and simultaneously support continuous acquisition of the original image data of the image sensor to perform image algorithm simulation.
Furthermore, unless specifically stated or indicated otherwise, the description of the terms "first," "second," and the like in the specification merely serve to distinguish between various components, elements, steps, etc. in the specification, and do not necessarily represent a logical or sequential relationship between various components, elements, steps, etc.
It will be appreciated that although the invention has been described above in terms of preferred embodiments, the above embodiments are not intended to limit the invention. Many possible variations and modifications of the disclosed technology can be made by anyone skilled in the art without departing from the scope of the technology, or the technology can be modified to be equivalent. Therefore, any simple modification, equivalent variation and modification of the above embodiments according to the technical substance of the present invention still fall within the scope of the technical solution of the present invention.

Claims (8)

1. The real-time acquisition device of the original image data is characterized by comprising an image sensor, a GMSL module, an FPGA and an image signal processing unit,
The image sensor is used for obtaining original image data through exposure;
The GMSL module is used for transmitting and copying the original image data to obtain the same two original image data, transmitting the two original image data to the FPGA and the image signal processing unit respectively,
The GMSL module comprises a serializer, a deserializer and a controller which are connected with each other, wherein the serializer is used for receiving the original image data transmitted by the image sensor, serializing the original image data, and transmitting the original image data after serializing to the deserializer through a GMSL link;
The deserializer receives the serialized original image data transmitted by the deserializer, deserializes the original image data, copies the deserialized original image data to obtain two identical original image data, and transmits the two identical original image data to the FPGA and the image signal processing unit respectively, and
The controller is used for configuring the serializer and the deserializer so that the serializer is matched with the image sensor, and the deserializer is also matched with the FPGA and the image signal processing unit respectively;
the image signal processing unit is used for configuring exposure parameters of the image sensor according to the original image data, and the FPGA is used for acquiring and storing the original image data in real time.
2. The apparatus for acquiring raw image data in real time as claimed in claim 1, wherein said image sensor transmits said raw image data to said GMSL module via MIPI interface.
3. The apparatus according to claim 1, wherein the deserializer transmits two identical pieces of the original image data to the FPGA and the image signal processing unit, respectively, through MIPI interfaces.
4. The apparatus for acquiring original image data in real time according to claim 1, wherein the controller is connected to the serializer and deserializer through an I2C interface.
5. The apparatus for acquiring raw image data in real time according to claim 1, wherein the image signal processing unit is configured to acquire register parameters and exposure parameters required for maintaining an exposure state of the image sensor based on the raw image data transmitted by the deserializer, and to configure the register parameters to the image sensor, and to automatically control the exposure parameters of the image sensor based on the exposure parameters.
6. The apparatus for acquiring raw image data in real time as claimed in claim 1, wherein said image signal processing unit is connected to said image sensor through an I2C interface.
7. The apparatus for acquiring raw image data in real time according to claim 1, further comprising an external storage device, wherein the FPGA is connected to the external storage device, so that the FPGA is configured to acquire the raw image data transmitted by the deserializer in real time and store the raw image data in the external storage device.
8. A method for acquiring original image data in real time, which adopts the apparatus for acquiring original image data according to any one of claims 1 to 7, comprising the steps of:
Step S1, an image sensor performs exposure according to initial exposure parameters to obtain original image data, and transmits the original image data to a GMSL module;
Step S2, the GMSL module transmits and replicates the original image data to obtain two identical original image data, wherein the GMSL module comprises a serializer, a deserializer and a controller which are connected with each other, the serializer receives the original image data transmitted by the image sensor, performs serialization processing on the original image data, and transmits the serialized original image data to the deserializer through a GMSL link;
The deserializer receives the serialized original image data transmitted by the deserializer, deserializes the original image data, and copies the deserialized original image data to obtain two identical original image data;
The controller configures the serializer and the deserializer to enable the serializer to be matched with the image sensor and enable the deserializer to be matched with the FPGA and the image signal processing unit respectively;
S3, the deserializer of the GMSL module transmits two identical original image data to the FPGA and the image signal processing unit respectively;
Step S4, the image signal processing unit obtains new exposure parameters of the image sensor according to the original image data, the image sensor exposes according to the new exposure parameters to obtain the original image data and transmits the original image data to a GMSL module, meanwhile, the FPGA acquires and stores the original image data in real time, and
And S5, circularly executing the steps S2-S4.
CN202210323674.XA 2022-03-29 2022-03-29 A device and method for real-time acquisition of original image data Active CN114666515B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202210323674.XA CN114666515B (en) 2022-03-29 2022-03-29 A device and method for real-time acquisition of original image data

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202210323674.XA CN114666515B (en) 2022-03-29 2022-03-29 A device and method for real-time acquisition of original image data

Publications (2)

Publication Number Publication Date
CN114666515A CN114666515A (en) 2022-06-24
CN114666515B true CN114666515B (en) 2024-12-06

Family

ID=82033497

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202210323674.XA Active CN114666515B (en) 2022-03-29 2022-03-29 A device and method for real-time acquisition of original image data

Country Status (1)

Country Link
CN (1) CN114666515B (en)

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108924477A (en) * 2018-06-01 2018-11-30 北京图森未来科技有限公司 A kind of long-distance video processing method and system, video processing equipment

Family Cites Families (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7464877B2 (en) * 2003-11-13 2008-12-16 Metrologic Instruments, Inc. Digital imaging-based bar code symbol reading system employing image cropping pattern generator and automatic cropped image processor
WO2016161563A1 (en) * 2015-04-07 2016-10-13 SZ DJI Technology Co., Ltd. System and method for storing image data in parallel in a camera system
KR20170142708A (en) * 2016-06-20 2017-12-28 주식회사 디엠씨시스 Camera system for adas
US20190132555A1 (en) * 2017-10-30 2019-05-02 Qualcomm Incorporated Methods and systems to broadcast sensor outputs in an automotive environment
CN110620903B (en) * 2019-08-28 2021-01-29 华为技术有限公司 Camera module, image acquisition system and method for outputting image data
CN111601019B (en) * 2020-02-28 2021-11-16 北京爱芯科技有限公司 Image data processing module and electronic equipment
CN111641765B (en) * 2020-06-03 2022-04-01 远峰科技股份有限公司 Camera module, vehicle-mounted camera system and remote photographing method
CN213846868U (en) * 2020-12-31 2021-07-30 北京都是科技有限公司 Signal acquisition system

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108924477A (en) * 2018-06-01 2018-11-30 北京图森未来科技有限公司 A kind of long-distance video processing method and system, video processing equipment

Also Published As

Publication number Publication date
CN114666515A (en) 2022-06-24

Similar Documents

Publication Publication Date Title
US9778315B2 (en) Testbench builder, system, device and method having agent loopback functionality
WO2020215182A1 (en) Image data processing method, transmission method, transmission apparatus and data interface switching apparatus
CN115190254A (en) Synchronous exposure circuit, exposure method and exposure device for multiple vision sensors
JP7766597B2 (en) Communication device and communication system
CN101562544B (en) Data packet generator and data packet generating method
TWI445374B (en) Remote management system and remote management method
CN114666515B (en) A device and method for real-time acquisition of original image data
WO2017218090A1 (en) Transmission of image data and camera management commands
WO2023236696A1 (en) Image acquisition card and image acquisition method
CN112449137A (en) Data transmission system and data transmission method based on mobile industry processor interface
CN112422835B (en) High-speed image acquisition method, system, equipment and storage medium
Khalilzad et al. FPGA implementation of real-time Ethernet communication using RMII interface
CN119599865A (en) A multi-channel camera image stitching system and method based on FPGA and ARM
CN117156074B (en) Video processing device and system with multiple circuit boards
CN106650006B (en) Debugging method and system of programmable logic device and electronic design automation terminal
CN216673171U (en) Image acquisition system and image acquisition device
JP2021036425A (en) Equipment with test function and its test method
CN119621186B (en) Industrial cameras, industrial camera program loading methods and data transmission methods
JP6026001B2 (en) Data transfer apparatus and data transfer method
CN118606257B (en) Multi-core system, communication method thereof, control device and storage medium
Augustine et al. FPGA based image readout from Camera using Ethernet protocol and Python
CN112422229B (en) Physical layer reconstruction system
Wang et al. A PCIe-based Hardware Acceleration Architecture of the Communication Protocol Stack
Ilvesmäki CAMERA DRIVER DEVELOPMENT FOR EMBEDDED LINUX
CN117156244A (en) Video data processing system, method and vehicle

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant