TWI407774B - Multi-view real time 3-d image processing system - Google Patents

Abstract

A multi-view real time 3-D image processing system including a data buffering module, a display device control module, a multi-view depth processing module, a multi-view data correction module, a multi-view cavity compensation module, a multi-view interpolation module and an output module. The data buffering module pre-stores and outputs image data. The display device control module outputs multiple display device control signals and the image data. The multi-view depth processing module generates multiple view images according to the display device control signals and the image data. The multi-view data correction module eliminates delay cycles between the view images. The multi-view cavity compensation module compensates cavity phenomenon of the view images. The multi-view interpolation module processes the view images to obtain a multi-view image. The output module outputs the multi-view image.

Description

Multi-view instant stereo image processing system

The present invention relates to a multi-view instant stereo image processing system, and more particularly to a multi-view instant stereo image processing system that can be applied to an image product that requires immediate processing of stereo images.

With the maturity of flat-panel display technology, planar images can no longer meet the needs of users, so stereoscopic display technology is gradually developed and evolved and is regarded as the new generation of display technology. In the traditional 3D display, the user needs to attach a pair of polarized glasses to generate parallax images of the left and right eyes to see the stereoscopic image. However, the loading of the polarized glasses often causes the user to feel uncomfortable, and is inconvenient for the user who has already loaded the glasses, which affects the viewing quality. Recently, the display industry has developed a 3D display that does not require polarized glasses. However, in the processing of stereoscopic images, stereoscopic images in the prior art are still required to be pre-processed by software, and software pre-processing is used. The processing flow usually takes a long time to process, which is not conducive to providing stereoscopic images to the user in real time, and indirectly making it more difficult to popularize 3D displays without polarized glasses. Therefore, the development of a technology system that can accelerate the processing of stereoscopic images and enable users to instantly view stereoscopic images has become one of the important topics in the industry.

The invention relates to a multi-view real-time image processing system, which can greatly reduce the time required for stereo image processing, and thus can improve the industrial competitiveness of the stereo image.

According to a first aspect of the present invention, a multi-view instant stereo image processing system is provided, including a data buffer module, a display device control module, a multi-view depth processing module, a multi-view data correction module, and a multi-view data correction module. A viewing angle cavity repairing module, a multi-viewing interfacing module, and an output module. The data buffer module is used to pre-store and output an image data. The display device control module is configured to receive image data and output a plurality of display device control signals and image data. The multi-view depth processing module is configured to generate a plurality of view images according to the display device control signals and image data. The multi-view data correction module is used to eliminate the delay time caused by the multi-view depth processing modules of the view images, so that the view images reach the same clock action. The multi-view cavity repair module is used to repair the view images to remove the void phenomenon of the view images. The multi-view interpolating module is configured to sort the synchronized and repaired view images to obtain a multi-view image. The output module is used to output a multi-view image.

In order to make the above-mentioned contents of the present invention more comprehensible, a preferred embodiment will be described below, and in conjunction with the drawings, a detailed description is as follows:

The present invention provides a multi-view real-time stereo image processing system, which can process multi-view real-time stereo images by a hardware structure without requiring a large amount of memory, and can greatly reduce the required processing time, thereby improving The industrial competitiveness of stereoscopic images.

Please refer to FIG. 1 , which is a functional block diagram of a multi-view instant stereo image processing system in accordance with a preferred embodiment of the present invention. In the first embodiment, the multi-view real-time image processing system 100 includes a data buffering module 110, a display device control module 120, and a multi-view depth processing module (multi -view depth processing module) 130, a multi-view data correction module 140, a multi-view cavity compensation module 150, and a multi-view cavity interpolating module (multi -view interpolation module) 160, an output module 170 and a control module 102. The control module 102 essentially generates a plurality of related parameters to each of the modules 110-170 to facilitate operation of the multi-view instant stereo image processing system 100. The output module 170 can be connected to an image storage module or a display device, and is not limited. The image storage module or the display device can be a common memory card, a liquid crystal display, or the like.

The data buffer module 110 is configured to receive external image data, perform pre-storage operations on the image data, and output the image data to the display device control module 120 when the multi-view real-time image processing system 100 operates. In this embodiment, the data buffer module 110 can use two registers of about 512 bytes to perform the operation of reading/transmitting the image data. In this way, part of the image data of a single image frame can be read/transmitted in real time, instead of storing the image data of the entire image frame size in the conventional software processing. . The setting of the data buffer module 110 of the present invention not only saves time for waiting for data, but also saves memory.

The control module 102 generates a plurality of display display parameters (DDPs) to the display device control module 120 to obtain a plurality of display device control signals. The display device control module 120 receives the image data output by the data buffer module 110 and outputs a plurality of display device control signals and image data to the multi-view depth processing module 130. The display device display parameters are substantially related to a display device specification, and the display device specifications may be specifications of the display device connected to the output module 170, or may be a plurality of preset display device specifications. First, and can be further selected by the user according to his needs, it is not limited to a single display device specification. The control module 102 generates a multi-view depth parameter (MVDP) to multi-view depth processing module 130. The multi-view depth processing module 130 generates a plurality of view images according to the multi-view depth parameter and the plurality of display device control signals and image data output by the display device control module 120.

Since the plurality of view images generated by the multi-view depth processing module 130 have different delay cycles, the control module 102 generates a data synchronization parameter (DSP) to a multi-view data correction mode. The group 140, the multi-view data correction module 140 eliminates the delay time between the plurality of view images according to the data alignment parameter, and causes the plurality of view images to reach the same clock action. In addition, in the plurality of view images generated by the multi-view depth processing module 130, a void phenomenon occurs between the foreground and the background of each view image, so the control module 102 generates a multi-view cavity compensation parameter (multi-view cavity compensation). The parameter (referred to as MVCP) to the multi-view cavity repair module 150 is used to repair the image of each view to eliminate the void phenomenon of each view image, so that the view images are relatively complete.

The control module 102 generates a multi-view interpolation parameter (MVIP) to multi-view interpolating module 160. The multi-view interpolating module 160 sorts the synchronized and patched plurality of view images according to the multi-view interpolating parameters. Processed to get a complete multi-view image. Wherein, the display parameters are the same as the display device display parameters, and the multi-view depth parameter, the multi-view cavity repair parameter, and the multi-view interpolation parameter are substantially related to a display device specification. The output module 170 is configured to output the obtained multi-view image to an image storage module (not shown) for storing the multi-view image for use by the image processing component of the subsequent segment or output to the connected image. The display device displays the multi-view image.

Please refer to FIG. 2, which is a detailed block diagram of a multi-view instant stereo image processing system 100 in accordance with a preferred embodiment of the present invention. In the multi-view real-time image processing system 100, the control module 102 includes a clock generator 104 and a clock shift generator 106. The clock generator 104 is configured to generate a clock signal (CLK) to the data buffer module 110, the display device control module 120, the multi-view depth processing module 130, the multi-view data correction module 140, and multiple viewing angles. The cavity repair module 150 and the multi-view interpolation module 160. The clock signal can be used to activate the data buffer module 110, the display device control module 120, the multi-view depth processing module 130, the multi-view data correction module 140, the multi-view cavity repair module 150, and the multi-view interspersed module. 160. In addition, display device display parameters, multi-view depth parameters, data alignment parameters, multi-view cavity repair parameters, and multi-view interpolated parameters may also be integrated into the clock signal as needed. The clock shift generator 106 adjusts the clock signal to a shift clock signal (CLK') to the output module 170. The shifting clock signal can be used to enable the multi-view image output by the output module 170 to conform to display device specifications (such as display device resolution, etc.) and can be displayed by the display device.

The data buffer module 110 includes an input buffer 112, an address controller 114, a data buffer 116, and a read/write controller 118. The input buffer 112 is configured to receive external image data and pre-store the image data to the data buffer 116. The address controller 114 is configured to provide the address data so that the data buffer 116 can correctly transfer the pre-stored image data to the data temporary storage processor 128 of the display device control module 120. The read/write controller 118 is used to control the read and write operations of the data buffer 116.

The display device control module 120 includes a horizontal signal generator 122, a data enable controller 124, a vertical signal generator 126, and a data temporary storage processor 128. The horizontal signal generator 122 is used to generate a horizontal synchronization signal (Hsync). The data enable controller 124 is configured to generate a data enable signal (DE). The vertical signal generator 126 is configured to generate a vertical synchronization signal (Vsync) according to the data enable signal. The horizontal signal and the vertical signal are related to the display device specifications corresponding to the output module 170. The data temporary storage processor 128 is configured to receive and output horizontal signals, vertical signals, and image data to the multi-view depth processing module 130.

The multi-view depth processing module 130 includes at least m view processor, where m is a positive integer and is related to the aforementioned display device specifications. In the second embodiment, the multi-view depth processing module 130 includes five viewing angle processors 131 to 135 as an example, but is not limited thereto. The number of operations of the view processor depends on the user's needs, and the multi-view depth parameter outputted by the control module 102 is used to control the multi-view depth processing module 130 to select a corresponding number of view processors (as implemented herein). In the example, the multi-view depth processing module has five viewing angle processors, but it also supports less than five viewing angle processors, in other words, it can use only four or three viewing angle processors. The view processors 131-135 perform image operations on the image data according to the multi-view depth parameters, so that each of the view processors 131-135 obtain respective corresponding view images.

In this embodiment, the multi-view data correction module 140 can be implemented by a buffer 142, but is not limited thereto. Since the delay times of the respective view processors 131-135 of the previous stage are different, the buffer 142 adjusts the plurality of view images according to the data alignment parameters to eliminate the delay time of each view image and output the same to the multi-view hole repair module 150. The multi-view depth processing module 130 includes m view processor, and the multi-view hole repair module 150 also includes m view patch processors. In FIG. 2, the multi-view cavity repair module 150 includes five view repair processors 151-155. The angle-of-view patching processors 151-155 respectively repair the corresponding perspective images according to the multi-view hole repair parameters. For example, the angle of view patching processor 153 repairs the angle of view image produced by the angle of view processor 133. In addition, as in the foregoing view processor, in this embodiment, the multi-view cavity repair module has five view repair processors, but it also supports less than five cases.

The multi-view interspersing module 160 includes an interpolating processor 162. Since the display specifications of the display devices of different brands are different, the interpolating processor 162 is synchronized and repaired according to the multi-view interpolating parameters related to the specifications of the display device. A plurality of view images are interpolated to obtain a multi-view image. In this embodiment, the output module 170 is implemented by an output buffer 172, which outputs multi-view images to a display device for display or output to an image storage module for storage.

The multi-view real-time stereo image processing system disclosed in the above embodiments of the present invention has a plurality of advantages. The following only some of the advantages are described as follows: The multi-view real-time stereo image processing system of the present invention does not require a large amount of memory, and is provided by hardware. The architecture can obtain multi-view real-time stereo images, so it can meet the needs of real-time processing. In addition, the multi-view instant stereo image processing system of the present invention supports a plurality of stereoscopic image display formats, thereby improving market acceptance of stereoscopic images and promoting the industrial competitiveness of stereoscopic images. In addition, the real-time operation by the hardware architecture does not require the use of a large amount of memory and does not consume the advantages of the software operation flow, and is more advantageous for integrating the multi-view real-time stereo image processing system into the system single chip, which is suitable for the handheld device. Demand.

In the above, the present invention has been described above by way of a preferred embodiment. The embodiments of the present invention are merely for the purpose of illustration and are not intended to limit the invention. A person skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, the scope of the invention is defined by the scope of the appended claims.

100‧‧‧Multi-view instant stereo image processing system

102‧‧‧Control Module

104‧‧‧ Clock Generator

106‧‧‧clock shift generator

110‧‧‧Data buffer module

112‧‧‧Input buffer

114‧‧‧ Address Controller

116‧‧‧Data buffer

118‧‧‧Reading and writing controller

120‧‧‧Display device control module

122‧‧‧Horizontal signal generator

124‧‧‧ Data Enable Controller

126‧‧‧Vertical signal generator

128‧‧‧data temporary storage processor

130‧‧‧Multi-view depth processing module

131~135‧‧‧ view processor

140‧‧‧Multi-view data correction module

142‧‧‧buffer

150‧‧‧Multi-view cavity repair module

151~155‧‧‧View patch processor

160‧‧‧Multi-view interspersed module

162‧‧‧Interpolation processor

170‧‧‧Output module

172‧‧‧Output buffer

1 is a functional block diagram of a multi-view instant stereo image processing system in accordance with a preferred embodiment of the present invention.

2 is a detailed block diagram of a multi-view instant stereo image processing system in accordance with a preferred embodiment of the present invention.

100. . . Multi-view instant stereo image processing system

102. . . Control module

110. . . Data buffer module

120. . . Display device control module

130. . . Multi-view depth processing module

140. . . Multi-view data correction module

150. . . Multi-view cavity repair module

160. . . Multi-view interspersed module

170. . . Output module

Claims (8)

A multi-view real-time image processing system includes: a data buffer module for pre-storing and outputting an image data; a display device control module for receiving the image data, and outputting a plurality of display device control signals and The image data; a multi-view depth processing module for generating a plurality of view images according to the display device control signals and the image data; and a multi-view data correction module for eliminating the plurality of view images The depth of view processing module causes the delay time, so that the view images reach the same clock action; a multi-view cavity repair module is used to repair the view images to remove the cavity phenomenon of the view images; a view insertion module for sorting the synchronized and repaired view images to obtain a multi-view image; an output module for outputting the multi-view image; wherein the multi-view instant stereo image processing system The utility model comprises a control module, which can generate a plurality of display device display parameters, a multi-view depth parameter, and a capital Aligning parameters, a multi-view cavity repair parameter, and a multi-view interpolating parameter are respectively transmitted to the display device control module, the multi-view depth processing module, the multi-view data correction module, the multi-view cavity repair module, and The multi-viewpoint interpolating module causes the display device control module to generate the display device control signals, so that the multi-view depth processing module generates the view images, and the multi-view data correction module adjusts the view images. The multi-view cavity repairing module repairs the view images, and the multi-view interspersing module sorts the view images; The control module further includes a clock generator and a clock shift generator, wherein the clock generator is configured to generate a clock signal to the data buffer module, the display device control module, and the multi-view depth processing module. The multi-view data correction module, the multi-view cavity repair module, and the multi-view interpolating module, the clock shift generator adjusts the clock signal to a shift clock signal and transmits the signal to the output mode group. The multi-view real-time stereo image processing system of claim 1, wherein the display device display parameters, the multi-view depth parameter, the multi-view cavity repair parameter, and the multi-view interpolating parameter are related to the multi-view angle The specifications of the display device to be displayed by the image correspond. The multi-view real-time image processing system of claim 1, wherein the data buffer module comprises: an input buffer for receiving the external image data, and pre-storing the image data to the data buffer a site controller for providing address data; a data buffer for outputting the pre-stored image data to the display device control module according to the address data; and a read/write control The device is used to control the operation of the data buffer. The multi-view real-time image processing system of claim 1, wherein the display device control module comprises: a horizontal signal generator for generating a horizontal signal; and a data enabling controller for generating a data enable signal; a vertical signal generator for generating a vertical signal based on the data enable signal; a data temporary storage processor for receiving and outputting the horizontal signal, the vertical signal and the image data to the multi-view depth processing module; wherein the horizontal signal and the vertical signal are related to a display device specification to be displayed Corresponding. The multi-view real-time image processing system of claim 1, wherein the multi-view depth processing module comprises a plurality of view processor for generating the image data according to the multi-view depth parameter respectively. These perspective images. The multi-view cavity stereoscopic image processing system of claim 5, wherein the multi-view cavity repairing module includes a plurality of viewing angle repairing processors for repairing the viewing angle images according to the multi-view cavity repairing parameters. . The multi-view data stereoscopic image processing system of claim 1, wherein the multi-view data correction module has a buffer for adjusting the view images according to the data alignment parameter to eliminate the view images. The delay time. The multi-view interpretent image processing system of claim 1, wherein the multi-view interspersing module includes an interpolation processor for interpolating the synchronized and repaired view images. The multi-view image is obtained.