TWI465105B - Automatic alignment method for three-dimensional image, apparatus and computer-readable recording medium - Google Patents
Automatic alignment method for three-dimensional image, apparatus and computer-readable recording medium Download PDFInfo
- Publication number
- TWI465105B TWI465105B TW100138860A TW100138860A TWI465105B TW I465105 B TWI465105 B TW I465105B TW 100138860 A TW100138860 A TW 100138860A TW 100138860 A TW100138860 A TW 100138860A TW I465105 B TWI465105 B TW I465105B
- Authority
- TW
- Taiwan
- Prior art keywords
- image
- value
- region
- feature
- right image
- Prior art date
Links
- 238000000034 method Methods 0.000 title claims description 51
- 238000004364 calculation method Methods 0.000 claims description 28
- 238000006073 displacement reaction Methods 0.000 claims description 20
- 238000004590 computer program Methods 0.000 claims description 4
- 230000001360 synchronised effect Effects 0.000 claims description 2
- 238000010586 diagram Methods 0.000 description 10
- 230000006870 function Effects 0.000 description 3
- 239000013598 vector Substances 0.000 description 3
- 241000282414 Homo sapiens Species 0.000 description 2
- 238000009825 accumulation Methods 0.000 description 2
- 239000003086 colorant Substances 0.000 description 2
- 230000000007 visual effect Effects 0.000 description 2
- 230000004075 alteration Effects 0.000 description 1
- 210000004556 brain Anatomy 0.000 description 1
- 230000007613 environmental effect Effects 0.000 description 1
- 230000003068 static effect Effects 0.000 description 1
Landscapes
- Testing, Inspecting, Measuring Of Stereoscopic Televisions And Televisions (AREA)
- Measurement Of Optical Distance (AREA)
Description
本發明有關一種立體影像自動校準方法、裝置及其電腦可讀取之記錄媒體,特別是指校準左右影像呈現立體影像位置誤差的自動校準方法與裝置。The invention relates to a stereo image automatic calibration method and device and a computer readable recording medium thereof, in particular to an automatic calibration method and device for calibrating left and right images to present a stereo image position error.
立體影像(three-dimensional image)主要是模擬人類左右兩眼所見的視差產生具有景深的影像,而這類立體影像可透過一種立體顯示器顯示,原理是根據兩眼視差的設計在人類大腦視覺處理部份重建影像,產生視覺上影像之立體感。The three-dimensional image mainly simulates the parallax seen by the left and right eyes of human beings to produce images with depth of field. Such stereoscopic images can be displayed through a stereoscopic display. The principle is based on the design of the two-eye parallax in the human brain visual processing department. Reconstruction of images produces a three-dimensional sense of visual imagery.
在建立立體影像時,可透過左右有一段距離差異的兩個影像感測器拍攝同一物件或景色,之後透過影像處理而結合出一單張立體影像,其中要解決的問題是左右兩個影像間的校準問題。When a stereoscopic image is created, the same object or scene can be captured by two image sensors having a difference in distance between the left and right, and then a single stereo image is combined by image processing, wherein the problem to be solved is between the left and right images. Calibration problem.
兩個影像感測器(如攝影機)所取得不同角度的兩張影像,一般會執行影像處理,將同一時間的兩張影像作一校準的步驟,包括亮度(luminance)、顏色(color)與白平衡(white balance)等參數,另可包括系統上的校正、環境參數與像差,都可能是校準步驟的一部份。Two image sensors (such as cameras) take two images at different angles, generally perform image processing, and perform two steps of calibration at the same time, including brightness, color, and white. Parameters such as white balance, including calibrations on the system, environmental parameters and aberrations, may be part of the calibration procedure.
習知技術在立體影像校準的實施上,通常是透過一電腦系統擷取各分頁的影像圖框(frame),逐筆進行影像處理,各頁可先分別暫存於圖框緩衝器(frame buffer),經左右影像對照後,執行校正。In the implementation of stereo image calibration, the conventional technology generally captures the image frames of each page through a computer system, and performs image processing one by one, and each page can be temporarily stored in the frame buffer (frame buffer). ), after the left and right images are compared, the correction is performed.
有鑑於習知技術在立體影像處理上耗費硬體資源的問題,本揭露書提出一種立體影像自動校準方法、裝置及其電腦可讀取之記錄媒體,技術涉及一個執行立體影像自動校準方法之硬體電路,並包括利用特定裝置執行自動校準的電腦程式,亦涉及記載程式之記錄媒體,藉此可提供有效且節省記憶體的立體影像校準技術。本發明之目的之一在於校準立體影像中不同位置取得的影像間的距離,使之可以在一適當範圍內呈現精準的立體影像。In view of the problem that the conventional technology consumes hardware resources in stereo image processing, the present disclosure proposes a stereo image automatic calibration method and device and a computer readable recording medium thereof, and the technology relates to a hard method for performing stereo image automatic calibration. The body circuit, and includes a computer program that performs automatic calibration using a specific device, and a recording medium that records the program, thereby providing an effective and memory-saving stereo image calibration technique. One of the objects of the present invention is to calibrate the distance between images taken at different positions in a stereoscopic image so that it can present a precise stereoscopic image in an appropriate range.
根據發明實施例之一,立體影像自動校準方法包括先經左右兩個方向拍攝特定物件,透過影像感測器取得一左影像與一右影像,之後,選取左影像與右影像個別之興趣區域(ROI),形成對應且具有一位移量之左影像區域與右影像區域。According to one embodiment of the present invention, a method for automatically calibrating a stereoscopic image includes first capturing a specific object through two directions, and obtaining a left image and a right image through the image sensor, and then selecting an individual region of interest of the left image and the right image ( ROI), forming a left image area and a right image area corresponding to and having a displacement amount.
方法接著參考硬體限制,如線暫存器之大小,取得左影像區域與右影像區域畫素值之一維投影值,即分別表示左影像區域與右影像區域之特徵值,再儲存至線暫存器。The method then refers to the hardware limitation, such as the size of the line register, and obtains one-dimensional projection value of the left image area and the right image area pixel value, that is, the characteristic values of the left image area and the right image area respectively, and then stored to the line. Register.
接著計算左影像區域與右影像區域之複數個特徵關聯值,其中包括計算左影像區域與右影像區域之特徵值的平均值、計算左影像區域之特徵值與平均值之差異,並右影像區域之特徵值與平均值之差異、再將左影像區域與右影像區域之特徵值與平均值之差異相減,經加總此差異相減的值,加總值即為在不同的特定位移量的複數個特徵關聯值。Then calculating a plurality of feature correlation values of the left image region and the right image region, including calculating an average value of the feature values of the left image region and the right image region, calculating a difference between the feature value and the average value of the left image region, and the right image region The difference between the eigenvalue and the average value, and then the difference between the eigenvalue and the average value of the left image region and the right image region is subtracted, and the summed value is added, and the total value is the specific displacement amount at different A plurality of feature association values.
最後,經比對取得對應不同位移量的複數個特徵關聯值中的最小特徵關聯值,根據此最小特徵關聯值所代表的視差距離校準左影像與右影像之位置。藉此可以呈現出較佳的立體影像。Finally, the minimum feature correlation value of the plurality of feature correlation values corresponding to different displacement amounts is obtained by comparison, and the positions of the left image and the right image are calibrated according to the parallax distance represented by the minimum feature correlation value. Thereby, a better stereoscopic image can be presented.
上述步驟更可包括將所取得之興趣區域的左影像區域與右影像區域中三原色之畫素值統一為單一顏色頻道的畫素值,較佳為綠色頻道的值,其中引用一雙線性內插法估計出畫素中紅色頻道對應為綠色頻道的畫素值,與藍色頻道對應為綠色頻道的畫素值。The above steps may further include unifying the pixel values of the three primary colors in the left image region and the right image region of the obtained region of interest into a pixel value of a single color channel, preferably a value of a green channel, wherein a bilinear inner reference is used. The interpolation method estimates that the red channel corresponds to the pixel value of the green channel, and the blue channel corresponds to the pixel value of the green channel.
為求運算的效率,並考慮硬體的限制與成本,可以引用一畫素合併(binning)的技術,將輸入較大畫素的影像有效降低實際執行畫素數量,以利維持在特定精確度的要求下,進行影像處理。In order to calculate the efficiency of the operation, and consider the limitations and costs of the hardware, you can use the technique of binning to effectively reduce the number of pixels actually input by the image of the larger pixel to maintain the specific accuracy. Image processing is performed under the request.
根據發明實施例之一,相關立體影像自動校準裝置包括有暫存立體影像自動校準裝置取得之左影像與右影像的記憶單元、將影像分別出紅色頻道、綠色頻道與藍色頻道的濾色單元,與執行取得影像之畫素值數值運算的數值運算單元。According to one of the embodiments of the present invention, the related stereoscopic image automatic calibration device includes a memory unit for temporarily storing the left image and the right image obtained by the stereo image automatic calibration device, and a color filter unit for separating the image from the red channel, the green channel, and the blue channel. And a numerical operation unit that performs numerical calculation of the pixel value of the image.
其中,數值運算單元根據功能具有將取得之影像的不同顏色頻道的畫素值統一為特定顏色頻道的內插計算模組、降低取得的影像畫素的合併計算模組、將影像數據投影至一維座標上以取得影像之特徵值的一維投影計算模組,與一特徵關聯計算模組,此特徵關聯計算模組將分別計算左影像與右影像之特徵平均值,再取得個別之特徵值與分別之特徵平均值之差異,經加總後得出左右影像之特徵關聯值。The numerical operation unit has a function of integrating the pixel values of the different color channels of the acquired image into an interpolation calculation module of a specific color channel, a combined calculation module for reducing the obtained image pixels, and projecting the image data to the image data. A one-dimensional projection calculation module for obtaining image feature values, and a feature correlation calculation module, the feature correlation calculation module respectively calculates the feature average values of the left image and the right image, and then obtains individual feature values. The difference between the mean values of the respective features and the sum of the features of the left and right images are obtained.
裝置特別包括一個使用線暫存單元的記憶體與比對加總得出的特徵關聯值,得出在不同位移下的一最小特徵關聯值,以及根據最小特徵關聯值所代表的視差距離進行左影像與右影像之校準的校正單元。The device specifically includes a memory associated with the memory of the line temporary storage unit and the correlation value obtained by the comparison, obtaining a minimum feature correlation value under different displacements, and performing left image according to the parallax distance represented by the minimum feature correlation value. Correction unit for calibration with the right image.
本發明實施例更涉及記載有執行上述立體影像自動校準方法的電腦程式的一種電腦可讀取之記錄媒體。The embodiment of the present invention further relates to a computer readable recording medium in which a computer program for performing the above-described stereo image automatic calibration method is described.
本揭露書提出的立體影像自動校準方法、裝置及其電腦可讀取之記錄媒體涉及一個執行立體影像自動校準方法之硬體電路,其利用特定裝置執行自動校準的電腦程式,亦涉及記載程式之記錄媒體。發明之目的之一在於校準立體影像中不同位置取得的影像間的距離,使之可以在一適當範圍內呈現精準的立體影像。The stereoscopic image automatic calibration method and device and the computer readable recording medium thereof disclosed in the disclosure relate to a hardware circuit for performing a stereo image automatic calibration method, which uses a specific device to perform an automatic calibration of a computer program, and also relates to a program Record media. One of the objects of the invention is to calibrate the distance between images taken at different locations in a stereoscopic image so that it can present a precise stereoscopic image in an appropriate range.
圖1描述利用兩個不同角度拍攝同一物件之設施示意圖,其中示意有左右兩個影像擷取裝置101,102(如照相機或攝影機),分別對同一物件103進行拍攝,取得如圖示之左右兩個影像(左影像L,右影像R),拍攝之內容可為靜態影像或是可以逐幀(frame)處理的動態影片。Figure 1 depicts a schematic diagram of a facility for photographing the same object using two different angles, wherein two left and right image capturing devices 101, 102 (such as a camera or a camera) are illustrated, respectively photographing the same object 103 to obtain two images as shown. (Left image L, right image R), the captured content can be a still image or a dynamic movie that can be processed frame by frame.
圖2(a)(b)(c)接著描述取得左右影像特徵曲線的示意圖。經利用特定裝置(如電腦系統)取得左右影像,如圖2(a)所示,可由一處理程式取得同一個物件或/與同一時間擷取的左影像L與右影像R的影像資料。執行立體影像自動校準的裝置經取得左右兩個影像(L,R)後,可計算兩個影像具有一定差異,特別是左右水平位移,為取得此位移量,可先設一基準線,依此為準得出兩個影像間的影像位移量。2(a), (b) and (c) are next schematic diagrams for obtaining the left and right image characteristic curves. By using a specific device (such as a computer system) to obtain left and right images, as shown in FIG. 2(a), the same object or/and the image data of the left image L and the right image R captured at the same time can be obtained by a processing program. After the two images (L, R) are obtained by the device for performing automatic calibration of the stereo image, the two images can be calculated to have a certain difference, especially the horizontal displacement. To obtain the displacement, a reference line can be set first. The amount of image displacement between the two images is obtained.
根據本揭露書提出的立體影像自動校準方法實施例,可如圖2(b)所示,在左影像L與右影像R中各以方塊取得具有一定面積的興趣區塊(region of interest,ROI),分別標示為左影像區域201與右影像區域202。According to the embodiment of the method for automatically calibrating a stereoscopic image according to the disclosure, as shown in FIG. 2(b), a region of interest (ROI) having a certain area is obtained in each of the left image L and the right image R. ) are labeled as left image area 201 and right image area 202, respectively.
在執行影像處理時,先將各興趣區域的畫素值經一維投影的演算,逐欄累加(此例為垂直方向,同理可應用於橫向累加)的畫素值,得出一特徵值(signature),經整個影像區域的累加值計算之後,可呈現如圖2(c)所示意顯示的一種分佈曲線,即為用興趣區域取得特徵值的示意圖。本發明即據特徵分佈估計兩張影像間的視差(parallax)距離。In the image processing, the pixel values of each region of interest are firstly calculated by one-dimensional projection, and the pixel values are added column by column (in this case, the vertical direction, the same reason can be applied to the lateral accumulation), and a feature value is obtained. (signature), after the calculation of the accumulated value of the entire image area, a distribution curve as shown in FIG. 2(c) may be presented, that is, a schematic diagram for obtaining the feature value by using the region of interest. The present invention estimates the parallax distance between two images based on the feature distribution.
立體影像自動校準方法係計算出左右影像間的視差距離,並據此校準兩張影像以取得符合一定需求的精準的立體影像,如圖3所示之流程,此流程接著描述本發明立體影像自動校準方法之步驟。The stereo image auto-calibration method calculates the parallax distance between the left and right images, and calibrates the two images accordingly to obtain a precise stereoscopic image that meets certain requirements, as shown in the flow of FIG. 3, which then describes the stereoscopic image automatic of the present invention. The steps of the calibration method.
開始如步驟S301,透過影像感測器取得左右影像資料,實施例可以相距一定距離的兩個或以上數量的攝影機同時拍攝一個目標物件,另不排除利用一部攝影機分別在不同位置拍攝同一物件。左右影像數據則先暫存於記憶體中。Beginning with step S301, the left and right image data are acquired by the image sensor. In the embodiment, two or more cameras at a certain distance can simultaneously capture one target object, and the same object is photographed at different positions by using one camera. The left and right image data is temporarily stored in the memory.
根據實施例,在硬體效能可應付的狀態下,整張取得的影像可直接進行後續演算與校準步驟,但仍可如步驟S303所述,利用軟體手段在左右影像中分別選擇興趣區域(ROI),各興趣區域可以x,y等座標值所描述,並可以根據需要調整。According to the embodiment, in the state that the hardware performance can cope, the entire acquired image can directly perform the subsequent calculation and calibration steps, but the ROI can be separately selected in the left and right images by using the software means as described in step S303. ), each area of interest can be described by coordinates such as x, y, and can be adjusted as needed.
根據實施例,在執行立體影像位置校準的需要上,並不考慮影像顏色,因此可以執行如步驟S305,經取得特定座標範圍的興趣區域的影像數據後,從記憶體中取得由影像感測器所感測的各興趣區域內的影像顏色頻道(channel),實施例可經一使用拜耳圖(Bayer pattern)的濾色片取得各畫素的三原色(R,G,B)畫素值,可參考圖4本發明採用之濾色片示意圖。根據實施例之一,若取得之影像非為彩色影像,則可忽略將影像經濾色片取得畫素值之步驟。According to the embodiment, the image color is not considered in the need of performing the stereo image position calibration. Therefore, after the image data of the region of interest of the specific coordinate range is obtained in step S305, the image sensor is obtained from the memory. The image color channel in each region of interest sensed, and the embodiment can obtain the three primary colors (R, G, B) pixel values of each pixel through a color filter using a Bayer pattern, which can be referred to. Figure 4 is a schematic view of a color filter used in the present invention. According to one of the embodiments, if the acquired image is not a color image, the step of obtaining the pixel value through the color filter may be ignored.
經取得彩色左右影像之三原色畫素值後,如步驟S307,各顏色頻道值可經週邊的數值取得之平均值所取代,以此方式統一顏色頻道,以利運算。舉例來說,可同時參考圖4內容,影像畫素由R(紅),G(綠),B(藍)等顏色頻道所組成,但可統一使用其中之一顏色頻道進行後續運算,此例以G(綠色)頻道為準進行校正。After obtaining the three primary color pixel values of the color left and right images, in step S307, the color channel values can be replaced by the average value obtained by the surrounding values, thereby unifying the color channels in order to facilitate the operation. For example, referring to FIG. 4 at the same time, the image pixels are composed of color channels such as R (red), G (green), and B (blue), but one of the color channels can be uniformly used for subsequent operations. The correction is based on the G (green) channel.
根據實施例之一,為了方便從影像的原始資料(raw data)計算出用於計算視差距離的方塊特徵(block signature),本揭露書所提之校準方法採用畫素中G(綠色)頻道值,並利用內插法用G頻道值取代R(紅色)與B(藍色)頻道值,如一種雙線性內插法(bilinear interpolation)。雙線性內插法應用一種拜耳顏色濾色陣列圖(Bayer CFA pattern,CFA: Color Filter Array)作為內插計算的基礎,也就是以雙線性內插法估計出畫素中紅色頻道對應為綠色頻道的畫素值,與藍色頻道對應為綠色頻道的畫素值。According to one of the embodiments, in order to facilitate calculation of a block signature for calculating a parallax distance from raw data of an image, the calibration method proposed in the present disclosure adopts a G (green) channel value in a pixel. And use the interpolation method to replace the R (red) and B (blue) channel values with the G channel value, such as a bilinear interpolation. The bilinear interpolation method uses a Bayer CFA pattern (CFA: Color Filter Array) as the basis of the interpolation calculation, that is, the bilinear interpolation method is used to estimate the red channel corresponding to the pixel. The pixel value of the green channel corresponds to the blue channel as the pixel value of the green channel.
舉例來說,請參閱圖4,若要以G值取代圖中R1值,也就是R1(G(R1)),此例使用畫素G1,G3,G4與G7值以內插法來估計R1的值,方程式如下:For example, please refer to Figure 4. If you want to replace the R1 value in the graph with the G value, that is, R1(G(R1)), this example uses the pixels G1, G3, G4, and G7 values to estimate R1 by interpolation. Value, the equation is as follows:
G(R1)=(G1+G3+G4+G7)/4 -------------方程式(2)G(R1)=(G1+G3+G4+G7)/4 ------------- Equation (2)
再以畫素中B4值為例,若要以G值取代B4值,則以其周圍的G值進行內插,以估計出B4的值,方程式如下:Taking the B4 value of the pixel as an example, if the B4 value is to be replaced by the G value, the G value around it is interpolated to estimate the value of B4. The equation is as follows:
G(B4)=(G4+G7+G8+G10)/4 ------------方程式(3)G(B4)=(G4+G7+G8+G10)/4 ------------ Equation (3)
經統一興趣區域的影像顏色頻道後,若影像大小過於硬體的處理能力,或是經過刻意設計,本發明將可進一步利用一種畫素合併(binning)的技術(步驟S309)。在面對輸入影像大於硬體可處理或是經過設計的範圍時,可使用一種畫素合併(binning)技術有效降低實際執行影像處理的數量,節省記憶體使用。畫素合併的技術是一種數據前置處理程序,可將感測器取得的影像畫素依據記憶體的大小進行畫素合併,將較大的輸入圖像縮成較小圖像,並可維持接近原影像的精準度。此畫素合併技術可以應用於任何畫素大小的輸入影像上,合併範圍則視實際硬體限制而定。此步驟視需要而定,若影像大小並非硬體無法處理,則可忽略此步驟。After unifying the image color channel of the region of interest, if the image size is too hard to process, or is deliberately designed, the present invention can further utilize a pixel binning technique (step S309). In the face of the input image is larger than the hardware can be processed or designed range, a pixel binning technology can be used to effectively reduce the number of image processing actually performed, saving memory usage. The technique of pixel combination is a data pre-processing program, which can combine the image pixels obtained by the sensor according to the size of the memory, and reduce the larger input image to a smaller image and maintain Close to the accuracy of the original image. This pixel combining technique can be applied to any pixel size input image, and the scope of the combination depends on the actual hardware limitations. This step is optional and can be ignored if the image size is not hardware that cannot be processed.
參考硬體限制(如線暫存器大小),畫素合併係將較大尺寸的影像縮為適合線暫存器的大小。舉例來說,若有一個4個畫素的影像,執行2x2畫素合併步驟,可成為一個畫素,有效降影像處理的硬體需要,特別是記憶體的使用。根據一實例,若線暫存器為640bit,影像為1280pixels,則可兩兩加總,將像素減少一半,再儲存至線暫存器。Referring to hardware limitations (such as line register size), pixel combining reduces the size of a larger image to fit the size of the line register. For example, if there is a 4 pixel image, the 2x2 pixel combination step can be used as a pixel to effectively reduce the hardware requirements of image processing, especially the use of memory. According to an example, if the line register is 640 bits and the image is 1280 pixels, the total number of pixels can be increased by two, and the pixels are reduced by half, and then stored in the line register.
經上述統一顏色頻道與畫素合併的步驟之後,接著方法將如步驟S311所述,根據硬體所提供的線緩衝器(line buffer)大小,利用演算法逐欄(此例為縱向)計算取得影像的一維投影值,較佳可直接引用興趣區域的影像資料。After the step of combining the unified color channel and the pixel, the method will be calculated according to the size of the line buffer provided by the hardware according to the size of the line buffer provided by the hardware (in this case, the vertical direction). The one-dimensional projection value of the image preferably directly refers to the image data of the region of interest.
相關運算如下。此例以線緩衝器為儲存影像各行畫素之全部或特定部份(如中間)64畫素(pixel,0~63)的大小為例,利用方程式(1)分別取得一維投影值,即分別表示ROI內之左影像區域之特徵值與右影像區域之特徵值,可反應出左右影像區域的特徵曲線。The correlation operation is as follows. In this example, the line buffer is used to store all or a specific part (such as the middle) of 64 pixels (pixel, 0 to 63) of each pixel of the image as an example, and the one-dimensional projection value is obtained by using equation (1), that is, The feature values of the left image region and the feature value of the right image region in the ROI are respectively reflected, and the characteristic curves of the left and right image regions can be reflected.
上述由不同或相同的感測器取得左右兩個方向的影像,分別標示為左影像(L)與右影像(R),可以在取得影像的同時,取得其中興趣區域作為後續運算,亦可於經上述初步處理之後,再根據其中興趣區域執行後續步驟。興趣區域的取得包括先設有一個方塊(比如是一個正方形、長方形),也就是定義出一個興趣區域(ROI),其中X與Y兩個方向為可調整,可參考圖2。兩個圖框分別以方塊標示,方塊中每一欄(如垂直畫素之加總)以曲線Cx 所描述(此例為橫向i與縱向64畫素為例的影像):The images obtained by the different or the same sensors in the left and right directions are respectively labeled as the left image (L) and the right image (R), and the image can be obtained while the region of interest is used as a follow-up operation. After the preliminary processing described above, the subsequent steps are performed according to the region of interest. The acquisition of the interest area includes first setting a square (for example, a square, a rectangle), that is, defining a region of interest (ROI), wherein the X and Y directions are adjustable, and can be referred to FIG. The two frames are respectively indicated by squares, and each column in the block (such as the sum of vertical pixels) is described by the curve C x (in this case, the image of the horizontal i and the vertical 64 pixels):
其中x係指水平座標方向;i索引指出所選取影像之橫向畫素之索引;j則為縱向的畫素索引,此例採用影像中間或其他部份64畫素之加總;P(i,j,L)與P(i,j,R)分別為在左右圖框內座標(i,j)的畫素值。透過加總運算,可將整個平面影像投影以一維數據來表示,也就是各影像的特徵值,以分佈來看,即形成一特徵曲線。Where x is the horizontal coordinate direction; i index indicates the index of the horizontal pixel of the selected image; j is the vertical pixel index, in this case the sum of 64 pixels in the middle of the image or other parts; P(i, j, L) and P(i, j, R) are the pixel values of the coordinates (i, j) in the left and right frames, respectively. Through the summation operation, the entire planar image projection can be represented by one-dimensional data, that is, the characteristic values of the respective images, and a characteristic curve is formed in terms of distribution.
可同時參考圖5(a)(b)所示取得一維投影值的示意圖,圖5(a)描述一個影像範圍的垂直方向畫素值累加的示意圖,接著各欄的累加值將反映如圖5(b)的特徵曲線上,即方程式(1)所算出的特徵值的變化曲線(Cx )。A schematic diagram of obtaining a one-dimensional projection value as shown in FIG. 5(a)(b) can be simultaneously referred to, and FIG. 5(a) is a schematic diagram showing an accumulation of pixel values in a vertical direction of an image range, and then the accumulated values of the respective columns will be reflected as shown in the figure. On the characteristic curve of 5(b), that is, the variation curve (C x ) of the eigenvalue calculated by equation (1).
經一維投影之後,影像特徵可以一曲線表示,相關數值也可以符合一線緩衝器的大小,如步驟S313所描述,經計算的特徵值暫存於此線緩衝器中。利用此一方向投影在線暫存器上的方式可大幅減少所需的記憶體。After one-dimensional projection, the image features can be represented by a curve, and the correlation value can also conform to the size of the line buffer. As described in step S313, the calculated feature values are temporarily stored in the line buffer. Using this method of projecting on the online register can greatly reduce the required memory.
繼續參考圖3之步驟S315,計算最小特徵關聯值。With continued reference to step S315 of FIG. 3, the minimum feature correlation value is calculated.
為了得到左圖框(L)與右圖框(R)間的視差距離,本揭露書所提出的方法係利用上述左右圖框的特徵值與平均值取得特徵關聯值,再得出一最小特徵關聯(minimum signature correlation),相關的特徵關聯係由上述左右圖框分別取得的特徵曲線得出,如採用以下方程式(4)所計算的結果:In order to obtain the parallax distance between the left frame (L) and the right frame (R), the method proposed by the present disclosure obtains the feature correlation value by using the feature values of the left and right frames and the average value, and then obtains a minimum feature. The correlation signature is related to the characteristic curve obtained by the above left and right frames respectively, as calculated by the following equation (4):
此例(但實際實施並不限於此例)同樣採用所選取的影像區域橫向(x方向)中全部或部份64個畫素(0~63)的畫素值,並藉此將經加總之一維投影值(特徵值)再計算影像區域的橫向特徵平均值(Averagex (L/R));其中q值為左右兩個影像區域之間的一位移量,此範例中可為192畫素(0~191)的範圍,為表示一個區域內(如ROI)中兩個影像的水平方向特徵值(如Cx )的位移量,可為一位移向量(shift vector),此位移向量即造成左右兩個影像間水平方向的視差。此方式係先分別得出左右圖框的特徵平均值(Averagex (L)與Averagex (R)),再由方程式計算得出各橫向畫素特徵值與特徵平均值之差異(Cx (i,L)-Averagex (L)與Cx (i+q,R)-Averagex (R))。In this case (but the actual implementation is not limited to this example), the pixel values of all or part of the 64 pixels (0 to 63) in the horizontal (x direction) of the selected image area are also used, and the summed The one-dimensional projection value (eigenvalue) is used to calculate the transverse feature average of the image region (Average x (L/R)); where q is a displacement between the two image regions, which can be 192 in this example. The range of prime (0 to 191) is a displacement amount indicating a horizontal eigenvalue (such as C x ) of two images in one region (such as ROI), which may be a shift vector, and the displacement vector is Causes the parallax in the horizontal direction between the left and right images. In this way, the mean values of the left and right frames (Average x (L) and Average x (R)) are obtained separately, and then the difference between the horizontal and horizontal feature values is calculated by the equation (C x ( i, L) - Average x (L) and C x (i + q, R) - Average x (R)).
再將左影像區域與右影像區域之特徵值與特徵平均值差異相減,之後將此相減後的值加總,也就是加總左右圖框的特徵值與特徵平均值的差異相減的值,此加總值即為在不同的q值(位移量)得出多個特徵關聯值R(q),此為在左影像區域(L)的特徵值與有一個距離以外的右影像區域(R)特徵值之間位移(shift)為q的關聯值,此例可有192個。Then subtract the difference between the feature value of the left image region and the right image region and the mean value of the feature, and then add the subtracted value to the total value, that is, the difference between the feature value of the left and right frame and the mean value of the feature is subtracted. Value, the total value is a plurality of feature correlation values R(q) at different q values (displacement amounts), which is the right image region outside the eigenvalue of the left image region (L) and having a distance (R) The shift between the eigenvalues is the associated value of q. In this case, there are 192.
經比對對應不同的位移量的特徵關聯值,可得出一最小特徵關聯值,以Rmin (q)表示,由上述計算出的關聯值取出在一影像區域中之最小值,也就是原始左右圖框間設計具有一個位移q,但此位移q在水平方向產生此最小特徵關聯值所代表的視差距離(parallax distance)(步驟S317),也就是在呈現立體影像時需要補償的視差距離,之後依據此最小特徵關聯值Rmin (q)代表的視差距離進行校準(步驟S319)。By comparing the feature correlation values corresponding to different displacement amounts, a minimum feature correlation value can be obtained, represented by R min (q), and the correlation value calculated by the above is taken out as a minimum value in an image region, that is, original The left and right frames are designed to have a displacement q, but the displacement q produces a parallax distance represented by the minimum feature correlation value in the horizontal direction (step S317), that is, a parallax distance that needs to be compensated when the stereoscopic image is presented. Then, calibration is performed in accordance with the parallax distance represented by the minimum feature correlation value R min (q) (step S319).
以上所揭示本揭露書提出的立體影像自動校準方法,其目的之一是用以估計從兩個感測器取得的兩個影像的物件視差(parallax)距離,據此進行校準,其中引用的一維投影技術係將特定範圍內的畫素值投影為一特徵曲線上,其中步驟可參考圖6流程所描述之步驟。One of the purposes of the method for automatically calibrating a stereoscopic image proposed by the above disclosure is to estimate the parallax distance of two images taken from two sensors, and perform calibration according to the reference. The dimensional projection technique projects a pixel value in a specific range onto a characteristic curve, and the steps can be referred to the steps described in the flow of FIG.
若影像為彩色,可以如步驟S601所作,先輸入統一頻道之影像,比如將R,G,B顏色空間中的紅色與藍色畫素利用內插法來以綠色值表示;若為非彩色的影像,或可忽略此步驟。If the image is colored, you can input the image of the unified channel first, as in step S601. For example, the red and blue pixels in the R, G, and B color spaces are represented by green values by interpolation; if they are non-colored Image, or you can ignore this step.
接著,依據實際需要,可決定出左右影像中的特定興趣區域(步驟S603),並對特定範圍內的畫素值逐行累加畫素值(步驟S605),產生後續計算特徵關聯需用的一維投影值(步驟S607)。Then, according to actual needs, a specific region of interest in the left and right images may be determined (step S603), and the pixel values are accumulated row by row for the pixel values in the specific range (step S605), and a subsequent need for calculating the feature association is generated. The dimension projection value (step S607).
圖7所示之流程則描述本發明計算特徵關聯值之步驟。步驟S701描述使用經計算得出的左右影像興趣區域之一維投影值,再如上述方程式(4)所示之範例,計算左右影像興趣區域之橫向特徵之平均值,可分別定義為左影像的第一平均值與右影像的第二平均值(Averagex (L/R),步驟S703),並與左右影像中橫向特徵值相減,取得各行左右影像中各一維投影值與其相對平均值之差異,可分別定義為左影像各投影值的第一差異與右影像各投影值的第二差異(步驟S705)。The flow shown in Figure 7 describes the steps of the present invention for calculating feature associated values. Step S701 describes using the calculated one-dimensional projection value of the left and right image interest regions, and then calculating the average value of the lateral features of the left and right image interest regions according to the example shown in the above equation (4), which can be respectively defined as the left image. The first average value and the second average value of the right image (Average x (L/R), step S703) are subtracted from the horizontal feature values in the left and right images, and the one-dimensional projection values and their relative average values in the left and right images of each line are obtained. The difference may be defined as a second difference between the first difference of each projection value of the left image and each projection value of the right image (step S705).
之後如步驟S707,累加各橫向特徵值與特徵平均值的差異量(第一、第二差異),依此取得各影像橫向位置上在不同的位移向量(q值)上的特徵關聯值(步驟S709),經反覆計算後取得最小特徵關聯值(步驟S711),此值被視為視差距離,也就是需要校準的值,相關左右影像拍攝的影像感測器或是相關設備應針對此需要校準的值調整裝置距離。Then, in step S707, the difference amount (first and second difference) between each lateral feature value and the feature average value is accumulated, and thus the feature correlation value on different displacement vectors (q values) in each image lateral position is obtained (step S709), after the repeated calculation, obtain the minimum feature correlation value (step S711), the value is regarded as the parallax distance, that is, the value to be calibrated, and the image sensor or related device for the left and right image shooting should be calibrated for this need The value adjusts the device distance.
圖8則接著描述實現上述校準方法的立體影像自動校準裝置之功能方塊圖。Fig. 8 is a block diagram showing the function of the stereoscopic image automatic calibration apparatus for realizing the above calibration method.
裝置本身特別包括有記憶單元803、濾色單元805、數值運算單元807、線暫存單元809、比對單元811與校正單元813,數值運算單元807則包括可以軟體或是硬體實現的畫素合併計算模組821、內插計算模組823、一維投影計算模組825與特徵關聯計算模組827,再包括一個連結於記憶單元803上用於採用影像中特定部份的資料的興趣區域模組831。The device itself includes a memory unit 803, a color filter unit 805, a numerical operation unit 807, a line temporary storage unit 809, a comparison unit 811 and a correction unit 813, and the numerical operation unit 807 includes a pixel that can be implemented by software or hardware. The combined computing module 821, the interpolation computing module 823, the one-dimensional projection computing module 825 and the feature correlation computing module 827 further include an area of interest coupled to the memory unit 803 for using data of a specific portion of the image. Module 831.
根據實施例,可分別由不同位置的影像感測器同時取得特定物件的左右兩個方向的影像,若拍攝靜態物件,則可以相同影像感測器在不同位置進行拍攝。如圖所示,立體影像自動校準裝置由影像感測器801取得左右影像,並暫存於裝置之記憶單元803,暫存之影像可經濾色單元805將畫素分別出其中紅色頻道(R)、綠色頻道(G)與藍色頻道(B)。其中在取得影像數據時,可利用軟體或硬體實現的興趣區域模組831將記憶單元803內儲存的影像在左右影像R中各以方塊取得具有一定面積的興趣區塊(ROI),可以方便計算距離誤差,並節省運算資源。According to the embodiment, the images in the left and right directions of the specific object can be simultaneously acquired by the image sensors at different positions, and if the static objects are photographed, the same image sensor can be photographed at different positions. As shown in the figure, the stereo image auto-calibration device obtains left and right images by the image sensor 801 and temporarily stores them in the memory unit 803 of the device. The temporarily stored images can be separated from the pixels by the color filter unit 805. ), green channel (G) and blue channel (B). When the image data is acquired, the region of interest module 831 implemented by the software or the hardware can acquire the image of interest (ROI) having a certain area in the left and right images R by using the image stored in the memory unit 803. Calculate distance errors and save computing resources.
由於本揭露書提出的校準方法並無利用彩色的特性,因此可以將畫素經統一頻道為同一顏色頻道上,比如上述經內插法將各顏色頻道統一為綠色頻道一般。經濾色單元805處理的畫素值再經數值運算單元807處理。Since the calibration method proposed in the present disclosure does not utilize the color characteristic, the pixels can be unified on the same color channel, for example, the above-mentioned interpolating method is used to unify each color channel into a green channel. The pixel values processed by the color filter unit 805 are processed by the numerical operation unit 807.
之後經數值運算單元807將取得影像之畫素值進行數值運算,根據本發明所需之功能主要區分為以軟體或硬體實現的多個模組,包括將取得之影像進行畫素合併的畫素合併計算模組821、計算影像單一顏色頻道的內插計算模組823、將影像畫素累加取得一維特徵值的一維投影計算模組825與根據左右影像之特徵計算特徵關聯值的特徵關聯計算模組827。Then, the numerical operation unit 807 performs numerical operation on the pixel values of the acquired image, and the functions required according to the present invention are mainly divided into a plurality of modules implemented by software or hardware, including a picture in which the acquired images are combined by pixels. The merging calculation module 821, the interpolation calculation module 823 for calculating a single color channel of the image, the one-dimensional projection calculation module 825 for accumulating the image pixels to obtain the one-dimensional eigenvalue, and the feature for calculating the correlation value of the feature according to the features of the left and right images Correlation calculation module 827.
其中利用內插計算模組823將取得之影像的不同顏色頻道的畫素值統一為特定顏色頻道,如上述較佳轉換為綠色頻道,但實際運作根據需求也不排除統一至其他的顏色頻道。計算方式可參考上述方程式(2)(3)。The interpolation calculation module 823 is used to unify the pixel values of the different color channels of the obtained image into a specific color channel, and the above is preferably converted into a green channel, but the actual operation does not exclude unifying to other color channels according to requirements. For the calculation method, refer to equation (2) (3) above.
接著,因為硬體資源(特別是記憶體)的限制,或是為求節省硬體資源,可利用數值運算單元807內的畫素合併計算模組821將取得的影像數據經畫素合併(binning)步驟有效降低實際執行影像處理的畫素,也就是將較大的輸入圖像縮成較小圖像。Then, because of limitation of hardware resources (especially memory), or in order to save hardware resources, the obtained image data may be binned by binning using the pixel combination calculation module 821 in the numerical operation unit 807. The steps effectively reduce the pixels that actually perform the image processing, that is, reduce the larger input image to a smaller image.
在處理左右影像時,可以利用數值運算單元807內的一維投影計算模組825將整個經選取的部份(如ROI)的影像數據投影至一維座標上的數據,藉此取得影像的一種特徵曲線(有一序列的特徵值),計算方式如方程式(1)。裝置具有一暫存數值運算單元807之處理數據的暫存單元,如圖示之線暫存單元809,經投影的影像數據則可以儲存於一線暫存單元809上,不同於一般需要將整張影像暫存於特定記憶體中(如裝置內之同步動態隨機存取記憶體,Synchronous Dynamic Random Access Memory,SDRAM),使用線暫存單元則更可節省內部運算用的記憶體使用。When processing the left and right images, the one-dimensional projection calculation module 825 in the numerical operation unit 807 can project the image data of the entire selected portion (such as ROI) onto the data on the one-dimensional coordinates, thereby obtaining a kind of image. The characteristic curve (having a sequence of eigenvalues) is calculated as Equation (1). The device has a temporary storage unit for temporarily storing the processing data of the numerical operation unit 807, such as the line temporary storage unit 809 shown in the figure, and the projected image data can be stored in the first-line temporary storage unit 809, which is different from the general need. The image is temporarily stored in a specific memory (such as Synchronous Dynamic Random Access Memory (SDRAM) in the device), and the use of the line temporary storage unit can save memory for internal computing.
經數值運算單元807取得在一維座標上的影像數據後,其中特徵關聯計算模組827再計算左右影像畫素橫向的特徵平均值,取得左右影像個別之特徵值與此平均值之差異(分別可定義為第一差異與第二差異),此差異即兩者之特徵關聯值,可參考上述方程式(4)的計算。After the image data on the one-dimensional coordinates is obtained by the numerical operation unit 807, the feature-related calculation module 827 calculates the average value of the horizontal and horizontal image pixels, and obtains the difference between the feature values of the left and right images and the average value (respectively It can be defined as the first difference and the second difference), and the difference is the characteristic correlation value of the two, and can be referred to the calculation of the above equation (4).
之後,經比對單元811比對各特徵關聯值,得出左右影像相對之最小特徵關聯值,最小特徵關聯值即表示為一視差距離,也就是需要校準補償左右影像形成立體影像所需的值,透過校正單元813進行左右影像位置之校準,以取得較佳可以呈現立體影像的視差,影像經調整後,將輸出至外部裝置815。Then, the comparison unit 811 compares the correlation values of the features to obtain a minimum feature correlation value of the left and right images, and the minimum feature correlation value is represented as a parallax distance, that is, a value required to calibrate and compensate the left and right images to form a stereoscopic image. The calibration of the left and right image positions is performed by the correction unit 813 to obtain a parallax that can preferably present a stereoscopic image. After the image is adjusted, the image is output to the external device 815.
本揭露書所提出執行立體影像自動校準的裝置即如上述圖8之描述,物件經影像感測器取得影像數據後,將透過濾色片取得各畫素的顏色頻道資料,再由興趣區域中計算並統一其中顏色頻道、計算一維投影值,以方便暫存於線暫存單元,最後算出最小特徵關聯值,依此為基準進行影像或是設備之校正。The device for performing automatic calibration of the stereo image proposed by the disclosure is as described in FIG. 8 above. After the image data is acquired by the image sensor, the color channel data of each pixel is obtained through the color filter, and then the region of interest is Calculate and unify the color channel and calculate the one-dimensional projection value to facilitate temporary storage in the line temporary storage unit, and finally calculate the minimum feature correlation value, and then perform image or device correction based on this.
綜上所述,本揭露書所提出的立體影像自動校準方法與相關裝置目的是透過估計兩個感應器得到的兩張影像的物件視差距離,以進行自動校準,其中特別透過統一顏色、畫素合併與最小特徵關聯的執行有效得出視差距離,再據此執行校準。自動校準方法有效使用線暫存器,透過投影為一維空間的方式進行誤差判斷,可以減少記憶體使用。In summary, the stereo image auto-calibration method and related device proposed by the present disclosure aims to perform automatic calibration by estimating the parallax distance of the two images obtained by the two sensors, in particular through uniform color and pixel. The execution of the merge associated with the minimum feature effectively yields the parallax distance and performs calibration accordingly. The automatic calibration method effectively uses the line register to make error judgment by projecting into a one-dimensional space, which can reduce memory usage.
惟以上所述僅為本發明之較佳可行實施例,非因此即侷限本發明之專利範圍,故舉凡運用本發明說明書及圖示內容所為之等效結構變化,均同理包含於本發明之範圍內,合予陳明。However, the above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Therefore, equivalent structural changes that are made by using the specification and the contents of the present invention are equally included in the present invention. Within the scope, it is combined with Chen Ming.
101,102...影像擷取裝置101,102. . . Image capture device
103...物件103. . . object
L...左影像L. . . Left image
R...右影像R. . . Right image
201...左影像區域201. . . Left image area
202...右影像區域202. . . Right image area
801...影像感測器801. . . Image sensor
803...記憶單元803. . . Memory unit
805...濾色單元805. . . Filter unit
807...數值運算單元807. . . Numerical arithmetic unit
809...線暫存單元809. . . Line temporary storage unit
811...比對單元811. . . Alignment unit
813...校正單元813. . . Correction unit
821...畫素合併計算模組821. . . Pixel combined computing module
823...內插計算模組823. . . Interpolation calculation module
825...一維投影計算模組825. . . One-dimensional projection computing module
827...特徵關聯計算模組827. . . Feature correlation calculation module
831...興趣區域模組831. . . Interest area module
815...外部裝置815. . . External device
步驟S301~步驟S319 立體影像自動校準流程Step S301 to step S319 automatic calibration process of stereo image
步驟S601~步驟S607 一維投影值計算流程Step S601 to step S607 One-dimensional projection value calculation flow
步驟S701~步驟S711 特徵關聯值計算流程Step S701 to step S711 feature correlation value calculation process
圖1描述利用兩個不同角度拍攝同一物件之設施示意圖;Figure 1 depicts a schematic diagram of a facility for photographing the same object using two different angles;
圖2(a)(b)(c)所示為本發明利用興趣區域取得特徵值的示意圖;2(a), (b) and (c) are schematic diagrams showing the use of the region of interest to obtain feature values according to the present invention;
圖3所示之流程描述本發明立體影像自動校準方法之步驟;The flow shown in FIG. 3 describes the steps of the automatic image calibration method of the present invention;
圖4所示為本發明採用之濾色片示意圖;Figure 4 is a schematic view showing a color filter used in the present invention;
圖5(a)(b)所示為本發明取得一維投影值的示意圖;Figure 5 (a) (b) shows a schematic diagram of obtaining a one-dimensional projection value according to the present invention;
圖6所示之流程描述本發明計算一維投影值之步驟;The flow shown in FIG. 6 describes the steps of the present invention for calculating a one-dimensional projection value;
圖7所示之流程描述本發明計算特徵關聯值之步驟;The flow shown in Figure 7 describes the steps of calculating the associated value of the feature of the present invention;
圖8所示為本發明立體影像自動校準裝置之功能方塊圖。FIG. 8 is a functional block diagram of a stereo image automatic calibration apparatus according to the present invention.
S301...取得左右影像資料S301. . . Get left and right image data
S303...選擇興趣區域S303. . . Select interest area
S305...取得影像顏色頻道S305. . . Get image color channel
S307...統一顏色頻道S307. . . Unified color channel
S309...畫素合併S309. . . Pixel merge
S311...逐欄取得一維投影值S311. . . Get one-dimensional projection values column by column
S313...暫存於線暫存器S313. . . Temporarily stored in the line register
S315...計算最小特徵關聯值S315. . . Calculate the minimum feature correlation value
S317...得出左右視差距離S317. . . Left and right parallax distance
S319...進行左右影像校準S319. . . Perform left and right image calibration
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW100138860A TWI465105B (en) | 2011-10-26 | 2011-10-26 | Automatic alignment method for three-dimensional image, apparatus and computer-readable recording medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| TW100138860A TWI465105B (en) | 2011-10-26 | 2011-10-26 | Automatic alignment method for three-dimensional image, apparatus and computer-readable recording medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| TW201318407A TW201318407A (en) | 2013-05-01 |
| TWI465105B true TWI465105B (en) | 2014-12-11 |
Family
ID=48872152
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| TW100138860A TWI465105B (en) | 2011-10-26 | 2011-10-26 | Automatic alignment method for three-dimensional image, apparatus and computer-readable recording medium |
Country Status (1)
| Country | Link |
|---|---|
| TW (1) | TWI465105B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9918010B2 (en) | 2015-06-30 | 2018-03-13 | Industrial Technology Research Institute | Method for adjusting vehicle panorama system |
Families Citing this family (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104252302A (en) * | 2013-06-26 | 2014-12-31 | 富泰华工业(深圳)有限公司 | Self-adaptive image adjusting system and method |
| TWI602420B (en) * | 2016-10-06 | 2017-10-11 | 晶睿通訊股份有限公司 | Stereo vision image calibration method and related image capturign device |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5652616A (en) * | 1996-08-06 | 1997-07-29 | General Instrument Corporation Of Delaware | Optimal disparity estimation for stereoscopic video coding |
| US8045793B2 (en) * | 2007-08-06 | 2011-10-25 | Samsung Mobile Display Co., Ltd. | Stereo matching system and stereo matching method using the same |
-
2011
- 2011-10-26 TW TW100138860A patent/TWI465105B/en not_active IP Right Cessation
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5652616A (en) * | 1996-08-06 | 1997-07-29 | General Instrument Corporation Of Delaware | Optimal disparity estimation for stereoscopic video coding |
| US8045793B2 (en) * | 2007-08-06 | 2011-10-25 | Samsung Mobile Display Co., Ltd. | Stereo matching system and stereo matching method using the same |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US9918010B2 (en) | 2015-06-30 | 2018-03-13 | Industrial Technology Research Institute | Method for adjusting vehicle panorama system |
Also Published As
| Publication number | Publication date |
|---|---|
| TW201318407A (en) | 2013-05-01 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US10958892B2 (en) | System and methods for calibration of an array camera | |
| JP5997645B2 (en) | Image processing apparatus and method, and imaging apparatus | |
| TWI554976B (en) | Surveillance systems and image processing methods thereof | |
| US20150301351A1 (en) | Image processing apparatus, image processing method, and program for generating a three dimensional image to be stereoscopically viewed | |
| CN112070845A (en) | Calibration method and device of binocular camera and terminal equipment | |
| US11863888B2 (en) | Image signal processing for reducing lens flare | |
| US9282253B2 (en) | System and method for multiple-frame based super resolution interpolation for digital cameras | |
| CN105205796A (en) | Wide-area image acquisition method and apparatus | |
| US10349040B2 (en) | Storing data retrieved from different sensors for generating a 3-D image | |
| US20110064298A1 (en) | Apparatus for evaluating images from a multi camera system, multi camera system and process for evaluating | |
| KR102801383B1 (en) | Image restoration method and device | |
| RU2690757C1 (en) | System for synthesis of intermediate types of light field and method of its operation | |
| CN109785390B (en) | Method and device for image correction | |
| WO2019054304A1 (en) | Imaging device | |
| JP2009284188A (en) | Color imaging apparatus | |
| KR101140953B1 (en) | Method and apparatus for correcting distorted image | |
| CN118233764B (en) | Image demosaicing method and device | |
| KR20220121533A (en) | Image restoration method and image restoration apparatus for restoring images acquired through an array camera | |
| TWI465105B (en) | Automatic alignment method for three-dimensional image, apparatus and computer-readable recording medium | |
| TWI599809B (en) | Lens module array, image sensing device and fusing method for digital zoomed images | |
| US9538161B2 (en) | System and method for stereoscopic photography | |
| CN117939291A (en) | Method, device, equipment and storage medium for determining image pickup parameters of endoscope system | |
| US10332269B2 (en) | Color correction of preview images for plenoptic imaging systems | |
| JP5446285B2 (en) | Image processing apparatus and image processing method | |
| CN117278821A (en) | A lateral chromatic aberration correction method for biprism virtual binocular vision system |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| MM4A | Annulment or lapse of patent due to non-payment of fees |