WO2019112096A1 - Viewpoint image mapping method for integrated imaging system using hexagonal lens - Google Patents

Abstract

Provided is a method for generating an image to be displayed on a screen in an integrated imaging system that uses a hexagonal lens hexagonally tiled through a lateral shift, the image being generated through mapping from a plurality of viewpoint images. A viewpoint image mapping method according to an embodiment of the present invention includes: a first calculation step for calculating each hexagonal lens inside which each sub-pixel on a screen belongs; and a second calculation step for calculating each pixel of each viewpoint image to be mapped to each sub-pixel on the basis of the relationship between each sub-pixel and each hexagonal lens calculated in the first calculation step. Accordingly, an image to be displayed on a display panel can be generated by mapping a viewpoint image with respect to an integrated image which uses an arbitrary hexagonal lens array that can remove a Moiré pattern.

Description

Viewpoint Mapping Method of Integrated Image System Using Hexagonal Lens

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to 3D image display technology, and more particularly to an integrated image display.

In the case of an integrated image using a two-dimensional lens array that provides the conventional vertical and horizontal parallax, a square lens array is used to efficiently use the limited amount of information.

However, if the periodic structure of the lens array is similar to the periodic structure of the subpixel of the display panel, there is a problem that the moiré pattern appears in the three-dimensional image recognized by the observer.

Therefore, in the case of a multi-view display using only the left and right parallax using a conventional lenticular lens or the like, the method of solving the moiré pattern is employed by tilting the lenticular lens so that the periodic structure of the lens array is different from the periodic structure of the sub-pixels of the panel.

Therefore, in order to remove the Moire pattern even in the case of the two-dimensional lens array, a method of changing the periodic structure of the lens array and the periodic structure of the subpixel is required.

As one of the methods, if a hexagonal shape constituting a two-dimensional lens array is not a regular shape but an arbitrary hexagonal shape capable of hexagonal tiling is used, the fill factor of the lens array is maximized, It is possible to remove the Moire pattern by making it significantly different from the periodic structure.

However, in the case of a lens array using such arbitrary hexagons, it is difficult to find a mapping rule in the mapping process of assigning view images to subpixels.

It is an object of the present invention to provide an integrated image system using a hexagonal tiling which is hexagonal tiled through a lateral shift, in which an image to be displayed on a screen is mapped from a plurality of viewpoint images, In a method for generating a plurality of images.

According to an aspect of the present invention, there is provided a method of mapping a viewpoint image, the method comprising: a first calculation step of calculating each hexagon lens in which each subpixel on the screen belongs; And a second calculation step of calculating each pixel of each viewpoint image to be mapped to each subpixel on the basis of the relationship between each subpixel and each hexagonal lens calculated in the first calculation step.

The first calculation step may include a first calculation step of calculating each grating to which each subpixel belongs; And a second calculation step of calculating which of the six hexagonal lenses included in each grid calculated in the first calculation step belongs to each of the subpixels.

The gratings are divided into a first basic vector connecting 'the center of the specific hexagonal lens' to the 'center of the first hexagonal lens adjacent to the specific hexagonal lens', and a second basic vector 'the center of the specific hexagonal lens' Can be generated by linearly combining the second fundamental vectors connecting " center "

The first basic vector is s = (s _x , s _y ), the second basic vector is t = (t _x , t _y ), the center coordinate of each hexagon lens is? S +? T, (?,?).

The lowest leftmost coordinate of each grid is? S +? T, and each grid may be indexed with (?,?).

If the indexes of the respective gratings calculated in the first calculation step are (?,?), The hexagonal lenses partially including the gratings calculated in the first calculation step are (?,?), beta], [alpha], [beta] + 1), and [alpha] + 1, [beta] + 1.

The second calculation step may be to calculate each hexagonal lens to which each subpixel belongs by using an equation of a straight line connecting points shared by the hexagonal lenses partially included in each grid calculated in the first calculation step have.

The second calculation step may be to calculate each pixel of each viewpoint image to be mapped to each subpixel based on the relative positional difference between 'each subpixel' and the 'center of each hexagonal lens to which each subpixel will belong' have.

The hexagonal lens may be a hexagonal lens other than regular hexagonal.

According to another aspect of the present invention, a viewpoint image mapping method includes providing a viewpoint image; And a second calculation step of calculating each hexagonal lens belonging to each subpixel on the screen and calculating each pixel of each viewpoint image to be mapped to each subpixel on the basis of the relationship between each subpixel and each hexagonal lens .

According to another aspect of the present invention, there is provided a multi-view image display method including: a first calculation step of calculating each hexagonal lens in which each sub pixel on the screen belongs to; A second calculation step of calculating each pixel of each viewpoint image to be mapped to each subpixel on the basis of the relationship between each subpixel and each hexagonal lens calculated in the first calculation step; And mapping and displaying each pixel of each viewpoint image calculated in the second calculation step on each subpixel on the display panel screen.

Meanwhile, the integrated image system according to another embodiment of the present invention calculates each hexagonal lens in which each subpixel on the screen belongs, and calculates a difference between each subpixel based on the relationship between each subpixel and each calculated hexagonal lens A processor for calculating each pixel of each viewpoint image to be mapped; A display panel in which each pixel of each viewpoint image calculated by the processor is mapped and displayed in each subpixel on the screen; And a hexagonal lens array positioned in front of the display panel.

As described above, according to the embodiments of the present invention, it is possible to generate an image to be displayed on a display panel by mapping a viewpoint image to an arbitrary hexagon lens array using integrated image capable of removing a moiré pattern.

Fig. 1 is a view showing a basic configuration and principle of an integrated image and multi-point display, in which (a) shows a configuration of a display system, Fig. 2 (b)

FIG. 2 is a view showing the difference between the integrated image display and the multi-view display, in which (a) shows the ray distribution characteristics of the multi-view display, (b)

3 is a view showing a viewpoint image rendering method of a multi-view display,

4 is a diagram illustrating a viewpoint image rendering method of an integrated image display,

Figure 5 illustrates a method of parameterizing a hexagonal lens array,

6 is a diagram illustrating a basic vector and a lattice structure for positioning a hexagonal lens array,

FIG. 7 shows a method of defining a basic vector for positioning a hexagonal lens array and examples of coordinates,

8 is a diagram illustrating indexes of a grid including subpixels and corresponding subpixels,

9 is a diagram illustrating hexagonal lenses that are likely to belong to subpixels for the (?,?) Lattice,

10 is a diagram illustrating a coordinate calculation method for finding a hexagonal lens to which a subpixel belongs to a (?,?) Lattice,

11 is a diagram illustrating an index definition method of a viewpoint image,

12 is a diagram showing a situation in which subpixels at (x _p , y _p ) belong to the (? ',?') Th lens,

FIG. 13 is a flow chart for explaining a viewpoint image mapping method of an integrated image system using a hexagonal lens,

14 is a diagram summarizing specific contents of a viewpoint image mapping method of an integrated image system using a hexagonal lens,

15 is a block diagram of an integrated imaging system using a hexagonal lens.

Hereinafter, the present invention will be described in detail with reference to the drawings.

Figure 1 shows the basic structure and principle of an integrated image or multi-view display. In the case of the integrated image and the multi-view display providing the vertical, horizontal, and left-right parallax, the two-dimensional lens array may be attached only in front of the display panel as shown in FIG.

When the focal distance of the attached lens array and the distance between the display panel and the attached lens array are similar to each other, light rays emitted without direction in each pixel of the display panel as shown in FIG. 1 (b) Depending on the relative position of the center, it proceeds in a specific direction with directionality.

As a result, in the integrated image or multi-view display, each pixel of the display panel expresses a specific light ray of a light field after passage through the lens.

Figure 2 shows the difference between an integrated image display and a multi-view display. In the case of the multi-point display, as shown in FIG. 2A, the bundles of pixels passing through each lens are defined to be shifted so that the light rays, which are taken by the respective pixels, converge at specific points in time (V1, V2, ...) It is a display that is designed.

On the other hand, in the case of the integrated image display, as shown in FIG. 2 (b), a bundle of pixels passing through each lens is arranged in a horizontal direction with respect to each lens, so that there is no point where rays gather in space.

In this case, it is necessary to determine what information the rays represented by the integrated image display or the multi-view display represent. For the multi-view display, when a perspective camera is positioned at each of the points of view as shown in FIG. 3, Can be obtained.

Also, in the case of the integrated image display, as shown in FIG. 4, a view image can be rendered using an orthographic camera for a collection of rays according to directions. The image to be displayed on the display panel should be appropriately mapped into a single image in consideration of the hardware structure such as the lens array structure and parameters of the display panel.

In the embodiment of the present invention, a viewpoint image mapping method of an integrated image or a multi-view display using an arbitrary hexagonal lens array is presented. The arbitrary hexagonal lens array includes not only an array of regular hexagonal lenses but also a lens array composed of hexagonal lenses other than regular hexagonal lenses.

In order to maximize the fill factor of the hexagonal lens array, a structure of a hexagonal lens array capable of hexagonal tiling is required. Therefore, in the embodiment of the present invention, a mapping method for a hexagon that can be hexagonal tiling only by lateral shift .

These hexagons have a central symmetric characteristic. Therefore, when the center of the hexagon is moved to the origin of the xy plane as shown in FIG. 5, the hexagon can be parameterized by expressing three vertexes continuous in the counterclockwise direction in the coordinates of the polar coordinate system.

(L ₁ , θ ₁ ), (l ₂ , θ ₂ ), and (l ₃ , θ ₃ ), respectively, in the polar coordinate system of the three vertexes continuous in the counterclockwise direction are hexagonal tiling Any hexagonal lens array can be determined.

As shown in FIG. 6, if a vector connecting the centers of two adjacent hexagonal lenses at the center of each hexagonal lens is defined as a primitive vector, the positions of the centers of all the hexagonal lenses can be determined by a linear combination of these basic vectors , And each hexagon lens can be indexed with coefficients of a linear combination.

For example, if each basic vector is s = (s _x , s _y ), t = (t _x , t _y ) and the center coordinate of any hexagonal lens is αs + βt, Lt; RTI ID = 0.0 > index. ≪ / RTI > An example of the basic vector definition method and the coordinates of each vector according to the convention of FIG. 5 is shown in FIG. FIG. 7 shows a method and coordinates for defining a basic vector for positioning a hexagonal lens array.

For the viewpoint image mapping, it is necessary to calculate, for each subpixel on the screen, how the subpixel belongs to the inside of the hexagon lens having an index, and the relative position difference with respect to the center of the hexagon lens.

The first step for this calculation is to find out how many sub-pixels belong to the lattice in the lattice generated by the basic vector as shown in FIG. Assuming that the leftmost and lowermost coordinates of the lattice are? S +? T, this lattice can be characterized by a lattice having an index of (?,?). At this time, the coordinates of the subpixel having the current interest (x _p, y _p) is the index of the lattice belonging to a sub-pixel when d (α, β) can be calculated by the following equation (1).

(1)

(One)

The subpixels belonging to the grid having the indices of (?,?) At this time are (?,?), (? + 1,?) 1), and it is necessary to calculate exactly which hexagonal lens the corresponding subpixel belongs to.

For this calculation, four hexagonal lenses having indices of (?,?), (? + 1,?), (?,? + 1), The coordinates of the points have to be calculated, and the line equation connecting these points can be used to calculate which subpixel belongs to which of the four hexagonal lenses.

Letting (α ', β') be the index of the hexagonal lens to which the subpixel belongs and (x _T , y _T ) = (x _p , y _p ) Can be calculated by the following equation (2).

(2)

(2)

After calculating the subpixel to which the hexagonal lens belongs, it is necessary to determine whether to map and fetch the information from the viewpoint image having an index by using the relative distance difference between the subpixel and the center of the hexagonal lens.

In this case, the index of the viewpoint image is defined in the same manner as in FIG. 11 in the embodiment of the present invention. That is, the subpixel corresponding to the center point of the hexagonal lens utilizes the information of the viewpoint image having the index of (0, 0), and the first index is -1 or +1 whenever the subpixel moves left and right by one pixel, Each time you move one subpixel, the second index is +1 or -1.

At this time, the viewpoint image index (v _x , v _y ) of the subpixel (x _p , y _p ) can be calculated by the following equation (3) in the situation as shown in FIG.

(3)

(3)

Information haejueoya mapped to the sub-pixel in this case the screen (v _x, v _y) of the pixel of the point image (α ', β') of the screen by using color information of the pixel containing the center of the hexagonal lens of the index sub- Record the information corresponding to the color of the pixel. If the index of the corresponding pixel of the point image (i _v, j _v) la and la the offset of the point image (o _vx, o _vy), the pixel size of the point image (p _vx, p _vy), the following equation: Can be calculated according to equation (4).

(4)

(4)

13 is a flowchart provided for explaining a viewpoint image mapping method of an integrated image system using a hexagonal lens.

In order to map the viewpoint image in the integrated image system using the hexagonal lens, first, subpixels constituting the display panel are set (S110). Specifically, the size and the number of subpixels are set.

The hexagonal lenses constituting the hexagonal lens array are indexed (S120), the grids are generated and indexed (S130), and the subpixels are indexed (S140). The hexagonal lens indexing method is shown in Fig. 6, the grid generation & indexing method is shown in Fig. 8, and the subpixel indexing method is shown in Fig. 11, respectively.

Then, viewpoint images to be displayed on the display panel are obtained (S150).

Then, each lattice to which each subpixel belongs is calculated (S160), and it is calculated (S170) whether each subpixel belongs to one of the hexagonal lenses partially included in each grid calculated in the step S160.

Each viewpoint image to be mapped to each subpixel is calculated based on the relative positional difference between 'each subpixel' and 'center of each hexagonal lens to which each subpixel will belong' (S180) Each pixel of each viewpoint image is calculated (S190).

Thus, each pixel of each viewpoint image is mapped to each subpixel to display an integrated image through the hexagon lens array.

The detailed contents of the viewpoint image mapping method of the integrated image system using the hexagonal lens described above are summarized in FIG.

15 is a block diagram of an integrated imaging system using a hexagonal lens. The integrated imaging system using a hexagonal lens includes a communication unit 210, a processor 220, a display 230, and a storage unit 240, as shown in FIG.

The communication unit 210 is a communication unit that receives a plurality of viewpoint images from an external device or an external network, and the storage unit 240 stores viewpoint images. The communication unit 210 and the storage unit 240 function as image providing means for providing view images.

The processor 220 maps the pixels of the viewpoint images to the subpixels according to the viewpoint image mapping method described above. In this process, the storage unit 240 provides the necessary storage space for the processor 220.

The display 230 includes the hexagonal lens array and the display panel described above. Pixels of each viewpoint image are mapped and displayed on each subpixel on the display panel screen, and the display panel screen is transmitted to the user as an integrated image by a hexagon lens array located in front.

It goes without saying that the technical idea of the present invention can also be applied to a computer-readable recording medium having a computer program for performing the functions of the apparatus and method according to the present embodiment. In addition, the technical idea according to various embodiments of the present invention may be embodied in computer-readable code form recorded on a computer-readable recording medium. The computer-readable recording medium is any data storage device that can be read by a computer and can store data. For example, the computer-readable recording medium may be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical disk, a hard disk drive, or the like. In addition, the computer readable code or program stored in the computer readable recording medium may be transmitted through a network connected between the computers.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present invention.

Claims (12)

A method of mapping a viewpoint image of an integrated imaging system, A first calculation step of calculating each hexagonal lens in which each sub pixel on the screen belongs to; And And a second calculation step of calculating each pixel of each viewpoint image to be mapped to each subpixel based on the relationship between each subpixel and each hexagonal lens calculated in the first calculation step, Way. The method according to claim 1, The first calculating step includes: A first calculation step of calculating each grating to which each subpixel belongs; And a second calculation step of calculating whether each subpixel belongs to any hexagonal lens among the hexagonal lenses partially included in each grid calculated in the first calculation step. The method of claim 2, The gratings, The first basic vector connecting the center of the specific hexagonal lens to the center of the first hexagonal lens adjacent to the specific hexagonal lens and the center of the specific hexagonal lens and the center of the second hexagonal lens adjacent to the specific hexagonal lens And the second basic vectors are linearly combined with each other. The method of claim 3, The first basic vector is s = (s _x , s _y ) The second basic vector is t = (t _x , t _y ) The center coordinate of each hexagonal lens is? S +? T, And each hexagonal lens is indexed with (?,?). The method of claim 4, The lowest leftmost coordinate of each lattice is? S +? T, Wherein each grid is indexed with (?,?). The method of claim 5, If the index of each grid calculated in the first calculation step is (?,?), The hexagonal lenses, which are partially included in the respective gratings calculated in the first calculation step, are arranged in the order of (?,?), (? + 1,?) ) Of the viewpoint image. The method of claim 2, In the second calculation step, Wherein each hexagon lens belonging to each subpixel is calculated through calculation using a straight line equation that connects points shared by hexagon lenses partially included in each grid calculated in the first calculation step . The method according to claim 1, In the second calculation step, Each pixel of each viewpoint image to be mapped to each subpixel is calculated on the basis of a relative positional difference between 'each subpixel' and the 'center of each hexagonal lens to which each subpixel will belong' . The method according to claim 1, The hexagonal lens, Wherein the image is a hexagonal lens not a regular hexagon. A providing unit for providing viewpoint images; And And a second calculation step of calculating each hexagonal lens belonging to each subpixel on the screen and calculating each pixel of each viewpoint image to be mapped to each subpixel on the basis of the relationship between each subpixel and each hexagonal lens Wherein the viewpoint image mapping method comprises: A first calculation step of calculating each hexagonal lens in which each sub pixel on the screen belongs to; A second calculation step of calculating each pixel of each viewpoint image to be mapped to each subpixel on the basis of the relationship between each subpixel and each hexagonal lens calculated in the first calculation step; And And mapping and displaying each pixel of each viewpoint image calculated in the second calculation step to each subpixel on the display panel screen. A processor for calculating each hexagonal lens belonging to each sub pixel on the screen and calculating each pixel of each view image to be mapped to each sub pixel based on a relationship between each sub pixel and each of the calculated hexagonal lenses; A display panel in which each pixel of each viewpoint image calculated by the processor is mapped and displayed in each subpixel on the screen; And And a hexagonal lens array positioned in front of the display panel.

Images