KR20140103017A - User Terminal and Method for displaying image thereof - Google Patents

Abstract

And a method of displaying an image of the user terminal. The image display method of the present user terminal displays a plurality of display layers having different depths in a superimposed manner. When a user input is detected, the depth of at least one display layer of the plurality of display layers is changed according to user input, Express the physical action by input.

Description

Technical Field [0001] The present invention relates to a user terminal,

The present invention relates to a user terminal and an image displaying method thereof, and more particularly, to a user terminal displaying an image using a plurality of display layers and an image displaying method thereof.

In recent years, various user terminals (e.g., smartphones or tablet PCs) have been used. Such a user terminal typically includes a touch screen for sensing user input.

Meanwhile, when the user input is detected on the touch screen of the user terminal, the existing user terminal provides only the two-dimensional image effect at the point where the user input is detected. Since existing user terminals provide only two - dimensional image effects, image effects provided by existing user terminals have non - realistic and unnatural limitations.

Therefore, there is a need to search for ways to provide more realistic and natural image effects in response to user input.

SUMMARY OF THE INVENTION The present invention has been made to solve the above-mentioned problems occurring in the prior art, and it is an object of the present invention to provide a display device capable of providing a three-dimensional image effect responding to user input using a plurality of display layers having different depths, And an image display method of the same.

According to an aspect of the present invention, there is provided a method of displaying an image of a user terminal, the method comprising: displaying a plurality of display layers having different depths in a superimposed manner; And modifying the depth of at least one of the plurality of display layers according to the user input to provide an image effect responsive to the user input when a user input is sensed.

The plurality of display layers may include a first display layer for displaying an image and a second display layer having transparency of a water surface on the first display layer.

In addition, the first display layer has a fixed depth, and the depth of the second display layer may be changed according to the user input.

The providing step may include: changing the depth of the second display layer at which the user input is sensed when the user input is sensed; And expressing the physical action by the user input at a point where the user input is sensed using the first display layer and the second display layer having the changed depth.

Also, the displaying may include calculating a refractive index and a reflectance at a point where the user input is sensed using the first display layer and the second display layer having the changed depth, The physical action by input can be expressed.

And, the image effect may be a riffle effect.

If the display screen is a screen displaying a plurality of display items, the area of the first display layer where the plurality of display items are displayed has a high depth value, and the plurality of display items of the first display layer An unmarked area may have a low depth value.

The second display layer may have a depth value between a depth value of an area where the plurality of display items are displayed and a depth value of an area where the plurality of display items are not displayed.

If the user input is detected in one of the plurality of display items, the providing step may decrease a depth value of a point at which the user input is detected in the first display layer to a depth value of the second display layer And the display item in which the first user input is sensed can provide an image effect of being immersed in water.

According to another aspect of the present invention, there is provided a user terminal including: a display unit for displaying a plurality of display layers having different depths in a superimposed manner; And a controller for changing the depth of at least one display layer of the plurality of display layers according to the user input when a user input is detected through the user input unit, And a control unit for controlling the display unit to provide the display unit.

The plurality of display layers may include a first display layer for displaying an image and a second display layer having transparency of a water surface on the first display layer.

In addition, the first display layer has a fixed depth, and the depth of the second display layer may be changed according to the user input.

When the user input is detected through the user input unit, the control unit changes the depth of the point where the user input is detected in the second display layer, and changes the depth of the first display layer and the second display layer having the changed depth, The display layer can express the physical action by the user input at the point where the user input is sensed.

Also, the controller calculates a refractive index and a reflectance at a point where the user input is sensed using the first display layer and the second display layer having the changed depth, and outputs the calculated refractive index and reflectance to the user input Can express the physical action by.

And, the physical action may be a riffle effect.

If the display screen is a screen displaying a plurality of display items, the area of the first display layer where the plurality of display items are displayed has a high depth value, and the plurality of display items of the first display layer An unmarked area may have a low depth value.

The second display layer may have a depth value between a depth value of an area where the plurality of display items are displayed and a depth value of an area where the plurality of display items are not displayed.

When the user input is detected in one of the plurality of display items, the controller may lower the depth value of the point where the user input is detected in the first display layer to a depth value of the second display layer, 1 < / RTI > user input can provide an image effect in which the sensed display item is submerged.

According to various embodiments of the present invention as described above, the user terminal can provide the user with a realistic and natural three-dimensional image effect in response to user input.

1 is a block diagram schematically illustrating a configuration of a user terminal according to an embodiment of the present invention;
FIG. 2 is a block diagram illustrating a detailed configuration of a user terminal according to an embodiment of the present invention; FIG.
FIGS. 3A and 3B illustrate a depth map of a first display layer and a first display layer according to an embodiment of the present invention;
FIGS. 4A and 4B illustrate a first display layer and a second display layer before and after a user input is sensed, according to an embodiment of the present invention;
5 is a diagram illustrating a method of rendering a display layer according to reflectance and refractive index according to an embodiment of the present invention;
6 is a diagram for explaining a method of generating a specular highlight according to an embodiment of the present invention;
FIGS. 7A and 7B are diagrams for explaining a ripple effect in response to a user input according to an embodiment of the present invention;
8A-8C illustrate a first display layer and a second display layer, when a plurality of display items are displayed, according to another embodiment of the present invention;
9A and 9B are views for explaining a water-immersion effect in response to a user input according to another embodiment of the present invention,
10 is a flowchart illustrating a method of displaying an image of a user terminal according to an embodiment of the present invention.

Hereinafter, the present invention will be described in more detail with reference to the drawings. 1 is a block diagram briefly showing a configuration of a user terminal 100 according to an embodiment of the present invention. 1, the user terminal 100 includes a display unit 110, a user input unit 120, and a control unit 130. [ At this time, the user terminal 100 may be implemented as a smart phone, but it may be implemented by various electronic devices such as a tablet PC, a notebook PC, a desktop PC, a smart TV, and the like.

The display unit 110 outputs the image data under the control of the control unit 130. In particular, the display unit 110 may output image data using a first display layer and a second display layer having different depths.

At this time, the first display layer is an image display layer for displaying an image, and may have a fixed depth. The second display layer is a semi-transparent display layer having a certain transparency (e.g., transparency of the water surface), and the depth can be changed according to user input. Accordingly, the display unit 110 can display the effect of the image displayed on the first display layer being water-immersed. Also, the display unit 110 calculates the reflectance and the refractive index according to the variation of the depth of the second display layer, and applies a physical phenomenon (for example, Snell's law or specular highlight) according to the calculated reflectance and refractive index And can output image data of the first display layer.

The user input unit 120 may sense a user input for controlling the user terminal 100. At this time, the user input unit 120 may be implemented by a touch screen, but it may be realized by another input device such as a mouse, a pointing device or the like.

The control unit 130 controls the overall operation of the user terminal 100 according to a user command input through the user input unit 120. [

In particular, when the display unit 110 detects a user input through the user input unit 120 while displaying a plurality of display layers having different depths, the controller 130 displays a plurality of display layers The display unit 110 may be controlled to change the depth of at least one display layer to provide an image effect responsive to user input.

Specifically, when a user input is detected through the user input unit 120, the control unit 130 changes the depth of a point where the user input is detected in the second display layer, and displays the first display layer and the second display having the changed depth Layer can be used to express the physical action by user input at the point where the user input is detected.

For example, when the second display layer is a semitransparent display layer having transparency of a water surface, the control unit 130 uses a first display layer and a second display layer having a changed depth to determine a refractive index And reflectance, and can express a physical action (e.g., Snell's law, specular highlight, etc.) by user input according to the calculated refractive index and reflectance.

At this time, the controller 130 may control the display unit 110 to display a ripple effect at a point where the user input is sensed by the physical action.

On the other hand, when the display screen is a wall paper screen displaying a plurality of display items, a region of the first display layer where a plurality of display items are displayed has a high depth value, and a plurality of display items among the first display layers are not displayed The region may have a low depth value. Also, the second display layer may have a depth value between a depth value of an area where a plurality of display items are displayed and a depth value of an area where a plurality of display items are not displayed.

When a user input is detected in one of the plurality of display items while the display unit 110 displays a start screen displaying a plurality of display items, the controller 130 controls the display unit 110 to display, The depth value may be lower than the depth value of the second display layer to provide an image effect in which the sensed display item is immersed in water.

According to an embodiment of the present invention as described above, the user terminal 100 can provide the user with a realistic and natural three-dimensional image effect in response to user input.

Hereinafter, the user terminal 100 will be described in more detail with reference to FIGS. 2 to 9B.

FIG. 2 is a block diagram illustrating the configuration of a user terminal 200 according to an exemplary embodiment of the present invention. Referring to FIG. 2, the user terminal 200 includes a communication unit 210, a display unit 220, a storage unit 230, a user input unit 240, and a control unit 250.

2 is a block diagram illustrating various components in the case where the user terminal 200 is a device having various functions such as a communication function, an image output function, and an image effect providing function. Therefore, depending on the embodiment, some of the components shown in Fig. 2 may be omitted or changed, and other components may be further added.

The communication unit 210 is configured to perform communication with various types of external devices or external servers according to various types of communication methods. The communication unit 210 may include various communication chips such as a Wi-Fi chip, a Bluetooth chip, an NFC chip, and a wireless communication chip. At this time, the Wi-Fi chip, the Bluetooth chip, and the NFC chip communicate with each other using the WiFi method, the Bluetooth method, and the NFC method. Among these, the NFC chip refers to a chip operating in an NFC (Near Field Communication) system using 13.56 MHz band among various RF-ID frequency bands such as 135 kHz, 13.56 MHz, 433 MHz, 860 to 960 MHz and 2.45 GHz. When a Wi-Fi chip or a Bluetooth chip is used, various connection information such as an SSID and a session key may be transmitted and received first, and communication information may be used to transmit and receive various information. The wireless communication chip refers to a chip that performs communication according to various communication standards such as IEEE, ZigBee, 3G (3rd Generation), 3rd Generation Partnership Project (3GPP), LTE (Long Term Evolution)

In particular, the communication unit 210 can receive image data from an external server or device.

The display unit 220 outputs image data under the control of the controller 250. In particular, the display unit 220 may output image data using a plurality of display layers having different depths.

Specifically, the display unit 220 may output image data using an image display layer displaying an image and a translucent display layer having transparency of a water surface.

At this time, the image display layer can have depth according to the displayed image. For example, if the image display layer displays an image as shown in FIG. 3A, the image display layer can have a depth map as shown in FIG. 3B. On the other hand, the image display layer may not change the depth according to the user input, but this is only an embodiment, and according to the embodiment, the depth of the image display layer may also be changed according to the user input. The translucent display layer is located on the image display layer and can change the depth according to user input. In particular, a semi-transparent display layer can have a higher depth value than the image display layer as a whole, but in some areas it can have a depth value lower than the image display layer.

Particularly, by outputting an image using an image display layer displaying an image and a translucent display layer having transparency of a water surface, the display unit 220 displays an image effect in which an image displayed on the image display layer is immersed in water, . That is, according to the shape of the surface of the semitransparent display layer calculated by the control unit 250, the display unit 220 displays various physical effects on the image displayed by the image display layer according to the reflectance and the refractive index (for example, Image display by the following refractive index, specular highlight, etc.) can be provided.

When a user input is detected through the user input unit 240, the display unit 220 displays an image displayed by the image display layer using a semitransparent display layer whose depth (or shape) is changed by the control of the control unit 250 Can be displayed. Accordingly, the display unit 220 can provide a ripple effect to the user at the point where the user input is sensed.

The display unit 220 may be implemented as an LCD (Liquid Crystal Display), but it may be an organic light-emitting diode (OLED), a thin film transistor (TFT), a cathode ray tube ), And the like.

The storage unit 230 stores various modules for driving the user terminal 200. For example, the storage unit 230 may store software including a base module, a sensing module, a communication module, a presentation module, a web browser module, and a service module. In this case, the base module is a base module for processing a signal transmitted from each hardware included in the user terminal 200 and transmitting the signal to the upper layer module. The sensing module is a module for collecting information from various sensors and analyzing and managing the collected information, and may include a face recognition module, a voice recognition module, a motion recognition module, and an NFC recognition module. The presentation module is a module for constructing a display screen, and may include a multimedia module for reproducing and outputting multimedia contents, a UI, and a UI rendering module for performing graphics processing. The communication module is a module for performing communication with the outside. A web browser module refers to a module that accesses a web server by performing web browsing. A service module is a module including various applications for providing various services.

As described above, the storage unit 230 may include various program modules, but it goes without saying that the various program modules may be omitted, modified or added depending on the type and characteristics of the user terminal 200. For example, when the user terminal 200 is implemented as a smart phone, the base module may further include a position determination module for determining a GPS-based position, and the sensing module may further include a sensing module can do.

The user input unit 240 may receive a user command for controlling the user terminal 200. The user input unit 240 may sense a user input for selecting one of the plurality of display items displayed on the display unit 220. [

In particular, when the user input unit 240 is implemented as a touch screen, the user input unit 220 may sense a user touch on a part of the entire area of the touch screen.

Meanwhile, the user input unit 240 may be implemented as a touch screen, and may be implemented as another input device capable of controlling the user terminal 200, such as a pointing device, a mouse, and the like.

The control unit 250 controls the overall operation of the display device 200 using various programs stored in the storage unit 230.

2, the control unit 250 includes a RAM 251, a ROM 252, a graphics processing unit 253, a main CPU 254, first through n interfaces 255-1 through 255-n, Bus < / RTI > At this time, the RAM 251, the ROM 252, the graphics processor 253, the main CPU 254, the first to n interfaces 255-1 to 255-n, etc. may be connected to each other via the bus 256 .

The ROM 252 stores a command set for booting the system and the like. The main CPU 254 copies the O / S stored in the storage unit 230 to the RAM 251 according to the instruction stored in the ROM 252, and executes the O / S To boot the system. When the booting is completed, the main CPU 254 copies various application programs stored in the storage unit 230 to the RAM 251, executes the application program copied to the RAM 251, and performs various operations.

The graphic processing unit 253 generates a screen including various objects such as an icon, an image, and a text using an operation unit (not shown) and a rendering unit (not shown). The operation unit calculates an attribute value such as a coordinate value, a shape, a size, and a color to be displayed by each of the objects according to the layout of the screen using the control command received from the user input unit 240. The rendering unit generates screens of various layouts including the objects based on the attribute values calculated by the operation unit. The screen generated by the rendering unit is displayed within the display area of the output unit 240.

The main CPU 254 accesses the storage unit 230 and performs booting using the O / S stored in the storage unit 230. [ The main CPU 254 performs various operations using various programs, contents, data stored in the storage unit 230, and the like.

The first to n interfaces 255-1 to 255-n are connected to the various components described above. One of the interfaces may be a network interface connected to an external device via a network.

In particular, when the display unit 220 detects a user input through the user input unit 240 while displaying a plurality of display layers having different depths, the controller 250 displays a plurality of display layers The display unit 220 may be controlled to change the depth of at least one display layer to provide an image effect responsive to user input.

4A, the display unit 220 may display an image by superimposing the image display layer 310 and the translucent display layer 320 having different depths. At this time, the controller 220 controls the display unit 220 to provide an image effect in which an image displayed in an area having a depth value smaller than the depth value of the semi-transparent display layer 320 of the image display layer 310 is immersed in water, Can be controlled.

More specifically, the controller 250 calculates a reflectance and a refractive index for an area having a depth value smaller than a depth value of the semi-transparent display layer 320 of the image display layer 310, The display unit 220 can be controlled to provide a submerged image effect. A method of rendering the display layer according to the reflectance and the refractive index by the controller 250 will be described with reference to FIG.

First, the control unit 250 generates light (S510). At this time, the light can be located in an arbitrary spatial region.

Then, the controller 250 calculates a normal vector of the translucent display layer 320 (S520). At this time, the controller 250 may calculate a vertical vector of each pixel as a normal vector for each pixel of the translucent display layer 320. [

Then, the controller 250 calculates the reflectance and the refractive index of each pixel according to the surface of the translucent display layer 320 and the position of the camera (S530). Specifically, the control unit 240 can calculate the reflectance and the refractive index of each pixel according to the position of the camera and the normal vector direction of the display layer using Fresnel's formulas.

The control unit 250 generates a specular highlight according to the reflectance, and renders the first display layer according to the index of refraction (S540). Specifically, the control unit 230 may process the highlighted specular image according to the calculated reflectance, or project it onto the semitransparent display layer using the environment map. For example, the control unit 230 generates a three-dimensional surface mesh using a height field of a riffle as shown in FIG. 6, adds light to an arbitrary position, , The camera can perform 3D shading according to the position and angle of the water surface to generate specular highlights. Also, the controller 230 may render the image display layer using the refractive index calculated according to the Snell's law.

Then, the controller 250 synthesizes the rendering results according to the reflectance and the refractive index (S550). The control unit 250 may control the display unit 220 to output the rendered image according to the reflectance and the refractive index.

Through the above-described process, the image displayed on the image display layer can provide a water-immersion effect.

4A, while the image display layer 310 having the different depths and the translucent display layer 320 overlap and display an image, if a user touch is detected in an arbitrary area of the touch screen, the controller 250 May reduce the depth value of the point where the user touch is detected in the semitransparent display layer 320, as shown in FIG. 4B. At this time, the decreasing depth value may be different according to the strength of the user's touch. Specifically, the control unit 250 decreases the depth value more as the intensity of the user's touch increases, and decreases the depth value as the intensity of the user's touch decreases. Meanwhile, the translucent display layer 320 shown in FIG. 4B shows a moment when a user's touch is detected, and the depth value of the translucent display layer 320 can be changed as a ripple occurs for a predetermined time.

When the depth value of the translucent display layer 320 is changed, the controller 250 may calculate the reflectance and the refractive index using the method described with reference to FIG. 5 to render and output the display layer.

Accordingly, the control unit 250 can control the display unit 220 to generate a riffle effect at the point where the user touch is detected. For example, as shown in FIG. 7A, when the user touches a specific region in a state that the image is rendered, the controller 250 provides a riffle effect as shown in FIG. 7B The display unit 220 can be controlled. At this time, the controller 250 may provide a wavelet effect based on the depth of the image display layer. That is, as shown in FIG. 7B, the controller 250 can control the display unit 220 to display and display more wavelets in a direction in which the depth value is lower than the direction in which the depth value is higher.

According to the embodiment as described above, the user terminal can provide a more realistic and natural three-dimensional ripple effect to the user.

Hereinafter, another embodiment of the present invention will be described with reference to FIGS. 8A to 9B.

Specifically, as shown in FIG. 9A, while a plurality of display items 910 to 990 are displayed, the image display layer 810 displays the display items 910 to 990, Area has a high depth value, and the area where the display items 910 to 990 are not located has a low depth value. 8B, the semitransparent display layer 820 has a depth between an area where the display items 910 to 990 are located and an area where the display items 910 to 990 are not located in the image display layer 810 Value. ≪ / RTI >

8B, when a user command is input in which one of the plurality of display items 910 to 990 is touched while the image display layer 810 and the translucent display layer 820 overlap and display an image , The controller 250 may reduce the depth value of the area where the touched display item is located to be lower than the depth value of the translucent display layer 820 as shown in FIG. Accordingly, the controller 250 can provide the user with the effect that the touched display item is wrapped in water.

For example, when the fifth display item 950 is touched while a plurality of display items 910 to 990 are displayed as shown in FIG. 9A, the control unit 250 controls the fifth display item The depth value of the area in which the first display item 950 is located may be lower than the depth value of the translucent display layer so that the fifth display item 950 may provide an image effect to the user as shown in FIG.

According to the various embodiments of the present invention as described above, the user terminal 200 can provide a realistic and natural three-dimensional image effect to the user.

Hereinafter, an image output method of the user terminal 100 will be described with reference to FIG.

First, the user terminal 100 superimposes a plurality of display layers having different depths (S1010). Specifically, the user terminal 200 may display an image by superimposing a first display layer displaying an image and a second display layer having transparency of a water surface.

Then, the user terminal 100 determines whether a user input is detected (S1020).

When the user input is detected, the user terminal 100 changes the depth of at least one of the plurality of display layers according to the user input, thereby providing an image effect responding to the user input (S 1030). Specifically, the user terminal 100 may provide a ripple effect to the user, as described in FIGS. 3A-B, and may provide the user with the effect of immersing the display item in the water as described in FIGS. 8A-B .

With the image output method as described above, the user terminal 100 can provide the user with a realistic and natural three-dimensional image effect in response to user input.

Meanwhile, the image output method of the user terminal 100 according to the above-described various embodiments may be implemented as a program and provided to a display device. In particular, a program including an image output method of the user terminal 100 may be stored in a non-transitory computer readable medium.

A non-transitory readable medium is a medium that stores data for a short period of time, such as a register, cache, memory, etc., but semi-permanently stores data and is readable by the apparatus. In particular, the various applications or programs described above may be stored on non-volatile readable media such as CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM,

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the present invention.

110, 220: display unit 120, 240: user input unit
130, 250: a control unit 210:
230:

Claims (18)

A method of displaying an image of a user terminal,
Displaying a plurality of display layers having different depths in an overlapping manner;
And changing the depth of at least one of the plurality of display layers according to the user input to provide an image effect responsive to the user input when a user input is detected. The method according to claim 1,
Wherein the plurality of display layers comprises:
A first display layer for displaying an image and a second display layer having transparency of a water surface on the first display layer. 3. The method of claim 2,
Wherein the first display layer has a fixed depth,
Wherein the depth of the second display layer is changed according to the user input. The method of claim 3,
Wherein the providing step comprises:
Changing a depth of a point where the user input is sensed in the second display layer when the user input is sensed; And
And expressing the physical action by the user input at a point where the user input is sensed using the first display layer and the second display layer having the changed depth. 5. The method of claim 4,
Wherein said expressing comprises:
Calculating a refractive index and a reflectance at a point where the user input is sensed by using the first display layer and the second display layer having the changed depth, and expressing the physical action by the user input according to the calculated refractive index and reflectance And displaying the image. 6. The method of claim 5,
Wherein the image effect comprises:
Wherein the image is a ripple effect. 3. The method of claim 2,
When the display screen is a screen displaying a plurality of display items,
Wherein an area in which the plurality of display items of the first display layer is displayed has a high depth value and an area in which the plurality of display items are not displayed in the first display layer has a low depth value Way. 8. The method of claim 7,
Wherein the second display layer has a depth value between a depth value of an area where the plurality of display items are displayed and a depth value of an area where the plurality of display items are not displayed. 9. The method of claim 8,
Wherein the providing step comprises:
When a user input is detected in one of the plurality of display items, a depth value of a point where the user input is detected in the first display layer is lower than a depth value of the second display layer, Wherein the display item provides a water-immersion image effect. A display unit for displaying a plurality of display layers having different depths in a superimposed manner;
A user input section for detecting user input;
And a control unit for controlling the display unit to change the depth of at least one of the plurality of display layers according to the user input when the user input is detected through the user input unit and provide an image effect responsive to the user input, And a user terminal. 11. The method of claim 10,
Wherein the plurality of display layers comprises:
A first display layer for displaying an image and a second display layer having transparency of a water surface on the first display layer. 12. The method of claim 11,
Wherein the first display layer has a fixed depth,
Wherein the depth of the second display layer is changed according to the user input. 13. The method of claim 12,
Wherein,
Wherein when the user input is detected through the user input unit, a depth of the point where the user input is sensed is changed in the second display layer, and the first display layer and the second display layer having the changed depth are used, And expresses the physical action by the user input at a point where the user input is detected. 14. The method of claim 13,
Wherein,
Calculating a refractive index and a reflectance at a point where the user input is sensed using the first display layer and the second display layer having the changed depth to express the physical action by the user input according to the calculated refractive index and reflectance Lt; / RTI > 15. The method of claim 14,
Wherein the image effect comprises:
Wherein the user terminal is a riffle effect. 12. The method of claim 11,
When the display screen is a screen displaying a plurality of display items,
Wherein the area of the first display layer in which the plurality of display items are displayed has a high depth value and the area of the first display layer in which the plurality of display items are not displayed has a low depth value. 17. The method of claim 16,
Wherein the second display layer has a depth value between a depth value of an area where the plurality of display items are displayed and a depth value of an area where the plurality of display items are not displayed. 18. The method of claim 17,
Wherein,
When a user input is detected in one of the plurality of display items, a depth value of a point where the user input is detected in the first display layer is lower than a depth value of the second display layer, Wherein the display item provides a water-immersion image effect.

Images