KR102706803B1 - Virtual reality device - Google Patents

Abstract

The present invention relates to a virtual reality device, which generates interpolation frame data of a preset region of interest when a user is stationary, varies the region of interest in real time to reflect the user movement when the user moves, and generates interpolation frame data of the real-time varied region of interest. The virtual reality device combines the interpolation frame data of the preset region of interest with non-interest region data of current frame data when the user is stationary, and outputs interpolation frame data of the entire screen on each of first and second screens, and combines the interpolation frame data of the region of interest that varies in real time with non-interest region data of the current frame data when the user moves, and outputs interpolation frame data of the entire screen on each of the first and second screens.

Description

{VIRTUAL REALITY DEVICE}

The present invention relates to a virtual reality device that implements virtual reality.

Virtual reality technology is being applied in the fields of national defense, architecture, tourism, movies, multimedia, and games. Virtual reality refers to a specific environment or situation that feels similar to the real environment by using stereoscopic image technology.

To maximize the immersion of virtual reality, virtual reality technology is being applied to personal immersive devices. Representative personal immersive devices include HMD (Head Mounted Display), FMD (Face Mounted Display), and EGD (Eye Glasses-type Display).

Personal immersive devices have not improved as much as expected in terms of inconvenient external design, stereoscopic effect of stereoscopic images, immersion, fatigue, etc. Recently, virtual reality devices that display stereoscopic images on the display device of a smartphone and wear the smartphone on the HMD device worn by the user have been commercialized.

The virtual reality device has a screen that is close to the user's eyes and a wide field of view using a fish-eye lens. When the screen moves according to the user's movement in the virtual reality device, screen dragging and judder phenomena may occur.

The present invention provides a virtual reality device that allows a user to experience no judder when the screen moves.

The virtual reality device of the present invention comprises first and second screens divided into a region of interest and a region of no interest, a sensor module detecting movement of a user, a first frame rate converter generating interpolated frame data of a preset region of interest when the user is stationary, a region of interest determination unit varying the region of interest in real time by reflecting the movement of the user when the user moves, a second frame rate converter generating interpolated frame data of the region of interest varied by the region of interest determination unit, a data synthesis unit combining the interpolated frame data of the region of interest received from the first frame rate conversion unit or the second frame rate conversion unit with non-interest region data of the current frame data to output interpolated frame data of the entire screen, and a display driver writing data output from the data synthesis unit to pixels of the first and second screens.

The above first frame rate converter calculates a first motion vector based on the object movement of the input image, and generates interpolated frame data of the region of interest using the motion vector.

The above first frame rate converter switches to a standby state when the user moves.

The second frame rate converter calculates a second motion vector reflecting the direction and speed of the user's movement, and generates interpolated frame data of the region of interest using the second motion vector.

The above second frame rate converter switches to a standby state when the user is at a standby state.

The image data displayed on the first and second screens moves in the opposite direction to the user's movement direction. The region of interest, which is variable by the region of interest determination unit, moves in the same direction as the user's movement direction. The second motion vector faces the opposite direction to the user's movement direction.

The virtual reality device of the present invention comprises: first and second screens divided into a region of interest and a region of no interest, a user movement detection unit for detecting the movement of the user, a first region of interest extraction unit for extracting data within a preset region of interest from current frame data of an input image, a first frame rate conversion unit for generating region of interest data of an interpolated frame using data received from the first region of interest extraction unit when the user is stationary in response to an output signal of the user movement detection unit, a region of interest determination unit for determining a region of interest that follows the movement of the user when the user moves in response to an output signal of the user movement detection unit, a second region of interest extraction unit for extracting data within the region of interest determined by the region of interest determination unit, a second frame rate conversion unit for generating region of interest data of an interpolated frame using data received from the second region of interest extraction unit when the user moves in response to an output signal of the user movement detection unit, a data synthesis unit for combining interpolated frame data of a region of interest received from the first frame rate conversion unit or the second frame rate conversion unit and data of a non-interest region of the current frame data to output interpolated frame data, and a data synthesis unit for outputting interpolated frame data from the data synthesis unit. It has a display driver that writes data into pixels of the first and second screens.

The method for driving the virtual reality device includes the steps of detecting a user's movement, generating interpolated frame data of a preset region of interest when the user is stationary, varying the region of interest in real time to reflect the user's movement when the user moves, generating interpolated frame data of the region of interest varied in real time, combining the interpolated frame data of the preset region of interest and non-interest region data of current frame data when the user is stationary to output interpolated frame data of the entire screen on each of the first and second screens, combining the interpolated frame data of the region of interest varied in real time when the user moves and non-interest region data of the current frame data to output interpolated frame data of the entire screen on each of the first and second screens, and writing the interpolated frame data of the entire screen into pixels of the first and second screens.

The present invention can implement a virtual reality screen in which the user does not perceive screen dragging, motion blur, judder, etc. by varying the region of interest (ROI) according to the user's movement and generating interpolated frame data only within the ROI region to increase the frame rate.

FIG. 1 is an exploded perspective view showing a virtual reality device according to an embodiment of the present invention.
FIG. 2 is a block diagram showing a display-related circuit in the virtual reality device illustrated in FIG. 1.
Figures 3 and 4 are drawings showing an example of frame rate up-conversion using the MEMC algorithm.
Figure 5 is a block diagram showing in detail the MEMC processing unit illustrated in Figure 2.
Figure 6 is a diagram illustrating an ROI area when there is no user movement.
Figures 7 and 8 are diagrams illustrating ROI areas when a user moves.
Figure 9 is a drawing showing an example in which the driving status of the first and second frame rate converters on the time axis changes according to the user's movement.
Figure 10 is a diagram showing an example of a region of interest (ROI) changing in real time as a user moves.
Figure 11 is a diagram showing the user's movement and the second motion vector.
Figure 12 is a drawing showing an example in which reverse judder occurs when the direction of movement of an object in an image and the direction of a second motion vector are opposite.

The advantages and features of the present invention, and the method for achieving them, will become clear with reference to the embodiments described in detail below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and these embodiments are provided only to make the disclosure of the present invention complete and to fully inform a person having ordinary skill in the art to which the present invention belongs of the scope of the invention, and the present invention is defined only by the scope of the claims.

The shapes, sizes, ratios, angles, numbers, etc. disclosed in the drawings for explaining embodiments of the present invention are exemplary, and therefore the present invention is not limited to the matters illustrated. Like reference numerals refer to like elements throughout the specification. In addition, in describing the present invention, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the present invention, the detailed description will be omitted. When the terms “includes,” “has,” “consists of,” etc. are used in this specification, other parts may be added unless “only” is used. When a component is expressed in the singular, it includes a case where the plural is included unless there is a specifically explicit description.

When interpreting a component, it is interpreted as including the error range even if there is no separate explicit description.

When describing a positional relationship, for example, when the positional relationship between two parts is described as 'on ~', 'above ~', 'below ~', 'next to ~', there may be one or more other parts located between the two parts, unless 'right' or 'directly' is used.

Although the terms first, second, etc. may be used to describe various components, these components are not limited by these terms. These terms are only used to distinguish one component from another. Accordingly, a first component referred to below may also be a second component within the technical concept of the present invention.

The individual features of the various embodiments of the present invention may be partially or wholly combined or combined with each other, and various technical connections and operations may be possible, and the individual embodiments may be implemented independently of each other or may be implemented together in a related relationship.

Hereinafter, preferred embodiments according to the present invention will be described in detail with reference to the attached drawings. Throughout the specification, the same reference numerals refer to substantially the same components. In the following description, if it is judged that a detailed description of a known function or configuration related to the present invention may unnecessarily obscure the gist of the present invention, the detailed description will be omitted.

FIG. 1 shows an example of a personal immersive device having a head mounted display (HMD) structure, but the structure of the personal immersive device of the present invention is not limited thereto since various modifications are possible. FIG. 2 is a block diagram showing a display-related circuit in the virtual reality device illustrated in FIG. 1.

Referring to FIGS. 1 and 2, the personal immersive device of the present invention includes a lens module (12), a display module (13), a main board (14), a head gear (11), a side frame (15), a front cover (16), etc.

The display module (13) displays an input image received from the main board (14) including a display driver (22) for driving two screens. The two screens may be implemented on one display element (20) or implemented with two display elements (20). The first screen displays an image seen by the user's left eye, and the second screen displays an image seen by the user's right eye. In FIGS. 6 to 8, reference numeral "20L" denotes the first screen, and "20R" denotes the second screen. The display module (13) displays image data input from the main board (14) on the screens of the display panel. The image displayed on the screens moves according to the user's movement. For example, the image moves in the opposite direction to the user's movement direction. The image data displayed on the screens may be 2D/3D image data implementing a video image of virtual reality (VR). The display module (13) may display various information input from the main board in the form of text, symbols, etc.

The display element (20) may be implemented as a display panel of a flat panel display device such as a liquid crystal display (LCD) or an organic light emitting diode display (OLED). The display panel includes data lines to which a data voltage is applied, gate lines (or scan lines) to which a gate pulse (or scan pulse) is applied, and pixels that are arranged in a matrix form by an orthogonal structure of the data lines and the gate lines and are electrically connected to the data lines and the gate lines. Each of the pixels is divided into a red sub-pixel, a green sub-pixel, and a blue sub-pixel for color implementation. Each of the pixels may further include a white sub-pixel. Each of the sub-pixels may include one or more TFTs (Thin Film Transistors).

The display driver (22) drives the display element (20) and writes image data received from the MEMC (Motion Estimation Motion Compensation) processing unit (24) of the main board (14) into the pixels of the display element. The display driver (22) includes a data driver, a gate driver, and a timing controller. The data driver converts data of an input image into a gamma compensation voltage to generate a data voltage and outputs the data voltage to data lines. The gate driver sequentially outputs a gate pulse synchronized with the data voltage to the gate lines. The timing controller transmits data of an input image received from the MEMC processing unit (24) to the data driver. The timing controller controls the driving timing of the data driver and the gate driver so as to be synchronized with the input image data.

The lens module (12) includes a pair of ultra-wide-angle lenses, i.e., fisheye lenses, for widening the angle of view of the user's left and right eyes. The pair of fisheye lenses includes a left-eye lens positioned in front of the first screen and a right-eye lens positioned in front of the second screen.

The headgear (11) includes a back cover exposing fish-eye lenses, and a band connected to the back cover. The back cover, side frame (15), and front cover (16) of the headgear (11) are assembled to secure an internal space in which components of the personal immersive device are arranged and to protect the components. The components include a lens module (12), a display module (13), and a main board (14). The band is connected to the back cover. A user wears the personal immersive device on his or her head with the band. When the user wears the personal immersive device on his or her head, he or she views different display panels with his or her left and right eyes through the fish-eye lenses.

The side frame (15) is fixed between the head gear (11) and the front cover (16) to secure a gap in the internal space where the lens module (12), display module (13), and main board (14) are placed. The front cover (16) is placed on the front of the personal immersive device.

The main board (14) includes an image processor that runs virtual reality software and supplies left-eye images and right-eye images to the display module (13). The main board (14) further includes an interface module connected to an external device, a sensor module (26), etc. The interface module is connected to an external device through an interface such as a Universal serial bus (USB), a High definition multimedia interface (HDMI), etc. The sensor module (26) includes various sensors such as a gyro sensor and a 3-axis acceleration sensor.

The main board (14) can receive input image data from the outside through an interface module that is not shown. The image processor of the main board (14) moves the images displayed on the screens according to the movement of the user's head by correcting the left and right eye image data in response to the output signal of the sensor module (26). The frame data of the input image is moved in the opposite direction to the direction of the user's head movement by the image processor. The image processor can generate left and right eye images that match the resolution of the display panel based on the analysis result of the depth information of the 2D image and transmit them to the display module (13).

The image processor of the main board (14) includes a MEMC (Motion Estimation Motion Compensation) processing unit (24). The MEMC processing unit (24) can analyze the output signal of the sensor module (26) to determine the movement of the user's head or body (hereinafter, referred to as "user movement"). The MEMC processing unit (24) executes a MEMC algorithm to increase the frame rate in order to prevent judder when an image on the screen moves in conjunction with the movement of the user's head. The MEMC algorithm compares and analyzes frame data of the original image to extract a motion vector and generates new interpolation frame data to be inserted between the frame data of the original image based on the motion vector. Increasing the frame rate of the input image data can prevent judder because new interpolation frame data reflecting the movement of an object displayed on the screen is added.

The general MEMC algorithm operates on the image data of the entire screen, which is independent of the user's movement and dependent on the object movement of the input image. Therefore, it causes the latency required for data operation processing, which reduces the user's movement and the corresponding image response speed. As a result, when the general MEMC algorithm is applied, the data operation processing time is not small, so screen dragging or motion blur may be perceived even if the frame rate is increased. In virtual reality images reproduced as stereoscopic images, screen dragging or motion blur not only reduces the image quality but also increases user fatigue.

The MEMC processing unit (24) selects a region of interest (hereinafter referred to as “ROI region”) as a local area within the screen using a MEMC algorithm that reflects the user’s movement, and generates interpolated frame data only within the ROI region, thereby reducing the data calculation time of the MEME algorithm and increasing the response speed of the image according to the user’s movement. The data of the non-ROI region (hereinafter referred to as “Non-ROI region”) is copied from the current frame data of the original image to the data of the Non-ROI region. Therefore, the present invention can implement a virtual reality screen in which the user cannot perceive judder or motion blur, such as screen dragging, by varying the ROI region according to the user’s movement and generating interpolated frame data only within the ROI region.

Figures 3 and 4 are drawings showing an example of frame rate up-conversion using the MEMC algorithm.

Referring to FIGS. 3 and 4, frame rate up conversion can be largely divided into motion estimation (ME) and motion compensated interpolation (MCI). Motion estimation (ME) is a process of calculating the motion of an object between consecutive frame data and producing a motion vector (MV1) corresponding to a change in motion. Motion compensated interpolation (MCI) is a process of generating new interpolated frame data by an interpolation method using a motion vector (MV1) obtained through motion estimation (ME).

There are various known algorithms for motion judgment (ME), but the block matching algorithm (BMA) as shown in Fig. 3 is widely used due to its ease of implementation.

The block matching algorithm (BMA), as illustrated in FIG. 3, divides the current frame data (Fn) into M×N (M and N are positive integers, generally M=N) sized blocks, and for each block, compares it with blocks within a search range (SR) of a predetermined size in the previous frame data (Fn-1) to find a motion vector (MV) that indicates the block with the highest similarity. In FIG. 3, Bi,j is a block existing in the i-th row and j-th column in the current frame data (Fn). The search range (SR) is defined as the maximum distance that blocks or pixels matching the current frame data (Fn) can move during one frame period. The search range (SR) is the maximum distance that a block existing in the current frame data (Fn) can exist in the previous frame (Fn-1), and is determined by considering the computational and hardware complexity issues in the motion judgment process.

The matching criterion of the block matching algorithm (BMA) is to move the block of the previous frame data (Fn-1) by one pixel based on the block (Bi,j) of the current frame data (Fn) and calculate the similarity as the sum of absolute differences (SAD), and then calculate the motion vector (MV1) based on the sum of absolute differences (SAD).

Motion compensated frame rate up conversion using block matching algorithm (BMA) calculates motion vectors (MV), which indicate the direction and speed of movement of an object, between the current frame data (Fn) and the previous frame data (Fn-1) in preset block units.

Unlike the block matching algorithm (BMA), the bilateral motion estimation method (Bilateral ME, BME) directly estimates the motion vector (MV1) based on the interpolated frame, as illustrated in Fig. 4.

As shown in Fig. 4, assuming that a block or pixel moves at a constant speed during one frame period, and when finding a block passing through the corresponding block (Bi,j) of interpolated frame data (Fn-1/2), the blocks that match each other in the previous frame (Fn-1) and the current frame data (Fn) are symmetrical with respect to the corresponding block (Bi,j). The bidirectional motion determination method (BME) determines a motion vector (MV1) by utilizing the symmetry of the previous frame data (Fn-1) and the current frame data (Fn) based on the interpolated frame data (Fn-1/2).

The MEMC algorithm of the present invention is not limited to FIGS. 3 and 4. For example, the present invention utilizes a known MEMC algorithm, but selects a ROI region in real time by considering the user's movement, and varies the ROI region in real time when the user moves, and increases the frame rate only within the ROI region. In addition, the algorithm of the present invention generates a motion vector (MV2) in the opposite direction to the user's movement, and generates interpolated frame data within the ROI region by using the motion vector (MV2) generated when the user moves.

Fig. 5 is a block diagram showing the MEMC processing unit (24) in detail. Fig. 6 is a drawing showing an ROI area (60) when there is no user movement. Figs. 7 and 8 are drawings showing an ROI area (60) when the user moves.

Referring to FIGS. 5 to 8, the MEMC processing unit (24) includes a first ROI extraction unit (31), a first frame rate conversion unit (32), a user movement detection unit (33), an ROI determination unit (34), a second ROI extraction unit (35), a second frame rate conversion unit (36), and a data synthesis unit (40).

The first ROI extraction unit (31) receives the current frame data (Fn) of the input image, extracts pixel data within a preset ROI area, and transmits the extracted pixel data to the first frame rate conversion unit (32). The ROI area (60) set by the first ROI extraction unit (31) is designated as the ROI area when the user does not move.

Users are sensitive to changes in the center of the screen, but are insensitive to changes in the edges of the screen. The ROI area (60) of the first ROI extraction unit (31) can be set to the center of each of the first and second screens (20L, 20R) excluding the edges, as shown in Fig. 6, because the user views the center of the screen as the center. This ROI area (60) can be designated as the center of the screen of approximately 50% of the entire screen, but is not limited thereto.

The first frame rate conversion unit (32) executes the MEMC algorithm on pixel data within the ROI region (60) input from the first ROI extraction unit (31) when a flag signal from the user movement detection unit (33) is received. The flag signal can indicate whether the user moves or not according to its logic value. For example, when the user is stationary, the flag signal can be generated as 0 (=low), and when the user moves, the flag signal can be generated as 1 (=high). The first frame rate conversion unit (32) is activated when there is no user movement and executes the MEMC algorithm. The first frame rate conversion unit (32) calculates an object movement-based motion vector (MV1, hereinafter referred to as “first motion vector”) from image data within the ROI region (60) regardless of the user’s movement, and uses the first motion vector (MV1) to generate ROI region data of interpolated frame data (Fn-1/2) between the previous frame data (Fn-1) and the current frame data (Fn), thereby increasing the frame rate. The previous frame data (Fn-1) and the current frame data (Fn) are actual frame data in the input image, and the interpolated frame data is data generated using the first motion vector (MV1) calculated based on the object’s movement through a comparison between the frame data of the input image. The first frame rate conversion unit (32) switches to a standby mode and does not operate when the user moves by the user movement detection unit (33).

The user movement detection unit (33) detects the user's movement using sensors such as a gyro sensor, a 3-axis acceleration sensor, a magnetometer, and an IR camera. If the user does not move, the user movement detection unit (33) generates a flag signal to activate the first frame rate conversion unit (32). On the other hand, if the user moves, the user movement detection unit (33) controls the first frame rate conversion unit (32) to a standby mode and transmits information indicating the direction and speed of movement according to the user's movement to the ROI determination unit (34).

The frame data of the input image moves in the opposite direction to the user's movement direction. Since the user mainly looks at the center of the screen, the ROI area moves along the user's movement direction. The ROI determination unit (34) resets the ROI area in consideration of the user's movement and changes the ROI area in real time in conjunction with the user's movement. The ROI area (60) determined by the ROI determination unit (34) follows the user's movement. The ROI determination unit (34) faces the same direction as the user's movement direction as shown in FIG. 7 and FIG. 8 and moves the ROI area (60) at the user's movement speed.

The second ROI extraction unit (34) receives the current frame data (Fn) of the input image, extracts pixel data within the ROI area (60) determined by the ROI area determination unit (34), and transmits the data to the second frame rate conversion unit (36).

The second frame rate conversion unit (36) executes the MEMC algorithm on pixel data within the ROI region (60) input from the second ROI extraction unit (35) when a flag signal from the user movement detection unit (33) is received. The second frame rate conversion unit (36) is activated when the user moves and executes the MEMC algorithm. The second frame rate conversion unit (36) calculates a first motion vector (MV1) from the image data within the ROI region (60) and reflects the user movement direction and speed to the first motion vector (MV1) to generate a user movement-based motion vector (MV2, hereinafter referred to as “second motion vector”) that follows the user movement as shown in FIGS. 7 and 8. The second motion vector (MV2) faces the opposite direction to the user movement direction and has a size proportional to the user movement speed as shown in FIGS. 7 and 8. And the second frame rate conversion unit (36) increases the frame rate by generating ROI area data of interpolation frame data (Fn-1/2) between the previous frame data (Fn-1) and the current frame data (Fn) using the second motion vector (MV2). The second frame rate conversion unit (36) is switched to standby mode and does not operate when the user is stationary by the user motion detection unit (33).

The data synthesis unit (40) can determine whether the user is moving based on a flag signal from the user movement detection unit (33). When the user is stationary, the data synthesis unit (40) combines the ROI area data generated by the first frame rate conversion unit (32) and the Non-ROI area data of the current frame data (Fn) into one frame data to generate interpolation frame data (Fn-1/2) of the entire screen in each of the first and second screens (20L, 20R). The data synthesis unit (40) outputs image data in the order of the previous frame data (Fn-1), the interpolation frame data (Fn-1/2), and the current frame data (Fn). The image data output from the data synthesis unit (40) is transmitted to the display driving unit (22).

FIG. 9 is a diagram showing an example in which the driving states of the first and second frame rate converters on the time axis change according to the user's movement. In FIG. 5, “MEMC” represents a MEMC algorithm executed by the first frame rate converter (32) that is activated when the user is stationary. “M-MEMC” represents a MEMC algorithm executed by the second frame rate converter (36) that is activated when the user moves.

Referring to Fig. 9, when the user is stationary, the first frame rate conversion unit (32) operates to generate interpolated frame data of a preset ROI area based on the object movement of the input image. At this time, the second frame rate conversion unit (32) does not operate and switches to standby mode.

When the user moves, the second frame rate conversion unit (36) operates to generate interpolated frame data of the ROI area that changes in real time based on the object movement of the input image and the user movement. At this time, the first frame rate conversion unit (32) does not operate and switches to standby mode.

Figure 10 is a diagram showing an example of a region of interest (ROI) changing in real time when a user moves. According to the present invention, when a user moves, the ROI area can change in real time according to the direction and speed of the user's movement.

The present invention generates interpolated frame data by executing the MEMC algorithm only within the ROI region within a screen divided into an ROI region and a non-ROI region. As a result, the present invention can implement a virtual reality screen in which the user cannot perceive judder or motion blur, such as screen dragging, by minimizing the latency according to the data operation of the MEMC algorithm and increasing the frame rate.

The image displayed on the screen (20L, 20R) moves in the opposite direction of the user's movement. Considering this, the second motion vector can be determined in the user's movement direction along the opposite direction of the user's movement. However, if the object of the input image moves in the opposite direction to the direction of the second motion vector (MV2), the direction of the second motion vector and the object movement direction of the actual image may conflict, resulting in reverse judder as in the example of Fig. 12.

Fig. 11 is a drawing showing a user movement and a second motion vector. In Fig. 11, “OBJ” represents a moving object of an image displayed on a screen (20L, 20R). “USER” represents a user wearing a virtual reality device, and “m” represents a motion vector of the user (USER). Fig. 12 is a drawing showing an example in which reverse judder occurs when the direction of movement of an object of an image and the direction of the second motion vector are opposite.

When an object (OBJ) displayed on the screen (20L, 20R) moves in the opposite direction with respect to the second motion vector (MV2), reverse judder (RJD) may be observed as illustrated in FIG. 12 because the object image generated in the ROI area of the interpolation frame is generated in the opposite direction to the actual movement direction of the object.

The present invention can prevent reverse judder (RJD) by adjusting the weight (α) multiplied to the second motion vector (MV2) by considering the moving object (OBJ) of the image as shown in the following mathematical expression 1, taking into account the reverse judder (RJD).

Here, m is a motion vector according to the user's movement and x represents the second motion vector (MV2). The weight (α) is determined as a value between 0 and 1 depending on the user's movement speed and the direction of the moving object in the image.

< Case 1 >

Regardless of the movement of the object (0BJ) in the image, the second motion vector (MV2) is determined in the opposite direction of the motion vector (m) of the user (USER). The faster the time at which the head or gaze of the user (OBJ) moves, the larger the second motion vector (MV2) becomes, and the movement of the object (OBJ) in the image feels relatively small. In this case, since the movement of the object in the image will be relatively small, the movement of the object is not considered. Therefore, if the user's movement speed is faster than a predetermined reference value, the weight (α) in mathematical expression 1 is determined as 1, and is determined in the opposite direction of the user's movement direction.

< Case 2 >

By comparing the interpolated frame with the next frame (current frame) that actually exists, the weight (α) is adjusted to a value less than 1 in the area where the second motion vector (MV) occurs in the reverse direction with respect to the movement direction of the moving object (OBJ) in the image, thereby minimizing reverse judder (RJD).

< Case 3 >

By comparing the interpolated frame with the next frame, if the movement of the object (OBJ) is in the reverse direction with respect to the second motion vector (MV2) in approximately 50% or more of almost all parts of the image, the ROI region data of the interpolated frame is generated with the first motion vector (MV) without applying the second motion vector (MV2). In this case, the weight (α) in mathematical expression 1 is adjusted to 0 (zero).

Through the above explanation, those skilled in the art will be able to see that various changes and modifications are possible without departing from the technical idea of the present invention. Therefore, the technical scope of the present invention should not be limited to the contents described in the detailed description of the specification, but should be determined by the scope of the patent claims.

20, 20L, 20R: Display element 22: Display driver
24: MEMC processing unit 26: Sensor module
31: 1st ROI extraction unit 32: 1st frame rate conversion unit
33: User movement detection unit 34: ROI determination unit
35: 2nd ROI extraction unit 36: 2nd frame rate conversion unit
40: Data synthesis section

Claims (15)

First and second screens divided into areas of interest and areas of no interest;
A sensor module that detects the user's movements;
A first frame rate converter for generating interpolated frame data of a preset region of interest when the user is stationary;
A region of interest determination unit that changes the region of interest in real time to reflect the movement of the user when the user moves;
A second frame rate converter unit that generates interpolated frame data of a region of interest varied by the region of interest determining unit;
A data synthesis unit that combines interpolated frame data of an area of interest received from the first frame rate conversion unit or the second frame rate conversion unit and non-interested area data of the current frame data to output interpolated frame data of the entire screen; and
A display driving unit is provided that writes data output from the data synthesis unit into pixels of the first and second screens,
A virtual reality device in which the first frame rate converter is switched to a standby state when the user moves. In the first paragraph,
A virtual reality device in which the first frame rate converter calculates a first motion vector based on the object movement of the input image and generates interpolated frame data of the region of interest using the motion vector. delete In the first paragraph,
A virtual reality device in which the second frame rate converter calculates a second motion vector reflecting the direction and speed of the user's movement, and generates interpolated frame data of the region of interest using the second motion vector. In paragraph 4,
A virtual reality device in which the second frame rate converter is switched to a standby state when the user is stationary. In paragraph 4,
The above first frame rate converter calculates a first motion vector based on the object movement of the input image, and generates interpolated frame data of the region of interest using the motion vector.
The second frame rate converter calculates a second motion vector reflecting the user's movement direction and speed, and generates interpolated frame data of the region of interest using the second motion vector.
The above first frame rate converter switches to a standby state when the user moves,
A virtual reality device in which the second frame rate converter is switched to a standby state when the user is stationary. In clause 5 or 6,
The image data displayed on the first and second screens moves in the opposite direction to the user's movement direction,
The region of interest, which is variable by the region of interest determining unit, moves in the same direction as the user's movement direction.
A virtual reality device wherein the second motion vector points in the opposite direction to the user's movement direction. First and second screens divided into areas of interest and areas of no interest;
A user motion detection unit that detects the user's movement;
A first region of interest extraction unit for extracting data within a preset region of interest from current frame data of an input image;
A first frame rate converter for generating region of interest data of an interpolated frame using data received from the first region of interest extraction unit when the user is stationary in response to an output signal of the user motion detection unit;
A region of interest determination unit that determines a region of interest that follows the user's movement when the user moves in response to an output signal of the user's movement detection unit;
A second region of interest extraction unit for extracting data within a region of interest determined by the region of interest determination unit;
A second frame rate converter for generating region of interest data of an interpolated frame using data received from the second region of interest extraction unit in response to an output signal of the user movement detection unit when the user moves;
A data synthesis unit that combines interpolated frame data of an area of interest received from the first frame rate conversion unit or the second frame rate conversion unit and data of an area of no interest of the current frame data to output interpolated frame data; and
A display driving unit is provided that writes data output from the data synthesis unit into pixels of the first and second screens,
A virtual reality device in which the first frame rate converter is switched to a standby state when the user moves. In Article 8,
A virtual reality device in which the first frame rate converter calculates a first motion vector based on the object movement of the input image and generates interpolated frame data of the region of interest using the motion vector. delete In Article 8,
A virtual reality device in which the second frame rate converter calculates a second motion vector reflecting the direction and speed of the user's movement, and generates interpolated frame data of the region of interest using the second motion vector. In Article 11,
A virtual reality device in which the second frame rate converter is switched to a standby state when the user is stationary. In Article 12,
The image data displayed on the first and second screens moves in the opposite direction to the user's movement direction,
The region of interest, which is variable by the region of interest determining unit, moves in the same direction as the user's movement direction.
A virtual reality device wherein the second motion vector points in the opposite direction to the user's movement direction. delete delete