KR20000032647A - 3-dimensional reading support system and method thereof - Google Patents

Abstract

An object of the present invention is to provide a reading support system and method for supporting not only reading content but also 3D additional information related thereto when a user reads.

According to the present invention, a book is attached with a marker that can be detected by a computer through a digital camera; A digital camera for photographing a page read by a user and a movement of a finger; A computer that analyzes the images of the pages of the book and the finger motion detected by the digital camera and retrieves additional information from the database according to the tracked finger movement; And a display device for displaying the result of the search by the computer.

Description

3D reading support system and method

The present invention relates to a reading support system and method, and more particularly, to a system and method for obtaining 3D information and moving picture information which are not included in a book in reading.

Conventional books contain text information and still image information, that is, pictures, but since such still image information is two-dimensional information rather than three-dimensional information, the image may be rotated or changed depending on the user's will. There was a disadvantage.

Recently, due to the popular dissemination of multimedia computers, multimedia documents including various still images, sound information, and moving image information have been developed, but they must see all the information through a monitor. There was a problem in that the commands must be entered in advance in advance. In addition, the monitor has a limitation of resolution and cannot display more information than a book generally used.

The present invention has been made to solve the problems of the prior art as described above, using a book with a marker and a head mounted display device to provide three-dimensional additional information in real time in the most convenient way for the user in three-dimensional The purpose is to provide a reading support system and method.

1 is a block diagram of a reading support system according to an embodiment of the present invention;

2 is a module configuration diagram of the reading support system shown in FIG. 1,

3 is a flowchart illustrating a pretreatment process occurring in the pretreatment module shown in FIG. 2;

4 is a flowchart illustrating a location tracking process occurring in the location tracking module illustrated in FIG. 2;

FIG. 5 is a flowchart of a visualization process occurring in the visualization module illustrated in FIG. 3.

According to the present invention for achieving the object as described above, the book is attached with a marker that can be detected by a computer through a digital camera; A digital camera for photographing a page read by a user and a movement of a finger; A computer that analyzes the images of the pages of the book and the finger motion detected by the digital camera and retrieves additional information from the database according to the tracked finger movement; And a display device for displaying the result of the search by the computer.

In addition, the first step of recognizing the identifier from the book with a marker that can be detected by the computer by a digital camera, and tracking the movement of the user's finger reading the book; A second step of analyzing the identifier recognized in the first step and a finger position of the user, synchronizing the analyzed result, and extracting a synchronized index from a previously stored database; And a third step of displaying the content corresponding to the index extracted in the second step.

In addition, a first step of recognizing an identifier from a book with a marker that can be detected by the computer by a digital camera on the computer, and tracking the movement of the finger of the user who is reading the book; A second step of analyzing the identifier recognized in the first step and a finger position of the user, synchronizing the analyzed result, and extracting a synchronized index from a previously stored database; A computer-readable recording medium capable of executing the third step of displaying contents corresponding to the index extracted in the second step on the head mounted display device HMD is provided.

In the following, with reference to the accompanying drawings, a preferred embodiment according to the present invention will be described in detail.

1 is a block diagram illustrating a reading support system according to an embodiment of the present invention.

The reading support system is equipped with a computer 150 for all data processing, database management, and image analysis, a digital camera 100 for digitally photographing and transmitting an object to a computer at high speed, and a marker for positioning in a page. A book 140, a brightly colored desk 120 to serve as a background of a digital photograph, a lamp 110 for adjusting brightness, and a translucent head mounted display device 130 for displaying additional information to the user 130, HMD: Head Mounted Display).

2 is a module configuration diagram of the reading support system shown in FIG. 1, which will be described in detail as follows.

The reading support system includes a preprocessing module 200 for performing an initialization operation, a location tracking module 220 for grasping a user's movement and the position and contents of a marker attached to the book for the preprocessed book, and grasping the book. Synchronized data with the additional information database 210 previously constructed based on the collected information, and the additional information generating module 230 for retrieving necessary additional information in the database and synchronizing with the angle and direction of the translucent head mounted display device. It is composed of a visualization module 240 for visualizing the translucent head mounted display device worn by the user.

3 is a flowchart illustrating a preprocessing process occurring in the preprocessing module illustrated in FIG. 2.

First, in step S100, an additional information database for the book is constructed to obtain additional information, and in step S110, a marker is attached to the book page so that the computer can locate it.

The additional information database content includes a 3D image and a video by computer graphic modeling of the content within the page. The markers used can also determine what information should be shown in the additional information database in the form of a small barcode. These barcodes are made of materials with high retroreflectivity so that they can be easily distinguished from other objects when shooting with a digital camera.

FIG. 4 is a flowchart illustrating a location tracking process occurring in the location tracking module shown in FIG. 2.

First, in step S200, a frame is taken by using a digital camera and the user's finger on the book is photographed at a speed of about 20 frames per second, and the frame is applied by using a computer vision algorithm in step S210. Analyze

On the other hand, the computer vision algorithm generates a frame using a digital camera, and then finds the location of each object, which can be applied to interactive applications such as games, for example William T. Freman, David B Anderson, Paul A. Beardsley, Chris N. Dodge, Michael Roth, Craig D. Weissman, and William S. Yerazunis published a paper titled 'Computer Vision for Interactive Computer Graphics' in an IEEE Computer Graphics and Application publication.

Subsequently, using the results of the frame analysis in step S210, in step S220 and step S230, the position and direction of the user's fingertip position and the position of the page and additional information of the book that the user is currently reading are needed. . Here, the finger acts as a pointer and is used as a tool for notifying the system where the current user needs additional information. The reading support system determines whether the position of the current finger and the marker match and determines whether to display as additional information.

On the other hand, to determine the location and direction of the portion of the user fingertips need to know the location of the user's fingertips is the object in US Patent Application No. 299,093 'System and method for tracking object using a detection system' It is described in detail as a technique for determining the position of the present invention, it may be used.

Thereafter, in step S240, a database index in the computer is finally extracted based on the analyzed marker data.

FIG. 5 is a flowchart illustrating a visualization process occurring in the visualization module illustrated in FIG. 3.

First, on the basis of the result obtained in the position tracking process described in FIG. 4, in step S300, the database stored in the computer is searched for necessary additional information, and in step S310, the position of the marker and the current head mount display device are written. Synchronize your eyes.

Next, in step S320, the information required by the user is displayed at an appropriate position of the head mounted display device. At this time, the translucent head-mounted display device can not only look outside the glass like the glasses, but also text and graphic information can be superimposed on the real image at the same time, so that the user can conveniently obtain the desired information. .

As described in detail above, the present invention automatically provides additional information on the content of a book that the user is reading by using a book with a marker and a head mounted display device, even if the user does not know the basic usage of the computer. There is an effect that it is possible to conveniently obtain three-dimensional information and video information.

The technical spirit of the present invention has been described above with reference to the accompanying drawings, but this is by way of example only for describing the best embodiment of the present invention and not for limiting the present invention. In addition, it is obvious that any person skilled in the art can make various modifications and imitations without departing from the scope of the technical idea of the present invention.

Claims (5)

A book with a marker that can be detected by a computer through a digital camera; A digital camera for photographing a page read by a user and a movement of a finger; A computer that analyzes the images of the pages of the book and the finger motion detected by the digital camera and retrieves additional information from the database according to the tracked finger movement; And And a display device for displaying the searched result by the computer. The method of claim 1, And the display device is a head mounted display device (HMD). A first step of recognizing an identifier from a book with a marker that can be detected by a computer by a digital camera and tracking the movement of a finger of the user who is reading the book; A second step of analyzing the identifier recognized in the first step and a finger position of the user, synchronizing the analyzed result, and extracting a synchronized index from a previously stored database; And displaying a content corresponding to the index extracted in the second step on a display device. The method of claim 3, wherein And the display device of the third step is a head mounted display device (HMD). On your computer, A first step of recognizing an identifier from a book with a marker that can be detected by a computer by a digital camera and tracking the movement of a finger of the user who is reading the book; A second step of analyzing the identifier recognized in the first step and a finger position of the user, synchronizing the analyzed result, and extracting a synchronized index from a previously stored database; A computer-readable recording medium capable of executing a third step of displaying contents corresponding to the index extracted in the second step on a head mounted display device (HMD).