JP2003186596A - Display device and input method of display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a display device and input method of the same, which is capable of performing the input of an advanced display control by a simple motion of a man, even in the distance out of his range in a display device conducting bulletin of information. <P>SOLUTION: The device has an image input means for detecting the man's motion, a separating means for separating body motion into the motion of a plurality of regions, and a motion information output means for outputting motion information for every motion region. Each motion information output is combined as a selective input instruction, which performs the motion of selection and movement, and a decision instruction to the selective input instruction. On the basis of the results, the control of displaying is conducted. <P>COPYRIGHT: (C)2003,JPO

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a display device for displaying various information, and more particularly to an input method for controlling the display device from a distance out of reach.

[0002]

2. Description of the Related Art With the progress of display panel technology and digital information technology, an environment is being prepared in which a plurality of people can use information distributed or stored in a device using a thin display panel that can be hung on a wall. is there. For this reason, the display device requires input operations such as switching of information sources and indication of display points.

As a conventional input method for controlling such a display device in space from a distance, a remote controller having a keyboard has been generally used. However, it is inconvenient for a fine pointing position input and a display device commonly used by a plurality of people for the following reasons.

It is difficult to input a fine position such as an operation with a mouse and give an instruction for continuous movement from the keyboard. In a device shared by unspecified users, it is difficult to publicize where the remote control is placed and how to use it. It is desirable to be able to give input instructions without holding or wearing special equipment.

As a solution to the above, a method using a human gesture as an input means has been proposed, and a conventional example will be described.

The block diagram of FIG. 6 is disclosed in Japanese Patent Laid-Open No. 10-3155.
In one example, voice recognition technology is used in addition to gestures to enable advanced input such as display device control. Image sensor 6 for body action of human 61
2 and the voice input means 63 as an auxiliary means thereof are also used, and the recognition means 64 determines the intention of the operator and controls the display device or the like as the controlled device 65.

[0007]

However, the use of voice input as in the above-mentioned conventional example causes some inconvenient problems. The first is that voice input is a nuisance to other unspecified users. For example, a display device for explaining an exhibit in an environment such as a museum is very inconvenient. Secondly, in an environment where noise and other sounds already exist, it is very technically advanced and expensive to separate the input voice by natural utterance into the input device. For example, when it is installed outdoors or on the street. In addition, it is more common to generate voices and sounds for explanation on the display device itself, even when not outdoors, and even in the case of a game device, a sound effect or the like corresponds thereto.

Furthermore, it must be taken into consideration that making speech for voice recognition for control purposes is a painful task for humans, unlike ordinary conversation.

Therefore, in view of the above problems, the present invention provides a display device and an input method capable of performing advanced input control by human body movements without using other auxiliary input means such as voice input. With the goal.

[0010]

In order to solve the above problems, the present invention provides a display device main body for displaying electronic information, an image inputting means for detecting a motion of a human body, and an image recorded by the image inputting means. A motion analysis unit that analyzes body motion from information and a display control unit that controls display are provided, and the motion analysis unit separates the body motion into motions of a plurality of parts, and each motion part. It is characterized by further comprising motion information output means for outputting motion information, and performing display control based on a result of combining the respective motion information outputs.

According to such a structure as described in claims 1 and 7, it is possible to perform high-level display control only by an input by the movement of the body.

Further, the present invention according to claims 2 and 8 realizes a method of specifying a user who performs input from a plurality of users standing in front of a display device.

Further, in the present invention as set forth in claims 3 and 9, the image information is separated for the motion of the human right hand and the motion of the left hand, and one motion information output performs the operation of selection or movement. It is a selective input instruction, and it is desirable that the other operation information output is at least a determination instruction for the selective input instruction.

Further, in the present invention as set forth in claims 4 and 10, the movement of the hand by the human right hand or left hand,
The image information is separated into the action of moving the face, and the action information output of one is the selective input instruction for performing the operation of selection or movement,
It is desirable that the other operation information output is at least a decision instruction for the selective input instruction.

Further, according to the present invention as set forth in claims 5 and 11, a method of extracting the position information of the face first, and obtaining and separating the target operation part based on the position information of the face. It is desirable to take

Further, according to the present invention as set forth in claims 6 and 12, a pattern in which a human's right hand and left hand motions are combined in specifying a user who inputs from a plurality of users. Is desirable.

[0017]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to the drawings.

FIG. 1 is an overall view of a display device according to an embodiment of the present invention. The display device main body 1 has an IP address as an independent network terminal, is composed of a control circuit 2 (described in detail in FIG. 2), a display panel 3 and a setting input means 4 for setting initial settings, reception addresses, etc., and is distributed. A communication line 5 is connected to a communication interface, which is a means for receiving electronic information, and is connected to the Internet network. There are various types of display devices, from a large-scale display device such as an outdoor display panel to a small-scale display device for home use, and the size of the display panel also changes accordingly.
Although it is mainly shared by a plurality of people, it may be intended for an individual. As for the form of the device, it is not a stationary type like a personal computer, but it is natural that it is installed on such a wall surface.

The form of information is diversified from character information to still images and moving images, and addition of voice to images is also commonly used. Therefore, a display device for obtaining such information has a distribution form called PUSH by constant connection. In FIG. 1, a surveillance camera 6 is provided on the local network of the communication line 5.
Also connected to form a network system, the indoor or outdoor state at a distance can be seen by the display device of the present invention, and the state of a protected person such as an infant can be monitored.
Used for external security checks and other purposes.

Further, the content service information represented by 7 or 8 is received and displayed through the Internet network. These contents are diverse with news and magazines that have been selected according to the taste of the user and have a contract for charges, or natural scenery, and are regularly or irregularly updated and sent to be accumulated in the display device. .

Although not shown in FIG. 1, a home server, which is not shown in FIG. 1, is installed in the same local network as the home server, and the information obtained using the server as a window is selected by a predetermined filter means. The network configuration may be such that the information is transmitted to the display device or the large amount of information is stored in the server.

Reference numeral 9 denotes an image input means for detecting the movement of the human body, which is an image pickup device using a CCD or the like.

FIG. 2 is a block diagram of a control circuit according to an embodiment of the present invention. A CPU 11 and a D-CONT 16 which is a display controller, a VRAM 15 which is a video RAM, and a plurality of pages of received information are accumulated via an internal bus 12. PAGE-RAM 18 which is a page memory to be placed is connected,
Display data such as images of D-CONT 16 are displayed on the display panel 1.
It is output to 7 and displayed. As a display panel method, a liquid crystal, EL, plasma display method or the like which can be thinned can be used.

Further, the internal bus has a KIF 19 which is an interface of the setting input means 4, a COM-IF 21 which is a communication interface, a ROM 13 which has a control program of the present invention built therein, a RAM 14 which is an internal memory, and a specific device which the display device has. An GEN23 which is an information generating means, an SP24 which generates a voice and a sound, a TIM25 which is a timer for performing time information and timer processing are connected, and subsequently, an image of the image inputting means 28 or an image inputting means which is deeply related to the inputting method of the present invention. Command generating means 26 for forming a command for controlling display by R and L which are motion information outputs 27 and 27 of motion analyzing means DIS for analyzing body motion from information, and a specific comparison pattern are stored. It is composed of a pattern memory 29.

The motion analysis means 27 separates the body motion into motions of a plurality of parts by an algorithm stored in the CPU program, and outputs motion information R and L for each motion part. R and L may be, for example, three parts such as motion information of the right hand and the left hand, up / down / left / right orientation of the right hand / face, and further up / down / left / right orientation of the right hand / left hand and face. Then, in order to speed up the processing time of such analysis, a dedicated IC or a dedicated CPU may be assigned. It should be noted that the definition of the motion of the hand used in the present invention is not defined in a narrow sense such as a palm or a finger, but is defined broadly as a part beyond the arm.

The command generating means 26 outputs motion information R,
The display circuit is controlled by using the combination of L. In particular, the combined result is used as one command for specific control such as initial control, but for complicated control in the middle, different types of control instructions are assigned to the respective operation information outputs R, L, etc. . For example, as different control instructions, one operation information output is a selective input instruction for performing a selection or movement operation, and the other operation information output is at least a determination instruction for the selective input instruction. Further, if expressions are changed as a way of thinking for different control instructions, one may have a relationship of a master-slave combination such as an instruction for a direct operation and the other for determination, termination, and interruption for the direct instruction.

The pattern memory 29 is a memory for storing a comparison pattern in which the result of combining operations is one command for a specific control such as the initial control. For example, it is applied when selecting one from a plurality of users or when starting control of a specific function.
As a specific physical movement, it is natural to have a specific shape with the right and left hands.

GEN 23 is a specific information generating means by the display device itself, and is used as base display information before the arrival of the received information or when the received information is completely displayed. For example, the time display can be displayed like a wall clock with an image design of a needle type analog clock display.

The KIF 19 is connected to a KB 20 which is a keyboard of setting input means, and 20 performs initial setting of the display device and setting of an information destination for selecting reception. In addition to such initial setting, the present invention also sets a priority rank for each piece of received information. When there are multiple pieces of information that should be displayed at the same time by receiving multiple pieces of information according to the priority rank, the display device itself processes the display method and order based on the rank information, and the appropriate information for the user to view. You can post.

The communication interface 21 is the communication transmission means 2
Connected to 2. Reference numeral 22 is not limited to a telephone line or a LAN cable, and the display device of the present invention can be constructed by a wireless system such as a wireless system or a communication system using a power line. Especially by using the power line communication system,
Since the power supply required for the display device can be shared with the route taken from the power line outlet, it is very easy to install without the need for laying cables, etc. You don't have to worry about shields.

FIG. 3 is a flowchart of body motion analysis and control command formation in the embodiment of the present invention. An example of operation is a case where the left and right hands are used to move or select a displayed operated object.

First, in S-31, it is analyzed and detected whether a person exists in the image output from the image input means.
When the presence is detected, the analysis output which is separated into the right hand motion and the left hand motion is obtained by the process of S-32.

Next, in S-33, it is detected whether or not it is a body motion instruction indicating the start of control. In this case, the operation itself, which is a combination of the right hand and the left hand, may be the start instruction, or only the left hand may be the start instruction.

When the start of control is instructed, in S-34, movement or selection is performed with respect to the operated object on the display screen following the operation of the right hand. Specifically, it is the movement of the cursor, the movement of a specific part in the image, the page feed of the entire screen, or the selection of a specific instruction or character from the list that appears in the image. It is a go operation.

Further, in parallel with S-34, the operation of the left hand is monitored by S-35 to detect the presence / absence of a specific pattern. In the case where the left hand instructs the action indicating the decision of the operation, the result of the movement amount or the selection instructed by the right hand at that time is confirmed. The movement / selection based on the instruction indicated by the position of the right hand is continued until the action of the left hand indicates the decision. This is extremely excellent as a human interface because it is possible to instruct two-dimensionally in both the + direction and the-direction in an analog manner using a hand that is extremely easy for humans to instruct. The detection part of the hand may be the palm, or the angle of the elbow or the angle of the arm itself.

S-36 carries out a specific confirmation process based on the detection of S-35, and issues a command to the control routine of the display device.

FIG. 4 is a display mode diagram illustrating the relationship between body movements and operations in the embodiment of the present invention. 4A in the figure
Shows a human motion, and shows a state in which the left hand is extended at a substantially right angle toward the display device.

Reference numeral 4A indicates an operation for instructing the start of control. As shown in the figure, the state in which both hands have a certain shape is used as the start instruction, but it may be expressed only by the operation of the left hand. After the control is started, a specific control operation is performed, and the operations shown in 4B and 4C are performed. 4B on the right side of FIG.
1, 4C1 and 4C2 represent the screen states of the display device after the control operation mode is entered.

4B shows the screen state 4B depending on the position of the right hand 41.
A cursor 43 appears at 1 and shows a relationship in which the cursor 43 moves following 41, which is the tip of the hand. Also, 43 may be a component on the display screen without being a cursor.

4C is a screen state 4 using the angle of the right arm 42.
The relationship of moving the selection list 44 displayed on C1 up and down is shown. Further, 4C2 shows the relationship in which the page display unit 45 displayed in the display corner of the screen 4C2 is instructed to move by the vertical movement of the arm angle 42 for page feed and page return.

The relationship between the right hand and the left hand in the above display form diagram is not limited and may be reversed.
In addition, it is a movement part that is easy for humans to control not only the movement of the hand but also the movement of the hand and face. It is useful to move the face both vertically and horizontally.

Reference numeral 4Z indicates an operation for giving an instruction when the movement or selection is decided. In the example of the figure, the left hand 46 is changed from the bent state to the horizontal state, which indicates an instruction that the content of the control is determined. In the present invention, complicated movements and selections and decision actions are made to correspond to actions of different parts of human beings by the select / set method, so that complicated and sophisticated control is replaced by actions that are extremely easy for humans to operate. is doing. The display device is easy to discriminate even from a distance and does not cause a misjudgment by using a large movement by the arm or the like rather than a detailed movement such as grasping a hand or fingers.

The main purpose of the present invention is to control the display when the user is at a distance from the display device, but it goes without saying that the present invention is also effective for applications where the human cannot operate the keyboard for other operations. . In this respect, it is also suitable for a configuration in which it is difficult to attach a keyboard even when the distance from the display device is short, and a case where a simpler human interface is realized by eliminating the keyboard.

Further, since the above-described structure of the present invention detects human motions on a plane, only one sensor needs to be used for plane input as an image input means, and a special device is required. do not do. As a method of inputting a body motion, a method of separating a motion of a hand into a plane motion and a front-back motion can be considered, but in that case, two image input means are required and the device becomes complicated. In addition, if a three-dimensional element is introduced, the position and angle that can be controlled by the user are limited, and the practicality is reduced.

Reference numeral 4X shown in FIG. 4 shows an operation pattern for identifying the user who tried to control when a plurality of persons are standing in front of the display device and for identifying the user. ing. In the 4X example, the specific pattern is instructed in a state where both the right hand and the left hand are bent at substantially right angles. 4X
Although not limited to this, it is desirable that the operation pattern is an intentional operation as much as possible and that the operation pattern is easily recognized by the display device and other people around.

FIG. 5 is a part of a flowchart according to an embodiment of the present invention, and particularly shows a part for identifying a body part.

First, in S-51, the face is identified and its position is extracted from the features of the human face such as the eyes, nose, and mouth. Next, the outline of the face is identified in S-52, and the orientation of the face is identified in S-53. Therefore, it is possible to estimate the distance and range in which the image of the arm or hand exists from the position, orientation, and outline of the face, and the shape and movement of the target part can be estimated more efficiently than the expected range and the contour pattern of the actual image based on that. You can ask.

[0048]

As described above, according to the present invention, in a display device for displaying information, it is possible to perform advanced display control by operating only a human body movement. Further, according to the present invention, in a display device that displays information to a plurality of users, the user can be specified, and advanced display control can be performed by the body motion of the user.

[0049]

[Brief description of drawings]

FIG. 1 is an overall view showing an embodiment of a display device according to the present invention.

FIG. 2 is a block diagram of a control circuit showing an embodiment of a display device according to the present invention.

FIG. 3 is a flow chart of an embodiment of a display device according to the present invention.

FIG. 4 is a display form diagram of an embodiment of a display device according to the present invention.

FIG. 5 is a partial flowchart of a display device according to the present invention.

FIG. 6 is a block diagram of a conventional example.

[Explanation of symbols]

1..Display unit body 2 ... Control circuit Display panel 5 ... Communication line 6 ... Surveillance camera 7, 8 ... Source of content service information 9. Image input means 16 ... Display controller ..Page memory 21..Communication interface 22 .. Communication transmission means 23..Specific information generation means 26 .. Command generation means 27 .. Motion analysis means 28 .. Image input means ..Pattern memory

   ─────────────────────────────────────────────────── ─── Continued front page    F-term (reference) 5B087 AA10 AB02 BC06                 5C082 AA13 BA41 BB53 CA54 CB05                       DA22 DA42 DA73 DA86 MM02                       MM05 MM09                 5L096 BA02 CA14 DA02 FA06 FA67                       FA69 HA01 JA22

Claims (12)

[Claims] 1. A display device main body for displaying electronic information, an image input means for detecting a movement of a human body, a movement analysis means for analyzing a body movement from image information recorded in the image input means, and a display control. In the display device including display control means for performing, the motion analysis means further includes a separation means for separating a body motion into motions of a plurality of parts, and a motion information output means for outputting motion information for each motion part. A display device, characterized by comprising: controlling display based on a result of combining the respective operation information outputs. 2. A display device main body for displaying electronic information, an image input means for detecting a movement of a human body, a movement analysis means for analyzing a body movement from the image information recorded in the image input means, and a display control. In the display device including display control means for performing, the motion analysis means further includes a separation means for separating a body motion into motions of a plurality of parts, and a motion information output means for outputting motion information for each motion part. If the result of combining the operation information output for each operation part is a preset specific pattern, the image generator who generated the specific pattern is determined as the display controller, and thereafter, A display device, which detects and analyzes a motion only for the person who generated the motion. 3. The separating means separates image information into human right hand and left hand motions, and one motion information output is a selective input instruction for performing a selection or movement operation, and the other motion. The display device according to claim 1, wherein the information output is at least a determination instruction for the selective input instruction. 4. The selective input instruction, wherein the separating means separates image information into a hand motion of a human right or left hand and a motion of moving a face, and one motion information output performs a selection or movement operation. 3. The display device according to claim 1, wherein the other operation information output is at least a determination instruction for the selective input instruction. 5. The motion part separating means extracts the position information of the face first, obtains and separates the target motion part based on the position information of the face, and separates the target motion part. The display device according to any one of 1. 6. The display device according to claim 2, wherein the specific pattern is a combination of human right hand and left hand motions. 7. A first step of detecting an image of a human body by an image input means, and a second step of separating the image information into body parts and outputting motion information as a motion for each part. A third step of forming a control command by combining the respective operation information outputs and outputting the control command to the display control means; and a fourth step of performing display control of the display device based on the control command.
And an input method for a display device. 8. A first step of detecting an image of a human body by an image input means, and a second step of separating the image information into body parts and outputting motion information as a motion for each part. When it is detected that the result of combining the operation information outputs is a preset specific pattern, the image generator who generated the specific pattern is determined as a display controller, and thereafter, the image generator is determined. Only form control commands,
A third step of outputting to the display control means, and a fourth step of performing display control of the display device based on the control command.
And an input method for a display device. 9. In the second step, image information is separated for the motions of the right hand and the left hand of a human and the respective motion information is output, and in the third step, one motion information output is performed. 9. The input method of the display device according to claim 7, wherein the input operation is a selective input instruction for performing a selection operation or a movement operation, and the other operation information output is at least a determination instruction for the selective input instruction. 10. In the second step, image information is separated for a hand motion with a human right hand or a left hand and a motion of moving a face, and respective motion information is output, and in the third step, The operation information output of one is a selective input instruction for performing a selection or movement operation, and the operation information output of the other is at least a determination instruction for the selective input instruction. Input method of the display device. 11. The method according to claim 7, wherein in the second step, face position information is first extracted, and a target operation part is obtained and separated based on the face position information.
11. The input method of the display device according to any one of 1 to 10. 12. The input method for a display device according to claim 8, wherein in the third step, the specific pattern is a pattern in which a right hand motion and a left hand motion of a human are combined.