JP2013115644A - Display device and display method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To display contents in accordance with positions of users, thereby minimizing a hindrance to viewing.SOLUTION: A display device comprises: means for receiving input of plural video contents; means for identifying plural users; detection means for detecting the positions of the users; means for associating each of the users with each of the video contents to be viewed; means for determining priorities in accordance with the positions of the users; creation means for creating a display screen by doing layout of the video contents in accordance with the priorities; and means for displaying the display screen. The display device detects the positional relation among the plural users and displays the video contents superposed on one another so as to give the highest priority to the content of a user nearest to the screen.

Description

  The present invention relates to a display device and a display method, and more particularly to a technique suitable for use in a display device viewed by a plurality of users.

  Conventionally, a large screen display represented by a liquid crystal display or a projection display has been widely used. A display system that displays on the entire wall surface by combining one or a plurality of displays has been put into practical use.

  In the wall surface display system, as a method for a plurality of users to view different contents, a method of detecting the user's position each time the remote controller is operated and displaying a screen at the nearest position has been devised. According to this, when there are a plurality of users, display is performed in a size divided by the number of people (see, for example, Patent Document 1).

JP 2001-94900 A JP 08-153187 A JP 2001-250121 A Japanese Patent Laid-Open No. 2005-100366

However, in the above-described conventional example, when the positions of a plurality of users approach, there are cases where viewing is hindered by being overwritten with the contents of other users. For example, when another user crosses, there is a problem that the content screen that the user views is hidden by the content screen of the other user.
In view of the above-described problems, an object of the present invention is to enable content to be displayed following a user's position so that viewing is prevented from being hindered.

  The display device of the present invention includes a means for inputting a plurality of video contents, a means for identifying a plurality of users, a detecting means for detecting the user's position, a means for associating the video content to be viewed with the user, and the position of the user And a means for deciding the priority according to the priority, a creation means for laying out video content according to the priority and creating a display screen, and a means for displaying the display screen.

  According to the present invention, it is possible to display the content of the user viewing the closest location with the highest priority, and there is an effect that it is possible to minimize the disturbing viewing.

It is a block diagram which shows the example of a characteristic structure of the display apparatus of 1st Embodiment. It is a figure which shows the example of a display when there is no interference between windows. It is a figure which shows the example of a display when a user moves. It is a figure which shows the example of a display when a user crosses back. It is a figure which shows the example of a display when a user crosses the front. It is a block diagram which shows the detailed structural example of a display apparatus. It is a flowchart which shows the flow of a process of user position detection. It is a figure which shows the structural example of a user list. It is a flowchart which shows the flow of an association process of a user and a window. It is a figure which shows the structural example of a window list. It is a flowchart which shows the flow of a process of an image | video display. It is a figure which shows the example of a display when a user crosses back. It is a figure which shows the example of a display when a user crosses back. It is a block diagram which shows the example of a characteristic structure of the display apparatus of 2nd Embodiment. It is a block diagram which shows the detailed structural example of the display apparatus of 2nd Embodiment. It is a flowchart which shows the flow of a process of user position detection. It is a figure which shows the structural example of the user list of 2nd Embodiment. It is a flowchart which shows the flow of a process of the video display of 2nd Embodiment.

Hereinafter, the present invention will be described in detail based on preferred embodiments with reference to the accompanying drawings. In addition, the structure shown in the following embodiment is an example, and this invention is not limited to the structure shown in figure.
(First embodiment)
In the present embodiment, an example will be described in which when a plurality of users view a large display, the display position of the content screen moves following the movement of the users. In the present embodiment, when the display position of the content screen interferes, a characteristic operation is performed in which the user's position is detected and the viewing content of the user closest to the screen is displayed with priority.

<Example of operation window display (FIGS. 2 to 5)>
Hereinafter, an example of display control operation according to the present embodiment will be described with reference to FIGS. In the present embodiment, when the display position between the windows interferes, the viewing window of the user closest to the display is displayed so as to be the forefront.

FIG. 2 is a diagram illustrating a display example when there is no interference between windows. Two users are viewing the display.
First, regarding the configuration of the display display, 1 is a display display body, and 31 is a display panel. A plurality of remote control position detectors 21 are installed on the upper side of the display 1 in the horizontal direction. Reference numeral 22 denotes a camera, which is installed in the upper part of the room where the display display 1 is installed, and captures images around the display display 1. As a coordinate system representing the position of the user, the horizontal direction of the display surface is defined as the X axis, the vertical direction is defined as the Y axis, and the direction perpendicular to the display surface is defined as the Z axis.

  There are two users around the display 1, 50 is user A, and 51 is user B. On the display panel 31, a content window is displayed at a position closest to the viewing user. The window A (52) viewed by the user A is displayed at a position closest to the user A. A window B (53) viewed by the user B is displayed at a position closest to the user B.

  FIG. 3 shows a display example when the user moves. FIG. 3 shows a state where the user B (51) moves rightward toward the display 1. Following the movement of the user, the display position of the window B (53) is also moved to the right and displayed near the new position of the user B (51).

  FIG. 4 shows a display example when the user crosses behind another user. As a result of the proximity of user A (50) and user B (51), the display positions of window A and window B interfere. User A's body is in the way, preventing user B from viewing. In the present embodiment, the viewing of the user A (50) can be continued by displaying the window A (52) in the foreground.

FIG. 5 shows a display example when a user crosses in front of another user. Also in this case, the display positions of the window A (52) and the window B (53) interfere with each other. User B (51) 's body is in the way, preventing user A from viewing. In this case, the window B (53) is displayed in the foreground so that the viewing of the user B closest to the screen can be continued.
As described above, in the present embodiment, an operation for displaying the content screen viewed by the nearest user so that it is the foremost is performed.

<Configuration of Display Device (FIGS. 1 and 6)>
FIG. 1 is a block diagram illustrating a characteristic configuration example of the display device according to the present embodiment. The configuration of the display device will be described with reference to FIG.
In FIG. 1, reference numeral 1 denotes a main body of a display device, which is a display display having a large display surface and capable of displaying a plurality of video content screens. Reference numeral 2 denotes a user position detection unit that detects the position of the user around the display 1. Further, it is detected at which position the user who has operated the content is.

  A user identification unit 3 assigns an ID to the detected user to identify each user. A window association unit 4 associates the content window with the user. Associating the user who has generated and operated the content window as the owner of the window.

  Reference numeral 5 denotes a priority determination unit that determines the display priority of each window. The distance of the user associated with the window is used as a criterion, and a high priority is assigned to a window with a short distance. Reference numeral 6 denotes a video input unit for inputting video content data such as broadcast waves and video signals from the outside. A display screen creation unit 7 lays out the content window according to the determined priority and creates display screen data.

  In the present embodiment, when the display positions of the windows interfere with each other, the overlapping is performed so that the window with the highest priority comes to the forefront. A display unit 8 displays display screen data on the display panel 32.

FIG. 6 is a block diagram illustrating a detailed configuration example of the display device according to the present embodiment. Elements that are the same as those in FIG.
In the present embodiment, the position of the user viewing the content is specified by matching the position of the remote controller 20 detected by RFID with the position of the person detected by image recognition.
In FIG. 6, reference numeral 20 denotes a remote controller that outputs a command instructed by the user in a data format modulated into infrared light. Further, the remote controller 20 has a built-in RFID, and the ID of the remote controller 20 can be detected on the display side.

  A remote control light receiving unit 33 receives infrared light transmitted from the remote control 20 and converts it into an electrical signal. Reference numeral 21 denotes a remote control position detector, which is composed of a plurality of RFID receivers and detects the position of the remote controller 20. As described with reference to FIG. 2, a plurality are installed on the upper side of the display 1. The position of the RFID receiver having the highest reception sensitivity is detected as the position of the remote controller 20. The remote controller position detection may be performed by arranging a plurality of infrared light receiving units.

  Reference numeral 22 denotes a camera that captures a scene around the display. An image recognizing unit 23 recognizes a captured image and detects the position of a person. Any known algorithm can be used as the image recognition method. For example, as shown in Patent Document 2, it is possible to separate and extract a plurality of persons by creating a silhouette function of a person from the difference from the background image and comparing it with a threshold value of the dimensions of each part of the human body.

  In addition to this, the position of the person can be specified by comparing the foot position of each person with an object on the floor whose position is known. Reference numeral 24 denotes a user list, which records the position of each recognized user. Reference numeral 25 denotes a window list, which records the user ID associated with the window and information for displaying the window. Details of both lists will be described later.

Next, elements related to display of video content will be described.
Reference numeral 26 denotes a video input interface constituting the video input unit 6, which inputs video content from the outside. For example, an analog composite signal, an analog component signal, an interface of a digital video signal standard such as HDMI or DVI, and various video contents are input.

  A broadcast tuner 27 extracts a signal of a selected channel from a broadcast wave input from an antenna and demodulates it. Further, the demodulated stream data is decoded to output video data.

  Reference numeral 28 denotes a high image quality processing unit which performs image processing such as enlargement / reduction, contour correction, and color correction on the input image data. An overlay control unit 29 creates screen data to be displayed on the display panel 32 by superimposing and drawing the content window being viewed. Reference numeral 30 denotes an OSD composition unit that further superimposes various operation information such as a video port number and volume setting on the display screen data.

  A display panel control unit 31 performs control for displaying a screen on the display panel 32. A synchronization signal having a timing according to the characteristics of the display panel 32 is generated, and display screen data is read and output according to the synchronization signal. It also converts signal voltage levels and creates auxiliary signals. Reference numeral 32 denotes a display panel, which is configured by an arbitrary display method such as liquid crystal, plasma, projection, CRT, organic EL, or SED.

  An overall control unit 34 controls the entire apparatus of the display 1 including user position detection and association, and video display control. The overall control unit 34 includes a CPU, peripheral circuits, memory, and the like. Although not shown, the display 1 includes an audio processing unit for reproducing audio data, an amplifier, a speaker, a communication interface, a user interface, a power source, and the like.

<User position detection processing flow (FIGS. 7 and 8)>
A flow of user position detection processing will be described with reference to the flowchart of FIG. In user position detection, a person is extracted by recognizing an image around the display, and the position of the person is calculated and registered in the user list. The processing of each step in the flowchart of FIG. 7 is executed by a program stored in a memory such as a RAM or a ROM (not shown) in the overall control unit 34 of the present embodiment and read by a CPU or the like (not shown). .
In FIG. 7, in S701, the camera 22 is driven to capture an image around the display. In S702, the image recognition unit 23 is driven to extract a person. In step S703, the user position extraction unit 35 is driven to calculate the position of the person. In S704, the calculated position coordinates are registered in the user list 24, and the process returns to S701. By repeating the above process at regular intervals, the position of the user around the display can be tracked.

  FIG. 8 is a diagram illustrating a configuration example of a user list. The position coordinates for each person are recorded in the list. A user ID 81 is assigned a unique identification name for identifying a person. 82 is an X coordinate, and the distance in the horizontal direction from the left end of the display is recorded. Reference numeral 83 denotes a Y coordinate, in which a vertical distance from the upper end of the display is recorded. Reference numeral 84 denotes a Z coordinate, which records the distance in the vertical direction from the display surface. In the example of FIG. 8, it is recorded that the user A is at a place where the X coordinate is 1000, the Y coordinate is 500, and the Z coordinate is 1700.

<Flow of Content Window Association Processing (FIGS. 9 and 10)>
The flow of the association process between the user and the window will be described with reference to the flowchart of FIG. The processing of each step of the flowchart of FIG. 9 is executed by a program stored in a memory such as a RAM or a ROM (not shown) in the overall control unit 34 of the present embodiment and read by a CPU or the like (not shown).
When the remote controller 20 is operated, the position of the remote controller is detected, and the operated user is matched with the user in the user list. The window generated and operated by the remote controller 20 is associated with the user and registered in the window list.

In S901, it is determined whether there has been remote control reception. If affirmative, the process proceeds to S902, and if negative, the execution of S901 is repeated (standby state).
In S902, the remote control light receiving unit 33 is used to receive a command from the remote control 20.
In S903, the position of the remote controller 20 is detected using the remote controller position detector 21.
In S904, a user who is closest to the remote control position is searched from the user list 24.
In step S905, the searched user is added to the window list 25.
In S906, the command instructed by the remote controller 20 is executed for the window, and the process returns to S901.

FIG. 10 is a diagram illustrating a configuration example of the window list 25. In the window list 25, the associated user ID, display content name, display coordinates, and priority are recorded.
In FIG. 10, reference numeral 1001 denotes a window ID, which is assigned an ID number for identifying a window. Reference numeral 1002 denotes a user ID, which records a user ID associated with the window. Reference numeral 1003 denotes a content name in which an input source name and a channel name displayed in the window are recorded.

Reference numeral 1004 denotes an X coordinate, and reference numeral 1005 denotes a Y coordinate. The coordinates of an area where a window is laid out on the display screen are recorded. Reference numeral 1006 denotes a priority order, which records the priority order determined in the video display process described later. In the example of FIG. 10, window 1 is associated with user A and displays the source of VIDEO 3. The display position is 700 for the X coordinate and 400 for the Y coordinate, and the priority is 1.
Through the above processing, the user and the content instructed by the user can be associated with each other.

<Flow of video display processing (FIG. 11)>
The flow of video display processing will be described with reference to the flowchart of FIG. This video display process is executed every time the display displays one frame. After determining the display position and priority of each window, the contents are drawn and displayed in the order of priority.

In step S1101, window IDs are sequentially extracted from the window list 25. The user ID associated with the window is extracted.
In S1102, the user ID is searched from the user list 24, and the position coordinates of the user are extracted.
In S1103, the display position of the window is determined so that the position of the user is the center.
In step S <b> 1104, the window priorities are rearranged in order of increasing distance between the associated users. The distance of the user uses the value in the Z coordinate field 84 of the user list, and the sorted priority is recorded in the priority field 1006 of the window list 25.

In S1105, it is determined whether the priority-determined window is the last window. If the result is affirmative, the process proceeds to S1106. If the result is negative, the process proceeds to S1101.
In S1106, the content screens are drawn in the display memory in order from the window with the lowest priority. Windows with lower priority are drawn on the back.
In S1107, it is determined whether the drawn window is the last window. If the result is affirmative, the process proceeds to S1108. If the result is negative, the process proceeds to S1106.
In step S1108, screen data for one frame in which all content screens are drawn is output to the display panel and displayed on the display screen.

Through the above processing, the content screen can be displayed following the position closest to the user. When the window position interferes, the window viewed by the nearest user can be displayed superimposed on the foreground.
As described above, according to the present embodiment, the content of the user who is viewed closest is displayed preferentially, so that it is possible to minimize the disturbing viewing.

The present invention is not limited to the above-described embodiments, and can be widely applied.
The display layout of the prioritized window may be as follows. As shown in FIG. 12, the low-priority window (53) may be laid out at a position that bypasses the high-priority window (52). As shown in FIG. 13, the low priority window (53) may be reduced. The display of the low priority window may be made thinner.

In a display capable of displaying 3D video, low priority 3D content may be laid out 3D far away. Since the user has a high possibility of gazing at an object closer to the distance, it is possible to reduce the obstacle to viewing by laying out the low-priority three-dimensional content in the distance. Further, the content image may be blended and displayed so that the main subject is arranged far away, instead of laying out the window itself far away.
Further, the display layout methods described above may be combined. For example, it may be laid out so that a low priority window is reduced and then bypassed.

  Although the user position detection in this embodiment is performed by using both the position detection of the remote controller and the image recognition, it may be performed only by the image recognition. It is only necessary to match the shape of the remote control and detect where in the captured image. Further, the position may be detected by using a distance sensor such as a CMOS area sensor or an ultrasonic sensor together.

In the present embodiment, the user association is performed using both the remote control position detection and the image recognition. However, the user association may be performed using only the RFID of the remote control. When there are a plurality of remote controllers with unique IDs and each one has one, the association can be made with the RFID of the remote controller that has been operated.
Furthermore, each process of the present invention can also be realized by executing software (program) acquired via a network or various storage media by a processing device (CPU, processor) such as a personal computer.

(Second Embodiment)
In the present embodiment, an example will be described in which it is detected whether or not a window is being viewed and a window that is not viewed is excluded from the priority control targets. The captured image is recognized to detect the orientation of the user's face, and when the user is facing the window, it is determined that the user is viewing.

<Configuration of Display Device (FIGS. 14 and 15)>
FIG. 14 is a block diagram illustrating a characteristic configuration example of the display device of the present embodiment. Elements that are the same as those in FIG. In FIG. 14, a user viewing detection unit 9 is added. The user viewing detection unit 9 detects whether each user is looking at the window and notifies the user identification unit 3 of the detection result.

  FIG. 15 is a block diagram illustrating a detailed configuration example of the display device according to the present embodiment. In FIG. 15, a user viewing detection unit 9 is added to the configuration of FIG. The user viewing detection unit 9 includes a user face direction extraction unit 36. The user face direction extraction unit 36 detects the direction of the user's face from the recognition result of the image recognition unit 23. Any known algorithm may be used to detect the face direction. For example, as shown in Patent Document 3, the face orientation can be detected by performing matching using an average feature vector calculated for each face orientation and feature data extracted from image data to be recognized.

<User position detection processing flow (FIGS. 16 and 17)>
FIG. 16 is a flowchart showing a flow of user position detection processing. In the flowchart of FIG. 16, S1601 is added to the flowchart of FIG. In step S1601, the result of the image recognition unit 23 is used to determine the direction of the user's face. In subsequent S704, in addition to the previous field, the determination result of the user's face direction is registered in the viewing / non-viewing field.
FIG. 17 is a diagram illustrating a configuration example of a user list in the present embodiment. In FIG. 17, a viewing / non-viewing field 85 is added to the user list of FIG. The viewing presence / absence field 85 records the result of the user viewing detection unit 9. In the example of FIG. 17, the user A is looking at the window and the user B is not looking.

<Flow of video display processing (FIG. 18)>
FIG. 18 is a flowchart showing a flow of video display processing. In FIG. 18, S1801 and S1802 are added to the flowchart of FIG. In S1801, it is determined whether the associated user is looking at the window. If the result is affirmative, the process proceeds to S1104. If the result is negative, the process proceeds to S1802.

  In step S1104, the window priorities are rearranged in the same manner as in the above-described embodiment, and the process advances to step S1105. In S1802, the process is performed when the priority control is not performed, and the priority of the window is set to the lowest, and the process proceeds to S1105. As a result, the windows that are not subject to priority control are laid out on the back side. With the above processing, it is possible to determine whether the user is looking at the window and to remove the window that is not seen from the priority control target and lay it out on the back side.

  As described above, according to this embodiment, the same effect as that of the above-described embodiment can be obtained. Furthermore, since the user who is not viewing the screen can be excluded from the priority control target, it is possible to further reduce the obstruction of viewing of the user who is actually viewing.

The present invention is not limited to the above-described embodiments, and can be widely applied.
In this embodiment, whether or not the user is viewing is determined based on the face direction, but may be determined based on the line of sight. In this case, the user viewing detection unit 9 detects the line-of-sight direction from the image recognition result of the captured image. Any known algorithm may be used to detect the gaze direction. For example, as shown in Patent Document 4, the line-of-sight direction can be detected by performing calibration with the open / closed image of the subject's eyes and comparing the input image with the eye and eyebrow template.

  Further, viewing detection may be performed by using various sensors such as an angle sensor regardless of image recognition. A sensor that detects the movement of the eyeball may be attached to the user's head to detect the line of sight.

  In addition, a window that is not seen may be temporarily deleted from the display screen. It detects that it has not been seen for a certain period of time and records it in the window list. In the video display process, drawing of a window that has not been seen is skipped. When the user views the display screen again, the content screen may be redrawn at the nearest position. Further, the window display may be resumed by the remote control process again.

  In the present embodiment, the process of lowering the priority is performed when the user is not watching, but the user may lower the priority according to the moving speed. Since there is a high possibility of gazing at the scene in front while moving, the degree of influence on viewing is small even if the priority is lowered. The user position extraction unit 35 detects the moving speed of the user from the difference from the previous user position and records it in the user list 24. In the video display process, when the moving speed of the user is equal to or higher than a certain level, the user is excluded from the priority rearrangement target and laid out on the back side. As described above, priority control can be performed in consideration of the moving speed of the user.

(Other embodiments)
The present invention can also be realized by executing the following processing. That is, software (computer program) that implements the functions of the above-described embodiments is supplied to a system or apparatus via a network or various computer-readable storage media. Then, the computer (or CPU, MPU, etc.) of the system or apparatus reads out and executes the program.

DESCRIPTION OF SYMBOLS 1 Display apparatus, 2 User position detection part, 3 User identification part, 4 Window correlation part, 5 Priority determination part, 6 Video input part, 7 Display screen creation part, 8 Display part, 9 User viewing detection part, 20 Remote control, 21 Remote control position detection unit, 22 Camera, 23 Image recognition unit, 24 User list, 25 Window list, 26 Video input interface, 27 Broadcast tuner, 28 High image quality processing unit, 29 Overlay control unit, 30 OSD composition unit, 31 Display panel control unit, 32 Display panel, 33 Remote control light receiving unit, 34 Overall control unit, 35 User position extraction unit, 36 User face direction extraction unit, 50 User A, 51 User B, 52 Window A, 53 Window B

Claims (24)

Means for inputting a plurality of video contents;
Means for identifying a plurality of users;
Detecting means for detecting the position of the user;
Means for associating a user with video content for viewing;
Means for determining priority according to the position of the user;
Creating means for laying out video content according to the priority and creating a display screen;
Means for displaying the display screen;
A display device comprising:   The display device according to claim 1, wherein the creating unit displays the associated content at a position closest to the user.   3. The display device according to claim 1, wherein the priority determining unit performs control so as to give a higher priority to the associated content as the distance between the device and the user is shorter. A determination unit for determining whether the user is moving;
The means for determining the priority gives a higher priority to a user whose movement speed is lower in accordance with a result of determination by the determination means. Display device. A means for detecting whether the user is viewing video content;
5. The priority determination unit according to claim 1, wherein only the user who is viewing the video content is subject to priority control according to a detection result of the detection unit. The display device according to item.   The means for detecting whether or not the video content is viewed includes a photographing means and an image recognition means, and detects whether a user's face is facing a display screen of the video content. The display device described.   6. The means for detecting whether or not the video content is viewed includes a photographing means and an image recognition means, and detects whether the user's line of sight is directed toward the display screen of the video content. The display device described.   The display device according to claim 1, wherein the creation unit superimposes content screens such that video content with low priority is on the back.   The display device according to any one of claims 1 to 7, wherein the creation unit reduces and lays out a video content having a low priority.   The display device according to claim 1, wherein the creating unit lays out the video content having a low priority with a lighter color.   8. The creation unit according to claim 1, wherein the creation unit creates a three-dimensional display screen, and performs layout so that video content having a low priority is located far away. Display device. Inputting a plurality of video contents;
Identifying a plurality of users;
A detecting step for detecting the position of the user;
Associating a user with video content for viewing;
Determining priority according to the user's location;
A creation step of laying out video content according to the priority and creating a display screen;
Displaying the display screen;
A display method characterized by comprising:   The display method according to claim 12, wherein the creating step displays the associated content at a position closest to the user.   The display method according to claim 12 or 13, wherein the step of determining the priority performs control so that the closer the distance between the device and the user is, the higher the priority is given to the associated content. A determination step for determining whether the user is moving;
15. The method according to claim 12, wherein the step of determining the priority gives a higher priority to a user whose movement speed is lower according to a result of the determination in the determination step. Display method. Further comprising detecting whether the user is viewing video content;
The step of determining the priority includes only a user who is viewing video content as a target of priority control according to a detection result of the detection step. The display method according to the item.   The step of detecting whether the video content is being viewed includes a shooting step and an image recognition step, and detecting whether the user's face is facing a display screen of the video content. Display method of description.   The step of detecting whether or not the video content is viewed includes a shooting step and an image recognition step, and detecting whether or not the user's line of sight is directed toward the display screen of the video content. Display method of description.   The display method according to any one of claims 12 to 18, wherein in the creating step, the content screens are overlapped so that video content with low priority is on the back.   The display method according to any one of claims 12 to 18, wherein in the creation step, video content having a low priority is reduced and laid out.   The display method according to any one of claims 12 to 18, wherein, in the creating step, the video content having a low priority is laid out with a light color.   19. The creation step is for creating a three-dimensional display screen, and laying out such that video content with a low priority is located far away. How to display. Inputting a plurality of video contents;
Identifying a plurality of users;
A detecting step for detecting the position of the user;
Associating a user with video content for viewing;
Determining priority according to the user's location;
A creation step of laying out video content according to the priority and creating a display screen;
A program causing a computer to execute the step of displaying the display screen.   A computer-readable storage medium storing the program according to claim 23.

Images