JP2017173497A - Display system, display device, terminal device, and display system control method - Google Patents

Abstract

An object of the present invention is to operate a terminal device and a display device in conjunction with each other regardless of the connection mode. A terminal device (100) having an output interface (106) and a projector (200) having an input interface (251) are provided, the output interface (106) and the input interface (251) are connected, and the terminal device (100) has a preset The audio data of the audio pattern is output to the output interface unit 106, and when the audio pattern of the audio data input from the input interface unit 251 corresponds to a preset audio pattern, the projector 200 performs an operation corresponding to the audio pattern. A display system 1 characterized by executing [Selection drawing] Fig. 3

Description

  The present invention relates to a display system, a display device, a terminal device, and a display system control method.

2. Description of the Related Art Conventionally, a display system that connects a terminal device and a display device and displays information such as an image transmitted from the terminal device on the display device is known. In this system, a technique for detecting an operation on an image displayed by a display device and drawing an image corresponding to the detected operation is known (for example, see Patent Document 1).
The bidirectional display method of Patent Document 1 displays an image based on a display signal input from an external device on a screen, detects an operation on the screen, and combines an object corresponding to the operation with an image based on the display signal. indicate. In addition, the bidirectional display method detects an operation on an external device and an interlock operation that is an operation on an object, generates a device operation signal corresponding to the detected interlock operation, outputs it to the external device, and responds to the device operation signal. Edit the object to accommodate changes in the image.

Japanese Patent Laying-Open No. 2015-197586

However, when the terminal device and the display device are connected in such a manner that control data cannot be transmitted and received, it may be difficult to link the operations of the terminal device and the display device.
The present invention has been made in view of the above circumstances, and an object thereof is to operate a terminal device and a display device in conjunction with each other regardless of a connection mode.

In order to achieve the above object, a display system of the present invention is a display system including a terminal device and a display device, the terminal device including an output unit that outputs image data and audio data, and a predetermined audio pattern. A first control unit that outputs the audio data from the output unit, and the display device includes an input unit to which the image data and the audio data output from the output unit are input, and the predetermined unit A storage unit that stores a correspondence relationship between the voice pattern and a predetermined operation; and a second unit that executes the predetermined operation when the voice data input to the input unit corresponds to the predetermined voice pattern. A display system.
According to the present invention, the terminal device transmits the audio data of the predetermined audio pattern to the display device, so that the display device can execute the predetermined operation set corresponding to the predetermined audio pattern. Therefore, the terminal device and the display device can be operated in conjunction regardless of the connection mode.

In the display system according to the present invention, the output unit includes an output interface that outputs the image data and the audio data, and the input unit includes an input interface that inputs the image data and the audio data. It is characterized by that.
According to the present invention, image data and audio data can be transmitted from a terminal device to a display device through a single interface.

In the above notation system, the output unit includes an output interface including an interface for outputting the image data and an interface for outputting the audio data, and the input unit inputs the image data. And an input interface including an interface for inputting the audio data.
According to the present invention, image data and audio data can be transmitted from a terminal device to a display device through separate interfaces.

In the display system according to the present invention, when the image data output from the output interface changes in advance, the first control unit converts the audio data of the predetermined audio pattern into the audio data. Output from the output interface.
According to the present invention, the terminal device can detect the change of the image data output from the output interface and output the audio data of the predetermined audio pattern. Accordingly, it is possible to cause the display device to perform a predetermined operation in response to a change in image data.

In the display system according to the present invention, the display device includes a display unit that displays a first image based on the image data input to the input interface, a detection unit that detects an operation of a pointer, A drawing unit that draws a second image corresponding to the operation detected by the detection unit and displays the second image on the display unit, wherein the drawing unit receives the audio data input to the input interface The display of the second image on the display unit is erased when the sound pattern is true.
According to the present invention, the second image displayed on the display unit can be erased by the sound data of the predetermined sound pattern transmitted from the terminal device to the display device.

In the display system according to the present invention, the display device includes a display unit that displays a first image based on the image data input to the input interface, a detection unit that detects an operation of a pointer, A drawing unit that draws a second image corresponding to the operation detected by the detection unit and displays the second image on the display unit; and a storage unit that stores image data. The drawing unit is connected to the input interface. Storing the at least one image data of the first image and the second image displayed on the display unit in the storage unit when the input audio data corresponds to the predetermined audio pattern; Features.
According to the present invention, at least one of the first image and the second image can be stored in the storage unit based on audio data of a predetermined audio pattern transmitted from the terminal device to the display device.

In the display system according to the present invention, the display device includes a display unit that displays a first image based on the image data input to the input interface, a detection unit that detects an operation of a pointer, A drawing unit that draws a second image corresponding to the operation detected by the detection unit and displays the second image on the display unit, wherein the drawing unit receives the audio data input to the input interface When the sound pattern corresponds to the above, the second image is shifted to a state where it is not displayed on the display unit.
According to the present invention, the second image corresponding to the operation of the indicator can be prevented from being displayed on the display unit by the sound data of the predetermined sound pattern transmitted from the terminal device to the display device.

In the display system according to the aspect of the invention, the first control unit may output the audio data of the predetermined audio pattern when a preset time has elapsed since the output of the image data was started. Is output from the output interface.
According to the present invention, audio data having a predetermined audio pattern is output when a set time has elapsed since image data was output. Therefore, when the set time has elapsed since the image based on the image data is displayed on the display unit, the second image corresponding to the operation of the indicator can be prevented from being displayed on the display unit.

In the display system according to the present invention, the display device includes a display unit that displays a first image based on the image data input to the input interface, a detection unit that detects an operation of a pointer, A drawing unit that draws a second image corresponding to the operation detected by the detection unit and displays the second image on the display unit, wherein the drawing unit receives the audio data input to the input interface When the sound pattern falls under the above pattern, the second image is displayed on the display unit for a preset time, and after the time has elapsed, the second image is not displayed on the display unit. It is characterized by that.
According to the present invention, the second image corresponding to the operation of the indicator is displayed on the display unit for a preset time by the sound data of the predetermined sound pattern transmitted from the terminal device to the display device. When the preset time has elapsed, the second image corresponding to the operation of the indicator can be prevented from being displayed.

The display device of the present invention includes an input interface for inputting image data and audio data, or an input interface including an interface for inputting image data and an interface for inputting audio data, and predetermined audio data input to the input interface. And the operation to be executed when the audio data of the predetermined audio pattern is input to the input interface, and the audio data input to the input interface is changed to the predetermined audio pattern. If applicable, the operation set in correspondence with the predetermined voice pattern is executed.
According to the present invention, it is possible to cause the display device to execute an operation set corresponding to audio data of a predetermined audio pattern.

The terminal device according to the present invention includes an output interface that outputs image data and audio data, or an output interface that includes an interface that outputs image data and an interface that outputs audio data, and changes in the image data output from the output interface. And a first control unit that detects and outputs the audio data of a predetermined audio pattern from the output interface.
According to the present invention, it is possible to detect a change in image data output to the output interface and output audio data of a predetermined audio pattern to the output interface.

The display system control method of the present invention is a display system control method including a terminal device and a display device connected to the terminal device, and an output unit that outputs image data and audio data by the terminal device. The audio data of a predetermined audio pattern is output, and the audio data input to the input unit connected to the output unit of the terminal device by the display device corresponds to the predetermined audio pattern In addition, a predetermined operation set corresponding to the predetermined voice pattern is executed.
According to the present invention, when the terminal device transmits a predetermined sound pattern to the display device, it is possible to cause the display device to perform an operation corresponding to the sound pattern. Therefore, the terminal device and the display device can be operated in conjunction regardless of the connection mode.

The figure which shows a system structure. The block diagram of a terminal device. The block diagram of a projector. The flowchart which shows operation | movement of a terminal device. The flowchart which shows operation | movement of a projector.

Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.
FIG. 1 is a diagram showing a system configuration of a display system 1 according to the present embodiment.
The display system 1 includes a terminal device 100 and a projector 200. The terminal device 100 and the projector 200 are connected by a cable 30 that can transmit and receive image data. The projector 200 projects an image based on the image data supplied from the terminal device 100 onto a projection target. The projector 200 corresponds to the “display device” of the present invention.

  The terminal device 100 is a personal computer such as a notebook PC (Personal Computer), but may be a desktop PC, a tablet terminal, a smartphone, a PDA (Personal Digital Assistant), or an electronic book terminal. The terminal device 100 may be a video output device such as a video playback device, a DVD (Digital Versatile Disk) player, a Blu-ray disc player, a TV tuner device, a CATV (Cable television) set-top box, or a video game machine. It is.

  Examples of the image data that the terminal device 100 supplies to the projector 200 include, for example, moving images such as videos, still image data such as photographs, document files created using word processing software or spreadsheet software, slide images created using presentation software, and the like. Data is included.

The projector 200 projects an image based on the image data supplied from the terminal device 100 onto a projection target. This embodiment illustrates the case where the projection target is the screen SC, but the projection target may be any one having a flat surface such as the inner wall of a building. The projection target may be an outer wall of a building or an object that is not uniformly flat, such as an object.
The projector 200 is installed above the screen SC as a projection target, and projects an image toward the screen SC disposed below.

In addition to the image based on the image data supplied from the terminal device 100, the projector 200 projects a menu screen 50 on which the user can specify various operations, and detects the operation of the menu screen 50 by the user. An indicator 70 is used for operations on the menu screen 50. In this embodiment, the case where the indicator 70 is an electronic pen will be described. However, the indicator 70 may be a user's finger.
A method in which the projector 200 detects an operation of the indicator 70 on the screen SC will be described later.

  FIG. 2 is a configuration diagram of the terminal device 100. The terminal device 100 includes a storage unit 101, a control unit 102, a drive device 103, a display unit 104, an operation unit 105, and an output interface unit 106. The output interface unit 106 corresponds to the “output unit” of the present invention.

The storage unit 101 is an auxiliary storage device such as a hard disk device, for example. The storage unit 101 may be replaced with a flash memory capable of storing a large amount of information or an optical disc such as a CD (Compact Disc), a DVD (Digital Versatile Disc), or a BD (Blu-ray (registered trademark) Disc). . The storage unit 101 stores application programs executed by the control unit 102 and various data.
The storage unit 101 stores audio data 1011 having a preset audio pattern. The sound pattern of the sound data 1011 is composed of, for example, one sound or a plurality of continuous sounds, and specifies the sound pitch (frequency), volume, and / or changes thereof, the length of each sound, and the like. It is a pattern. The voice pattern may be configured with a value having a width so as to include a plurality of voice patterns as well as a voice pattern having a specific change.
The audio data 1011 includes, for example, audio data of an audio pattern (hereinafter referred to as a first audio pattern) that erases drawing data drawn on the screen SC using the indicator 70, and drawing data drawn on the screen SC. And audio data of an audio pattern (hereinafter referred to as a second audio pattern) that causes the projector 200 to store the image data. Details of the drawing data will be described later. The image represented by the drawing data corresponds to the “second image” of the present invention.
In addition, the storage unit 101 stores a correspondence table 1012. In the correspondence table 1012, an operation received by the operation unit 105 and an audio pattern of audio data output to the projector 200 when the operation is received are registered in association with each other.
Further, when the projector 200 executes a plurality of operations by one operation received by the terminal device 100, a plurality of sound patterns may be registered in the correspondence table 1012 in association with the one operation.

  The control unit 102 includes a CPU (Central Processing Unit), a ROM (Read Only Memory), a RAM (Random Access Memory), and the like. The CPU of the control unit 102 reads the firmware stored in the ROM or the application program stored in the storage unit 101 into the RAM and executes it.

The control unit 102 executes the application program and causes the display unit 104 to display an image corresponding to the executed application program.
For example, the control unit 102 executes word processing software, spreadsheet software, and presentation software as application programs, and causes the display unit 104 to display a document, a table file, and a slide image. The image displayed on the display unit 104 by the control unit 102 includes image data and files read by the drive device 103 from an optical disk such as a CD, DVD, and BD, and image data and files stored in the storage unit 101 in advance. included.
Further, when a mode for transmitting image data corresponding to the image displayed on the display unit 104 to the projector 200 is set, the control unit 102 outputs image data corresponding to the image displayed on the display unit 104 to the output interface unit. It outputs to 106.

Further, the control unit 102 detects an operation of the operation unit 105, detects a change in the display mode of the display unit 104, and transmits audio data of a preset audio pattern to the projector 200 through the output interface unit 106.
For example, when executing word processing software, spreadsheet software, or presentation software as an application program, the control unit 102 monitors the input of the operation unit 105 and switches the page or sheet of the file displayed on the display unit 104. Detect operations. When the control unit 102 detects a switching operation of the page or sheet of the file displayed on the display unit 104, the control unit 102 refers to the correspondence table 1012 and acquires audio data of an audio pattern corresponding to the detected operation. For example, the control unit 102 acquires audio data of an audio pattern that deletes the drawing data drawn on the screen SC using the indicator 70. The control unit 102 outputs the acquired audio data to the output interface unit 106.
In addition, when the control unit 102 detects image data read from the optical disk by the drive device 103 or the end of the image data stored in advance in the storage unit 101, the output unit 106 outputs audio data of a preset audio pattern. May be output.
Further, the control unit 102 may detect a change in the display mode of the display unit 104 by detecting a change in the image displayed on the display unit 104. For example, when the image displayed on the display unit 104 is a slide image that is switched every predetermined time, the control unit 102 detects a change in the image, and outputs the audio data of a preset audio pattern to the output interface unit 106. May be output.

  The drive device 103 reads information from an optical disk such as a CD, DVD, or BD that is an external storage medium. The drive device 103 records information on the optical disc under the control of the control unit 102.

  The display unit 104 is a display device that includes a display surface that displays an image. The display unit 104 has a display surface such as a liquid crystal display, an organic EL (Electro Luminescence) display, or a CRT (Cathode Ray Tube), and displays various types of information on the display surface in accordance with instructions from the control unit 102.

  The operation unit 105 is an input device such as a keyboard, a mouse, or a touch panel that receives an operation from the user. The operation unit 105 generates an operation signal corresponding to the received operation and outputs the operation signal to the control unit 102.

The output interface unit 106 includes a connector for connecting the cable 30 and an interface circuit, converts image data and audio data into packet data, and outputs the packet data to the cable 30.
As an interface with which the terminal device 100 can transmit image data and audio data, for example, HDMI (registered trademark) (High-Definition Multimedia Interface) and DisplayPort are cited. Moreover, when the terminal device 100 is a tablet terminal or a smartphone, a configuration including an MHL (Mobile High-definition Link) as an interface may be used.

  In addition, the output interface unit 106 may be configured to include an interface that outputs image data and an interface that outputs audio data. For example, the terminal device 100 can be configured to include a VGA terminal such as a D-Sub connector or a DVI (Digital Visual Interface) terminal as a connector for transmitting image data. Further, the terminal device 100 can be configured to include a composite terminal as a connector for transmitting audio data.

  Although the present embodiment shows a configuration in which the terminal device 100 and the projector 200 are connected by the cable 30, a configuration in which the terminal device 100 and the projector 200 are connected wirelessly may be used. As the interface in this case, for example, Miracast (registered trademark), WirelessHD (registered trademark), Widi (Wireless Display), AirPlay (registered trademark), WHDI (Wireless Home Digital Interface), or the like can be used.

FIG. 3 is a configuration diagram of the projector 200.
The projector 200 includes an input interface unit 251. The input interface unit 251 corresponds to the “input unit” of the present invention.
The input interface unit 251 includes a connector for connecting the cable 30 and an interface circuit, and inputs image data and audio data. Examples of the interface capable of receiving image data and audio data included in the projector 200 include HDMI (registered trademark) (High-Definition Multimedia Interface) and DisplayPort. Moreover, when the terminal device 100 is a tablet terminal or a smartphone, a configuration including an MHL (Mobile High-definition Link) as an interface may be used.

  The input interface unit 251 receives packet data transmitted via the cable 30 and extracts image data and audio data from the received packet data. The input interface unit 251 separates the extracted image data and audio data into image data and audio data. The input interface unit 251 outputs the separated image data to the image processing unit 255 and outputs audio data to the audio processing unit 252 and the control unit 270.

Further, the input interface unit 251 may be configured to include an interface for inputting image data and an interface for inputting audio data separately. For example, the projector 200 may be configured to include a VGA terminal such as a D-Sub connector or a DVI (Digital Visual Interface) terminal as a connector for receiving image data. Further, the projector 200 can be configured to include a composite terminal as a connector for receiving audio data.
Further, the projector 200 may be configured to be connected to the terminal device 100 wirelessly.

The projector 200 includes a display unit 210 that forms an optical image and projects the image onto the screen SC.
The display unit 210 includes a light source unit 211, a light modulation device 212, and a projection optical system 213. The light source unit 211 includes a light source including a xenon lamp, an ultra-high pressure mercury lamp, an LED (Light Emitting Diode), a laser light source, or the like. Further, the light source unit 211 may include a reflector and an auxiliary reflector that guide light emitted from the light source to the light modulation device 212. Further, the light source unit 211 includes a lens group for increasing the optical characteristics of the projection light, a polarizing plate, a light control element that reduces the amount of light emitted from the light source on the path to the light modulation device 212, and the like (both shown) Abbreviation) may be provided.

  The light source unit 211 is driven by the light source driving unit 221. The light source driving unit 221 is connected to the internal bus 280. The light source driving unit 221 turns on and off the light source of the light source unit 211 under the control of the control unit 270.

  The light modulation device 212 includes, for example, three liquid crystal panels corresponding to the three primary colors RGB. The light emitted from the light source unit 211 is separated into three color lights of RGB and each enters the corresponding liquid crystal panel. The three liquid crystal panels are transmissive liquid crystal panels, and modulate the transmitted light to generate image light. The image light modulated by passing through each liquid crystal panel is combined by a combining optical system such as a cross dichroic prism and emitted to the projection optical system 213.

The light modulation device 212 is driven by the light modulation device driving unit 222. The light modulator driving unit 222 is connected to the internal bus 280.
The light modulator driving unit 222 generates R, G, and B image signals based on synthesized image data input from an image synthesizing unit 257 described later. The light modulator driving unit 222 drives the corresponding liquid crystal panel based on the generated R image signal, G image signal, and B image signal, and draws an image on each liquid crystal panel.

  The projection optical system 213 includes a lens group that projects the image light modulated by the light modulation device 212 toward the screen SC and forms an image on the screen SC. Further, the projection optical system 213 may include a zoom mechanism for enlarging or reducing a projection image projected on the screen SC and a focus adjustment mechanism for adjusting the focus.

The projector 200 includes an operation panel 231 and an input processing unit 233. The input processing unit 233 is connected to the internal bus 280.
The operation panel 231 that functions as a user interface is provided with various operation keys and a display screen constituted by a liquid crystal panel.
When the operation key on the operation panel 231 is operated, the input processing unit 233 outputs an operation signal corresponding to the operated key to the control unit 270. Further, the operation panel 231 is integrally formed with a touch sensor that detects contact with the operation panel 231. The input processing unit 233 detects the position of the operation panel 231 touched by the user's finger or the like as an input position, and outputs an operation signal corresponding to the detected input position to the control unit 270. Further, the input processing unit 233 displays various screens on the operation panel 231 based on the control signal input from the control unit 270.

  Further, the projector 200 has a remote controller 5 used by the user. The remote controller 5 includes various buttons, and transmits an infrared signal corresponding to the operation of these buttons. The main body of the projector 200 is provided with a remote control light receiving unit 232 that receives an infrared signal emitted from the remote control 5. The remote control light receiving unit 232 receives an infrared signal transmitted from the remote control 5. The input processing unit 233 decodes the infrared signal received by the remote control light receiving unit 232, generates an operation signal indicating the operation content in the remote control 5, and outputs the operation signal to the control unit 270.

The projector 200 includes an indicator detection unit 240 that detects an operation by the pen-type indicator 70 and an operation position indicated by the operation. The indicator detection unit 240 includes an imaging unit 241 and a position detection unit 242. The indicator detection unit 240 is connected to the internal bus 280.
The imaging unit 241 includes a digital camera and images the screen SC to generate a captured image. The imaging range of the imaging unit 241, that is, the angle of view is preferably a range including the entire screen SC or the screen SC.
The camera mounted on the imaging unit 241 is not limited to a camera that performs imaging with visible light, and may be a camera that captures infrared light. For example, the indicator 70 may be provided with a light emitting unit that emits infrared light, and the light emission of the light emitting unit is captured by the imaging unit 241 to detect the operation position of the indicator 70.

  The position detection unit 242 detects the operation position of the indicator 70. The position detection unit 242 analyzes the captured image of the imaging unit 241 and detects the image of the indicator 70 from the captured image. When detecting the image of the indicator 70 from the captured image, the position detector 242 detects the position of the tip of the indicator 70 and specifies the coordinates of the detected operation position. The coordinates specified by the position detection unit 242 are the coordinates of the operation position of the indicator 70 in the captured image. The position detection unit 242 outputs the coordinates of the detected operation position to the control unit 270.

  The projector 200 includes an audio processing unit 252 and an audio output unit 253 that perform predetermined processing such as decoding processing on audio data. The audio processing unit 252 performs various types of audio processing on the audio data input from the input interface unit 251, and outputs the processed audio data to the audio output unit 253. The sound output unit 253 outputs sound based on input sound data.

  The projector 200 includes an image processing system. This image processing system is configured around a control unit 270 that controls the entire projector 200 in an integrated manner, and further includes a storage unit 260, an image processing unit 255, an image composition unit 257, and a frame memory 256. Each unit constituting the image processing system is connected to each other via an internal bus 280 so that data communication is possible.

The image processing unit 255 processes the image data input from the input interface unit 251 under the control of the control unit 270, and outputs the processed image data to the image composition unit 257 as display image data (first image). The processing executed by the image processing unit 255 includes, for example, resolution conversion (scaling) processing or resizing processing, shape correction processing such as distortion correction, digital zoom processing, color tone correction processing, and luminance correction processing. The image processing unit 255 executes processing designated by the control unit 270, and performs processing using parameters input from the control unit 270 as necessary. Of course, the image processing unit 255 can also execute a combination of a plurality of processes.
Note that image data supplied from the outside includes image data supplied from the terminal device 100 and stored in the storage unit 260. In other words, the image data stored in the storage unit 260 when the power of the projector 200 is turned on is also included.

  The image composition unit 257 expands the display image data input from the image processing unit 255 in the frame memory 256. Further, the image composition unit 257 acquires the image data (drawing data) generated by the control unit 270 and develops it in the frame memory 256. Here, the image synthesis unit 257 synthesizes the display image data input from the image processing unit 255 and the drawing data generated by the control unit 270 in the frame memory 256. The image synthesizing unit 257 reads the synthesized image data (hereinafter referred to as synthesized image data) from the frame memory 256 and outputs it to the light modulation device driving unit 222.

The storage unit 260 is an auxiliary storage device such as a hard disk device, for example. The storage unit 260 may be replaced with a flash memory capable of storing a large amount of information or an optical disc such as a CD (Compact Disc), a DVD (Digital Versatile Disc), or a BD (Blu-ray (registered trademark) Disc). . The storage unit 260 stores application programs executed by the control unit 270 and various data.
The storage unit 260 also stores audio data 2601 of a preset audio pattern. The sound pattern of the sound data 2601 is composed of, for example, one sound or a plurality of continuous sounds, and specifies the sound pitch (frequency), volume, and / or changes thereof, the length of each sound, and the like. It is a pattern. The voice pattern may be configured with a value having a width so as to include a plurality of voice patterns as well as a voice pattern having a specific change.
In the audio data 2601, for example, audio data of an audio pattern that deletes drawing data drawn on the screen SC using the indicator 70, and audio that causes the projector 200 to store drawing data and image data drawn on the screen SC. The voice data of the pattern is stored. In addition, the storage unit 260 stores a correspondence table 2602 in which identification information given to each audio data is associated with an operation to be executed by the projector 200. The correspondence table 2602 is a table in which a voice pattern of voice data is associated with an operation to be executed when voice data corresponding to the voice pattern is input. In the correspondence table 2602, one operation may be set in association with a plurality of sound patterns, or one operation may be associated with one sound pattern.

  The control unit 270 includes a CPU, a ROM, and a RAM (all not shown) as hardware. The ROM is a non-volatile storage device such as a flash ROM, and stores a control program and data. The RAM constitutes a work area of the CPU. The CPU expands the control program read from the ROM to the RAM, and executes the expanded control program to control each unit of the projector 200.

  The control unit 270 includes a projection control unit 271, an operation acquisition unit 272, a drawing processing unit 273, a data storage unit 274, and an audio pattern analysis unit 275 as functional blocks. These functional blocks are realized by executing a control program expanded in the RAM.

The projection control unit 271 controls each unit of the projector 200 to control the projection of the image onto the screen SC.
Specifically, the projection control unit 271 controls the image processing unit 255 to perform image processing on image data supplied from the terminal device 100. At this time, the projection control unit 271 may read out parameters necessary for processing by the image processing unit 255 from the storage unit 260 and output them to the image processing unit 255. In addition, the projection control unit 271 controls the image composition unit 257 to cause the image composition unit 257 to execute processing for combining display image data and drawing data. In addition, the projection control unit 271 controls the light modulation device driving unit 222 to draw an image on the liquid crystal panel of the light modulation device 212. Further, the projection control unit 271 controls the light source driving unit 221 to turn on the light source of the light source unit 211 and adjust the luminance of the light source. As a result, image light that is emitted from the light source and modulated by the light modulation device 212 is projected onto the screen SC by the projection optical system 213.

  The operation acquisition unit 272 performs coordinate conversion on the coordinates of the operation position input from the indicator detection unit 240. The indicator detection unit 240 specifies the operation position of the indicator 70 by coordinates in the captured image, and outputs the coordinates of the specified operation position to the control unit 270. The operation acquisition unit 272 converts the coordinates of the operation position input from the indicator detection unit 240 into coordinates in the frame memory 256 that are coordinates for the drawing processing unit 273 to draw in the frame memory 256.

  The drawing processing unit 273 performs processing for drawing characters, symbols, and graphics in accordance with the operation position detected by the indicator detection unit 240 or the locus of the operation position. The drawing processing unit 273 generates drawing data that is data of characters, symbols, and figures such as kanji, hiragana, katakana, alphabets, and numbers in accordance with the size of the display image data developed in the frame memory 256. The drawing data generated by the drawing processing unit 273 is output to the image composition unit 257. The image compositing unit 257 superimposes the image based on the drawing data generated by the drawing processing unit 273 on the frame memory 256 in which the image based on the display image data processed by the image processing unit 255 is developed, and outputs the composite image data. Generate.

When the audio pattern analysis unit 275 instructs the data storage unit 274 to store data, the data storage unit 274 generates a file when the file for storing the image data is not generated in the storage unit 260.
The timing at which the data storage unit 274 creates a file is, for example, when the creation date of the most recently created file is different from the current date, or when a preset time has elapsed from the creation time of the most recently created file. And the like. Further, when the user gives an instruction to create a file, when another terminal device 100 is connected to the projector 200, when transmission of image data from the terminal device 100 to the projector 200 is completed, from the terminal device 100, For example, the type of image data transmitted to the projector 200 is changed.
A file name for identifying the file is added to the file. As the file name, a date when the file is generated, a serial number (number given in the order of file generation), or the like is used. Alternatively, a file name input by the user operating the operation panel 231 or the remote controller 5 may be used. it can.
Next, the data storage unit 274 stores, in the file, the drawing data generated by the drawing processing unit 273, the display image data developed in the frame memory 256, and the composite image data obtained by combining the drawing data and the display image data. To do.
When the composite image data is generated by the image composition unit 257 and developed in the frame memory 256, the data storage unit 274 acquires the composite image data from the image composition unit 257 and stores it in a file. The data storage unit 274 may acquire the display image data from the image processing unit 255 and the drawing data from the drawing processing unit 273, and may store the display image data and the drawing data separately in a file. Further, when the drawing data is not expanded in the frame memory 256 and only the display image data is expanded, the data storage unit 274 acquires only the display image data from the image composition unit 257 and stores it in the file.
Furthermore, the data storage unit 274 associates the storage times of these data when storing drawing data, display image data, and composite image data in a file. The data storage unit 274 stores drawing data, display image data, and composite image data set at the same storage time in a file in association with each other.

  The voice pattern analysis unit 275 analyzes the voice pattern of the voice data input from the voice processing unit 252. For this analysis, an existing speech recognition technique can be used. The voice pattern analysis unit 275 analyzes the input voice data and determines whether or not a preset voice pattern (for example, a first voice pattern or a second voice pattern) is included. The voice pattern analysis unit 275 refers to the voice data 2601 stored in the storage unit 260 and determines whether or not a preset voice pattern is included. In addition, when the voice pattern analysis unit 275 detects a preset voice pattern, the voice pattern analysis unit 275 refers to the correspondence table 2602 of the storage unit 260 and acquires a process associated with the detected voice pattern. The voice pattern analysis unit 275 instructs the drawing processing unit 273 and the data storage unit 274 to execute the acquired processing. The process associated with the sound pattern includes an erasing process for erasing the drawing data drawn on the screen SC by the indicator 70, a saving process for saving the display image data and the drawing data in association with each other in the storage unit 260, and the like. It is.

FIG. 4 is a flowchart showing the operation of the control unit 102 of the terminal device 100. In this flowchart, an operation of detecting an operation of the operation unit 105 for switching sheets displayed on the display unit 104 and transmitting a first sound pattern as corresponding sound data to the projector 200 will be described.
First, the control unit 102 determines whether an application program is executed and an image based on image data (data file) generated by the application program is displayed on the display unit 104 (step S1). In the case of negative determination (step 1 / NO), the control unit 102 stands by until an image based on the image data is displayed on the display unit 104. If the determination is affirmative (step S1 / YES), the control unit 102 determines whether output of image data generated by the application program to an external device has been started (step S2).

When the determination in step S2 is negative (step S2 / NO), the control unit 102 determines whether or not the display of the image based on the image data displayed on the display unit 104 has ended (step S7). If the determination is affirmative (step S7 / YES), the control unit 102 ends this processing flow. In the case of a negative determination (step S7 / NO), the control unit 102 returns to the determination in step S2 and determines whether output of image data to an external device is started (step S2).
When the determination in step S2 is affirmative (step S2 / YES), the control unit 102 determines whether or not an operation for switching sheets by the operation unit 105 has been received (step S3). In the case of a negative determination (step S3 / NO), the control unit 102 determines whether or not the output of the image data to the external device has ended (step S6). In the case of a positive determination (step S6 / YES), the control unit 102 performs the above-described determination of step S7. In the case of a negative determination (step S6 / NO), the control unit 102 returns to the determination in step S3 and determines whether or not an operation for switching sheets by the operation unit 105 has been received.

If the determination in step S3 is affirmative (step S3 / YES), the control unit 102 refers to the correspondence table 1012 stored in the storage unit 101, and sets a preset voice pattern corresponding to the accepted operation. The included voice data is acquired (step S4). For example, when receiving an operation of switching sheets, the control unit 102 acquires, from the storage unit 101, audio data of a first audio pattern that deletes drawing data drawn on the screen SC. The control unit 102 outputs the acquired audio data of the first audio pattern to the output interface unit 106. The output interface unit 106 converts the input audio data of the first audio pattern into packet data, and transmits the converted packet data to the projector 200 via the cable 30 (step S5).
After the process of step S5, the control unit 102 determines whether or not the display of the image based on the image data is finished (step S6). In the case of a negative determination (step S6 / NO), the control unit 102 returns to the determination in step S3 and determines whether or not an operation for switching sheets by the operation unit 105 has been received. In the case of a positive determination (step S6 / YES), the control unit 102 performs the above-described determination of step S7. If the determination in step S7 is negative (step S7 / NO), the control unit 102 returns to the determination in step S2. If the determination is affirmative (step S7 / YES), the control unit 102 ends this processing flow. .

In the above-described flowchart, an example has been described in which the control unit 102 determines whether the operation unit 105 has received an operation for switching sheets, and transmits audio data of a predetermined audio pattern to the projector 200. As an operation other than this, the control unit 102 outputs audio data of a preset audio pattern to the projector 200 when a preset setting time has elapsed after starting to supply image data to the projector 200. You may make it do. By operating the control unit 102 in this way, the projector 200 can execute a predetermined operation after a predetermined time has elapsed since the image was displayed on the screen SC. For example, the terminal device 100 starts supplying image data to the projector 200, and an image based on the image data is displayed on the screen SC. The display device 70 displays the image on the screen SC only for a predetermined time from the display of the image. Drawing of the image is allowed. When a predetermined time elapses, the control unit 102 of the terminal device 100 transmits audio data of an audio pattern that prohibits drawing of an image on the screen SC by the indicator 70 to the projector 200. When the control unit 270 of the projector 200 receives audio data of an audio pattern that prohibits drawing of an image on the screen SC by the indicator 70 from the terminal device 100, for example, the indicator detection unit 240 stops the generation of the captured image. The detection of the operation by the indicator 70 is stopped.
Thereby, drawing of the image using the indicator 70 is permitted only for a predetermined time after the image based on the image data is displayed on the screen SC, and after the predetermined time has passed, the image using the indicator 70 is displayed. It is possible to prevent drawing.

FIG. 5 is a flowchart showing the operation of the projector 200.
The control unit 270 of the projector 200 determines whether image data is supplied from the terminal device 100 and an image based on the supplied image data is projected on the screen SC by the display unit 110 (step S11). In the case of a negative determination (step S11 / NO), the control unit 270 ends this processing flow.
If the determination is affirmative (step S11 / YES), the control unit 270 determines whether audio data is input from the input interface unit 251 (step S12). In a negative determination (step S12 / NO), the control unit 270 returns to the determination in step S12.

  If the determination in step S12 is affirmative (step S12 / YES), the control unit 270 causes the audio processing unit 252 to decode the input audio data (step S13), and inputs the processed audio data. .

  The voice pattern analysis unit 275 of the control unit 270 analyzes the input voice data (step S14), and determines whether the voice data includes a preset voice pattern (step S15). If the determination is negative (step S15 / NO), the voice pattern analysis unit 275 instructs the voice processing unit 252 to output the voice data to the voice output unit 253 (step S16). Thereafter, the sound pattern analysis unit 275 returns to the determination in step S12 and determines whether sound data is input from the input interface unit 251 or not.

If the determination in step S15 is affirmative (step S15 / YES), the voice pattern analysis unit 275 controls the voice processing unit 252 to stop the voice output from the voice output unit 253 (step S17). Then, the drawing processing unit 273 is instructed to erase the drawing data.
The drawing processing unit 273 causes the image composition unit 257 to delete the drawing data developed in the frame memory 256 and delete the drawing data drawn on the screen SC (step S18).

  When the drawing data is erased, the voice pattern analysis unit 275 causes the voice processing unit 252 to output the voice data to the voice output unit 253 and restarts the voice output (step S19).

  In the above-described flowchart, the case where the drawing data drawn on the screen SC is erased has been described. However, the processing executed by the projector 200 by inputting the audio data of the preset audio pattern is limited to the deletion of the drawing data. Is not to be done.

  For example, if the voice pattern analysis unit 275 determines that the input voice data is voice data of a preset voice pattern (voice data of the second voice pattern), the data storage unit 274 stores the drawing data And instructing to store at least one of the display image data. The data storage unit 274 causes the storage unit 260 to store at least one of drawing data drawn by the drawing processing unit 273, display image data processed by the image processing unit 255, and composite image data developed in the frame memory 256.

If the voice pattern analysis unit 275 determines that the input voice data is voice data of a preset voice pattern, the voice pattern analysis unit 275 causes the drawing processing unit 273 to stop generating drawing data. The drawing processing unit 273 does not generate drawing data corresponding to the detected operation even if the indicator detection unit 240 detects the operation of the indicator 70. In addition, the voice pattern analysis unit 275 may cause the indicator detection unit 240 to stop generating the captured image and stop detecting the operation by the indicator 70.
Further, when the sound pattern analysis unit 275 determines that the input sound data is sound data of a preset sound pattern, the sound pattern analysis unit 275 does not cause the image composition unit 257 to synthesize the drawing data with the display image data. You may instruct. Even if the drawing data generated by the drawing processing unit 273 is input, the image composition unit 257 does not draw the drawing data on the display image data developed in the frame memory 256.
Accordingly, it is possible to prevent an image based on the drawing data corresponding to the operation of the indicator 70 from being projected onto the screen SC by the audio data of the preset audio pattern transmitted from the terminal device 100 to the projector 200.
For example, when the display system 1 of this embodiment is used in a school or the like, the teacher can operate the terminal device 100 so that the student cannot write on the screen SC.

Further, the voice pattern analysis unit 275 causes the pointer detection unit 240 to detect the operation of the pointer 70 for a preset time after determining that voice data of a preset voice pattern has been input, The drawing processing unit 273 may generate drawing data. When the preset time has elapsed, the voice pattern analysis unit 275 causes the pointer detection unit 240 to stop detecting the pointer 70 and causes the drawing processing unit 273 to stop generating drawing data.
For example, when the display system 1 of the present embodiment is used at a school or the like, the student can draw on the screen SC using the indicator 70 only for a preset time. be able to.

As described above, the embodiment to which the present invention is applied includes the terminal device 100 and the projector 200.
The terminal device 100 outputs audio data of a preset audio pattern from the output interface unit 106. The projector 200 is set with a sound pattern set in advance and an operation to be executed when sound data corresponding to the sound pattern is input to the input interface unit 251. When the audio data input to the input interface unit 251 corresponds to an audio pattern, the projector 200 executes an operation set in accordance with the audio pattern. Accordingly, the terminal device 100 and the projector 200 can be operated in conjunction with each other regardless of the connection mode.

  The output interface unit 106 included in the terminal device 100 includes an output interface that outputs image data and audio data. The input interface unit 251 provided in the projector 200 includes an input interface for inputting image data and audio data. Therefore, image data and audio data can be transmitted from the terminal device 100 to the projector 200 through one interface.

  The output interface unit 106 included in the terminal device 100 includes an output interface including an interface that outputs image data and an interface that outputs audio data. The input interface unit 251 included in the projector 200 includes an input interface including an interface for inputting image data and an interface for inputting audio data. Therefore, image data and audio data can be transmitted from the terminal device to the display device through separate interfaces.

Further, the terminal device 100 includes a control unit 102 that outputs audio data of a preset audio pattern from the output interface unit 106 when image data output from the output interface unit 106 changes in advance.
Accordingly, it is possible to detect a change in image data output from the output interface unit 106 and output audio data having a preset audio pattern to the projector 200. The projector 200 can respond to the change in image data. A predetermined operation can be performed.

The projector 200 includes a display unit 210 that displays a first image based on image data input to the input interface unit 251, an indicator detection unit 240 that detects an operation of the indicator, and an indicator detection unit 240. A drawing processing unit 273 that draws a second image corresponding to the operation to be detected and displays the second image on the display unit 210 is provided. The voice pattern analysis unit 275 erases the display of the second image that the display unit 210 projects on the screen SC when the voice data input to the input interface unit 251 corresponds to a preset voice pattern.
Therefore, the second image projected on the screen SC can be erased by the sound data of the preset sound pattern transmitted from the terminal device 100 to the projector 200.

In addition, when the audio data input to the input interface unit 251 corresponds to a preset audio pattern, the audio pattern analysis unit 275 causes the data storage unit 274 to display the first image and the first image projected on the screen SC. The image data of at least one of the two images is stored in the storage unit 260.
Therefore, at least one of the first image and the second image projected on the screen SC is stored in the storage unit 260 based on the sound data of the preset sound pattern transmitted from the terminal device 100 to the projector 200. be able to.

The voice pattern analysis unit 275 displays the drawing data generated by the drawing processing unit 273 on the image synthesis unit 257 when the voice data input to the input interface unit 251 corresponds to a preset voice pattern. Instructs not to combine with image data. Also, the audio pattern analysis unit 275 causes the drawing processing unit 273 to stop generating drawing data when the audio data corresponds to a preset audio pattern.
Thereby, it is possible to prevent the image based on the drawing data corresponding to the operation of the indicator 70 from being projected on the screen SC by the audio data of the preset audio pattern transmitted from the terminal device 100 to the projector 200.

In addition, the control unit 102 of the terminal device 100 outputs the audio data of the audio pattern from the output interface unit 106 when a preset time has elapsed after starting the output of the image data.
Therefore, when the set time has elapsed since the image based on the image data is displayed on the screen SC, the second image corresponding to the operation of the indicator 70 can be prevented from being displayed on the screen SC.

In addition, when the audio data input to the input interface unit 251 corresponds to a preset audio pattern, the drawing processing unit 273 causes the second image to be displayed on the screen SC for a preset time. After the elapse, the second image is shifted to a state where it is not displayed on the screen SC.
Therefore, the second image corresponding to the operation of the indicator 70 is displayed on the screen SC for a preset time by the voice data of the preset voice pattern transmitted from the terminal device 100 to the projector 200. be able to. In addition, it is possible to prevent the second image corresponding to the operation of the indicator 70 from being displayed after a preset set time has elapsed.

The above-described embodiment shows a preferred embodiment of the present invention, does not limit the present invention, and various modifications can be made without departing from the gist of the present invention.
For example, in the above-described embodiment, the projector 200 has been described as a liquid crystal projector using a transmissive liquid crystal panel, but may be a projector using a reflective liquid crystal panel or a digital mirror device. Further, the projector is not limited to a projector that projects an image on the screen SC, but a liquid crystal monitor or liquid crystal television that displays an image on a liquid crystal display panel, or a monitor device or television receiver that displays an image on a plasma display panel (PDP), OLED Various display devices such as a monitor device that displays an image on an organic EL display panel called (Organic light-emitting diode), OEL (Organic Electro-Luminescence), or a self-luminous display device such as a television receiver are also included in the present invention. It is included in the display device.

  Further, each functional unit of the terminal device 100 illustrated in FIG. 2 and each functional unit of the projector 200 illustrated in FIG. 3 indicate a functional configuration realized by cooperation of hardware and software. The specific implementation is not particularly limited. Therefore, it is not always necessary to mount hardware corresponding to each function unit individually, and it is of course possible to realize a function of a plurality of function units by one processor executing a program. In addition, in the above embodiment, a part of the function realized by software may be realized by hardware, or a part of the function realized by hardware may be realized by software.

5. Remote control, 30 ... cable, 50 ... menu screen, 70 ... indicator, 100 ... terminal device, 101 ... storage unit, 102 ... control unit, 103 ... drive device, 104 ... display unit, 105 ... operation unit, 106 ... output Interface unit 200 ... Projector (display device) 210 ... Display unit 211 ... Light source unit 212 ... Light modulation device 213 ... Projection optical system 221 ... Light source drive unit 222 ... Light modulation device drive unit 231 ... Operation Panel, 232 ... Remote control light receiving unit, 233 ... Input processing unit, 240 ... Indicator detection unit, 241 ... Imaging unit, 242 ... Position detection unit, 251 ... Input interface unit, 252 ... Audio processing unit, 253 ... Audio output unit, 255: Image processing unit, 256: Frame memory, 257 ... Image composition unit, 260 ... Storage unit, 270 ... Control unit, 271 ... Projection control unit, 2 72 ... Operation acquisition unit, 273 ... Drawing processing unit, 274 ... Data storage unit, 275 ... Audio pattern analysis unit, SC ... Screen.

Claims (12)

A display system comprising a terminal device and a display device,
The terminal device
An output unit for outputting image data and audio data;
A first control unit that outputs the audio data of a predetermined audio pattern from the output unit,
The display device
An input unit to which the image data and the audio data output from the output unit are input;
A storage unit for storing a correspondence relationship between the predetermined sound pattern and a predetermined operation;
And a second control unit that executes the predetermined operation when the audio data input to the input unit corresponds to the predetermined audio pattern. The output unit includes an output interface that outputs the image data and the audio data.
The display system according to claim 1, wherein the input unit includes an input interface for inputting the image data and the audio data. The output unit includes an output interface including an interface that outputs the image data and an interface that outputs the audio data.
The display system according to claim 1, wherein the input unit includes an input interface including an interface for inputting the image data and an interface for inputting the audio data.   The first control unit outputs the audio data of the predetermined audio pattern from the output interface when the image data output from the output interface changes in a preset manner. The display system according to claim 2 or 3. The display device displays a first image based on the image data input to the input interface;
A detection unit for detecting the operation of the indicator;
A drawing unit that draws a second image corresponding to the operation detected by the detection unit and displays the second image on the display unit;
3. The drawing unit, when the audio data input to the input interface corresponds to the predetermined audio pattern, erases the display of the second image on the display unit. 5. The display system according to any one of 4 to 4. The display device displays a first image based on the image data input to the input interface;
A detection unit for detecting the operation of the indicator;
A drawing unit for drawing a second image corresponding to the operation detected by the detection unit and displaying the second image on the display unit;
A storage unit for storing image data,
The drawing unit is configured to display at least one of the first image and the second image displayed on the display unit when the audio data input to the input interface corresponds to the predetermined audio pattern. 5 is stored in the storage unit. The display system according to claim 2. The display device displays a first image based on the image data input to the input interface;
A detection unit for detecting the operation of the indicator;
A drawing unit that draws a second image corresponding to the operation detected by the detection unit and displays the second image on the display unit;
The drawing unit shifts to a state where the second image is not displayed on the display unit when the audio data input to the input interface corresponds to the predetermined audio pattern. Item 5. The display system according to any one of Items 2 to 4.   The first control unit outputs the audio data of the predetermined audio pattern from the output interface when a preset time has elapsed after starting the output of the image data. The display system according to claim 2 or 3. The display device displays a first image based on the image data input to the input interface;
A detection unit for detecting the operation of the indicator;
A drawing unit that draws a second image corresponding to the operation detected by the detection unit and displays the second image on the display unit;
The drawing unit displays the second image on the display unit for a preset time when the audio data input to the input interface corresponds to the predetermined audio pattern, and the elapsed time The display system according to any one of claims 2 to 4, wherein the second image is later shifted to a state in which the second image is not displayed on the display unit. An input interface for inputting image data and audio data, or an input interface including an interface for inputting image data and an interface for inputting audio data;
The voice pattern of the predetermined voice data input to the input interface and the operation to be executed when the voice data of the predetermined voice pattern is input to the input interface are set.
The display device, wherein when the audio data input to the input interface corresponds to the predetermined audio pattern, the operation set corresponding to the predetermined audio pattern is executed. An output interface that outputs image data and audio data, or an output interface that includes an interface that outputs image data and an interface that outputs audio data;
A first control unit that detects a change in the image data output from the output interface and outputs the audio data of a predetermined audio pattern from the output interface;
A terminal device comprising: A control method of a display system having a terminal device and a display device connected to the terminal device,
The terminal device outputs the audio data of a predetermined audio pattern from an output unit that outputs image data and audio data,
When the audio data input to the input unit connected to the output unit of the terminal device corresponds to the predetermined audio pattern, the display device is set corresponding to the predetermined audio pattern A control method of a display system, characterized by executing a predetermined operation.

Images