JP2004213570A - Information provision method - Google Patents

Abstract

An object of the present invention is to provide various information to a user while utilizing an existing system.
In an information providing system, an audio signal output from a television device is transmitted to an information management server as collation target audio feature data via a notebook personal computer and the Internet in order. Then, a matching process is performed for each broadcast channel audio feature data corresponding to each broadcast channel, and various types of provided information corresponding to the broadcast channel specified based on the matching rate of the matching result are transmitted to the notebook computer 3. Accordingly, even the existing television device 2 can supply various types of information corresponding to the audio signal to the user, and thus provide various information to the user while utilizing the existing system. be able to.
[Selection diagram] FIG.

Description

[0001]
TECHNICAL FIELD OF THE INVENTION
The present invention relates to an information providing method, and is suitably applied to, for example, an information providing system for automatically providing a user with a homepage on the Internet related to a television program currently viewed on a television device.
[0002]
[Prior art]
Conventionally, in an information providing system, a broadcasting station transmits viewing channel information including an electronic program guide (Electronic Program Guide) and a URL (Uniform Resource Locator) added to each television program in the electronic program guide. In a television device having a function of connecting to the Internet, a URL related to a television program currently being viewed is specified based on the viewing channel information, and the specified URL is automatically accessed to correspond to the URL. Is displayed on a part of the display screen (for example, Patent Document 1).
[0003]
[Patent Document 1]
JP-A-10-177532
[0004]
[Problems to be solved by the invention]
By the way, in the information providing system having such a configuration, the type of information related to a television program is not limited to a URL, and an existing television device other than a television device having a connection function has a homepage related to the URL. There was a problem that can not be provided.
[0005]
The present invention has been made in view of the above points, and aims to propose an information providing method in an information providing system capable of providing various types of information to a user while utilizing an existing system.
[0006]
[Means for Solving the Problems]
In order to solve such a problem, in the present invention, an information terminal device and an information management server that manages a plurality of types of provided information respectively associated with a plurality of broadcast channels for each predetermined unit are connected via a network. In the information providing method of the information providing system, the information terminal device collects an audio signal transmitted from a predetermined broadcast device, and, based on the audio signal, determines whether the information is to be collated on the information management server side. The collation target audio information is generated and transmitted to the information management server via the network, while the information management server receives the collation target audio information and generates a plurality of The matching rate is calculated by collating each audio signal of each broadcast channel obtained from the broadcast wave related to the broadcasting format, and the broadcast specified based on the matching rate is determined. It transmits each providing information associated with the channel to the information terminal device.
[0007]
In this case, in the information providing system, since both the information terminal device and the information management server execute each process based on the existing audio signal included in the broadcast channel, the information providing system is, for example, a broadcasting device such as an existing television device. In this case, a plurality of types of provided information corresponding to the audio signal output from the broadcasting device can be supplied to the user, and the plurality of types of provided information can be supplied to the user without changing the format of various broadcasting systems. Can be supplied to
[0008]
BEST MODE FOR CARRYING OUT THE INVENTION
Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.
[0009]
(1) Configuration of information provision system
In FIG. 1, reference numeral 1 denotes an information providing system as a whole, which includes a television device 2 installed at a predetermined place such as a home or a station yard, and a portable personal computer having a notebook shape (hereinafter referred to as a notebook personal computer). ) 3 and an information management server 4 that manages information provided by a broadcaster who broadcasts a predetermined broadcast channel or another provider, and the notebook computer 3 and the information management server 4 are connected via the Internet 5. Various data can be mutually transmitted and received.
[0010]
In the information providing system 1, when the user of the notebook computer 3 is watching a broadcast program currently being broadcast on the television device 2, the user uses the notebook computer 3 to obtain provided information on the broadcast program. When the predetermined information acquisition operation is performed, the notebook computer 3 extracts the characteristics of the audio waveform based on the audio signal of the broadcast program, and transmits the extraction result to the information management server 4 via the Internet 5.
[0011]
In this case, the information management server 4 specifies the broadcast channel and the time zone of the broadcast program based on the characteristics of the audio waveform extracted by the notebook computer 3, and provides the provided information previously associated with the time zone of the broadcast program. Is transmitted via the Internet 5.
[0012]
(2) Notebook PC internal configuration
As shown in FIG. 2, in the notebook computer 3, a ROM (Read Only Memory) 12 via a bus 11 to a CPU (Central Processing Unit) 10 for totally controlling the notebook computer 2, and a work area A RAM (Random Access Memory) 13, a hard disk drive (hereinafter, referred to as an HDD) 14, a system clock 15, a display unit 16, an operation unit 17, digitize audio signals collected from a microphone 22, and volume of a speaker 23. And a voice control unit 18 for controlling the operation, etc., a network interface 19 for transmitting and receiving various data to and from the information management server 4, and a USB (Universal Serial Bus) interface 20. .
[0013]
The ROM 12 stores a basic program for realizing indirect control of hardware and the like, and the HDD 14 stores a data acquisition program for acquiring various data from the information management server 4, various application programs, and the like. Have been.
[0014]
The CPU 10 appropriately reads out various programs stored in the ROM 12 and the HDD 14, expands them in the RAM 13, and executes various processes by controlling each unit via the bus 11.
[0015]
For example, when another electronic device such as a 5.1 channel compatible speaker is connected via the USB interface 20, the CPU 10 controls the USB interface 20 to exchange data with the electronic device. It has been made executable.
[0016]
By the way, when a predetermined information acquisition operation is performed via the operation unit 17, the CPU 10 reads the data acquisition program stored in the HDD 14, expands the data acquisition program in the RAM 13, and acquires the data according to a predetermined procedure according to the data acquisition program. The processing is performed.
[0017]
(3) Internal configuration of information management server
As shown in FIG. 3, in the information management server 4, a ROM 32 via a bus 31, a RAM 33 as a work area, an HDD 34, a reference clock unit 35, a notebook 30 A network interface 36 for transmitting and receiving various data to and from the personal computer 3 and a feature extraction unit 37 for extracting features of audio signals of broadcast channels are connected to each other.
[0018]
A broadcast channel receiving unit 38 is connected to the feature extracting unit 37. The broadcast channel receiving unit 38 includes a plurality of broadcast channels corresponding to a plurality of broadcast systems, such as television broadcasts and satellite broadcasts, broadcasted by broadcasters. (Not shown), a tuner, and the like are provided.
[0019]
The ROM 32 stores a basic program for realizing indirect control of hardware and the like. The HDD 34 stores a data supply program for supplying various data to the notebook computer 3 and a plurality of broadcast channels. A program date and time table indicating the date and time of each broadcast program for one month, for example, is stored, and a plurality of types of provided information related to the broadcast program are stored in association with the program date and time table.
[0020]
As shown in FIG. 4, the provided information includes a first data group that is always transmitted to the information terminal device (notebook computer 3) that has requested the acquisition of the provided information, and an information terminal device that has requested the acquisition. And a third data group that is transmitted only when the broadcast channel specified by the information management server 4 is a pay broadcast. You.
[0021]
The first data group is stored by being divided into categories according to the data content, and specifically includes a title of a predetermined broadcast program, a broadcast time zone, a synopsis of the content, presence / absence of rebroadcast, and other broadcast program contents. Program content data representing relevant information, a URL (Uniform Resource Locator) representing an access method and an access location of a homepage related to the content of the broadcast program, a sales destination and a photograph of clothes in each scene related to the broadcast program, The advertisement data represents information related to a recipe of a dish, a photograph, and other information used for each scene related to a broadcast program.
[0022]
In addition, the second data group is also stored by being divided into categories according to the data content, and specifically, moving image data (hereinafter referred to as video data) composed of video at a camera angle different from the camera angle broadcasted by a predetermined broadcast program. , This is called different viewpoint moving image data), sign language moving image data of a sign language image corresponding to the sound of the broadcast program, high-quality moving image data having an image quality higher than that of the broadcast program, and other moving images related to the broadcast program. Data, audio data of a theme song broadcasted in the broadcast program, and 5.1 channel audio data that provides higher quality audio than the audio broadcasted in the broadcast program (hereinafter referred to as 5.1 audio data). ) And other audio data related to the broadcast program.
[0023]
Incidentally, each of the moving image data and the audio data relating to the second data group is compressed and stored by, for example, an encoding method called MPEG (Moving Picture Experts Group).
[0024]
Further, the third data group stores, for each broadcaster, an opinion to a broadcaster that broadcasts a predetermined pay broadcast, and questionnaire data representing a question item prepared by the payer.
[0025]
Incidentally, in the information management server 4, the category of the first or second data group can be newly established or deleted by the administrator of the information management server 4, and the program date and time table stored in the HDD 34 and the The data is updated or deleted, and a new type of data can be stored in the HDD 34.
[0026]
The CPU 30 reads out the basic program stored in the ROM 32 and develops it in the RAM 33, and constantly controls the reference clock unit 35 and the feature extracting unit 37 via the bus 31 according to the developed basic program.
[0027]
In this case, the feature extraction unit 37 always receives the audio signals of all the broadcast channels related to the respective broadcast waves corresponding to the plurality of broadcast schemes via the antenna and the tuner of the broadcast channel reception unit 38 sequentially.
[0028]
Then, the feature extraction unit 37 sequentially performs analog-to-digital conversion processing, FFT (Fast Fourier Transform) processing, and normalization processing on the audio signal of each broadcast channel in units of, for example, 5 seconds, and obtains the resulting audio signal of each broadcast channel. The data of the frequency components respectively corresponding to the signals (hereinafter referred to as broadcast channel audio feature data) is stored in the HDD 34 for, for example, 5 minutes, and then sequentially updated.
[0029]
In this way, the feature extraction unit 37 updates the frequency components representing the waveform features of the audio signals in all the broadcast channels related to each of the broadcast waves corresponding to the plurality of broadcast systems to the HDD 34 as the broadcast channel audio feature data. It is made to remember while.
[0030]
Further, the reference clock unit 35 manages the time so that it always keeps an accurate time based on the time transmitted from a predetermined reference clock device that keeps a reference time in Japan, for example.
[0031]
Also, the CPU 30 reads out the data supply program stored in the HDD 34, expands the data supply program in the RAM 33, and controls each unit via the bus 31 so as to execute the data supply process according to a predetermined procedure according to the data supply program. Has been done.
[0032]
(4) Information provision processing
As shown in FIG. 5, regarding the procedure of the information providing process in the information providing system, the processing procedure in the CPU 10 of the notebook computer 3 that executes the data acquiring process (hereinafter referred to as the acquiring CPU 10) and the data supplying process are executed. This will be described with reference to a flowchart divided into a processing procedure in the CPU 30 of the information management server 4 (hereinafter, referred to as a supply-side CPU 30).
[0033]
That is, when a predetermined information acquisition operation is performed via the operation unit 17, the acquisition-side CPU 10 proceeds from the start step of the routine RT1 to the next step SP11, and determines whether or not an audio signal has been input via the microphone 22. Judge. If a negative result is obtained here, this means that the audio signal has not yet been input to the audio control unit 18 via the microphone 22. At this time, the acquisition-side CPU 10 waits until the audio signal is input. Await.
[0034]
On the other hand, if a positive result is obtained, this indicates that the audio signal has begun to be input to the audio control unit 18 via the microphone 22. At this time, the acquisition-side CPU 10 executes the next acquisition request data generation processing. Move to routine SRT1.
[0035]
As shown in FIG. 6, the acquisition-side CPU 10 moves from the start step of the acquisition request data generation routine SRT1 to the next step SP21, and starts the “year” and “month” at the time when the audio signal starts being input via the microphone 22. , "Hour", "minute", "second", "millisecond" and "microsecond" are recognized based on the system clock 15, and the recognized time is generated as input time data. Move to the next step 22.
[0036]
In step SP22, the acquisition-side CPU 10 controls the audio control unit 18 and digitizes the audio signal input to the audio control unit 18 via the audio control unit 18 for one minute based on the system clock 15, for example.
[0037]
Then, the acquisition-side CPU 10 sequentially performs FFT processing and normalization processing on the same time unit (for example, 5 seconds) as the feature extraction unit 37 in the information management server 4 on the audio data for the unit time obtained as a result of the digitization. Data to be subjected to collation on the information management server 4 with frequency component data corresponding to the resulting audio signal (the audio signal input to the audio control unit 18) (hereinafter referred to as a collation target voice) After that, the process proceeds to next step SP23.
[0038]
In step SP 23, the acquisition-side CPU 10 determines whether or not the notebook PC 3 has the processing capability, the capacity of the RAM 12, the performance of each unit connected via the bus 11, the electronic device connected via the USB interface 20, and the like. After recognizing the image processing capability and the audio processing capability and generating the recognized processing capability as processing capability data, the process proceeds to the next step SP24.
[0039]
In step SP24, the acquisition-side CPU 10 transmits the input time data, collation target voice feature data, and processing capability data sequentially generated in steps SP21 to SP23 to the information management server 4 via the network interface 19 as acquisition request data. Move to the next step SP12 (FIG. 5).
[0040]
On the other hand, when a predetermined power-on operation is performed, the supply-side CPU 30 proceeds from the start step of the routine RT2 to the next step SP101, and determines whether or not acquisition request data has been received. If a negative result is obtained here, this means that the acquisition request data has not yet been received from the notebook computer 3 via the network interface 36, and at this time, the supply CPU 30 waits for the acquisition request data. .
[0041]
On the other hand, if a positive result is obtained in step SP101, this means that the acquisition request data has been received from the notebook computer 3 via the network interface 36. At this time, the supply CPU 30 sets The specific subroutine SRT11 is executed.
[0042]
As shown in FIG. 7, the supply-side CPU 30 proceeds from the start step of the program time specification subroutine SRT11 to the next step SP31, and checks the matching target audio feature data in the acquisition request data received in step SP101 (FIG. 5). The matching process is executed for each of the plurality of broadcast channel audio feature data stored in the HDD 34.
[0043]
Specifically, the supply-side CPU 30 compares the matching-target audio feature data with only one of the broadcast channel audio feature data selected from the plurality of broadcast channel audio feature data, and checks the matching target audio feature data for the selected broadcast channel audio feature data. Calculate the matching rate of the voice feature data.
[0044]
Then, the supply-side CPU 30 similarly calculates the coincidence rate for the remaining broadcast channel audio feature data, and proceeds to the next step SP32.
[0045]
In step SP32, the supply-side CPU 30 determines whether or not the coincidence rates calculated in step SP31 each exceed a predetermined threshold. If a negative result is obtained here, this means that the broadcast channel audio feature data corresponding to the collation target audio feature data generated on the notebook personal computer 3 side is not stored in the HDD 34 of the information management server 4, that is, the notebook personal computer 3 Indicates that the audio signal collected by the microphone 22 is noise or the like and is not an audio signal of a predetermined broadcast channel. At this time, the supply-side CPU 30 proceeds to the next step SP33 and specifies the broadcast channel. After notifying the failure, the process returns to step SP101 (FIG. 5).
[0046]
On the other hand, if a positive result is obtained in step SP32, this means that the audio signal collected by the microphone 22 of the notebook computer 3 is an audio signal of a predetermined broadcast channel. The CPU 30 moves to the next step SP34.
[0047]
After specifying the broadcast channel corresponding to the broadcast channel audio feature data whose coincidence rate exceeds the predetermined threshold in step SP34, the supply-side CPU 30 inputs the input time data of the acquisition request data received in step SP101 (FIG. 5). , The program time zone in the specified broadcast channel is also specified, and the next data selection processing routine SRT12 (FIG. 5) is executed.
[0048]
Here, if there are two or more broadcast channel audio feature data whose matching rate exceeds a predetermined threshold, this means that the audio signal collected by the microphone 22 of the notebook computer 3 has two or more types of audio of the broadcast channel. In this case, the supply side CPU 30 specifies each broadcast channel corresponding to each of the two or more broadcast channel audio feature data, and then specifies the program of each broadcast channel. The time zone is also specified.
[0049]
As shown in FIG. 8, the supply-side CPU 30 proceeds from the start step of the data selection processing routine SRT12 to the next step SP41, and corresponds to the program time zone specified in step SP34 among the provided data stored in the HDD 34. Recognize the various data (FIG. 4) relating to the attached first and second data groups, and proceed to the next step SP42.
[0050]
In step SP42, the supply CPU 30 recognizes the image processing capability and the audio processing capability of the notebook computer 3 based on the processing capability data in the acquisition request data received in step SP101 (FIG. 5), and then proceeds to the next step SP43. Move on to
[0051]
In step SP43, the supply-side CPU 30 outputs one of various types of data related to the second data group recognized by the notebook computer 3 in step SP41 based on the image processing capability and the audio processing capability of the notebook computer 3 recognized in step SP42. It is determined whether or not all or a part can be processed.
[0052]
If a negative result is obtained here, this indicates that the processing capability of the notebook computer 3 is, for example, low in the processing capability of the acquisition-side CPU 10, small in RAM capacity, or hardware for reproducing moving image data and audio data. This indicates that all data related to the second data group cannot be processed with the processing capability of the notebook computer 3 such as not having a software configuration. At this time, the supply-side CPU 30 proceeds to the next step SP44. Then, after only various data (FIG. 4) relating to the first data group are recognized as selection candidates, the flow proceeds to the next step SP46.
[0053]
On the other hand, if a positive result is obtained in step SP43, this means that a part or all of various data relating to the second information can be processed by the processing capability of the notebook computer 3, and at this time, the supply side CPU 30 Proceeds to the next step SP45, recognizes various data related to the first data group and, for example, all various data related to the second data group as selection candidates, and proceeds to the next step SP46.
[0054]
Incidentally, the supply-side CPU 30 has a capability of reproducing various types of audio data including, for example, 5.1-channel type audio data, but has hardware and software for decoding various moving image data conforming to MPEG2. If it is determined that there is no ability to reproduce the moving image data, the audio data among the various data relating to the first data group and the various data relating to the second data group (FIG. 4), 5.1Ch Voice data and other voice data are recognized as selection candidates.
[0055]
In step SP46, if the questionnaire data relating to the third data group is associated with the program time slot specified in step SP34, the supply-side CPU 30 determines that the broadcast channel specified in step SP34 (FIG. 6) It is determined that the broadcast is a pay broadcast, the questionnaire data is recognized as a selection candidate, and the routine goes to the next step SP47.
[0056]
In step SP47, the supply-side CPU 30 determines whether or not the system clock 15 of the notebook computer 3 is accurately counted. Specifically, the supply-side CPU 30 determines the time of the input time data received in step SP101 (FIG. 5) and the broadcast channel audio feature data (step SP32 (FIG. 7)) corresponding to the matching target audio feature data received together with the input time data. ), The broadcast channel related to the broadcast channel audio feature data determined to have a match rate with the matching target audio feature data exceeding the predetermined threshold is input to the feature extraction unit 37 based on the reference clock unit 35 in advance. Compare the measured times (“year”, “month”, “hour”, “minute”, “second”, “millisecond”, and “microsecond”) to determine whether they match. .
[0057]
If an affirmative result is obtained here, this means that the sound of the broadcast program broadcast in the same time zone is synchronized with the system clock 15 of the notebook personal computer 3 which is the basis of the time when the time input through the microphone 22 is measured. The time at which the time when the audio is inputted to the feature extracting unit 37 via the broadcast channel receiving unit 38 is coincident with the reference clock unit 35 of the information management server 4 which is the basis for measuring the time is indicated. This indicates that the system clock 15 keeps the same time as the reference clock unit 35 managed so as to keep accurate time. At this time, the supply CPU 30 moves to the next step SP102 (FIG. 5).
[0058]
On the other hand, if a negative result is obtained, this means that the time at which the system clock 15 on the notebook computer 3 and the reference clock unit 35 on the information management server 4 coincide is not shown, that is, the reference clock unit 35 This indicates that the system clock 15 is ticking a time different from the above. At this time, the supply-side CPU 30 proceeds to the next step SP48.
[0059]
In step SP48, the supply-side CPU 30 generates clock correction data for correcting the system clock 15 of the notebook computer 3 to a clock clocked by the reference clock unit 35 which is managed so as to keep accurate time. The data is temporarily stored in the HDD 34, and the process proceeds to the next step SP102 (FIG. 5).
[0060]
In step SP102, the supply-side CPU 30 stores, for example, various data (FIG. 4) relating to the first, second, and third data groups recognized as selection candidates in the data selection processing routine SRT12 (FIG. 6), and clock correction data. After generating the selection screen data for selecting the data desired to be received on the basis of the above, the selected screen data is transmitted to the notebook computer 3 through the network interface 36, and the next desired data supply processing routine SRT13 is executed.
[0061]
By the way, in step SP12, the acquiring CPU 10 waits until the selection screen data is received, and upon receiving the selection screen data transmitted from the information management server 4, moves to the next step SP13.
[0062]
In step SP13, the acquisition-side CPU 10 displays a selection screen CP, for example, as shown in FIG. 9 on the display unit 16 based on the selection screen data received in step SP12, and obtains each item IT1 corresponding to the information desired to be received. The user of the notebook computer 3 is prompted to select any or a combination of IT5 to IT5 and to select the item IT6 only when desired, and executes the next desired data acquisition processing routine SRT2.
[0063]
In this case, the acquisition-side CPU 10 generates item determination data representing any or a combination of the items IT1 to IT6 selected in step SP13 among the items IT1 to IT6 on the selection screen CP, and acquires the information determination server 4. The supply-side CPU 30 of the information management server 4 makes a selection decision item out of various data (FIG. 4) related to the first, second, and third data groups stored in the HDD 34 and the clock correction data. The data corresponding to the data is read and supplied to the notebook computer 3.
[0064]
Specifically, for example, when the user of the notebook computer 3 selects the item IT1, the acquisition-side CPU 10 sets the program title, the program time, the outline of the program content, and the rebroadcast based on the program content data supplied from the information management server 4. A screen (not shown) indicating the presence / absence and other information about the program contents is displayed on the display unit 16.
[0065]
Further, for example, when the user of the notebook computer 3 selects the items IT2, IT3a and IT5, the acquisition-side CPU 10 sets the advertisement data and the URL among the advertisement data, the URL and the clock correction data supplied from the information management server 4 as the advertisement data and the URL. A screen (not shown) and a URL screen (not shown) showing a sales destination, a photograph, and the like of clothes broadcast in each scene of the program are displayed on the display unit 16, and a system clock is displayed based on the clock correction data. 15 is corrected so as to keep the same time as a predetermined reference clock device.
[0066]
Further, for example, when the user of the notebook computer 3 selects the item IT4a, the supply CPU 30 of the information management server 4 reads out the different viewpoint moving image data according to the second data group and supplies the data to the notebook computer 3. At this time, the supply CPU 30 periodically reads the clock correction data regardless of whether or not the item IT6 is selected on the selection screen CP, and supplies the clock correction data to the read notebook computer 3.
[0067]
In this case, the acquisition-side CPU 10 of the notebook computer 3 corrects the system clock 15 so as to keep the same time as the reference clock unit 35 that keeps the correct time based on the clock correction data, and also sets the delay time when passing through the Internet 5. By temporarily storing the different-view video data for a predetermined time in the internal memory of the display unit 16 and sequentially decoding the video data in consideration of the delay time in decoding and the like, the television device 2 (FIG. 1) The video having a camera angle different from the camera angle of the broadcast program can be reproduced (streamed) via the display unit 16 in a state almost synchronized with the broadcast program being broadcast.
[0068]
Further, for example, when the user of the notebook computer 3 selects the item IT4b, similarly to the case where the item IT4a is selected, the supply-side CPU 30 of the information management server 4 reads the sign language moving image data according to the second data group and sets the clock. The correction data is also read and supplied to the notebook computer 3.
[0069]
In this case, the acquisition-side CPU 10 of the notebook personal computer 3 determines the different viewpoints for a predetermined time in consideration of the delay time in accordance with the system clock 15 corrected so as to keep an accurate time, similarly to the case of processing the different viewpoint moving image data. By temporarily storing the moving image data in the internal memory of the display unit 16 and then sequentially decoding the moving image data, the broadcast program is almost certainly synchronized with the broadcast program broadcast on the television device 2 (FIG. 1). The corresponding sign language image can be reproduced via the display unit 16 and a barrier-free effect can be realized.
[0070]
Further, for example, when the user of the notebook computer 3 selects the item IT4f, similarly to the case where the item IT4a is selected, the supply CPU 30 of the information management server 4 reads out 5.1 audio data according to the second data group. At the same time, the clock correction data is read out and supplied to the notebook computer 3.
[0071]
In this case, the acquisition-side CPU 10 of the notebook personal computer 3 converts the 5.1 audio data into the USB interfaces 20 and 5 in accordance with the system clock 15 corrected so as to keep an accurate time, as in the case of processing the different viewpoint moving image data. By sequentially outputting the sound through the speakers corresponding to one channel, high-quality sound full of a sense of reality can be output from the speaker in a state almost surely synchronized with the program displayed on the television device 2 (FIG. 1). It is made to be able to output.
[0072]
For example, when the user of the notebook computer 3 selects the item IT6, the acquisition-side CPU 10 of the notebook computer 3 displays a questionnaire screen (not shown) on the display unit 16 based on the questionnaire data supplied from the information management server 4. Then, when the contents of the questionnaire are answered via the operation unit 15, questionnaire answer data is generated and returned to the information providing server 4.
[0073]
In this case, the supply-side CPU 30 of the information management server 4 generates electronic discount coupon data as electronic discount information for discounting a monthly information distribution fee of a broadcast channel to be paid-broadcast on the television device 2, for example. The coupon data is subjected to a predetermined encryption process and then supplied to the notebook computer 3, so that the user of the notebook computer 3 can use the pay-broadcast program more easily.
[0074]
As described above, the information providing system 1 is configured to provide various types of provided information in various combinations according to the user's preference. Depending on the combination of the various types of provided information, the information providing system 1 includes not only visual or audible elements. It is possible to provide even maintenance-related elements and fee form elements as provided information, and thus it is possible to further increase the user's willingness to watch television broadcasting in general as compared with the related art.
[0075]
(5) Operation and effect
In the above-described configuration, the information providing system 1 uses the notebook computer 3 to collect audio signals broadcast by the television device 2 through the microphone 22, and to generate frequency components representing characteristics of an audio waveform in a part of the audio signals. Is generated as the voice feature data to be collated, and transmitted to the information providing server 4 via the Internet 5.
[0076]
On the other hand, the information providing server 4 corresponds to each of the broadcast channels stored while being updated in the HDD 34 through the broadcast channel receiving unit 38 and the feature extracting unit 37 with respect to the collation target audio feature data received via the Internet 5. The matching process is executed for each broadcast channel audio feature data to be transmitted, and various types of provided information associated with the broadcast channel specified based on the matching rate of the matching result are transmitted to the notebook computer 3.
[0077]
In this case, in the information providing system 1, since both the notebook computer 3 and the information management server 4 execute various processes based on the existing audio signals included in the broadcast channel, the information providing system 1 may be the existing television device 2. It is possible to supply not only various kinds of provided information corresponding to audio signals output from the television device 2 to the user, but also to provide various kinds of provided information to the user without changing the format of each broadcasting system. Can be.
[0078]
Therefore, in the information providing system 1, as long as the user has the notebook computer 3, not only a television device installed at home but also a television device installed on a station premises or a wall surface of a building, etc. Various kinds of information can be supplied regardless of the installation location of the television apparatus 2 or the viewing location of the user.
[0079]
In the information providing system 1, the notebook computer 3 generates processing capability data representing the processing capability of itself (notebook computer 3) after generating the matching target voice feature data, and the matching target voice feature data and the processing capability data. Is transmitted to the information providing server 4, and the information management server 4 discards the provided information corresponding to the broadcast channel specified based on the matching rate of the matching result in accordance with the processing capability of the notebook computer 3. Send after selecting.
[0080]
Therefore, the information providing system 1 transmits in advance only the provided information that can be processed by the notebook computer 3 out of the various provided information in advance, so that the provided information selected by the user of the notebook computer 3 cannot be received. This can be avoided almost certainly, and thus various information can be provided without impairing the expectation of the user.
[0081]
Further, in the information providing system 1, the notebook personal computer 3 converts the time at the time of input through the microphone 22 when generating the collation target voice feature data, and the collation target voice feature data and the input time data into the information providing server. 4, the information management server 4 generates clock correction data for correcting the system clock 15 of the notebook computer 3 based on the time when the broadcast channel specified based on the matching rate of the collation result is received. Is transmitted to the notebook computer 3 together with various kinds of provided information.
[0082]
Therefore, the information providing system 1 can accurately correct the system clock 15 of the notebook computer 3 regardless of the intention of the user of the notebook computer 3.
[0083]
Further, in the information providing system 1, on the information management server 4 side, data (such as moving image data) to be synchronized with video or audio of a broadcast program actually broadcasted by the television device 2 is included in various types of provided information to be transmitted. When the data includes the various data in the second data group), the clock correction data is also automatically transmitted, and the notebook computer 3 adds the clock correction data to the clock correction data before and during the streaming. The moving image data is sequentially decoded in accordance with the system clock 15 corrected so that an accurate time is ticked based on the time.
[0084]
Therefore, the information providing system 1 reproduces (streams) the moving image data in a state where the moving image data is almost certainly synchronized with the broadcast program broadcast on the television device 2 (FIG. 1) regardless of the intention of the user of the notebook computer 3. )can do.
[0085]
According to the above configuration, in the information providing system 1, the audio signal output from the television device 2 is transmitted to the information management server 4 as collation target audio feature data via the notebook computer 3 and the Internet 5 sequentially. The information management server executes a matching process for each broadcast channel audio feature data corresponding to each broadcast channel, and sends to the notebook computer 3 various types of provided information corresponding to the broadcast channel specified based on the matching rate of the matching result. By transmitting the information, even the existing television device 2 can supply various kinds of information corresponding to the audio signal to the user, and thus various information can be provided to the user while utilizing the existing system. Can be provided.
[0086]
(6) Other embodiments
In the above embodiment, the case where the audio signal output from the television device 2 is input via the microphone 22 has been described. However, the present invention is not limited to this, and the audio output terminal may be connected to the television device 2. When the notebook computer 3 is provided with an audio input terminal, an audio signal is input from the audio output terminal of the television device 2 via a predetermined cable and the audio input terminal of the notebook computer 3 sequentially. You may do it.
[0087]
In this case, the information providing system 1 can generate collation target audio feature data that faithfully represents the waveform characteristics of the audio signal output from the television device 2 on the notebook computer 3 side. As a result, the information management server On the side 4, the broadcast program can be specified almost certainly, and thus various kinds of provided information related to the broadcast program desired by the user currently being broadcast on the television device 2 can be provided.
[0088]
In the above-described embodiment, the case has been described in which the collation target speech feature data obtained by sequentially performing the FFT processing and the normalization processing is generated as the collation target speech information. However, the present invention is not limited to this. For example, data obtained by various other methods such as data obtained by filtering with one or a plurality of narrow-band filters or detecting a waveform obtained by the filtering is generated as collation target audio information. Alternatively, if it is within a range allowed by the bandwidth of the Internet 5, a voice signal for a predetermined unit time input via the microphone 22 may be directly generated as verification target voice information. .
[0089]
In this case, the information management server 4 performs collation for each audio information of each broadcast channel obtained from broadcast waves of a plurality of broadcast formats by the same method as the method of generating the collation target audio information on the notebook computer 3 side. By doing so, the same effect as in the above-described embodiment can be obtained.
[0090]
Furthermore, in the above-described embodiment, the case has been described where the feature of the audio signal output from the television device 2 is used as a comparison target. However, the present invention is not limited to this. The feature of the video signal of the broadcast program being broadcast may be used as a collation target. As a means for inputting such a video signal to the notebook computer 3, for example, a video signal is input from a video output terminal of the television device 2 via a predetermined cable and a video input terminal of the notebook computer 3 in order, The screen of the television device 2 captured by the imaging unit may be input as a video signal. Incidentally, when a video signal imaged by the imaging unit is input, it is needless to say that it is necessary to use the feature of the video signal as a comparison target in consideration of the distortion of the screen of the television device 2 and the imaging unit. Absent.
[0091]
Further, in the above-described embodiment, the case where the data acquisition process (FIG. 5) is executed when a predetermined information acquisition operation is performed has been described. If a unit is provided, the data acquisition process (FIG. 5) may be executed when a predetermined infrared signal transmitted from the remote controller is received via the infrared receiving unit.
[0092]
In this case, the data acquisition process (FIG. 5) can be performed without causing the user to perform a predetermined information acquisition operation on the notebook computer 3, and the broadcast signal is specified by the infrared signal received via the infrared receiver. If it is used as an auxiliary means in this case, the specification of the broadcast channel can be hastened as compared with the case where it is not used.
[0093]
Further, in the above-described embodiment, “year”, “month”, “hour”, and “year” at the time when an audio signal starts to be input through the microphone 22 and when a predetermined broadcast channel is input to the feature extraction unit 37. Although a case has been described in which a time consisting of “minutes”, “seconds”, “milliseconds”, and “microseconds” is measured, the present invention is not limited to this, and “year”, “month”, “hour” , "Minutes", "seconds" and "milliseconds" may be counted, and at least a time consisting of "year", "month", "hour", "minute" and "second" You only need to keep time.
[0094]
In this case, if the information management server 4 measures the time when the information is input to the feature extraction unit 37 in correspondence with the time measured on the notebook computer 3 side, the same effect as in the above-described embodiment can be obtained. be able to.
[0095]
Furthermore, in the above-described embodiment, a case has been described in which the notebook personal computer 3 outputs 5.1 audio data sequentially through the USB interface 20 and the 5.1 channel compatible speaker, but the present invention provides However, the present invention is not limited to this. When outputting the sound of the 5.1 sound data, the sound in the television device 2 may be muted. May use sound in the television device 2. In this case, it is possible to provide the user with a higher quality sound that is more realistic.
[0096]
Furthermore, in the above-described embodiment, a case has been described where the provided information is stored in the HDD 34 in the system shown in FIG. 4, but the present invention is not limited to this, and the provided information is stored in the HDD 34 in various other systems. You may do it.
[0097]
Furthermore, in the above-described embodiment, a case has been described in which various types of provided information related to the contents of a broadcast program are associated with the program date and time table. However, the present invention is not limited to this. Is also good.
[0098]
Furthermore, in the above-described embodiment, a case has been described in which various types of provided information are associated with each of the contents of a broadcast program. However, the present invention is not limited to this. Various types of provided information may be associated with each other.
[0099]
Further, in the above-described embodiment, only when the broadcast program specified by the information management server 4 is a pay broadcast, questionnaire data for discounting an information distribution fee for the broadcast program is provided to the notebook computer 3 as provision information. Although the case where transmission is performed has been described, the present invention is not limited to this. Regardless of whether or not the broadcast program specified on the information management server 4 side is a pay broadcast, a data acquisition request from the notebook computer 3 is transmitted. This is an interactive service linked with the broadcast program, such as transmitting questionnaire data for providing an answer to the quiz to a viewer during a broadcast program of the quiz program as provided information. Other various types of viewer participation type questionnaire information may be transmitted as the provision information.
[0100]
In this case, even when the broadcast program specified on the information management server 4 side is an analog broadcast, interactivity (interactive information transmission / reception) can be realized.
[0101]
Further, since the transmission and reception of the viewer participation type questionnaire information is performed via the Internet 5 which is a transmission path different from the broadcast wave, interactivity (interactive transmission and reception of information) can be realized without changing the broadcast format. . Further, as the audience participation type questionnaire information, not only a questionnaire in which a predetermined item is selected to obtain an answer, but also a response such as transmitting a predetermined photo or voice data, for example, is requested. Questionnaires can also be applied.
[0102]
Further, in the above-described embodiment, when the information management server 4 receives the acquisition request data (step SP101), it executes the program time specification subroutine SRT11 (FIG. 7) to execute the verification target audio in the acquisition request data. The case where the program time of the broadcast program corresponding to the characteristic data is specified has been described. However, the present invention is not limited to this. When the acquisition request data is received, the program time specification subroutine SRT11 is executed. It may be determined whether or not the collation target voice feature data is data corresponding to a commercial.
[0103]
Specifically, for example, analog-to-digital conversion processing, FFT processing, and normalization processing are sequentially performed on the audio data corresponding to various commercials in units of 5 seconds, and the resulting waveform characteristic data of commercials representing frequency components is stored in the HDD 34. In advance.
[0104]
In this state, the supply-side CPU 30 of the information management server 4, when receiving the acquisition request data, collates the collation target audio characteristic data with the waveform characteristic data of each commercial, similarly to the above-described program time identification subroutine SRT11, When it is determined that the collation target voice feature data is data relating to a commercial, the user of the notebook computer 3 is notified that a predetermined acquisition operation is to be performed again. The next program time specifying subroutine SRT11 is executed only when it is determined that is not data relating to commercials.
[0105]
In this way, even when commercials are being broadcast on different broadcast channels in the same time zone, it is possible to reliably specify the program time of the broadcast program currently being broadcast on the television device 2.
[0106]
Further, in the above-described embodiment, a case has been described in which the moving image data and the like relating to the second data group managed by the HDD 34 of the information management server 4 are reproduced (streamed) while being downloaded. The present invention is not limited to this, and only the download may be performed without the reproduction.
[0107]
Further, in the above-described embodiment, a case has been described in which the notebook personal computer 3 and the information management server 4 are connected via the Internet 5, but the present invention is not limited to this. The connection may be made via a network.
[0108]
Further, in the above-described embodiment, the case where the notebook personal computer 3 is configured as described above with reference to FIG. 2 has been described. However, the present invention is not limited to this, and the ROM 10, the RAM 13, and the network Various other configurations may be used as long as the configuration includes the interface 20 and a microphone or a voice input terminal.
[0109]
Furthermore, in the above-described embodiment, a case has been described in which the television device 2 is applied as a broadcasting device. However, the present invention is not limited to this. For example, a personal digital assistant such as a radio receiver or a cable television receiver may be used. The present invention may be applied to a computer or the like. In short, various broadcast devices other than receiving and broadcasting broadcast waves of a predetermined broadcast system can be applied.
[0110]
In this case, in the information management server 4, an antenna, a tuner, and the like for receiving a broadcast wave of each broadcast system transmitted from the broadcast device are provided in the broadcast channel receiving unit 38, and the broadcast is received via the broadcast channel receiving unit 38. If the characteristics of the audio signals of all the broadcast channels related to the waves are stored in the HDD 34 as broadcast channel characteristic data, for example, the moving image data corresponding to the audio signals may be synchronized with the audio signals broadcast in the radio program. For example, the same effect as the above-described embodiment can be obtained.
[0111]
Furthermore, in the above-described embodiment, a case has been described in which the notebook computer 3 is applied as the information terminal device. However, the present invention is not limited to this, and for example, a portable digital assistant such as a PDA (Personal Digital Assistants) or a mobile phone. The present invention may be applied not only to the information terminal device but also to various other information terminal devices such as a desktop personal computer. The point is that a sound collecting means for collecting surrounding sounds and a predetermined network are provided. In addition, various other information terminal devices having at least transmission / reception means for transmitting / receiving various data to / from the information management server 4 can be applied.
[0112]
A plurality of such information terminal devices may be connected to the information management server 4 via a predetermined network, and the types of the plurality of information terminal devices may be different.
[0113]
Further, in the above-described embodiment, the case where the information management server 4 is configured as described above with reference to FIG. 3 has been described. However, the present invention is not limited to this. , A broadcast channel and a broadcast program can be specified, and various other configurations can be used as long as various types of provided information related to the specified broadcast program can be transmitted.
[0114]
Further, in the above-described embodiment, a case has been described where the information providing system 1 in which the television device 2 and the information management server 4 are connected via the Internet 5 is applied as the information providing system. The present invention is not limited to this. In addition to such a configuration, for example, a predetermined word corresponding to the broadcast program specified by the information management server 4 is searched as a keyword based on an electronic program guide or the like stored in advance in the HDD. An information providing system or the like in which a search engine server configured to supply the result to the notebook computer 3 as provided information is connected to the information management server 4. In short, at least a broadcasting device such as the television device 2 and the information management server 4 is connected via a predetermined network such as the Internet 5. The beam can be applied to the present invention.
[0115]
【The invention's effect】
As described above, according to the present invention, an information terminal device and an information management server that manages a plurality of types of provided information associated with a plurality of broadcast channels for each predetermined unit are connected via a network. In the information providing method of the information providing system, the information terminal device collects a sound signal transmitted from a predetermined broadcasting device and becomes a target when the information management server collates based on the sound signal. The collation target audio information is generated, and the collation target audio information is transmitted to the information management server via the network. On the other hand, the information management server receives the collation target audio information, and performs multiple broadcasts on the collation target audio information. The matching rate is calculated by collating for each audio signal of each broadcasting channel obtained from the broadcasting wave of the format, and the matching is performed on the broadcasting channel specified based on the matching rate. Each providing information attached to be transmitted to the information terminal device.
[0116]
In this case, in the information providing system, since both the information terminal device and the information management server execute each process based on the existing audio signal included in the broadcast channel, the information providing system is, for example, a broadcasting device such as an existing television device. In this case, a plurality of types of provided information corresponding to audio signals output from the broadcasting device can be supplied to the user, and the plurality of types of provided information can be supplied to the user without changing the format of various broadcasting systems. And thus various types of information can be provided to the user while utilizing the existing system.
[Brief description of the drawings]
FIG. 1 is a schematic diagram illustrating an overall configuration of an information providing system.
FIG. 2 is a block diagram showing an internal configuration of the notebook computer.
FIG. 3 is a block diagram showing an internal configuration of an information management server.
FIG. 4 is a schematic diagram illustrating a system of provided information.
FIG. 5 is a flowchart illustrating an information providing processing procedure in the information providing system.
FIG. 6 is a flowchart illustrating an acquisition request data generation processing procedure;
FIG. 7 is a flowchart illustrating a program time specification processing procedure.
FIG. 8 is a flowchart illustrating a data selection processing procedure.
FIG. 9 is a schematic diagram illustrating an example of a selection screen.
[Explanation of symbols]
DESCRIPTION OF SYMBOLS 1 ... Information provision system, 2 ... Television apparatus, 3 ... Laptop computer, 4 ... Information management server, 5 ... Internet, 10, 30 ... CPU, 14, 34 ... HDD, 15 ... System Clock 16, Display unit 19, 36 Network interface 20, USB interface 22, Microphone 35 Reference clock unit 37 Feature extracting unit 38 Broadcast channel receiving unit

Claims (6)

In an information providing method of an information providing system, an information terminal device and an information management server that manages a plurality of types of provided information respectively associated with a plurality of broadcast channels for each predetermined unit are connected via a network. ,
In the above information terminal device,
A first step of collecting an audio signal transmitted from a predetermined broadcast device;
A second step of generating collation target audio information to be collated on the information management server side based on the audio signal;
A third step of transmitting the collation target audio information to the information management server via the network.
In the above information management server,
A fourth step of receiving the verification target audio information;
A fifth step of collating the matching target audio information for each of the audio signals of the broadcast channels obtained from broadcast waves related to a plurality of broadcast formats and calculating a coincidence rate;
Transmitting the respective provided information associated with the broadcast channel specified based on the coincidence rate to the information terminal device. The information providing method according to claim 1, wherein the information terminal device is a portable information terminal device having at least a microphone. In the third step,
A processing capability information generating step of generating processing capability information representing the processing capability of the information terminal device;
Transmitting the processing capability information to the information management server via the network together with the verification target voice information;
In the sixth step,
A providing information selecting step of selecting each of the providing information corresponding to the broadcast channel specified based on the matching rate of the matching based on the processing capability information. Information provision method. In the third step,
A time counting step for counting a time at which the collation target audio information starts to be generated in the second step;
Transmitting the time to the information management server via the network together with the verification target voice information;
In the sixth step,
A clock correction step of generating correction clock information for correcting a system clock of the information terminal device based on the time,
2. The information providing method according to claim 1, wherein the correction clock information is transmitted to the information terminal device together with each of the provided information. 2. The information providing method according to claim 1, wherein each of the provided information is roughly classified into a plurality of data groups transmitted to the information terminal device and managed according to a predetermined condition. In the sixth step,
Only when the broadcast channel specified based on the matching rate of the collation is a pay broadcast, questionnaire information for discounting an information distribution fee in the pay broadcast is transmitted to the information terminal device together with each of the provided information. The information providing method according to claim 1, wherein:

Images