JP2017058978A - Information processing device and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing device for causing a user to recognize the importance of each part obtained by dividing time series information.SOLUTION: The information processing device includes division means for dividing time series information including voice information and image information and changing with time into a plurality of parts, feature image acquisition means for acquiring a feature image showing a feature of a divided part from image information included in the part, feature word acquisition means for acquiring a feature word showing a feature of the divided part on the basis of voice information included in the part, and summary image generation means for generating a summary image including a plurality of feature images displayed in a mode corresponding to importance of the feature word of the divided part.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an information processing apparatus and a program.

  For example, time series information such as a movie viewed by a user is divided into a plurality of parts, and obtained from feature words related to the part obtained from sound included in the part and images included in the part. There is a technique for creating a summary by associating feature images related to the part (Patent Document 1).

JP 2008-148121 A

  Since the importance level of the feature word in each part is different between each part, when the user browses the time series information according to the importance level of the feature word, it is necessary to show the importance level of the feature word of each part to the user . Therefore, when creating a summary including a plurality of feature images, if the size of the plurality of feature images can be varied according to the importance of the feature word, the convenience for the user is improved.

  One of the objects of the present invention is to provide an information processing apparatus that allows a user to grasp the importance of feature words of each part into which time-series information is divided.

  An information processing apparatus according to claim 1 of the present invention includes a classifying unit that classifies time-series information that includes audio information and image information and changes with time into a plurality of parts, and a feature image that indicates characteristics of the divided parts. Feature image acquisition means for acquiring from the image information included in the part, feature word acquisition means for acquiring a feature word indicating the characteristics of the classified part based on the audio information included in the part, and the classification And summary image generation means for generating a summary image including a plurality of the feature images displayed in a manner corresponding to the degree of importance of the feature words of the selected part.

  The information processing apparatus according to claim 2 of the present invention is the information processing apparatus according to claim 1, wherein the feature word acquisition unit is configured to perform the feature according to an appearance frequency in the audio information corresponding to the divided part. Get a word.

  An information processing apparatus according to a third aspect of the present invention is the information processing apparatus according to the first aspect, wherein the summary image including the plurality of feature images each displayed in a size corresponding to the importance is generated. To do.

  An information processing apparatus according to a fourth aspect of the present invention is the information processing apparatus according to the third aspect, wherein the feature image is displayed larger as the importance is higher, and smaller as the importance is lower.

  An information processing apparatus according to a fifth aspect of the present invention is the information processing apparatus according to the first aspect, wherein the importance is determined from the voice information of the feature word.

  An information processing apparatus according to a sixth aspect of the present invention is the information processing apparatus according to the fifth aspect, wherein the importance is determined by a volume of the voice information of the feature word or a silent time.

  An information processing apparatus according to a seventh aspect of the present invention is the information processing apparatus according to the first aspect, wherein the summary image includes the feature image associated with the feature word.

  An information processing apparatus according to an eighth aspect of the present invention is the information processing apparatus according to the third or fourth aspect, wherein the summary image is associated with the feature word having an aspect corresponding to a size of the feature image. Includes feature images.

  The information processing device according to claim 9 of the present invention is the information processing device according to claim 8, wherein the summary image generating means is configured to store the feature word associated with the feature image according to the size of the feature image. Limit the display.

  The information processing apparatus according to claim 10 of the present invention is the information processing apparatus according to claim 9, wherein the summary image generation means is configured to perform the processing when the size of the feature image is equal to or less than a predetermined threshold. The feature word associated with the feature image is not displayed.

  An information processing apparatus according to an eleventh aspect of the present invention is the information processing apparatus according to the ninth aspect, wherein the summary image generating means is configured such that the size of the feature image is less than or less than a predetermined threshold value. A part of the feature word associated with the feature image is displayed.

  An information processing apparatus according to claim 12 of the present invention further includes an operation instruction accepting unit that accepts an operation instruction by a user in the information processing apparatus according to claim 10 or 11, wherein the operation instruction accepting unit includes the operation instruction accepting unit, Upon receiving an operation for instructing the feature image for which the display of feature words is restricted, the summary image generation means generates the summary image in which the feature words for which the display is restricted are displayed.

  According to a thirteenth aspect of the present invention, there is provided a program for classifying time-series information that includes audio information and image information and changes with time into a plurality of parts, and a feature image indicating the characteristics of the divided parts as the part. Feature image acquisition means for acquiring from the included image information, feature word acquisition means for acquiring a feature word indicating the characteristics of the classified part based on the audio information included in the part, and characteristics of the classified part A program for causing a computer to function as summary image generation means for generating a summary image including a plurality of the feature images displayed in a manner according to the importance of a word.

  According to the first and thirteenth aspects of the present invention, it is possible to allow the user to grasp the importance of the feature word of each part into which the time series information is divided.

  According to claim 2 of the present invention, a feature word can be determined according to the appearance frequency in each part.

  According to the third aspect of the present invention, the importance of the feature word of each part can be recognized by the user based on the size of the feature image.

  According to the fourth aspect of the present invention, it is possible to allow the user to grasp that the importance of the feature word is higher as the feature image is larger and the importance of the feature word is lower as the feature image is smaller.

  According to claim 5 of the present invention, the importance of the feature word can be determined from the voice information.

  According to claim 6 of the present invention, the importance of the feature word can be determined from the volume of the voice information.

  According to claim 7 of the present invention, it is easy for the user to grasp the information of each part.

  According to claim 8 of the present invention, it is possible to make the user grasp the importance of the feature word of each part from the display mode of the feature word.

  According to the ninth to eleventh aspects of the present invention, it is possible to reduce the amount of information about a part having a low importance level of a feature word.

  According to the twelfth aspect of the present invention, it is possible to provide information about a part having a low importance level of a feature word only when the user requests it.

It is a figure which shows an example of the hardware constitutions of the information processing system which concerns on this embodiment. It is a functional block diagram which shows an example of the main functions which the server which concerns on this embodiment implement | achieves. It is a figure which shows an example of the summary image which concerns on this embodiment. It is a flowchart which shows an example of the summary image generation process which the server which concerns on this embodiment performs. It is a flowchart which shows an example of the characteristic information acquisition process which the server which concerns on this embodiment performs. It is a flowchart which shows an example of the display mode setting process which the server which concerns on this embodiment performs. It is a figure which shows an example of the summary image which concerns on this embodiment. It is a flowchart which shows an example of the display mode setting process which the server which concerns on this embodiment performs. It is a figure which shows an example of the summary image which concerns on this embodiment.

  Hereinafter, an embodiment of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram illustrating an example of a hardware configuration of an information processing system 1 according to the present embodiment. As shown in FIG. 1, the information processing system 1 according to the present embodiment includes a server 10 and a terminal device 20 connected via a network. In FIG. 1, the information processing system 1 includes only one terminal device 20, but may include a plurality of other terminal devices 20.

  The server 10 is an information processing apparatus such as a server computer, and includes a control unit 11, a storage unit 12, and a communication unit 13.

  The control unit 11 is, for example, a CPU or the like, and executes various types of information processing according to programs stored in the storage unit 12.

  The storage unit 12 includes, for example, a memory element such as a RAM or an EOM, a hard disk, and the like. The storage unit 12 holds programs executed by the control unit 11 and various data. The storage unit 12 also operates as a work memory for the control unit 11.

  The communication unit 13 is a network interface such as a LAN card, and transmits and receives information to and from the server 10 via communication means such as a LAN and a wireless communication network. In addition, the control part 11, the memory | storage part 12, and the communication part 13 are each connected via the bus | bath.

  The terminal device 20 is an information processing device such as a personal computer, and includes a control unit 21, a storage unit 22, a communication unit 23, a display unit 24, and an operation unit 25. The units 21 to 25 are connected via a bus. The control unit 21, the storage unit 22, and the communication unit 23 may have the same configuration as the control unit 11, the storage unit 12, and the communication unit 13, respectively.

  The display unit 24 is a liquid crystal display, a CRT display, an organic EL display, or the like, for example, and displays information according to an instruction from the control unit 21.

  The operation unit 25 is, for example, a keyboard, a mouse, a button, a touch panel, or the like. The operation unit 25 receives a user instruction operation and outputs the content of the instruction operation to the control unit 21.

  Next, functions realized by the information processing system 1 according to the present embodiment will be described. FIG. 2 is a functional block diagram illustrating an example of main functions realized by the server 10 according to the present embodiment. As shown in FIG. 2, the server 10 functionally includes a time-series information acquisition unit 50, a part classification unit 51, a feature word acquisition unit 52, a feature image acquisition unit 53, a feature information association unit 54, and an importance calculation unit. 55, a display mode setting unit 56, and a summary image generation unit 57. These functions are realized by the control unit 11 executing the program stored in the storage unit 12. This program includes, for example, a magnetic disk (HDD, FD (Flexible Disk), etc.), an optical recording medium (optical disk (CD (Compact Disk), DVD (Digital Versatile Disk), etc.)), a magneto-optical recording medium, a semiconductor memory ( The data is supplied to the server 10 via a computer-readable recording medium such as a flash ROM or via communication means such as the Internet.

  The time series information acquisition unit 50 acquires time series information. Here, the time-series information is information that changes with time, and in the present embodiment, it is information including audio information and image information including a plurality of frame images. Specific time-series information may be lecture information recording a lecture or distribution record information recording video distributed on a television or the Internet. The time series information may be stored in the storage unit 12 of the server 10 or may be provided from the terminal device 20.

  The part classification unit 51 classifies the time series information into a plurality of parts. The part classification unit 51 classifies audio information and image information included in the time series information into a plurality of parts. For example, the part classification unit 51 calculates a difference between color histograms of two consecutive frame images, a pixel difference, a conversion coefficient difference such as a discrete cosine transform, and the like, and when the calculated difference is equal to or greater than a predetermined threshold value. The two frame images are used as part separators.

  The feature word acquisition unit 52 acquires, for each part classified by the part classification unit 51, a feature word indicating the feature of the part. The feature word acquisition unit 52 according to the present embodiment acquires a feature word according to the voice information. First, the feature word acquisition unit 52 performs voice recognition on the voice information of each part and converts it into character information. Then, the feature word acquisition unit 52 acquires a feature word from the converted character information. For example, the feature word acquisition unit 52 extracts the most frequently occurring word from the converted character information as a feature word.

  The feature image acquisition unit 53 acquires a feature image indicating the feature of each part classified by the part classification unit 51. The feature image acquisition unit 53 acquires a frame image corresponding to the feature word acquired by the feature word acquisition unit 52 as a feature image. Specifically, the feature image acquisition unit 53 acquires the appearance position of the feature word in the voice information, and acquires the frame image corresponding to the appearance position as the feature image. The feature image acquisition unit 53 may acquire a feature image based on the image information. For example, the feature image acquisition unit 53 may acquire a frame image that first appears in a part as a feature image, or acquire a frame image that satisfies a predetermined criterion from a plurality of frame images included in the part as a feature image. Also good.

  The feature information association unit 54 stores, in the association information storage unit, association information in which the feature word acquired by the feature word acquisition unit 52 and the feature image acquired by the feature image acquisition unit 53 are associated with each other. Assume that the association information associates a feature word and a feature image for each part. Information related to feature words and feature images may be further associated with the association information for each part. For example, an appearance position where a feature word appears in time series information (for example, reproduction time), an appearance position where a feature image appears in time series information, and the like may be associated.

  The importance calculation unit 55 calculates the importance of each part classified by the part classification unit 51. The importance calculation unit 55 calculates the importance of each part based on the feature words acquired by the feature word acquisition unit 52. For example, the importance calculation unit 55 calculates the importance based on the volume of the time when the feature word appears in the voice information. Specifically, the importance calculation unit 55 calculates the importance higher as the volume of the time when the feature word appears is larger, and calculates the importance lower as the volume of the time when the feature word appears is lower. This is because the higher the volume, the more content you want to claim. Further, the importance level calculation unit 55 may calculate the importance level based on whether or not there is a silent time immediately before the time when the feature word appears in the voice information. Specifically, the importance calculation unit 55 calculates the importance high when there is a silent time immediately before the time when the feature word appears. This is because, for example, when the time-series information is a recording of a lecture, it is considered that the content that you want to insist on after silent time is included because it is judged to be silent to attract the attention of the audience . The importance level calculation unit 55 may calculate the importance level according to the number of appearances of the feature word in each part. This is because the feature word of the part is considered to be more important as the number of appearances of the feature word in the part increases. The importance calculation unit 55 may calculate the importance based on the feature image. For example, the importance calculation unit 55 may calculate the importance from a color histogram or image distribution of the feature image. The importance calculation unit 55 may calculate the importance of each part by combining two or more of the importance calculation methods described above. For example, the importance calculation unit 55 may calculate the importance based on the feature word and the feature image.

  The display mode setting unit 56 sets the display mode of each feature image when the summary image generation unit 57 generates a summary image representing a summary of time-series information using a plurality of feature images. The summary image generation unit 57 generates a summary image according to the display mode set by the display mode setting unit 56.

  The display mode setting unit 56 sets the display mode of each feature image based on the importance level of each part calculated by the importance level calculation unit 55. Specifically, the display mode setting unit 56 varies the sizes of the plurality of feature images according to the importance. That is, the display mode setting unit 56 sets the feature image of the part with high importance to be large and the feature image of the part with low importance to be displayed small.

  FIG. 3 is a diagram illustrating an example of the summary image 100 according to the present embodiment. The summary image 100 shown in FIG. 3 includes a plurality of frames 110, and a feature image is displayed on each frame 110. The plurality of frames 110 have different sizes. For example, the frame 110a is larger than the frame 110b. The plurality of feature images are displayed in different sizes depending on the size of each frame 110. The size of these frames and the size of the feature image are set according to the importance as described above. The display mode setting unit 56 may set the size of the feature image or the size of each frame 110 of the summary image 100. When the display mode setting unit 56 sets the size of each frame 110, the summary image generation unit 57 may convert each feature image into a size corresponding to the corresponding frame 110 and display it.

  A summary image 100 as shown in FIG. 3 is transmitted to the terminal device 20 via the network and displayed on the display unit 24 of the terminal device 20. A user of the terminal device 20 can see the summary image 100 displayed on the display unit 24 of the terminal device 20. When the user selects any feature image, the time-series information is reproduced from the appearance position of the feature image so that the user can view it. In this way, the user can determine the importance level from the size of the feature image divided into frames, and can select the reproduction position according to the importance level.

  Further, the display mode setting unit 56 may set the display position of each of the plurality of feature images in the summary image 100. For example, the display mode setting unit 56 may set the display position of each feature image so that the feature images are arranged in time series. The display mode setting unit 56 may set the display position of each feature image in accordance with the size of the feature image. For example, the display mode setting unit 56 may set the display position of each feature image so that the feature images are arranged in descending order, or display each feature image so that the largest feature image is positioned at the center of the summary image 100. The position may be set.

  Further, in the summary image 100 shown in FIG. 3, there may be a case where a character image indicating a feature word associated with the feature image is superimposed on each feature image and displayed. At this time, if the size of the feature image is less than or less than a predetermined threshold value, not all feature word characters may be displayed. For example, if the feature image is small relative to the number of characters in the feature word, it may not be possible to display all the characters in the feature word. Therefore, the display mode setting unit 56 adjusts the character image according to the size of each feature image. For example, the display mode setting unit 56 reduces the size of the character image so that all characters of the feature word are displayed according to the size of the feature image. Further, the display mode setting unit 56 may restrict the display of the character image when all the characters of the feature word cannot be displayed with respect to the size of the feature image. In this case, the character image may be displayed when the user who views the summary image 100 moves the mouse over a feature image in which display of the character image is restricted. These specific processes will be described later.

  Here, an example of the summary image generation process executed by the server 10 according to the present embodiment will be described with reference to the flowchart shown in FIG.

  First, the time series information acquisition unit 50 acquires time series information (S101). Here, the time series information includes audio information and image information including a plurality of frame images.

  The part classification unit 51 classifies the time series information acquired by the time series information acquisition unit 50 into a plurality of parts (S102). Here, the part classification unit 51 classifies the time series information into K parts, and assigns numbers 1 to K to the divided parts in time series order.

  Then, an initial value of n = 1 is set to the variable n (S103), and feature information acquisition processing is executed (S104). Here, the variable n is a counter variable that takes an integer value of 1 or more.

  An example of the feature information acquisition process executed by the server according to the present embodiment will be described with reference to the flowchart of FIG. Here, the feature information acquired by the feature information acquisition process is information indicating the feature of each part. In the present embodiment, a feature word and a feature image are acquired.

  First, the feature word acquisition unit 52 acquires the sound information of the nth part (S201).

  Next, the feature word acquisition unit 52 converts the voice information of the nth part into character information by voice recognition (S202).

  Then, the feature information acquisition unit extracts a feature word indicating the feature of the nth part from the character information obtained in step S202 (S203).

  Next, the feature image acquisition unit 53 acquires the appearance position of the feature word obtained in the process S203 in the audio information, and acquires the frame image corresponding to the appearance position as the feature image indicating the feature of the nth part ( S204).

  Then, the feature information associating unit 54 associates the feature word acquired in the process S203 with the feature image acquired in the process S204 (S205), and returns. Here, the feature information associating unit 54 stores the association information in which the number for identifying the part, the feature word, and the feature image are associated with each other in the storage unit.

  When the feature information extraction process in process S104 is completed, it is determined whether or not n = K is satisfied (S105). That is, it is determined whether the feature information extraction process has been performed for all K parts.

  As a result of the determination in step S105, when it is determined that n = K is not satisfied (S105: N), 1 is added to the variable n (S106), and the processing after step S104 is repeatedly executed.

  As a result of the determination in step S105, when it is determined that n = K is established (S105: Y), the importance calculation unit 55 calculates the importance for each of the K parts (S107). Here, the importance calculated by the importance calculation unit 55 is stored in association with the association information.

  Then, display mode setting processing is executed (S108). An example of the display mode setting process executed by the server according to the present embodiment will be described with reference to the flowchart of FIG.

  As shown in FIG. 6, first, the display mode setting unit 56 determines the size of the frame 110 that displays the feature image of each part based on the importance of each part calculated by the importance calculation unit 55 in step S107. Set (S301). Here, the display mode setting unit 56 sets the size of the K frames 110 corresponding to the K parts. For example, the display mode setting unit 56 sets the frame 110 corresponding to the part with high importance large and sets the frame 110 corresponding to the part with low importance small. The information set by the display mode setting unit 56 is added and associated with the association information of each part.

  Next, an initial value of m = 1 is set to the variable m (S302). Here, the variable m is a counter variable that takes an integer value of 1 or more.

  Then, the display mode setting unit 56 calculates the number of characters of the m-part feature word (S303). In addition, the display mode setting unit 56 calculates the number of characters that can be placed when placing characters on the frame 110 corresponding to the m part (S304). Here, the number of characters that can be arranged on the frame 110 is determined in advance according to the size of the frame 110 and the character size to be used.

  Then, it is determined whether the number of characters that can be arranged is equal to or greater than the number of characters in the feature word (S305). That is, it is determined whether all the characters of the feature word can be arranged on the frame 110 set in the process S301.

  As a result of the determination in step S305, when the number of characters that can be arranged is equal to or greater than the number of characters in the feature word (S305: Y), the display mode setting unit 56 associates the feature word character data indicating the character of the feature word with the feature image ( S306). Here, the display mode setting unit 56 adds the feature word character data to the association information corresponding to the m part and stores them in association with each other. The feature word character data may include information such as a character font and an arrangement position. The display mode setting unit 56 may newly generate and store a feature image associated with feature word character data.

  Then, it is determined whether m = K is satisfied (S307). That is, it is determined whether the feature word character data is associated with all K parts.

  As a result of the determination in step S307, if it is determined that m = K is not satisfied (S307: N), 1 is added to the variable m (S308), and the processing after step S303 is repeatedly executed.

  If it is determined that m = K is satisfied as a result of the determination in step S307 (S307: Y), the process returns.

  When the number of characters that can be arranged is smaller than the number of characters of the feature word as a result of the determination in step S305 (S305: N), the display mode setting unit 56 displays the feature word character data indicating the characters of the feature word to be arranged on the frame 110 Is adjusted (S309). For example, the display mode setting unit 56 changes the character size of the feature word character data to a size that can be arranged on the frame 110. And the process after process S307 is performed. The display mode setting unit 56 may associate a part of the feature word with the feature image as feature word character data. Further, the display mode setting unit 56 may set the character of the feature word indicated by the feature word character data to be displayed in telop.

  When the display mode setting process ends, the summary image generation unit 57 generates the summary image 100 according to the display mode set in the display mode setting process (S109), and the summary image generation process ends. Here, the summary image generation unit 57 may generate the summary image 100 according to the associated information for each part of the association information.

  FIG. 7 is a diagram illustrating an example of the summary image 100 according to the present embodiment. FIG. 7 shows an example of the summary image 100 generated by the summary image generation unit 57 in step S109. In the summary image 100 shown in FIG. 7, the feature word character image 200 corresponding to the feature word character data is superimposed on the feature image of each frame 110. The feature word character image 200b superimposed on the feature image of the small frame 110b has a smaller character size than the feature word character image 200a superimposed on the feature image of the large frame 110a. In this way, by displaying all the characters of the feature words in the feature image of the small frame 110b, it is easy for the user to grasp the information of each part.

  Next, another example of the display mode setting process executed by the server according to the present embodiment will be described with reference to the flowchart of FIG. Here, a process when the feature word character image is not superimposed on the feature image will be described.

  First, the display mode setting unit 56 sets the size of the frame 110 for displaying the feature image of each part based on the importance of each part calculated by the importance calculation unit 55 in step S107 (S401).

  Next, the display mode setting unit 56 selects a part on which the feature word character image is not superimposed (S402). Here, the display mode setting unit 56 selects a part whose number of characters that can be arranged is less than the number of feature word characters. The display mode setting unit 56 may select all the parts.

  And the display mode setting part 56 produces | generates the alternative data which superimpose instead of a feature word character image about the part which does not superimpose a feature word character image (S403). Here, the substitute data is for informing the user who views the summary image 100 that the character information is present although the feature word character image is not superimposed. The substitute data is, for example, an image icon or character data such as “...”.

  Then, the display mode setting unit 56 generates display character data (S404). The display character data is displayed when the user performs an operation instruction (for example, a mouse over operation) for selecting a feature image on which no feature word character image is superimposed. The display character data may indicate the character of the feature word, or may indicate character information obtained by converting speech information included in the part by speech recognition.

  The display mode setting unit 56 associates the substitute data generated in step S403 and the display character data with the feature image (S404), and returns. Here, the display mode setting unit 56 adds the substitute data and the display character data to the association information corresponding to each part and stores them in association with each other.

  FIG. 9 is a diagram illustrating an example of the summary image 100 according to the present embodiment. FIG. 9 shows an example of the summary image 100 generated by the summary image generation unit 57 according to the display mode set by the display mode setting process shown in FIG. In the summary image 100 shown in FIG. 9, the substitute data image 300 corresponding to the substitute data is superimposed on the feature image of the small frame 110b. Here, when the user selects the alternative data image 300 by a mouse-over operation or the like, the operation instruction is transmitted from the terminal device 20 to the server 10. When the server 10 receives the operation instruction, the summary image generation unit 57 generates a summary image 100 in which the display character image 400 corresponding to the display character data is superimposed. The generated summary image is transmitted to the terminal device 20. The summary image 100 on which the display character image 400 is superimposed is displayed on the display unit 24 of the terminal device 20. The display character image 400 may be configured such that characters indicated by the display character data are displayed in telop.

  Further, when the server 10 receives an operation instruction for selecting the substitute data image 300 by a user's mouse-over operation or the like, the summary image generation unit 57 generates a display character image 400 corresponding to the display character data and sends it to the terminal device 20. It is good also as transmitting.

  The present invention is not limited to the above embodiment.

  For example, the feature word may be changed when the number of characters that can be arranged in the process S305 is smaller than the number of characters of the feature word. Here, the feature word acquisition process may be executed, and the feature word acquisition unit 52 may acquire the feature words arranged in the frame of the size set by the display mode setting unit 56.

  Moreover, in said embodiment, although the server 10 and the terminal device 20 showed the example of a different body, the server 10 and the terminal device 20 may be an integrated apparatus.

  DESCRIPTION OF SYMBOLS 1 Information processing system, 10 server, 11, 21 Control part, 12, 22 Storage part, 13, 23 Communication part, 20 Terminal device, 24 Display part, 25 Operation part, 50 Time series information acquisition part, 51 Part classification part, 52 feature word acquisition unit, 53 feature image acquisition unit, 54 feature information association unit, 55 importance calculation unit, 56 display mode setting unit, 57 summary image generation unit, 100 summary image, 110, 110a, 110b frame, 200, 200a 200b Character word image, 300 alternative data image, 400 display character image.

Claims (13)

Classifying means for classifying time-series information that includes audio information and image information and changes with time into a plurality of parts;
Feature image acquisition means for acquiring a feature image indicating the characteristics of the divided part from the image information included in the part;
Feature word acquisition means for acquiring a feature word indicating the characteristics of the divided part based on the audio information included in the part;
Summary image generation means for generating a summary image including a plurality of the feature images displayed in a manner according to the importance of the feature words of the divided parts;
An information processing apparatus comprising: The feature word acquisition means acquires the feature word according to the appearance frequency in the audio information corresponding to the divided part.
The information processing apparatus according to claim 1. The summary image generating means includes
Generating the summary image including the plurality of feature images, each of which is displayed in a size corresponding to the importance.
The information processing apparatus according to claim 1. The feature image is
The higher the importance, the larger the display, and the lower the importance, the smaller the display,
The information processing apparatus according to claim 3. The importance is determined from the voice information of the feature word.
The information processing apparatus according to claim 1. The importance is determined by the volume or silent time of the voice information of the feature word.
The information processing apparatus according to claim 5. The summary image is
Including the feature image associated with the feature word;
The information processing apparatus according to claim 1. The summary image is
Including the feature image associated with the feature word in an aspect corresponding to the size of the feature image;
The information processing apparatus according to claim 3, wherein the information processing apparatus is an information processing apparatus. The summary image generating means includes
Restricting the display of the feature words associated with the feature image according to the size of the feature image;
The information processing apparatus according to claim 8. The summary image generating means includes
When the size of the feature image is less than or less than a predetermined threshold, the feature word associated with the feature image is not displayed.
The information processing apparatus according to claim 9. The summary image generating means includes
When the size of the feature image is equal to or less than a predetermined threshold value, a part of the feature word associated with the feature image is displayed.
The information processing apparatus according to claim 9. An operation instruction receiving means for receiving an operation instruction by a user;
When the operation instruction accepting unit accepts an operation for instructing the feature image for which the display of the feature word is restricted, the summary image generation unit displays the feature word for which the display is restricted. Generate images,
The information processing apparatus according to claim 10, wherein the information processing apparatus is an information processing apparatus. Classifying means for classifying time-series information that includes audio information and image information and changes with time into a plurality of parts,
Feature image acquisition means for acquiring a feature image indicating the characteristics of the divided part from the image information included in the part;
A feature word acquisition means for acquiring a feature word indicating the characteristics of the divided part based on the audio information included in the part;
Summary image generation means for generating a summary image including a plurality of the feature images displayed in a manner according to the importance of the feature words of the divided parts;
As a program to make the computer function as.

Images