JP2006085378A - Information processing apparatus, control method therefor, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an information processing apparatus capable of efficiently and suitably presenting a search result intended by a user, a control method therefor, and a program.
Multimedia data is searched based on a search condition. An analysis rule for analyzing the multimedia data obtained as the search result is selected, and the multimedia data obtained as the search result is analyzed according to the selected analysis rule. A display rule corresponding to the search condition is selected, and display contents to be displayed as a search result are determined based on the selected display rule and the analysis result. The search results are displayed according to the display contents.
[Selection] Figure 2

Description

  The present invention relates to an information processing apparatus that searches for multimedia data composed of at least one of document data, image data, audio data, or a combination thereof, a control method thereof, and a program. is there.

  With the high performance of small computer systems and the increase in capacity of storage devices, data such as document data, image data, audio data, and other mixed data have come to be distributed. Furthermore, with the spread of the Internet, which is a global computer network in recent years, and the lower prices of mobile communication devices, it has become an era when these data can be easily obtained and enjoyed. The amount of information handled by individuals has increased, and the function of searching for such information has become very important.

  In order to efficiently use such a large amount of multimedia data, various search methods and devices equipped with them have been proposed. For example, a method of searching for multimedia data that matches a search condition by managing the multimedia data in a storage device such as a database, adding metadata such as feature amounts and keywords, and providing a search condition There is.

  Since the amount of information handled by individuals is large and the contents are diversified and complicated, it is difficult to obtain only the necessary information from a large database according to search conditions. Normally, users narrow down candidates by search conditions, You have to search for the desired information. When displaying the search results, it is common to display a list of identification information such as candidate titles and thumbnails that meet the search conditions, and the user can select this identification information to display more detailed information. Let

  In a conventional multimedia data search device for searching multimedia data, a list of search results is displayed according to the suitability with the search query, such as being ranked according to the suitability and similarity with the search condition. There were many (for example, refer patent document 1).

In the future, it is desirable not only to provide a search function for multimedia data, but also to display search results so that the user can easily find and use desired information.
JP 2001-109779 A

  When a large amount of information is searched and used, such as information collection, necessary information must be searched, selected and arranged according to the purpose from the search results.

  In the conventional method, since a list of search results is displayed according to the compatibility with the search query, information of various contents related to the search query is displayed in an inconsistent manner. In other words, since the display considering the relevance and connection due to the contents of a large number of data obtained as a search result is not performed, when viewing the search result, the user may review the information once viewed many times. The contents must be memorized and organized in the head, and adjusted in consideration of the browsing order of information. In particular, when the number of search results is large, detailed information must be viewed one by one while repeating scrolling and page breaks, which requires much labor, time, and familiarity.

  The present invention has been made to solve the above-described problems, and an object thereof is to provide an information processing apparatus capable of efficiently and suitably presenting a search result intended by a user, a control method therefor, and a program. And

In order to achieve the above object, an information processing apparatus according to the present invention comprises the following arrangement. That is,
An information processing apparatus for searching multimedia data composed of at least one of document data, image data, or audio data, or a combination thereof,
Storage means for storing the multimedia data to be searched;
Input means for inputting search conditions for searching the multimedia data;
Search means for searching multimedia data from the storage means based on the search condition;
Analysis rule holding means for holding analysis rules for analyzing multimedia data obtained as a search result by the search means;
An analysis unit that selects an analysis rule corresponding to the search condition from the analysis rule holding unit, and analyzes the multimedia data obtained as a search result by the search unit according to the selected analysis rule;
Display rule holding means for holding display rules for determining the contents to be displayed as the search results;
Display content for selecting a display rule corresponding to the search condition from the display rule holding unit, and determining display content to be displayed as the search result based on the selected display rule and the analysis result of the analysis unit A determination means;
Search result display means for displaying the search result according to the display content determined by the display content determination means.

  ADVANTAGE OF THE INVENTION According to this invention, the information processing apparatus which can present the search result which a user intends efficiently and suitably, its control method, and a program can be provided.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  The present invention provides a search in a system that uses multimedia data composed of at least one of document data, image data (still image or moving image data), audio data, or a combination thereof. When displaying the search results searched by the engine using browsing software or search applications, the data obtained as search results is handled in a specific way so that it can be easily understood and effectively presented to the user. The configuration is provided.

[Embodiment 1]
FIG. 1 is a block diagram showing a basic configuration of a computer apparatus in which a multimedia data search system, which is an information processing apparatus according to Embodiment 1 of the present invention, is constructed.

  In FIG. 1, reference numeral 100 denotes a CPU comprising a microprocessor, which performs operations for various processing, logical judgments, etc., and is connected to each bus via an address bus AB, a control bus CB, and a data bus DB. Control the components. The control content is instructed by a program on the ROM 101 or RAM 102 described later.

  A plurality of computer programs can be operated in parallel by the function of the CPU 101 itself and the computer program mechanism.

  The address bus AB transfers an address signal indicating a component to be controlled by the CPU 101. The control bus CB transfers a control signal of each constituent element to be controlled by the CPU 100. The data bus DB performs data transfer between the component devices.

  Reference numeral 101 denotes a ROM, which is a read-only fixed memory. The ROM 101 stores a computer program area and a data area in which a control procedure by the CPU 101 described later is stored.

  Reference numeral 102 denotes a RAM, which is a writable random access memory that is used as a computer program area and a data area in which a control procedure by the CPU 100 described later is stored, and from each component other than the CPU 100. It is also used as a temporary storage area for various computer programs and various data.

  These storage media such as the ROM 101 and the RAM 102 store various data such as computer programs and data for realizing the data management of the first embodiment, and the CPU 100 reads out and executes the program codes stored in these storage media. The function is realized by doing so, but the type of the storage medium is not limited.

  Also, a storage medium storing the program and data according to the present invention is supplied to the system or apparatus, and the program is copied from the storage medium onto the rewritable storage medium such as the RAM 102. As a storage medium, a CD-ROM, a floppy (registered trademark) disk, a CD-ROM, a hard disk, a memory card, a magneto-optical disk, or the like can be used.

  A hard disk (DISK) 103 functions as an external memory for storing various data. The DISK 103 has a built-in storage medium that can read and write a large amount of information at a relatively high speed, and can store and read various data in the storage medium as necessary. The stored computer program and data are called up completely or partially on the RAM 102 when necessary according to an instruction from the keyboard (KB) 107 or instructions from various computer programs.

  Reference numeral 104 denotes a MemCard (memory card), which is a detachable storage medium. Information is stored in this storage medium, and the stored information is referenced and transferred by connecting the storage medium to another device. It becomes possible.

  Reference numeral 107 denotes a keyboard (KB) which includes various function keys such as alphabet key, hiragana key, katakana key, character symbol input key such as a punctuation point, and cursor movement key for instructing cursor movement. A pointing device such as a mouse can also be included.

  A cursor register (CR) 108 stores an address indicating a position displayed on the CRT 111. The CPU 100 can read and write the contents of the CR 108. A CRT controller (CRTC) 110, which will be described later, displays a cursor at a position on the CRT 111, which is a display device, corresponding to the address stored in the CR.

  Reference numeral 109 denotes a display buffer memory (DBUF), which stores a pattern of data to be displayed. Reference numeral 110 denotes a CRT controller (CRTC), which plays a role of displaying the contents stored in the CR 108 and the DBUF 109 on the CRT 111. The CRT 111 is a display device using a cathode ray tube or the like, and the dot pattern display pattern and cursor display on the CRT 111 are controlled by the CRTC 110. Note that another display device such as an LCD can be used instead of the CRT 111.

  A character generator (CG) 112 stores character and symbol patterns to be displayed on the CRT 111.

  Reference numeral 105 denotes an NCU, which is a communication device for communicating with an external device such as another computer device. By using this device, the program and data of the first embodiment can be shared with other devices. become.

  In FIG. 1, through a network (LAN), a personal computer (PC (personal computer)) 114, an image receiving / accumulating / displaying device (TV / VR) 115 such as a television broadcast or a video taken by itself, a home Is connected to a game computer (GC) 116 for free use, and information can be freely exchanged therewith. Needless to say, the device of the present invention and the device connected via the network may be anything. The network type may be anything, and the network may be connected to an external network instead of the closed network as shown in the figure.

  Reference numeral 106 denotes a receiving device (DTU) that realizes a broadcast communication receiving function using an artificial satellite or the like. The parabolic antenna (ANT) 113 receives a signal such as a radio wave broadcast via the artificial satellite. And has a function of acquiring broadcast data. There are various types of broadcast-type communications, such as those broadcast on terrestrial radio waves, those broadcast on coaxial cables and optical cables, and those distributed on LANs and large-scale networks. Any form can be considered as long as it is for broadcast communication.

  Next, the functional configuration of the multimedia data search system according to the first embodiment will be described with reference to FIG.

  FIG. 2 is a diagram showing a functional configuration of the multimedia data search / display system according to the first embodiment of the present invention.

  2 are realized by the CPU 100 in FIG. 1 executing a control program stored in the ROM 101 to control each component.

  Reference numeral 201 denotes a search query input processing unit for inputting requirements (search query (search condition)) relating to a desired search target. A search query holding unit 202 temporarily stores the search query input by the search query input processing unit 201. The search query holding unit 202 is realized by the RAM 102, for example.

  A search processing unit 203 refers to the search query held by the search query holding unit 202 and searches for multimedia data that matches the search query. A search result holding unit 204 temporarily holds a search result searched by the search processing unit 203. The search result holding unit 204 is realized by the RAM 102, for example.

  Reference numeral 205 denotes a metadata holding unit, which stores metadata attached to multimedia data stored in the multimedia data holding unit 206. A multimedia data holding unit 206 stores multimedia data to be searched.

  An analysis rule selection processing unit 207 refers to the search query held by the search query holding unit 202 and selects an analysis rule corresponding to the search query from the analysis rules stored in the analysis rule holding unit 208. select.

  An analysis rule holding unit 208 stores analysis rules for analyzing multimedia data obtained as a search result. Reference numeral 209 denotes a decision analysis rule holding unit that temporarily holds information for identifying the analysis rule selected by the analysis rule selection processing unit 207. The decision analysis rule holding unit 209 is realized by the RAM 102, for example.

  A search result analysis processing unit 210 analyzes the search result held in the search result holding unit 204 according to the analysis rule held in the decision analysis rule holding unit 209.

  An analysis result holding unit 211 temporarily stores the analysis result analyzed by the search result analysis processing unit 210. The analysis result holding unit 211 is realized by the RAM 102, for example.

  A display content determination processing unit 212 uses the display rule held by the determination display rule holding unit 217 to determine display content to be displayed as a search result according to the analysis result held in the analysis result holding unit 211. .

  A display content holding unit 213 temporarily stores the display content determined by the display content determination processing unit 212. The display content holding unit 213 is realized by the RAM 102, for example.

  Reference numeral 214 denotes a search result display unit that displays the search result with reference to the display content held by the display content holding unit 213. Reference numeral 215 denotes a display rule holding unit, which stores display rules for determining display contents to be displayed as search results.

  A display rule selection processing unit 216 refers to the search query held by the search query holding unit 202 and selects a display rule corresponding to the search query from the display rules stored in the display rule holding unit 215. select.

  A decision display rule holding unit 217 temporarily holds information for identifying the display rule selected by the display rule selection processing unit 216. The determination display rule holding unit 217 is realized by the RAM 102, for example.

  In the first embodiment, for example, when multimedia data such as a still image or a moving image shot at home is searched and a shot of the still image or moving image obtained as a search result is reproduced, the display is effectively performed. Is assumed.

<Search process>
In the first embodiment, a search is performed using metadata associated with each data to be searched. The metadata is given by manually assigning data such as titles and keywords, or by analyzing the multimedia data to be searched and extracting feature quantities such as media feature quantities.

  In search, users are given search conditions (search queries) such as search keywords and keyword lists, and the search conditions are compared with the metadata associated with each data, and metadata matching the search conditions is given. Get multimedia data. Further, the similarity between the search condition and the metadata of each acquired multimedia data may be calculated and acquired based on the calculation result.

  Here, the similarity is a value indicating how appropriate a specific character string in the search condition input by the user is as an expression indicating a relationship with each data.

  Next, an example of the search condition setting screen according to the first embodiment will be described with reference to FIG.

  FIG. 3 is a diagram showing an example of a search condition setting screen according to the first embodiment of the present invention.

  This search condition setting screen is illustrated as a screen when a window system is used. The search condition setting screen in FIG. 3 is a screen example when the window system is used in a dedicated application. However, when displaying on a small screen such as a digital camera or a video camera, the display according to the screen size is displayed. It is desirable to do the way.

  Reference numeral 31 denotes a title bar, which is a part for displaying the title of the search condition setting screen and operating the entire screen such as movement and size change. 32 and 33 are buttons for providing functions related to the search condition setting screen, 32 is a help display button, and 33 is a button for closing the search condition setting screen.

  Reference numeral 34 denotes an input area for inputting a search condition for performing a search. The input area 34 is a text input area for inputting a keyword, a phrase, or the like for searching for an image. A plurality of keywords or phrases are input separated by a delimiter (for example, a blank), and a logical operation AND. Either OR or OR can be specified with a radio button.

  Although not shown in the first embodiment, importance or similarity distance of a plurality of inputted keywords or phrases may be set, and information such as date range is specified as another search condition. You may do it.

  Reference numeral 35 denotes an OK button. When this OK button 35 is operated, the content input in the input area 34 as a search condition is finally confirmed, and the process proceeds to the next process. On the other hand, 36 is a cancel button. When the cancel button 36 is operated, the input contents in the input area 34 are discarded.

  Next, an example of metadata (held by the metadata holding unit 205) added to image data (multimedia data) such as still images and moving image shots to be searched will be described with reference to FIG. .

  FIG. 4 is a diagram showing an example of metadata according to the first embodiment of the present invention.

  FIG. 4 shows that a title, a photographer, a shooting date, a data type (moving image / still image), a category, a keyword, an importance level, and the like are assigned to each image data as metadata. Here, an example in which character information that can be handled as simple keywords is mainly stored is shown, but information other than character information such as an image feature amount can be given as metadata.

  In FIG. 4, as an example, in a certain image A, the title is “cake sword”, the photographer is “Information Taro”, the shooting date is “2003-04-01T12: 30”, the data type is “Movie”, and the category is "Wedding-wedding-cake sword", "Bride and groom", "Wedding cake" as keywords, and "High" as importance are assigned. These metadata are stored in a storage medium such as the DISK 103, the ROM 101, and the RAM 102, for example.

  By using this metadata, it is possible to search image data such as still images and moving image shots stored in a database on the storage medium.

<Display search results>
Next, a method for effectively displaying the search result will be described in detail.

  In the first embodiment, image data such as a still image or a moving image that is a search result is analyzed, and effective display contents are determined according to the analysis result.

1. Analysis of search results Data can be considered to be diverse in terms of data management, but the type and contents of data handled in a certain situation are predictable to some extent or limited to a certain range. In addition, depending on the type of content of each data, it is possible to predict to some extent empirically the specific content, things, people, and situations that appear in each data. The relationship etc. can be estimated to some extent.

  Therefore, in the first embodiment, data analysis rules corresponding to the data handling situation, data range, content type, etc. are defined in advance and stored in a storage medium such as the DISK 103, ROM 101, RAM 102, etc. The data obtained as a search result is analyzed. The analysis rule may be defined for each search engine or for each database in which data to be searched is stored.

  The data analysis method differs depending on the content and type of the data to be searched, and the situation and range of the search, but in the first embodiment, it is expressed in the data obtained as the search result. Classify and relate according to meaning and content.

  First, when classification is performed according to the content and type of data, for example, classification can be performed by paying attention to common factors (factors to which the data belongs) for events, categories, scenes, central event objects, and the like. Events, categories, scenes, etc. can be expected to be composed of a plurality of events, categories, scenes, etc. Therefore, a hierarchical structure of events, categories, scenes, etc. is defined as an analysis rule, and data is classified according to these hierarchical structures.

  Next, an example of the analysis rule having this hierarchical structure will be described with reference to FIG.

  FIG. 5 is a diagram showing an example of an analysis rule having a hierarchical structure according to the first embodiment of the present invention.

  FIG. 5 shows a hierarchical structure of classification elements such as events, categories, and scenes. FIG. 5 shows the inclusive relation / hierarchical relation of individual sub-classification elements XA to XI included in a certain classification element X. The classification element X first includes classification elements XA to C, and further indicates that the classification element XA includes classification elements XD to F. A keyword may be associated with the end element of the hierarchical structure and used for data classification. Information in this hierarchical structure is stored in a storage medium such as the DISK 103, the ROM 101, and the RAM 102, for example.

  For example, in the first embodiment, since image data such as videos and still images taken mainly at home is a search target, events such as “wedding”, “athletic meet”, etc. when shooting images at home A hierarchical structure can be defined in units of and themes.

  If the classification element X in FIG. 5 is an event called wedding, it can be classified into categories such as the introduction of the bride and groom, the process up to the wedding, the scenery of the venue, the wedding, and the reception. The wedding category in these categories consists of admission, vows, ring exchange, etc., and the reception category consists of admission, cake entry, speech, etc., and further speech. This category can be composed of scenes such as guest speech, friend representative speech, and the like.

  Further, if the classification element X is a theme of child growth, it can be classified into categories such as birth, Shichigosan, kindergarten era, elementary school era, and the like. For example, the category of the kindergarten era can be constituted by small events such as an entrance ceremony, an athletic meet, and a school performance.

  In the first embodiment, each piece of data obtained as a search result is classified into categories according to the above-described hierarchical structure using metadata attached to image data as shown in FIG. For example, in FIG. 4, category information is given to the data in advance according to the hierarchical structure of FIG. 5, and can be classified into categories using this.

  Also, as shown in FIG. 5, a keyword is associated with the end element of the hierarchical structure, and by comparing this with metadata such as a keyword and a title assigned to each data, the search result can be determined. It can also be classified into categories by estimating the content.

  Next, the individual data obtained as a search result and the group of data as the classification result are related.

  In the first embodiment, for example, spatial positional relationship, temporal and semantic context, similarity relationship, equality / parallel relationship, contrast (opposite) relationship, causal relationship such as cause and result, overview and details, materialization / example・ Association rules for finding mutual relationships such as supplement, modification, etc. are held as analysis rules.

  Here, an example of the association rule will be described with reference to FIG.

  FIG. 6 is a diagram showing an example of an association rule based on the meaning / content of image data according to the first embodiment of the present invention.

  FIG. 6 shows association rules for search results obtained from search queries belonging to the category “wedding”.

  For example, a “category exterior” and “inside the venue”, which are subcategories of the category “venue scenery” in the event “wedding”, have a spatial positional relationship of perspective, outside, inside, and part. In addition, it can be said that there is a causal relationship between the cause and the result of the category “Bride and groom's sorrow” and the category “wedding”. In addition, the category “state of waiting room” and “wedding” correspond to a modification relationship or a supplementary relationship.

  In the first embodiment, the association is performed according to the association rule of FIG. 6 using metadata such as titles and keywords assigned to the image data as shown in FIG. Further, the relationship between before and after may be performed according to the shooting date and time of the image data.

  In the first embodiment, data obtained as a search result is further classified, and individual data and a group of data as a classification result are related.

  Here, an example of group association will be described with reference to FIG.

  FIG. 7 is a diagram illustrating an example of classifying data obtained as a search result according to the first exemplary embodiment of the present invention, and associating individual data and a group of data that is the classification result.

  In FIG. 7, G represents a group of data, D represents data, and R represents a relationship. A line indicated by R represents a start point, and ● represents an end point.

  For example, the classification element G1 and the classification element G2 have a relationship of R13, and the classification element G1 and the classification element G3 have a relationship of R11. The classification elements G11 and G12 included in the classification element G1 have a relationship of R11.

  Further, the data D111 and D112 included in the classification element G11 have a relationship of R13 with D111 as the start point and D112 as the end point. D224 is an element included in the classification element G2, but is not related to any element.

  These pieces of information are temporarily stored in a storage medium such as the RAM 102.

  Next, details of the search result analysis processing by the search result analysis processing unit 210 will be described with reference to FIG.

  FIG. 8 is a flowchart showing search result analysis processing by the search result analysis processing unit according to the first embodiment of the present invention.

  First, in step S901, data obtained as a search result is classified according to an analysis rule. This classification method is as described above.

  Next, in step S902, the data obtained as the search result and the group of data obtained as the classification result are related. The relation method is as described above.

  As described above, in the first embodiment, as the analysis process of the search result analysis processing unit 210, the search result classification and the correlation between the search result and the classification result are performed, but the data to be searched is handled. Various other analysis methods are conceivable depending on the situation, the range of data, the type of content, characteristics, and the like.

2. Determination of Display Content Next, a method for determining the display content of the search result based on the obtained analysis result and displaying the search result in an easy-to-understand manner for the user will be described in detail.

  Humans connect and understand fragmentary information in their heads by integrating and linking various linguistic and non-verbal symbols. When complicated imaginary media is required, it is difficult to understand, and information presentation without time, space, and semantic continuity will be confusing. Therefore, it is important to present information in such a way that humans can interpret it naturally.

  In the first embodiment, a display rule corresponding to a search target is defined in advance by applying a human understanding pattern. By determining the display contents based on the above analysis results according to the display rules, the search results can be shown in an easily understandable manner.

  In particular, in the first embodiment, as display contents, the display elements and configuration, the expansion method (display element presentation order), the display element connection method, and the like are determined as display rules. This display rule may be defined for each search engine or may be defined for each type of database in which data to be searched is stored.

  When showing search results, it is easier for the user to understand if a block of data representing one content corresponding to a chapter or paragraph in a sentence and a sequence or scene in a video is composed step by step. The elements of each block include a central element that is the center of the block, a supplementary element that is content-linked to the central element, a linking element that connects each central element / supplemental element, and to help understand the content. There are explanatory elements, etc.

  Next, display content determination processing by the display content determination processing unit 212 will be described with reference to FIG.

  FIG. 9 is a flowchart showing display content determination processing by the display content determination processing unit according to the first embodiment of the present invention.

  First, in step S1401, the display target element and configuration are determined from the search results based on the analysis results described above.

  Details of the processing in step S1401 will be described with reference to FIG.

  FIG. 10 is a flowchart showing details of the process in step S1401 according to the first embodiment of the present invention.

  In the first embodiment, first, in step S1501, an element determined to be particularly important in the search result is selected as a central element. For this, the importance of the data is determined from the individual data obtained as a search result based on, for example, the similarity. Alternatively, the metadata shown in FIG. 4 may be used, or the user may specify at the time of search.

  In the first embodiment, each element to be displayed indicates a still image or a video shot obtained as a search result, and both the similarity obtained at the time of search and the importance given as metadata are high. Are determined to be particularly important elements, and are selected as the central elements.

  Next, in step S1502, a block of data centered on the determined central element is determined. In the first embodiment, the data group into which the search results are classified is determined as the data block that includes the central element. In particular, in the first embodiment, a data block is a set of shots of a plurality of still images and moving images indicating scenes and sequences determined by classification.

  In step S1503, an element having a relationship with the central element in each block is selected as a supplementary element. The element associated with the supplementary element of the central element is also selected as the supplementary element. Elements that are not related to any central element are not selected as supplementary elements. Further, when selecting the supplemental element, the importance of each data may be taken into consideration, and the element having low importance may not be selected.

  The central element and the supplementary element described above are display elements to be displayed, and the display element and the data block including the display element are used as constituent elements when displaying the search result. Here, the central element may not be related to any element.

  Here, an example of a display element, a configuration, and a relationship thereof will be described with reference to FIG.

  FIG. 11 is a diagram showing an example of the display elements, configuration, and relationship thereof according to Embodiment 1 of the present invention.

  In FIG. 11, G represents a group of data that is a block, D represents each data that is a display element, R represents a relationship, ○ of the line pointed to by R represents the start point, and ● represents the end point. What D is shown in bold and bold is a central element, and what is shown in thin and bold is a supplementary element.

  These pieces of information are temporarily stored in a storage medium such as the RAM 102.

  Returning to the description of FIG.

  In step S1402, the display element determined in step S1401 and a data block expansion method (presentation order) including the display element are determined.

  When presenting search results, expand each element in an order that humans can naturally understand, for example, from the overview or from the whole to the details, according to the spatial and temporal arrangement in the order in which the explanation is understood. Easy to understand. This can be derived from the relationship of each element obtained by analysis. Therefore, in the first embodiment, the presentation order based on the mutual relationship between the components is defined as a display rule, and this is stored in a storage medium such as the DISK 103, the ROM 101, and the RAM 102. By using this display rule, the presentation order of the components is determined.

  Here, an example of the presentation order will be described with reference to FIG.

  FIG. 12 is a diagram illustrating an example of the presentation order according to the first embodiment of the present invention.

  FIG. 12 defines the order of presentation and the priorities based on the data blocks and the interrelationships between the individual data. The priority indicates the strength of dependency and cohesion between the components when the components are presented according to the mutual relationship.

  When the priority is strong, it is difficult to understand if the components are presented separately, so that the data must be continuously arranged in the presentation order.

  If the priority is medium, the components do not necessarily have to be continuous if the order of presentation is observed, but if they are not continuous, elements that can understand the relationship must be supplemented. Represents that.

  When the priority is weak, it is indicated that the constituent elements do not need to be continued if the presentation order is observed.

  In FIG. 12, for example, when there is a space or a semantic whole and part relationship between data A and data B, data B (partial) is also included in the order of data A (whole) to data B (partial). ) To data A (whole) in order, both of which are strong.

  Also, if there is a causal relationship between data A and data B, when data A representing the cause is presented in the order of data B representing the result, the priority is medium, but conversely the data When presenting in order from B (result) to data A (cause), the priority is strong and data B and A must be consecutive.

  For example, in the first embodiment, since “the situation near the venue” and “the situation at the venue” have a relationship between perspective and outside, it is preferable to display “the situation near the venue” to “the situation at the venue”. I understand.

  This rule can be applied to data blocks as well, and if two data blocks are related, the data blocks can be correlated with each other even if the data in each data block is not associated. The presentation order of the data in the block can be determined by the relationship.

  In addition, when a component has a relationship with a plurality of elements, the presentation order is determined by the priority of the relationship and the importance of the data. When a display element is related to both the central element and the supplementary element, the relationship with the central element takes precedence.

  For example, in the data block G12 of FIG. 11, the supplementary element D121 has a relationship of R4 having a high priority with the central element D122, and has a relationship of R2 with a medium priority with the central element D123. Since the central element D123 has a relationship of R4 having a strong priority with the supplementary element D124, data can be displayed in the order of D121, D122, D124, and D123. The supplementary element D121 is also related to the supplementary element D125, but this relation is ignored because the relation with the central element is prioritized.

  Returning to the description of FIG.

  In step S1403, on the basis of the relationship between components and the development method, a connection method is determined for connection so that the separation between components, the relationship with the central component, and the relationship with other components are clear. That is, if it is clearly indicated that the components are different depending on the display and the mutual relationship between the components cannot be understood naturally, a connecting element for indicating the relationship is required. Further, when the development is difficult to understand, an explanatory element for further helping the user to understand is required.

  In the first embodiment, when an image is reproduced, a sequence and a scene are connected by a cut connection or a screen between white and black to clearly indicate a delimiter. If the relationship between the sequence and the scene is not clear, the mutual relationship is shown by using a material image, a transition effect, a sound effect, and the like. In addition, when it is difficult to understand the development and contents, it is used as an explanatory element such as a telop or a supermarket so that it can be easily understood.

  FIG. 13 is a diagram defining a connection ID indicating a connection method with respect to the mutual relationship and presentation order of the components according to the first embodiment of the present invention. The medium having the medium priority in FIG. 12 differs in connection method depending on whether or not the elements to be connected are continuous.

  FIG. 14 is a diagram showing display contents corresponding to each connection ID according to the first embodiment of the present invention as display rules.

  In FIG. 14, the display contents differ depending on whether the element to be connected is a shot, a scene, or a sequence. The display content corresponding to the link ID in FIG. 14 differs depending on the type and content of the data to be searched, and it is necessary to define the corresponding content.

  From FIG. 13 and FIG. 14, for example, if shots and scenes having a context are displayed continuously from front to back, they are connected by cut, and if sequences are displayed continuously from front to back, a fade-out or pause screen is inserted. You can see that it is better to connect. In addition, when displaying shots or scenes having contexts from the back to the front, it is necessary to clearly indicate the relationship by inserting a crossfade, a material image, BGM, or the like. Furthermore, in the case of a sequence, it can be seen that explanatory elements such as telop and supermarket are necessary.

  For example, since the data block G11 and the data block G12 in FIG. 11 are scenes and have a context of R11, when elements are displayed as D111, D112, D121,. Connect D121 with a cut. In addition, when elements are displayed as..., D123, D111, D112 in the order of G12 to G11, D123 and D111 are connected by a cross fade or the like.

  Next, an example of the search result display screen of Embodiment 1 will be described with reference to FIG.

  FIG. 15 is a diagram showing an example of a search result display screen according to the first embodiment of the present invention.

  FIG. 15 is an example of a search result display screen in which search results are displayed on the CRT according to the display contents described above. This is a screen example when the window system is used in a dedicated application. However, when displaying on a small screen such as a video camera, it is desirable to perform a display method according to the screen size.

  Reference numeral 161 denotes a title bar, which is a part for performing title display on the search result display screen and operation of the entire screen such as movement and size change. Reference numerals 162 and 163 denote buttons for providing functions related to the search result display screen, 162 denotes a help display button, and 163 denotes a button for closing the search result display screen.

  Reference numeral 164 denotes a display area for displaying a search condition input by the user, the number of search results, and the like. In the first embodiment, the keywords used as search conditions and the number of data obtained as search results are displayed.

  Reference numerals 165 and 166 denote display areas for displaying search results. In the display area 165, the image data is reproduced according to the display contents determined based on the data analysis result. The display area 166 displays text information such as a title, description, and comments.

  Reference numeral 167 denotes a playback button for instructing the start of playback of image data. Reference numeral 168 denotes a stop button for instructing to stop reproduction of the image data. Reference numeral 169 denotes an end button for ending the display of the search result display screen.

  In the first embodiment, an analysis rule and a display rule are defined in units of events, which is an example suitable for analyzing and displaying the content of home video content and the like. However, according to the present invention, it is possible to cope with content other than video easily by selecting a unit serving as a reference according to the content.

  Next, an overview of the entire processing executed in the first embodiment will be described with reference to FIG.

  FIG. 16 is a flowchart showing an overview of the entire processing executed in the first embodiment of the present invention.

  First, in step S2201, search result analysis processing is executed. The details are as shown in FIG.

  In step S2202, display content determination processing is executed based on the analysis result in step S2201. The details are as shown in FIG.

  Next, in step S2203, search result display processing is executed according to the display content determined in step S2202.

  As described above, according to the first embodiment, when the search result is displayed, the data obtained as the search result is analyzed, and the display content is dynamically determined based on the analysis result and displayed. . This makes it possible to display data that seems to be necessary for the user in an easy-to-understand manner according to the relevance of the data type and content, and is necessary even when searching a large amount of information such as collecting information. Information can be easily searched, selected, organized and used.

  In the first embodiment, on the basis of the result of analyzing the data, in particular, the display contents are determined by determining the configuration and constituent elements of the search result, the expansion method (presentation order of the constituent elements), and the connecting method of the constituent elements. However, it can also be determined according to the type and content of data to be searched, the analysis method, and the like.

[Embodiment 2]
In the second embodiment, a case will be described in which retrieved document data is effectively displayed.

<Search process>
As in the first embodiment, the search is performed using the metadata assigned to the search target. In the second embodiment, the metadata can be provided with character information extracted by analyzing document data, in addition to extracting and assigning manpower and media feature quantities. In addition, information such as the creation date and time and the creator can be given by extracting information automatically given by the OS.

  First, an example of the search condition setting screen according to the second embodiment will be described with reference to FIG.

  FIG. 17 is a diagram showing an example of a search condition setting screen according to the second embodiment of the present invention.

  This search condition setting screen is an example of a screen when a window system is used, and it is assumed that document data on a computer network is searched and the result is displayed. This search condition setting screen is a screen example when a window system is used in a dedicated application, but a similar function may be provided by a Web browser or the like.

  Reference numeral 171 denotes a title bar, which is a part for displaying the title of the search condition setting screen and for operating the entire screen such as movement and size change. Reference numerals 172 and 173 are buttons for providing functions related to the search condition setting screen, 172 is a help display button, and 173 is a button for closing the search condition setting screen.

  Reference numeral 174 denotes an input area for inputting a search condition for performing a search. The input area 174 is a text input area for inputting a keyword. A plurality of keywords are separated by a delimiter (such as a space) and input, and either a logical operation AND or OR can be designated by a radio button. It has become. In addition to keywords, phrases and sentences can be entered.

  As other search conditions, the number of search results to be displayed on one screen and the language in which information to be searched is described are specified.

  Although not shown in the second embodiment, importance or similarity distances of a plurality of inputted keywords and phrases may be set, and as other search conditions, a date range or a search target is set. Information such as a position may be designated.

  Reference numeral 175 denotes an OK button. When this OK button 175 is operated, the content set in the input area 174 as a search condition is finally confirmed, and the process proceeds to the next process. On the other hand, 176 is a cancel button. When the cancel button 176 is operated, the input contents in the input area 174 are discarded.

  In the search condition setting screen of FIG. 17, the user arbitrarily inputs a keyword, phrase, sentence, etc., and the search condition is specified, but the user specifies the genre, category, etc. to which the information belongs. You can also search from within. For example, the category type and genre provided in the second embodiment may be displayed in a list or tree, and the user may select a desired one.

  When the keyword “notebook PC” is input in the input area 174 in FIG. 17, related information is searched using metadata assigned to each data to be searched.

  Here, an example of metadata will be described with reference to FIG.

  FIG. 18 is a diagram illustrating an example of metadata according to the second embodiment of the present invention.

  FIG. 18 shows that a title, a creator, a creation date, a category, a keyword, and the like are assigned to each data as metadata. Here, an example is shown in which character information that can be handled as simple keywords is mainly stored, but information other than character information such as feature amounts according to each data characteristic can also be added as metadata. is there.

  In FIG. 18, as an example, in a certain document data A, “AAA note price information” as a title, “Information Taro” as a creator, “2003-04-01T08: 30” as a creation date, and “Economy-Enterprise— Computer-product-price "," notebook PC "as a keyword, and product name" AAA notebook "are assigned. These metadata are preserve | saved at storage media, such as DISK103, ROM101, RAM102, for example.

  In the second embodiment, character information is given as a search query, and a search is performed according to the character information. Images, sounds, etc. are designated as a search query, and the media feature amount extracted from the search query is compared with the media feature amount of each data. You can also search by doing this. Further, the search may be performed by associating the media feature amount with the character information given as the search query.

  In the second embodiment, the search is performed using the metadata assigned to each data. However, the search can be performed by directly analyzing the document data.

  As described above, data such as character information stored in the database on the storage medium can be searched.

<Display search results>
Next, a method for effectively displaying the search result will be described in detail.

  In the second embodiment, document data as a search result is analyzed, and effective display contents are determined according to the analysis result.

  The basic processing procedure is the same as that of the first embodiment, and the type of data to be searched, the analysis rule for analyzing the search result, and the display rule for determining the display contents are different from those of the first embodiment.

1. Analysis of Search Results Analysis of search results is performed by analyzing data obtained as search results by using data analysis rules stored in a storage medium such as DISK 103, ROM 101, and RAM 102, as in the first embodiment. To realize. In the second embodiment, a category related to a search query is estimated from a search query specified at the time of search, and an analysis rule corresponding to the category is selected.

  Next, as in the first embodiment, multimedia data obtained as a search result is classified according to an analysis rule having a hierarchical structure of categories as shown in FIG. For example, document data acquired through a computer network such as the Internet can generally be classified into categories such as economy, politics, art, and religion. The category of economy is corporate, finance, real estate, shopping, investment, etc. Further, the category of companies can be composed of smaller categories such as industry types such as electrical machinery, automobiles, and construction, product information, service information, and the like.

  In the second embodiment, each piece of data obtained as a search result is categorized according to the above-described hierarchical structure using metadata attached to document data as shown in FIG. For example, in FIG. 18, category information is given as metadata in advance according to the hierarchical structure of FIG. 5, and can be classified into categories using this.

  In addition, a keyword is assigned to each classification element of the hierarchical structure in FIG. 5, and what kind of search result is obtained by comparing this with metadata such as a keyword and a title assigned to each document data. The contents can be estimated and classified into categories.

  Further, a category or the like related to the search query may be estimated from the search query, and classification may be performed only for the category belonging to the lower layer.

  In the second embodiment, the data may be classified according to common factors such as the data type of data obtained as a search result, the data management structure (directory, folder, URI / URL, etc.), and the like.

  When classifying by data type, for example, it is classified by media type such as text, moving image, still image or the like, or even in a document, whether only text or character information and image are mixed. This may be preliminarily assigned with a media type as one of metadata, but can also be estimated from a file name (extension). In the case of an HTML document or an XML document, by analyzing a tag or text described in the document, it can be estimated whether the document is composed only of text or an image is embedded in the document. .

  In the case of a still image, it may be analyzed for whether it is a natural image or chart data and used for classification. This can also be applied in advance as metadata, or can be estimated by image feature quantities such as edges.

  When classifying based on the data management structure or the positional relationship in which the data is stored, for example, the classification is based on a hierarchical structure of locations based on directories, folders, URI / URLs, network addresses, and the like.

  In addition, it is possible to classify by analyzing the media feature amount of each content such as an image or sound, and the similarity or clustering technology may be used for the classification.

  In the second embodiment, as in the first embodiment, an association rule for associating individual data obtained as a search result and a group of data as a classification result is held as an analysis rule.

  Here, an example of the association rule will be described with reference to FIG.

  FIG. 19 is a diagram showing an example of an association rule based on the meaning / content of data according to the second embodiment of the present invention.

  FIG. 19 shows an association rule for a search result obtained from a search query belonging to the category “computer-product”.

  For example, the category “product information” and its subcategory “product specification” have a relationship of outline and details, and whole and part. In addition, the category “accessory” is complementary to the category “product specification”.

  In the second embodiment, as in the first embodiment, the association is performed according to the association rule in FIG. 19 by using the metadata attached to the data as shown in FIG. Further, the relationship between before and after may be performed according to the data creation date and update date.

  In the second embodiment, the analysis is performed by using the metadata assigned to each data. However, the document data is directly analyzed, and the media feature amount of each content such as an image or sound is also analyzed. By doing so, the correlation can be found.

  Further, as in the first embodiment, the data obtained as a search result is classified, and individual data and a group of data as a classification result are related (FIG. 7).

  In the second embodiment, the search result analysis processing by the search result analysis processing unit 210 is the same as that of the first embodiment except that the analysis rule to be used is different.

2. Determination of Display Contents Also in the second embodiment, the display contents of search results are determined based on the obtained analysis results. Here, in the second embodiment, the flow of the display content determination processing by the display content determination processing unit 212 is as shown in FIG. 9 as in the first embodiment, but each display rule for determining the display content is different. Different.

  Therefore, with reference to FIG. 9, the display content determination processing by the display content determination processing unit 212 in the second embodiment will be described.

  First, in step S1401, the elements and configuration to be displayed are determined from the search results. In the second embodiment, the display elements indicate document data obtained as search results, and are obtained at the time of search. Those having a high degree of similarity are determined to be particularly important elements, and this is set as a central element. Further, the data block in the second embodiment is a group of a plurality of document data representing categories and genres determined by classification.

  Next, in step S1402, the display element determined in step S1401 and the method of expanding the data block including the display element (presentation order) are determined using FIG. 12 as in the first embodiment.

  For example, in the second embodiment, the category “accessory” has a supplementary relationship with the category “product specification”.

  Next, in step S1403, based on the relationship and development method of each component, a connection method for connecting the components so that the division of each component, the relationship with the central element, and the relationship with other components are clarified. decide. Further, when the development is difficult to understand, an explanatory element for further helping the user to understand is required.

  In the second embodiment, for example, a break of each component is clearly shown by a line feed or a blank. In addition, a block of data may be indicated by surrounding it with a frame. If the relationship is not clear, use a design such as indentation, symbols, or colors to show these relationships. If the flow cannot be understood naturally, an explanation element for explaining the contents is required, such as an item name and a subtitle.

  FIG. 20 is a diagram showing display contents corresponding to each connection ID according to the second embodiment of the present invention as display rules.

  In FIG. 20, the display contents differ depending on whether the elements to be connected are individual data, paragraphs (paragraphs), or chapters. From FIG. 13 and FIG. 20, for example, when displaying data and paragraphs that are related to each other consecutively from front to back, they are connected by displaying “→” symbols, numbers, etc., and chapters are continuously displayed from front to back. It can be seen that it is better to connect by line feed and number display.

  In addition, when data or paragraphs having the opposite relationship are displayed continuously, it is necessary to clearly indicate the relationship by displaying a “⇔” symbol or the like. Furthermore, in the case of chapters, it can be seen that explanation elements such as subtitles are required.

  Also, if the number of search results is large or the amount of data displayed on the screen is large and it is difficult to see on one screen, the search results are composed of multiple screens, and the screen is changed by hyperlinks, buttons, etc. Or a paragraph break or interrelationship with other components. The screen transition method can be determined based on the block configuration and development, the interrelationship between each component, the number of elements that can be displayed on one screen, and the like.

  In addition, an appropriate screen transition method varies depending on a device for displaying a search result such as a PDA, a type of data to be searched, and the like. For example, when the hierarchical relationship is clearly indicated by screen transition, the screen transition is determined according to the hierarchical structure, and the elements of the lower hierarchy are indicated by screen switching. In addition, when the data has a clear order such as a context and it is difficult to grasp the contents unless it is viewed in an appropriate order, the following information may be limited and shown according to the above-described development.

  Next, an example of the search result display screen of the second embodiment will be described with reference to FIG.

  FIG. 21 is a diagram showing an example of a search result display screen according to the second embodiment of the present invention.

  FIG. 21 is an example of a search result display screen in which search results are displayed on the CRT according to the display contents described above. This is an example of a screen when a window system is used in a dedicated application. However, when displaying on a small screen such as a mobile phone or a PDA, it is desirable to perform a display method according to the screen size.

  Reference numeral 231 denotes a title bar, which is a portion for performing title display on the search result display screen and operations for the entire screen such as movement and size change. Reference numerals 232 and 233 are buttons for providing functions related to the search result display screen, 232 is a help display button, and 233 is a button for closing the search result display screen.

  Reference numeral 234 denotes a display area for displaying the search conditions input by the user, the number of search results, and the like. In the second embodiment, a keyword used as a search condition and the number of data obtained as a search result are displayed.

  Reference numeral 235 denotes a display area for displaying search results. The display area 235 displays display contents determined based on the data analysis result. In the second embodiment, first, “AAA”, “BBB”, and “CCC”, which are character strings representing the contents of data chunks, are displayed. Since there is an order in this hierarchy, a number indicating the order is displayed.

  Next, “AAA-a1”, “AAA-a2”, and “AAA-a3”, which are character strings representing the contents of data chunks, are displayed for the layer below “AAA”. Since these are in an equal or parallel relationship, “●” is added to the head.

  Next, as the search results belonging to “AAA-a1”, the document names “AAA-a11” to “AAA-a15” are the search results belonging to “AAA-a2”, and the document names “AAA-a21”, “AAA”. -A22 "is displayed. In the same way, the document names as search results are displayed for the other layers.

  Reference numeral 236 denotes a button for terminating the display of the search result display screen.

  In the example of FIG. 21, the search result is displayed on one screen. However, as described above, the search result may be composed of a plurality of screens, and the screen may be changed. Further, an HTML document may be generated according to the display contents described above and displayed on a Web browser.

  Next, an example of a search result display screen displayed using a Web browser will be described with reference to FIG.

  FIG. 22 is a diagram showing an example of a search result display screen according to the second embodiment of the present invention.

  FIG. 22 shows an example of a search result display screen in which search results are displayed on the CRT using a Web browser. In FIG. 22, instead of displaying the search results on one screen, the search results are configured by a plurality of screens by linking them.

  Reference numeral 241 denotes a title bar, which is a portion for performing title display on the search result display screen and operation of the entire screen such as movement and size change. Reference numerals 242 and 243 are buttons for providing functions related to the search result display screen, 242 is a help display button, and 243 is a button for closing the search result display screen.

  Reference numeral 244 denotes a display area for displaying the search conditions input by the user, the number of search results, and the like. In the second embodiment, keywords used as search conditions and the number of data obtained as search results are displayed.

  Reference numeral 245 denotes a display area for displaying search results. The display area 245 displays the display content determined based on the data analysis result. In the second embodiment, first, “AAA”, “BBB”, and “CCC”, which are character strings representing the contents of data chunks, are displayed. Since this hierarchy has an order, a number indicating the order is displayed. Yes.

  Next, “AAA-a1”, “AAA-a2”, and “AAA-a3”, which are character strings representing the contents of data chunks, are displayed for the layer below “AAA”. “AAA-a1” to “AAA-a3” are underlined (anchors), indicating that other HTML documents are linked. When the user clicks on the character string, the screen jumps to the search result display screen belonging to each.

  As described above, according to the second embodiment, the same effect as that of the first embodiment can be obtained for document data having character information.

[Embodiment 3]
In the third embodiment, an application example of the first embodiment will be described.

  First, an example of the search condition setting screen in the third embodiment will be described with reference to FIG.

  FIG. 23 is a diagram showing an example of a search condition setting screen according to the third embodiment of the present invention.

  In FIG. 23, the same components as those in FIG. 3 of the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

  Reference numeral 34a denotes an input area for inputting a search condition for performing a search. In addition to the configuration of the input area 34 in FIG. 3, the input area 34 a has four stages of display time of “long”, “medium”, “short”, and “highlight” as display conditions when displaying the search result. Can be specified with a checkbox.

  Next, display content determination processing by the display content determination processing unit 212 in the search condition setting screen of the third embodiment will be described. In the third embodiment, the flow of the display content determination process is as shown in FIG. 9 as in the first embodiment, but the display target element and configuration are determined from the search result based on the analysis result in step S1401. Different configuration.

  Hereinafter, details of the processing in step S1401 of the third embodiment will be described with reference to FIG.

  FIG. 24 is a flowchart showing details of the process in step S1401 according to the third embodiment of the present invention.

  In FIG. 24, the same processing as that in FIG. 10 of the first embodiment is denoted by the same step number, and detailed description thereof is omitted.

  After the processes in step S1501 and step S1502, a supplementary element is selected. The method for selecting the supplemental element differs depending on display conditions for performing display. For example, when a summary is displayed with highlights or the like, a supplementary element is not selected, and only a central element is configured. Further, supplementary elements to be selected differ depending on display conditions such as the overall data size and display time.

  Accordingly, in step S1513, display conditions for displaying the search results are determined. In the third embodiment, the user designates the entire display time in four stages of “long”, “medium”, “short”, and “highlight” at the time of search. Decide.

  First, in step S1513a, it is determined whether or not the display time is “long”. If the display time is “long” (YES in step S1513a), in step S1514, an element having a relationship with the central element in each block is selected as a supplementary element. Next, in step S1515, an element related to the supplementary element of the central element is also selected as the supplementary element. Elements that are not related to any central element are not selected as supplementary elements.

  On the other hand, if the display time is not “long” (NO in step S1513a), it is determined in step S1513b whether the display time is “medium”. If the display time is “medium” (YES in step S1513b), in step S1516, an element directly related to the central element in each block is selected as a supplementary element. Further, when the central element has a direct relationship with a plurality of supplementary elements, the element having the highest priority of the relationship with the central element may be selected. Next, in step S1517, an element related to the supplementary element of the central element is determined depending on whether the priority of the relationship with the supplementary element is high, or whether the importance can be determined by similarity or metadata. To do.

  On the other hand, if the display time is not “medium” (NO in step S1513b), it is determined in step S1513c whether the display time is “short”. When the display time is “short” (YES in step S1513c), in step S1518, the priority of the relationship with the central element among the elements directly related to the central element in each block is set. Elements that are determined to be high or important by similarity or metadata are selected as supplementary elements.

  On the other hand, if the display time is not “short” (NO in step S1513c), that is, if the display time is “highlight”, the process ends.

  The above-described central element or supplementary element is an element to be displayed, and this display element and a block of data including the display element are used as a component for displaying the search result. Here, the central element may not be related to any element.

  In the case of the third embodiment, FIG. 11 is an example of the display elements and configuration determined in step S1504, and the relationship between them. In addition, the presentation order determination method in step S1402 of FIG. 9 differs depending on display conditions such as the overall data size and display time.

  In the case of the third embodiment, in FIG. 12, when the presentation order is determined, the display element and the data block expansion method are determined according to the display conditions. For example, when the display time is “short” or “highlight”, the presentation order is determined so that the priority of the relationship is as strong as possible, thereby eliminating the need for connected elements and explanatory elements as much as possible.

  As described above, according to the third embodiment, in addition to the effects described in the first embodiment, based on the display conditions, data that is considered necessary for the user is related to the relevance such as the data type and contents. Accordingly, the information can be displayed in an easily understandable manner, and necessary information can be easily searched, selected, organized, and used even when a large amount of information is retrieved, such as information collection.

[Embodiment 4]
The fourth embodiment is an application example of the second embodiment under the configuration of the third embodiment.

  First, an example of the search condition setting screen in the fourth embodiment will be described with reference to FIG.

  FIG. 25 is a diagram showing an example of a search condition setting screen according to the fourth embodiment of the present invention.

  In FIG. 25, the same components as those in FIG. 17 of the second embodiment are denoted by the same reference numerals, and detailed description thereof is omitted.

  Reference numeral 174a denotes an input area for inputting a search condition for performing a search. In addition to the configuration of the input area 174 in FIG. 17, the input area 174 a displays the data size of the entire search result as “large”, “medium”, “small”, and “summary” as display conditions when displaying the search result. It can be specified with the check box in four stages.

  When this search condition setting screen is used, the processing of FIG. 24 is performed based on the entire data size (for example, “large”, “medium”, “small”) which is one of the display conditions. Decide how to select supplemental elements.

  As described above, according to the fourth embodiment, the same effect as that of the third embodiment can be obtained for document data having character information.

[Embodiment 5]
In the above embodiment, metadata such as keywords is assigned to multimedia data, and this is used for search and analysis of search results. However, what is used for search and analysis of search results? A method according to the type / characteristic of each multimedia data and the purpose of the user can be used.

  In the above embodiment, the DISK 103, the ROM 101 and the RAM 102 in FIG. 1, or the storage medium such as a memory card, the metadata shown in FIGS. 4 and 18, and the analysis rules and diagrams shown in FIGS. Although the display rules shown in FIGS. 12 to 14 and 20 are stored, these pieces of information can be acquired from an external device on the network, stored in a storage medium, and used.

  Although the embodiments have been described in detail above, the present invention can take an embodiment as, for example, a system, an apparatus, a method, a program, or a storage medium, and specifically includes a plurality of devices. The present invention may be applied to a system that is configured, or may be applied to an apparatus that includes a single device.

  In the present invention, a software program (in the embodiment, a program corresponding to the flowchart shown in the figure) that realizes the functions of the above-described embodiment is directly or remotely supplied to the system or apparatus, and the computer of the system or apparatus Is also achieved by reading and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, or the like.

  As a recording medium for supplying the program, for example, floppy (registered trademark) disk, hard disk, optical disk, magneto-optical disk, MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card ROM, DVD (DVD-ROM, DVD-R) and the like.

  As another program supply method, a client computer browser is used to connect to an Internet homepage, and the computer program of the present invention itself or a compressed file including an automatic installation function is downloaded from the homepage to a recording medium such as a hard disk. Can also be supplied. It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  In addition, the program of the present invention is encrypted, stored in a storage medium such as a CD-ROM, distributed to users, and key information for decryption is downloaded from a homepage via the Internet to users who have cleared predetermined conditions. It is also possible to execute the encrypted program by using the key information and install the program on a computer.

  In addition to the functions of the above-described embodiments being realized by the computer executing the read program, the OS running on the computer based on an instruction of the program is a part of the actual processing. Alternatively, the functions of the above-described embodiment can be realized by performing all of them and performing the processing.

  Furthermore, after the program read from the recording medium is written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer, the function expansion board or The CPU or the like provided in the function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are realized by the processing.

It is a block diagram which shows the basic composition of the computer apparatus with which the multimedia data search system of Embodiment 1 of this invention is constructed | assembled. It is a figure which shows the function structure of the multimedia data search display system of Embodiment 1 of this invention. It is a figure which shows an example of the search condition setting screen of Embodiment 1 of this invention. It is a figure which shows an example of the metadata of Embodiment 1 of this invention. It is a figure which shows an example of the analysis rule which consists of hierarchical structure of Embodiment 1 of this invention. It is a figure which shows an example of the association rule by the meaning and content of the image data of Embodiment 1 of this invention. It is a figure which shows an example which classify | categorizes the data obtained as a search result of Embodiment 1 of this invention, and performs correlation of the group of the data which is individual data and a classification result. It is a flowchart which shows the search result analysis process by the search result analysis process part of Embodiment 1 of this invention. It is a flowchart which shows the display content determination process by the display content determination process part of Embodiment 1 of this invention. It is a flowchart which shows the detail of a process of step S1401 of Embodiment 1 of this invention. It is a figure which shows an example of the display element of Embodiment 1 of this invention, a structure, and its relationship. It is a figure which shows an example of the presentation order of Embodiment 1 of this invention. It is the figure which defined connection ID which shows the connection method with respect to the mutual relationship of the component of Embodiment 1 of this invention, and a presentation order. It is the figure which showed the display content corresponding to each connection ID of Embodiment 1 of this invention as a display rule. It is a figure which shows an example of the search result display screen of Embodiment 1 of this invention. It is a flowchart which shows the outline | summary of the whole process performed in Embodiment 1 of this invention. It is a figure which shows an example of the search condition setting screen of Embodiment 2 of this invention. It is a figure which shows an example of the metadata of Embodiment 2 of this invention. It is a figure which shows an example of the association rule by the meaning and content of the data of Embodiment 2 of this invention. It is the figure which showed the display content corresponding to each connection ID of Embodiment 2 of this invention as a display rule. It is a figure which shows an example of the search result display screen of Embodiment 2 of this invention. It is a figure which shows an example of the search result display screen of Embodiment 2 of this invention. It is a figure which shows an example of the search condition setting screen of Embodiment 3 of this invention. It is a flowchart which shows the detail of a process of step S1401 of Embodiment 3 of this invention. It is a figure which shows an example of the search condition setting screen of Embodiment 4 of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 201 Search query input process part 202 Search holding part 203 Search processing part 204 Search result holding part 205 Metadata holding part 206 Multimedia data holding part 207 Analysis rule selection processing part 208 Analysis rule holding part 209 Decision analysis rule holding part 210 Search result Analysis processing unit 211 Analysis result holding unit 212 Display content determination processing unit 213 Display content holding unit 214 Search result holding unit 215 Display rule holding unit 216 Display rule selection processing unit 217 Determination display rule holding unit

Claims (44)

An information processing apparatus for searching multimedia data composed of at least one of document data, image data, or audio data, or a combination thereof,
Storage means for storing the multimedia data to be searched;
Input means for inputting search conditions for searching for the multimedia data;
Search means for searching multimedia data from the storage means based on the search condition;
Analysis rule holding means for holding analysis rules for analyzing multimedia data obtained as a search result by the search means;
An analysis unit that selects an analysis rule corresponding to the search condition from the analysis rule holding unit, and analyzes the multimedia data obtained as a search result by the search unit according to the selected analysis rule;
Display rule holding means for holding display rules for determining the contents to be displayed as the search results;
Display content for selecting a display rule corresponding to the search condition from the display rule holding unit, and determining display content to be displayed as the search result based on the selected display rule and the analysis result of the analysis unit A determination means;
An information processing apparatus comprising: a search result display unit that displays the search result according to the display content determined by the display content determination unit. The analyzing means classifies the multimedia data obtained as a search result by the search means according to the meaning and content thereof;
A multimedia data obtained as a search result by the search means, and a correlation means for performing a correlation for a group of multimedia data classified by the classification means;
2. The information processing according to claim 1, wherein the display content determination unit determines display content to be displayed as the search result based on a classification result by the classification unit and a correlation result by the correlation unit. apparatus. The classification means classifies the multimedia data obtained as a search result by the search means according to a hierarchical structure,
The association means relates the multimedia data obtained as a search result by the search means and the group of multimedia data hierarchically classified according to the hierarchical structure by the classification means. The information processing apparatus according to claim 2. The analysis means performs analysis when storing the multimedia data as a search target in the storage means, and associates the analysis result with the multimedia data as metadata,
The information processing apparatus according to claim 2, wherein the display content determination unit determines display content to be displayed as the search result based on metadata associated with the multimedia data. The display rule includes a definition for determining a presentation order according to a relationship between the multimedia data,
The said display content determination means determines the said presentation order of the multimedia data obtained as a search result of the said search means based on the correlation result of the said correlation means. Information processing device. The display rule includes a definition of a priority with respect to a relationship between the multimedia data and a presentation order,
The display content determination means includes a group of multimedia data classified by the classification means, a multimedia data obtained as a search result of the plurality of search means, and a group of multimedia data classified by the classification means; 6. The information processing apparatus according to claim 5, wherein if there is a relationship, the presentation order of multimedia data obtained as a search result of the search means is determined based on the priority. The multimedia data is still image data or moving image data,
The search result display unit continuously reproduces and displays still image data or moving image data as the search result according to the display content determined by the display content determination unit. Information processing device. The classifying unit classifies the still image or the moving image data, which is multimedia data obtained as a search result by the searching unit, according to a hierarchical structure in which a common factor to which the still image or the moving image data belongs is a unit of the hierarchy. And
The association means performs association between multimedia data obtained as a search result by the search means and a group of multimedia data hierarchically classified according to the hierarchical structure. The information processing apparatus described in 1. The display rule further includes a definition for connecting the multimedia data and other multimedia data according to the type of the multimedia data.
The display content determination means determines the presentation order of the multimedia data obtained as the search result of the search means based on the correlation result of the correlation means, and refers to the display rule, The information processing apparatus according to claim 5, wherein a connection method for connecting multimedia data is determined. The display content determination means further determines a connection method for connecting the groups based on the presentation order and relationship of the groups of multimedia data classified by the classification means. Information processing device. The classification means classifies the multimedia data obtained as a search result by the search means according to a hierarchical structure,
The association means relates the multimedia data obtained as a search result by the search means and the group of multimedia data hierarchically classified according to the hierarchical structure by the classification means. The information processing apparatus according to claim 10. The analysis means performs analysis when storing the multimedia data as a search target in the storage means, and associates the analysis result with the multimedia data as metadata,
The information processing apparatus according to claim 10, wherein the display content determination unit determines display content to be displayed as the search result based on metadata associated with the multimedia data. The display content determination means is connected in such a way that the division of multimedia data obtained as a search result of the search means or the division of a group of multimedia data classified by the classification means is clarified as the connection method. The information processing apparatus according to claim 10, wherein a connection method to be determined is determined. The display content determination means, as the connection method, further clarifies the relation with other related multimedia data and the relation with other related multimedia data groups classified by the classification means. The information processing apparatus according to claim 13, wherein a connection method to be connected is determined. The display content determination means may further determine whether the multimedia data obtained as a search result of the search means and the group of multimedia data classified by the classification means are displayed in succession as the connection method. The information processing apparatus according to claim 10, wherein a connection method to be connected is determined depending on whether or not. The display content determining means, as the connection method, further determines a connection method for connection according to a hierarchy of groups of multimedia data classified hierarchically according to the hierarchical structure by the classification means. The information processing apparatus according to claim 11. The multimedia data is still image data or moving image data,
The search result display unit continuously reproduces and displays still image data or moving image data as the search result according to the display content determined by the display content determination unit. Information processing device. The classifying unit classifies the still image or the moving image data, which is multimedia data obtained as a search result by the searching unit, according to a hierarchical structure in which a common factor to which the still image or the moving image data belongs is a unit of the hierarchy. And
18. The association means performs an association between multimedia data obtained as a search result by the search means and a group of multimedia data classified hierarchically according to the hierarchical structure. The information processing apparatus described in 1. An identification unit for identifying importance of the multimedia data obtained as a search result of the search unit;
The display content determination means selects a display rule corresponding to the search condition from the display rule holding means, the selected display rule, the analysis result of the analysis means, and the important information identified by the identification means The information processing apparatus according to claim 1, wherein display content to be displayed as the search result is determined based on a degree. The analyzing means classifies the multimedia data obtained as a search result by the search means according to the meaning and content thereof;
A multimedia data obtained as a search result by the search means, and a correlation means for performing a correlation for a group of multimedia data classified by the classification means;
The information processing according to claim 19, wherein the display content determination means determines display contents to be displayed as the search result based on a classification result by the classification means and a correlation result by the correlation means. apparatus. The classification means classifies the multimedia data obtained as a search result by the search means according to a hierarchical structure,
The association means relates the multimedia data obtained as a search result by the search means and the group of multimedia data hierarchically classified according to the hierarchical structure by the classification means. The information processing apparatus according to claim 20. The analysis means performs analysis when storing the multimedia data as a search target in the storage means, and associates the analysis result with the multimedia data as metadata,
The information processing apparatus according to claim 20, wherein the display content determination unit determines display content to be displayed as the search result based on metadata associated with the multimedia data. The identification means identifies the importance of the multimedia data using at least one of the similarity obtained at the time of the search by the search means and the importance associated with the multimedia data, or a combination thereof. The information processing apparatus according to claim 20. The display rule includes a definition for determining a presentation order according to a relationship between the multimedia data,
The display content determination means is a multimedia obtained as a search result by the search means based on the classification result of the classification means, the correlation result of the correlation means, and the importance level identified by the identification means. Determining the multimedia data to be displayed and its configuration from the data, and further determining the presentation order of the multimedia data to be displayed based on the correlation result of the correlation means and the importance. The information processing apparatus according to claim 20, characterized in that: The display content determination means selects, as a display element, multimedia data that is the center of the multimedia data based on the importance of the multimedia data obtained as a search result of the search means,
Based on the classification result of the classification means, a group of multimedia data including the central multimedia data is determined, and further, based on the correlation result of the correlation means, the center is selected from the groups. 25. The information processing apparatus according to claim 24, wherein multimedia data related to the multimedia data to be selected is selected as a display element. In the case where the display content determination means selects multimedia data related to the central multimedia data from the groups as a display element, the display content determination means further includes a search result of the multimedia data obtained as a search result of the search means. The information processing apparatus according to claim 25, wherein selection is performed based on importance. The display rule includes a definition of a priority with respect to a relationship between the multimedia data and a presentation order,
The display content determination means includes a group of multimedia data including the selected display element and the central multimedia data, and a plurality of display elements or a group of multimedia data including the central multimedia data. 25. The information according to claim 24, wherein when there is a relationship, the presentation order of the multimedia data to be displayed is determined based on the priority, or the priority and the importance of the multimedia data. Processing equipment. The multimedia data is still image data or moving image data,
The search result display unit continuously reproduces and displays still image data or moving image data as the search result according to the display content determined by the display content determination unit. Information processing device. The classifying unit classifies the still image or the moving image data, which is multimedia data obtained as a search result by the searching unit, according to a hierarchical structure in which a common factor to which the still image or the moving image data belongs is a unit of the hierarchy And
29. The association means associates multimedia data obtained as a search result by the search means with a group of multimedia data hierarchically classified according to the hierarchical structure. The information processing apparatus described in 1. Setting means for setting display conditions when displaying the search results;
Identification means for identifying the importance of multimedia data obtained as a search result of the search means;
The display content determination unit selects a display rule corresponding to the search condition from the display rule holding unit, the selected display rule, the analysis result of the analysis unit, and the display set by the setting unit. The information processing apparatus according to claim 1, wherein display content to be displayed as the search result is determined based on a condition and an importance level identified by the identification unit. The analyzing means classifies the multimedia data obtained as a search result by the search means according to the meaning and content thereof;
A multimedia data obtained as a search result by the search means, and a correlation means for performing a correlation for a group of multimedia data classified by the classification means;
The information processing according to claim 30, wherein the display content determination means determines display contents to be displayed as the search result based on a classification result by the classification means and a correlation result by the correlation means. apparatus. The classification means classifies the multimedia data obtained as a search result by the search means according to a hierarchical structure,
The association means relates the multimedia data obtained as a search result by the search means and the group of multimedia data hierarchically classified according to the hierarchical structure by the classification means. The information processing apparatus according to claim 31. The analysis means performs analysis when storing the multimedia data as a search target in the storage means, and associates the analysis result with the multimedia data as metadata,
The information processing apparatus according to claim 31, wherein the display content determination unit determines display content to be displayed as the search result based on metadata associated with the multimedia data. The identification means identifies the importance of the multimedia data using at least one of the similarity obtained at the time of the search by the search means and the importance associated with the multimedia data, or a combination thereof. 32. The information processing apparatus according to claim 31. 32. The information processing apparatus according to claim 31, wherein the display condition is at least one of a data size and a display time of the entire search result to be displayed, or a combination thereof. The display rule includes a definition for determining a presentation order according to a relationship between the multimedia data,
The display content determination means is based on the classification result of the classification means, the correlation result of the correlation means, the display condition set by the setting means, and the importance level identified by the identification means, Among the multimedia data obtained as a search result by the search means, determine the multimedia data to be displayed and its configuration,
The presentation order of the multimedia data to be displayed is determined based on the importance, at least one of the display conditions, or a combination thereof, and an association result of the association means. Item 32. The information processing device according to Item 31. The display content determination means selects, as a display element, multimedia data that is the center of the multimedia data based on the importance of the multimedia data obtained as a search result of the search means,
Based on the classification result of the classification means, determine a group of multimedia data including the central multimedia data,
Selecting multimedia data related to the central multimedia data from the groups as a display element based on the association result of the association unit and the display condition set by the setting unit. The information processing apparatus according to claim 36. The display content determination means further selects multimedia data obtained as a search result of the search means when selecting multimedia data related to the central multimedia data from the groups as a display element. The information processing apparatus according to claim 37, wherein selection is performed based on importance. The display content determination means does not select, as a display element, multimedia data related to the central multimedia data from the groups when the display condition is summary display. The information processing apparatus described. The display rule includes a definition of a priority with respect to a relationship between the multimedia data and a presentation order,
The display content determination means includes a group of multimedia data including the selected display element and the central multimedia data, and a plurality of display elements or a group of multimedia data including the central multimedia data. The multimedia data to be displayed based on the priority of the multimedia data, at least one of the display conditions set by the setting means, or a combination thereof, and the priority 37. The information processing apparatus according to claim 36, wherein the order of presenting is determined. The multimedia data is still image data or moving image data,
The search result display unit continuously reproduces and displays still image data or moving image data as the search result in accordance with the display content determined by the display content determination unit. Information processing device. The classifying unit classifies the still image or the moving image data, which is multimedia data obtained as a search result by the searching unit, according to a hierarchical structure in which a common factor to which the still image or the moving image data belongs is a unit of the hierarchy. And
42. The association means associates multimedia data obtained as a search result by the search means with a group of multimedia data hierarchically classified according to the hierarchical structure. The information processing apparatus described in 1. Control an information processing apparatus that retrieves multimedia data from a storage unit that stores multimedia data composed of at least one of document data, image data, audio data, or a combination thereof. A control method,
An input step for inputting a search condition for searching the multimedia data;
A search step of searching multimedia data from the storage unit based on the search condition;
An analysis rule holding step for holding an analysis rule for analyzing multimedia data obtained as a search result by the search step in a storage medium;
An analysis step of selecting an analysis rule corresponding to the search condition from the storage medium held in the analysis rule holding step, and analyzing multimedia data obtained as a search result in the search step according to the selected analysis rule When,
A display rule holding step for holding a display rule for determining the content to be displayed as the search result in a storage medium;
The display rule corresponding to the search condition is selected from the storage medium held in the display rule holding step, and the display is displayed as the search result based on the selected display rule and the analysis result of the analysis step A display content determination step for determining the content;
A search result display step of displaying the search result according to the display content determined in the display content determination step. Control an information processing apparatus that retrieves multimedia data from a storage unit that stores multimedia data composed of at least one of document data, image data, audio data, or a combination thereof. A program for realizing control,
A program code of an input process for inputting a search condition for searching the multimedia data;
Based on the search condition, a program code of a search step for searching multimedia data from the storage unit,
A program code of an analysis rule holding step for holding an analysis rule for analyzing multimedia data obtained as a search result by the search step in a storage medium;
An analysis step of selecting an analysis rule corresponding to the search condition from the storage medium held in the analysis rule holding step, and analyzing multimedia data obtained as a search result in the search step according to the selected analysis rule Program code,
A program code of a display rule holding step for holding a display rule for determining the content to be displayed as the search result in a storage medium;
The display rule corresponding to the search condition is selected from the storage medium held in the display rule holding step, and the display is displayed as the search result based on the selected display rule and the analysis result of the analysis step Program code of the display content determination process for determining the content,
And a program code of a search result display step for displaying the search result according to the display content determined in the display content determination step.

Images