CN101853253A - Device and method for managing multimedia content in mobile terminal - Google Patents

Abstract

Provided are an apparatus and method for managing multimedia content in a mobile terminal. The device includes: a data acquisition unit, used to convert multimedia content into text data, and extract corresponding keyword values from the converted text data according to predetermined keywords reflecting key attributes of the multimedia content; a database establishment unit, used to Establish the multimedia content database based on the keyword value obtained by the data acquisition unit; the storage unit is used to store the multimedia content database established by the database establishment unit; the query input unit is used to identify the query information input by the user as text data, and from extracting a key search term from the identified text data; a search unit for searching multimedia content related to the search term from a multimedia content database stored in the storage unit based on the search term extracted by the query input unit; and a search result output unit, Used to output the search results of the search unit to the user.

Description

Device and method for managing multimedia content in mobile terminal

technical field

The present invention relates to a device and method for managing multimedia content in a mobile terminal, more specifically, the present invention relates to a device for establishing a multimedia content database for managing and querying multimedia content by uniformly converting multimedia content into text data and methods.

Background technique

With the development of communication technology and digital signal processing technology, mobile terminals can perform various other functions in addition to basic voice call functions, such as taking photos, sending messages, sending and receiving emails, GPS (Global Positioning System) positioning wait. When using the above functions, a large amount of various multimedia contents or information will be generated and transmitted, such as short messages, emails, photos, voice calls and the like. Since various multimedia contents have respective different data formats, it is difficult to perform unified management of information among different kinds of multimedia contents.

Furthermore, the penetration rate of mobile terminals is so high that mobile terminals have become the main means by which people communicate with each other. Therefore, not only a large amount of contact information is stored in the address book of the mobile terminal, but also various interpersonal relationships between different contacts are reflected in the information of various multimedia contents, for example, in the e-mail sent by A to B C will be mentioned, and the photo of B and C may be stored in B's mobile terminal. In addition, another contact or the like may be mentioned in a voice call or a short message between two contacts. However, the interpersonal relationship embodied in such multimedia content cannot be properly reflected or queried in existing mobile terminals. Therefore, there is a need for a technical solution capable of reflecting the interpersonal relationship among various contacts by managing multimedia content in the mobile terminal.

Contents of the invention

The purpose of the present invention is to provide a device and method capable of managing various media information in a mobile terminal. Through the device and method, various multimedia information is converted into a unified text form and searched based on the unified form. And the interpersonal relationship of each contact person reflected in a large amount of multimedia information can also be well reflected.

According to an aspect of the present invention, there is provided a device for managing multimedia content in a mobile terminal, including: a data acquisition unit, used to convert multimedia content into text data, and according to a predetermined keyword reflecting the key attributes of the multimedia content Extract the corresponding keyword value from the converted text data; the database building unit is used to build the multimedia content database based on the keyword value obtained by the data acquisition unit; the storage unit is used to store the multimedia content database established by the database building unit; The query input unit is used to identify the query information input by the user as text data, and extracts key search terms from the identified text data; the search unit is used to extract the search terms based on the query input unit from the stored in the storage unit. The multimedia content database is searched for multimedia content related to the search word; and the search result output unit is used to output the search result of the search unit to the user.

The data acquisition unit includes: a text conversion unit, which is used to convert the multimedia content into text data; and a text analysis unit, which is used to extract corresponding keywords from the converted text data according to predetermined keywords reflecting the key attributes of the multimedia content value.

The text conversion unit includes at least one of the following items: a short message conversion unit for converting short messages into text data; an e-mail conversion unit for converting e-mails into text data; a voice call conversion unit for The voice call is converted into text data; the scene classification and clustering unit is used to classify photos according to a predetermined category and obtain photos similar to the photo through clustering processing, and record the scene classification and clustering results of the photos as Text data; a facial recognition unit, used to recognize the facial features and total number of people in the photo, and record the result of facial recognition as text data; and an electronic map conversion unit, used to refer to the electronic map to convert the position information of the positioning system Convert to text data.

The text analysis unit includes: a word division unit, which is used to divide the text text output by the text conversion unit into a plurality of words; a part-of-speech determination unit, which is used to determine the parts of speech of a plurality of words divided by the word division unit; a name extraction unit, It is used to extract the name of the person in the text body according to the output result of the part-of-speech determination unit; the time extraction unit is used to extract the time words in the text body according to the output result of the part-of-speech determination unit; the important word extraction unit is used to determine the unit according to the part-of-speech The output result of the text is used to extract important words in the text body; and other keyword value extraction units are used to extract keyword values from information other than the text body output by the text conversion unit.

The device further includes: a time information analyzing unit, configured to convert the time words extracted by the time extracting unit into time information.

The device further includes: a synonym generation unit, configured to generate corresponding synonyms for the important words extracted by the important word extraction unit.

The database building unit builds a multimedia content database based on keywords, keyword values, and related attribute descriptions.

The database establishing unit also establishes a contact database, and the contact records of the contact database are updated along with the multimedia content database.

The query input unit includes: a text recognition unit, which is used to convert the query information input by the user through a keyboard or handwriting pad into text data; a text capture unit and an optical character recognition OCR unit, which are used to capture and recognize character data on the queried photo a facial recognition unit, used to recognize the facial features of each person in the query photo, and record the result of the facial recognition as text data; and a voice recognition unit, used to convert the voice query input by the user into text data.

The search result output unit outputs the search result of the search unit to the user in a visualized manner.

The search unit searches the multimedia content related to the search term from the multimedia content database stored in the storage unit with reference to the contact database based on the search term extracted by the query input unit.

The searching unit searches for mutual relations of contacts in the multimedia content based on the multimedia content database and the contact database.

If the query input unit extracts multiple search terms, the search unit searches for the multiple search terms according to a predetermined logical relationship or a logical relationship set by the user.

According to another aspect of the present invention, a method for managing multimedia content in a mobile terminal is provided, including: converting the multimedia content into text data, and converting the converted text data according to a predetermined keyword reflecting a key attribute of the multimedia content Extract the corresponding keyword value; build a multimedia content database based on the keyword value; store the established multimedia content database; identify the query information input by the user as text data, and extract the key search key from the identified text data searching for multimedia content related to the search term from a stored multimedia content database based on the extracted search term; and outputting a search result to a user.

The step of converting multimedia content into text data includes at least one of the following steps: converting short messages into text data; converting emails into text data; converting voice calls into text data; classifying photos according to predetermined categories and passing Cluster processing to obtain photos similar to the photo, and record the classification and clustering results of the photos as text data; recognize the facial features and total number of people in the photos, and record the recognition results as text data; and The position information of the positioning system is converted into text data with reference to the electronic map.

The step of extracting the corresponding keyword value from the converted text data according to the predetermined keyword reflecting the key attribute of the multimedia content includes: dividing the text text into a plurality of words; determining the parts of speech of the divided words; The part of speech of the word is used to extract the name of the person in the text body; the time words in the text body are extracted according to the part of speech of the divided multiple words; the important words in the text body are extracted according to the part of speech of the divided multiple words; Information other than extracting key values.

In the method, it also includes: converting the extracted time words into time information.

In the method, it also includes: generating corresponding synonyms for the extracted important words.

The multimedia content records in the multimedia content database include keywords, keyword values and related attribute descriptions.

In the method, it also includes: establishing a contact database, and the contact records of the contact database are updated along with the multimedia content database.

The step of recognizing the query information input by the user as text data includes at least one of the following steps: converting the query information input by the user through a keyboard or tablet into text data; capturing and recognizing character data on the queried photo; identifying the query information The facial features of each person in the photo, and record the recognition result as text data; and convert the voice query input by the user into text data.

In the step of outputting the search result to the user, the search result is output to the user in a visualized manner.

In the method, it further includes: based on the extracted search term, referring to the contact database, and searching the stored multimedia content database for multimedia content related to the search term.

In the method, it further includes: searching for mutual relations of contacts in the multimedia content based on the multimedia content database and the contact database.

In the method, further comprising: if a plurality of search words are extracted, searching for the plurality of search words according to a predetermined logical relationship or a logical relationship set by a user.

Description of drawings

The above and/or other objects and advantages of the present invention will become more clear through the following description of the embodiments in conjunction with the accompanying drawings, wherein:

1 is a block diagram illustrating an apparatus for managing multimedia content in a mobile terminal according to an exemplary embodiment of the present invention;

2 is a flowchart illustrating a method of managing multimedia content in a mobile terminal according to an exemplary embodiment of the present invention;

FIG. 3 is a diagram illustrating a detailed structure of a data collection unit in the device for managing multimedia content in a mobile terminal shown in FIG. 1;

Figure 4 shows a photo input to the data acquisition unit shown in Figure 3;

Fig. 5 shows an example of a contact photo stored in a mobile terminal;

6 is a diagram illustrating a detailed structure of a query input unit in the device for managing multimedia content in a mobile terminal shown in FIG. 1; and

FIG. 7 illustrates an example of search results output by a search result output unit in the apparatus for managing multimedia content in a mobile terminal shown in FIG. 1 .

Detailed ways

Reference will now be made in detail to embodiments of the present invention, examples of which are illustrated in the accompanying drawings, wherein like numerals refer to like parts throughout. The embodiments are described below in order to explain the present invention by referring to the figures.

FIG. 1 is a block diagram illustrating an apparatus for managing multimedia content in a mobile terminal according to an exemplary embodiment of the present invention. As shown in FIG. 1 , the device for managing multimedia content in a mobile terminal according to an exemplary embodiment of the present invention includes: a data acquisition unit 10 configured to convert multimedia content into text data, and to reflect the key attributes of the multimedia content according to predetermined The keyword extracts the corresponding keyword value from the converted text data; the database building unit 20 is used to set up the multimedia content database based on the keyword value obtained by the data acquisition unit 10; the storage unit 30 is used to store the database building unit 20 The multimedia content database that builds; Inquiry input unit 40, is used for identifying the query information that the user inputs as text data, and extracts key search word from the text data of recognition; Search unit 50, is used for extracting based on inquiry input unit 40 The search term searches for multimedia content related to the search term from a multimedia content database stored in the storage unit 30; and a search result output unit 60 configured to output the search result of the search unit to the user.

An example of implementing the method for managing multimedia content according to the present invention by using the device for managing multimedia content shown in FIG. 1 will be described below with reference to FIG. 2 .

FIG. 2 is a flowchart illustrating a method of managing multimedia content in a mobile terminal according to an exemplary embodiment of the present invention. With reference to Fig. 2, in step S100, multimedia content is converted into text data by data acquisition unit 10, and extracts corresponding keyword value from the converted text data according to the keyword of predetermined reflection multimedia content key attribute; In step S200, Based on the keyword value that data collection unit 10 obtains by database establishment unit 20, build multimedia content database, and the multimedia content database of establishment is stored in storage unit 30; Recognize as text data, and extract key search word from the text data of recognition; In step S400, search and search from the multimedia content database stored in storage unit 30 by the search word that search unit 50 extracts based on inquiry input unit 40 word-related multimedia content; then, in step S500, the search result output unit 60 outputs the search result of the search unit 40 to the user.

Components and specific operations of the device for managing multimedia content in a mobile terminal shown in FIG. 1 will be described below with reference to FIGS. 3 to 7 .

FIG. 3 is a diagram showing a detailed structure of the data collection unit 10 in the device for managing multimedia content in a mobile terminal shown in FIG. 1 . As an example, the data acquisition unit 10 shown in FIG. 3 may include: a text conversion unit 101, which is used to convert multimedia content into text data; a text analysis unit 102, which is used to convert from Extract the corresponding keyword value from the text data of .

According to an exemplary embodiment of the present invention, the text conversion unit 101 converts various multimedia contents into text data. As an example, for a short message, the text conversion unit 101 includes a short message conversion unit 1011 for converting the short message into text data; for an email, the text conversion unit 101 includes an email conversion unit 1012 for converting the email into Text data; above-mentioned short message conversion unit 1011 and e-mail conversion unit 1012 are designed to convert complete short message or e-mail information into text format data, which includes sender, recipient, CC party, title, text, Sending and receiving time, etc. For voice calls, the text conversion unit 101 includes a voice call conversion unit 1013 for converting voice calls into text data. Existing speech recognition technology can be used to construct the voice call conversion unit 1013. For example, the voice call conversion unit 1013 can recognize the content of the voice call based on the pre-established language model and acoustic model, and convert it into text data, except In addition to the content of the voice call, the voice call conversion unit 1013 also converts the sender, receiver, time, etc. of the voice call into text data; for photos, the text conversion unit 101 includes a scene classification and clustering unit 1014 for Classify the photos and obtain photos similar to the photo through clustering processing, wherein the predetermined category refers to the preset photo category, for example, portrait (single photo, group photo, group photo), landscape (landscape, beach, sky), objects (flowers, still life), etc., and clustering is a common algorithm for pattern recognition, by extracting the features in the photo frame, according to a certain distance measure, to measure the similarity between photos, and Photos with small mutual distances are grouped into one category, and the scene classification and clustering unit 1014 records the scene classification and clustering results of the photos as text data; and the facial recognition unit 1015 is used to identify the facial features of each person in the photo And the total number of people, preferably, can refer to the photos in the contact database (to be described later) or the specific portrait database to identify the information of each character in the photo to be converted, and the facial recognition unit 1015 will Results are recorded as text data. In addition, preferably, the text conversion unit further includes an electronic map conversion unit 1016 for converting the location information of the positioning system into text data with reference to the electronic map, so as to provide text data of the address when generating or receiving multimedia content. Those skilled in the art should understand that: the above-mentioned units are only examples, and are not intended to limit the present invention. In fact, the types of multimedia content are flexible, and corresponding text conversion units can be set for the multimedia content that needs to be managed. The function of text conversion can be achieved, and any existing technology for realizing text conversion can be applied to the present invention.

According to an exemplary embodiment of the present invention, the text analysis unit 102 includes: a word division unit 1021, which is used to divide the text text output by the text conversion unit 101 into a plurality of words; a part-of-speech determination unit 1022, which is used to determine the Part of speech of a plurality of words divided; Name extraction unit 1023 is used to extract the name of the person in the text text according to the output result of the part of speech determination unit 1022; Time extraction unit 1024 is used to extract the text text according to the output result of the part of speech determination unit 1022 The time words in; preferably, after the time extraction unit 1024, there is a time information parsing unit 1026, which is used to convert the time words extracted by the time extraction unit 1024 into time information; The output result of 1022 extracts the important words in the text text; Preferably, after the important words extracting unit 1025, a synonym generation unit (not shown) can be provided for generating corresponding important words for the important words extracted by the important words extracting unit 1025 The "synonyms" of so that the search is more intelligent and comprehensive; other keyword value extracting unit 1027 is used for other information (such as sender, receiver, CC) from the text conversion unit 101 output except the text text , title, sending and receiving time, location, etc.) to extract keyword values.

As mentioned above, after the data acquisition unit 10 converts various multimedia contents into text data, and extracts corresponding keyword values from the converted text data according to predetermined keywords reflecting the key attributes of the multimedia contents, it will The obtained data collection results are sent to the database building unit 20 . Then, the database building unit 20 builds a multimedia content database based on the content output by the data collection unit 10 . As an example, in the multimedia content database, each multimedia content is recorded based on predetermined keywords reflecting the key attributes of the multimedia content, corresponding keyword values and related attribute descriptions, each multimedia content record has its own index, and the index can represent It is in the form of "letter + label", wherein the letter indicates the media type, and the label indicates the serial number of the multimedia record in the media type.

For example, at 19:55 on August 3, 2008, the short message "Xiao Li, are you free in the evening? Go to the movies together?" received from Zhang San in the International Trade Center, after passing through the data After the processing of the acquisition unit 10, it can be drawn that the receiving time of the short message is "2008-8-3 19:55", the receiving place is "International Trade Center", the sender is "Zhang San", and the name of the person in the text is "Xiaoli", the time in the text is "evening", which can be parsed by the time information analysis unit 1026 as "2008-8-318:00-24:00", the important word in the text is "movie", and the part of speech is noun, and the synonym generation unit can generate the synonym "movie" accordingly. Based on the above keywords and keyword values, the database building unit 20 generates corresponding multimedia content records in the multimedia content database, as shown in Table 1 for details.

Table 1, Record examples of receiving short messages in the multimedia content database

As described in Table 1, for the keyword <media type>, its keyword value is "short message", and the relevant attribute description indicates whether the short message is a sent short message or a received short message. In this example, the short message is Received short message; for the keyword <sender> <name>, its keyword value is "Zhang San", and the related attribute description first indicates whether "Zhang San" is the contact of the mobile terminal, if it is the contact of the mobile terminal person, then "contact"=Y, if it is not a contact of the mobile terminal, then "contact"=N; in addition, preferably, in the case of "contact"=Y, the relevant attribute description can further indicate the contact The index of the person in the contact database (it will be described later); for the keyword <time>, its keyword value is "2008-8-3 19:55"; for the keyword <location>, its keyword value is "International Trade Center", and the relevant attribute description indicates whether the location is the sending location or the receiving location. In this example, "International Trade Center" is the receiving location; ", the content of the relevant attribute description is the same as the keyword <sender> <person's name>; for the keyword <text> <time>, its keyword value is "evening", preferably, its related attribute description indicates the time word "evening" The parsing result of "2008-8-3 18:00-24:00"; for the keyword <body> <important words>, its keyword value is "movie", and the related attribute description indicates the part of speech of "movie", preferably , whose relative attribute description also indicates a synonym for "movie", ie, "film".

Similarly, for at 20:00 on August 3, 2008, the short message "movie ticket has been ordered, see you in the evening" sent to Zhang San in the International Trade Center, after passing through the data collection unit 10 After processing, it can be concluded that the sending time of the short message is "2008-8-3 20:00", the sending place is "International Trade Center", the receiver is "Zhang San", the time in the text is "evening", and its It can be parsed as "2008-8-3 18:00-24:00" by the time information parsing unit 1026, the important word in the text is "movie", the part of speech is a noun, and the synonym generation unit can generate the synonym "movie", Another important word in the text is "movie ticket", and the part of speech is a noun. Based on the above keywords and keyword values, the database building unit 20 generates corresponding multimedia content records in the multimedia content database, as shown in Table 2 for details.

Table 2, examples of records in the multimedia content database for sending short messages

As another example, at 9:38 on September 5, 2008, the email titled "Express delivery has arrived" received from Li Si at the China International Trade Center and copied to Xinxin, "Xiao Li, hello! The courier you sent to me has been received, and all the goods are in good condition, thank you! After the multimedia content of "Li Si" is processed by the data acquisition unit 10, it can be concluded that the receiving time of the email is "2008-9 -5 9:38", the receiving location is "International Trade Center", the sender is "Li Si", the receiver is "Xiaoli", the copying party is "Xinxin", the important word in the title is "Express", the part of speech is a noun, the names of the people in the text are "Xiaoli" and "Li Si", the important word in the text is "express delivery", the part of speech is a noun, and the other important word in the text is "goods", the part of speech is "noun" . Based on the above keywords and keyword values, the database building unit 20 generates corresponding multimedia content records in the multimedia content database, as shown in Table 3 in detail.

Table 3. Examples of records of received e-mails in the multimedia content database

Similarly, for the e-mail titled "Recommend a good movie" sent to Xinxin at 19:20 on August 4, 2008 in Wangjing New City, "I went to see "xxx" with Zhang San yesterday, it was very good. , you can go and see with Xiaoan when you have time", after the processing of the data acquisition unit 10, it can be drawn that the sending time of the email is "2008-8-4 19:20", and the sending The location is "Wangjing New City", the sender is "Xiaoli", the receiver is "Xinxin", the important word in the title is "Movie", the names of the people in the text are "Zhang San" and "Xiao'an", and the words in the text are The time is "yesterday", which can be resolved to "2008-8-3 00:00-24:00" by the time information analysis unit 1026. Based on the above keywords and keyword values, the database building unit 20 generates corresponding multimedia content records in the multimedia content database, as shown in Table 4 specifically.

Table 4. Examples of records in the multimedia content database for sending e-mails

As another example, for a voice call made to Li Si at the International Trade Center from 10:15 to 10:19 on September 5, 2008, "I: Hello, is it Li Si? Li Si: Hello Xiaoli Come on me: The loan you transferred has already arrived. Sorry for your trouble. Li Si: You’re welcome. Me: I’ll contact you when I’m free." The start time of the call is "2008-9-5 10:15", the end time is "2008-9-5 10:19", the sending location is "International Trade Center", the sender is "Xiaoli", and the receiver is " Li Si", the names of the people in the text are "Li Si" and "Xiaoli", the important word in the text is "loan", and the part of speech is "noun". Based on the above keywords and keyword values, the database building unit 20 generates corresponding multimedia content records in the multimedia content database, as shown in Table 5 in detail.

Table 5. Examples of records of outgoing voice calls in the multimedia content database

In Table 5, since the voice call lasts for a period of time, two specific times are generated for the keyword "time". At this time, the related attribute description of the time needs to indicate when the start time is and what time is the end time.

As another example, for the photo shown in (a) in Figure 4, the photo was taken at 14:44 on September 1, 2008, and the location of the photo was Chang'an Grand Theater. The left side of the photo is Xinxin, and the right side For Xiaoan. After the photo is processed by the data acquisition unit 10, it can be drawn that the shooting time of the photo is "2008-9-1 14:44", the shooting location is "Chang'an Grand Theater", the photo category is "portrait", and the photo in the photo is "portrait". The two persons are "Xinxin" and "Xiao'an", and the classification and clustering unit 1014 can also identify that there are two similar photos in this photo. Based on the above keywords and keyword values, the database building unit 20 generates corresponding multimedia content records in the multimedia content database, as shown in Table 6 specifically.

The record example of photo shown in (a) in table 6, Fig. 4 in the multimedia content database

In Table 6, "quantity" indicates the number of people in the portrait, and p1 and p2 respectively indicate the indexes of two photos similar to the photo in the multimedia content database.

Similarly, for the photo shown in (b) in FIG. 4, the photo was taken at 10:00 on October 3, 2008, and the photo was taken at the Basongcuo Scenic Area. After the photo is processed by the data acquisition unit 10, it can be drawn that the shooting time of the photo is "2008-10-3 10:00", the shooting location is "Basongcuo Scenic Area", the photo category is "landscape", and the classification The sum clustering unit 1014 may also identify that there is a similar photo to the photo. Based on the above keywords and keyword values, the database building unit 20 generates corresponding multimedia content records in the multimedia content database, as shown in Table 7 in detail.

Record examples of photos shown in (b) in table 7, Fig. 4 in the multimedia content database

keywords key value Related property description <media type> photo <photo category> landscape <similar photo> 1 piece "1" = p3 <time> 2008-10-3 10:00 <location> Basongcuo Scenic Area "attribute" = shooting location

In Table 7, p3 represents the index of the photo similar to the photo in the multimedia content database.

The keywords of each of the above records can be flexibly set based on the type of multimedia content and the requirements of the search, for example, certain keywords can be deleted, and new keywords can also be added, and the present invention is not limited to those given in Table 1 to Table 7 example. The database building unit 20 stores the multimedia content database built as described above in the storage unit 30 .

It can be seen from Table 1 to Table 7 that the various multimedia contents in the mobile terminal often involve the interconnection between various contacts, for example, the content of the short message, the CC party and content of the email, the content of the call, portraits in photos to reflect the relationship between different contacts. Since the embodiment of the present invention converts the above-mentioned various multimedia contents into text data, and establishes a corresponding database, it can clearly reflect various interrelationships between such different contacts, thereby facilitating mobile terminal users Manage his human resources.

In order to further correlate the interpersonal relationship involved in the multimedia content with the terminal user's contacts, as an optional method, the present invention establishes a relationship between the contact database and the multimedia content database through the database establishment unit 20, the specific operation Information related to multimedia content and other contacts is additionally recorded in each contact record of the contacts database.

For example, for the contact person Xinxin, its photo in the contact database is shown in (a) in Figure 5. In this embodiment of the present invention, the index of Xinxin in the contact database is "n3", and the contact person Each contact in the library is recorded based on predetermined keywords and their corresponding keyword values, in particular, in addition to name, avatar, mobile phone, office phone, home phone, ringtone, e-mail, address, etc. among the regular keywords In addition to at least one or more, the embodiment of the present invention also sets the keywords "communication record", "media record" and "associated record" specifically for the multimedia content database, wherein the "communication record" indicates that the contact is in the multimedia The number of times and in which record the contact appears as <Sender>, <Receiver>, or <Cc> in each record in the content database, and Media Record indicates that the contact appears in each record in the multimedia content database as < Title>, <text>, <portrait> and other positions appearing times and which record they appear in, "associated record" indicates the number of times the contact is associated with other contacts based on each record in the multimedia content database, and also That is to say, whenever the contact is associated with some other contact in a certain multimedia content record, the variable indicating the number of times the two are associated is incremented by 1. It can be seen that the specific keyword values of keywords "communication records", "media records" and "associated records" are updated in real time along with the increase of multimedia content records. The content database is also responsible for updating the contact database according to the addition of records in the multimedia content database.

As an example, Table 8 and Table 9 below show examples of records "Xinxin" and "Xiaoan" in the contact database, wherein the index of the record "Xinxin" is n3, and the index of the record "Xiaoan" is n4.

Table 8. The record of "Xinxin" in the contact database

keywords key value <name> Xinxin <Avatar> As shown in (a) in Figure 5 <mobile phone> 13800000000

keywords key value <office phone number> 65799989 <Home Phone> 89825643 <ringtone> Xxx.mp3 <email> Xinxin@abc.com <address> Room x, unit x, building x, xx community, xx road, xx district, xx city <Communication Record> "Quantity" = 35, "1" = e8, "2" = s2, "3" = c9, ... <Media Record> "Quantity" = 28, "1" = p5, "2" = e20, "3" = c9, ... <associated record> "n1" = 10, "n2" = "3", "n4" = 20, "n5" = 0, ...

In Table 8, the keyword value of the keyword <communication record> indicates that the number of occurrences of the contact as <sender>, <receiver> or <cc> in each record of the multimedia content database is 35 times, Specifically appear in e8, s2, c9... etc. records in turn, wherein e8, s2 and c9 are the indexes of relevant multimedia records in the multimedia content database, and the index notation indicates the category of multimedia content through the first letter, The following number indicates the serial number. The keyword value of the keyword <media record> indicates that the contact appears 28 times in various records of the multimedia content database as <title>, <text>, <portrait>, etc., specifically appearing in p5, e20, c9... etc. are being recorded. The keyword value of the keyword <associated record> indicates the number of times the contact has been associated with other contacts based on each record in the multimedia content database, that is, the contact with the first contact n1 has been contacted 10 times, and the contact with the second contact n2 has contacted 3 times, has contacted the fourth contact n4 20 times, and has contacted the fifth contact n5 0 times, wherein n1, n2, n4 and n5 are indexes of each contact in the contact database.

Table 9. The record of "Xiao'an" in the contact database

keywords key value <name> Xiaoan <Avatar> Shown in (b) in Figure 5 <mobile phone> 13400000000 <office phone number> 82956321

keywords key value <Home Phone> 63215487 <ringtone> Yyy.mp3 <email> Xiaoan@xyz.com <address> Room y, unit y, building yy, yy community, yy road, yy district, yy city <Communication Record> "Quantity" = 2, "1" = s60, "2" = c2 <Media Record> "Quantity"=8, "1"=p5, "2"=e20, "3"=s9,... <associated record> "n1"=0, "n2"=1, "n4"=20, "n5"=0, ...

The meanings of each keyword and its keyword value are the same as those in Table 8, and will not be described here one by one.

As an optional manner, the database establishing unit 20 may also store the above-mentioned contact database in the storage unit 30 .

FIG. 6 is a diagram illustrating a detailed structure of a query input unit 40 in the apparatus for managing multimedia content in a mobile terminal illustrated in FIG. 1 . As shown in Figure 6, the query input unit 40 according to an exemplary embodiment of the present invention may include a text recognition unit 401, which is used to convert the query information input by the user through a keyboard or tablet into text data; a text capture unit 402 and an OCR ( Optical character recognition) unit 403, used to capture and identify the character data on the inquired photo; face recognition unit 404, similar to the face recognition unit 1015 in the data collection unit 10, used to identify the character data in the inquired photo The facial features of each person, specifically, can refer to the photo in the contact database or a specific portrait database to identify the information of each person in the photo to be queried, and record the result of facial recognition as text data; the speech recognition unit 405 , for converting the voice query input by the user into text data, and its working mode is similar to that of the voice call conversion unit 1013 . The above-mentioned units are only exemplary, and the present invention is not limited thereto, and the way of query input can be simplified, or another way of query input can also be added. After the query input by the user is identified as text data, the search term extraction unit 406 extracts key search terms from the identified text data. The working mode of the query term extraction unit 406 is similar to the text analysis unit 102 of the data acquisition unit 10, using It is used to extract the key content in the query text and other information as search terms. For example, the search term may be a person's name, time, important words or other information.

The search unit 50 queries the corresponding multimedia content records from the database stored in the storage unit 30 based on the key search words extracted by the query input unit 40 . Specifically, the search unit 50 can search for items related to the input search word in the keyword values of each multimedia content record in the multimedia content database, and search for items in a specific way (such as according to time, place, media type, etc.) The results of the search are divided, and the processed results are output to the search result output unit 60 . Here, the search result output unit 60 may output the search result in a visualized manner.

An example of searching according to an exemplary embodiment of the present invention will be described below with reference to FIG. 7, which shows an example of search results output by the search result output unit 60 in the device for managing multimedia content in the mobile terminal shown in FIG. .

Specifically, as shown in (a) of Figure 7, at this time, the search word extracted by the query input unit 40 is "Zhang San", and the search unit 50 can search for <sender>, <receiver> in the multimedia content database. Or <cc party> is the multimedia content record of "Zhang San", and sort according to time, output the sorted results to the search result output unit 60, and the search result output unit 60 outputs (a) as shown in Figure 7 screen shown. As another optional way, if "Zhang San" is the contact person of the mobile terminal and has established the contact database according to the present invention, then the "communication record" item in the contact database of Zhang San can be used to search for related information. multimedia recording. The above method is just an example, and those skilled in the art can apply the established multimedia content database (and contact database) in different ways, search for various contents, and filter and sort them in different ways. For example, if "Zhang San" is a contact person of the mobile terminal and has set up a contact database according to the present invention, then the relevant multimedia records can also be searched for by Zhang San's "media record" item in the contact database, and Output the results of the search.

As another example, as shown in (b) of FIG. 7, at this time, the search term extracted by the query input unit 40 is "movie", and the search unit 50 can search for <title> and <text> in the multimedia content database as "" "movie" or its synonym "film" multimedia content record, and sort according to time, the sorted result is output to the search result output unit 60, and the search result output unit 60 outputs as shown in (b) among Fig. 7 Screen.

As another example, as shown in (c) of FIG. 7, at this time, the search term extracted by the query input unit 40 is "Zhang San", and "Zhang San" is a contact person of the mobile terminal and has established a contact person according to the present invention. In this case, the search unit 50 can obtain the contact frequency between "Zhang San" and other contacts according to the "associated records" recorded in "Zhang San" in the contact database. Preferably, such Contact can mean direct contact or indirect contact (that is, Zhang San establishes contact with another contact person through one of his related contacts), and it is even more difficult to get direct contact and indirect contact at the same time. Helping to fully reflect the interpersonal relationship involving Zhang San, the search unit 50 outputs the obtained contact frequency to the search result output unit 60, and the search result output unit 60 can output the search result in a visualized manner, for example, output as shown in Figure 7 In the screen shown in (c), Zhang San is connected with various contacts by lines, and the thickness of the lines indicates different contact frequencies, that is, contacts with higher contact frequencies are connected by thicker lines Indicates that contacts with lower contact frequency are represented by thinner lines, and the contact frequency between each contact other than Zhang San can also be displayed together, so that (c) in Figure 7 is It shows a relatively complete diagram of interpersonal relationships centered on Zhang San.

In particular, if the query input unit 40 extracts a plurality of search words, and these search words may include different situations such as person's name, time, place, and important words, the user may be requested through the search result output unit 60 to sequentially select the appropriate search words. The search conditions, that is, the logical relationship between different search terms can be set by the user. In addition, the logical relationship between multiple search terms can also be established according to preset conditions, for example, a logical AND relationship between multiple search terms can be defaulted. After setting the logical relationship between the plurality of search terms, the search unit 50 further searches for the corresponding multimedia content.

According to the present invention, various multimedia contents of the mobile terminal can be stored in the multimedia content database in the form of text data, thereby reflecting the implicit relationship between each multimedia content itself and each other, and the user can conveniently Database for diverse searches. In addition, the present invention also establishes a contact database related to the multimedia content database, so that the contact status among various contacts can be reflected in real time, and it is convenient for users to grasp their own interpersonal resources. The management equipment and method according to the present invention are realized based on the local machine, do not need an additional server, are relatively easy to implement in terms of schemes, and have low cost.

The above respective embodiments of the present invention are merely exemplary, and the present invention is not limited thereto. Those skilled in the art should understand: all search schemes that can be provided by the database that is established based on the various information after the multimedia content is converted into text data can be applied to the present invention, that is to say, the present invention converts the multimedia content into text data, Therefore, related information is retained, and a search relationship is established based on the information, so that the user can more conveniently manage the multimedia content of the mobile terminal and fully grasp the relationship between various contacts. Those skilled in the art will recognize that changes may be made in these embodiments without departing from the principles and spirit of the invention, the scope of which is defined in the claims and their equivalents.

Claims (25)

1. A device for managing multimedia content in a mobile terminal, comprising: The data acquisition unit is used to convert the multimedia content into text data, and extract corresponding keyword values from the converted text data according to predetermined keywords reflecting the key attributes of the multimedia content; A database establishment unit, configured to establish a multimedia content database based on the keyword value obtained by the data acquisition unit; a storage unit for storing the multimedia content database established by the database establishment unit; a query input unit, configured to recognize query information input by a user as text data, and extract key search terms from the recognized text data; a search unit for searching a multimedia content related to the search term from a multimedia content database stored in the storage unit based on the search term extracted by the query input unit; and The search result output unit is used to output the search result of the search unit to the user. 2. The device according to claim 1, wherein the data acquisition unit comprises: a text conversion unit for converting the multimedia content into text data; and The text analysis unit is used for extracting corresponding keyword values from the converted text data according to predetermined keywords reflecting the key attributes of the multimedia content. 3. The device of claim 2, wherein the text conversion unit comprises at least one of: A short message conversion unit is used to convert the short message into text data; an email conversion unit for converting email into text data; A voice call conversion unit is used to convert voice calls into text data; The scene classification and clustering unit is used to classify the photos according to predetermined categories and obtain photos similar to the photos through clustering processing, and record the scene classification and clustering results of the photos as text data; A facial recognition unit, configured to recognize the facial features and the total number of people in the photo, and record the results of the facial recognition as text data; and The electronic map conversion unit is used for converting the position information of the positioning system into text data with reference to the electronic map. 4. The device of claim 3, wherein the text analysis unit comprises: A word division unit, used to divide the text text output by the text conversion unit into a plurality of words; A part-of-speech determination unit is used to determine the parts of speech of a plurality of words divided by the word division unit; A name extraction unit, used to extract the name of the person in the text body according to the output result of the part-of-speech determination unit; A time extraction unit is used to extract the time words in the text body according to the output result of the part of speech determination unit; An important word extraction unit is used to extract important words in the text body according to the output result of the part-of-speech determination unit; and Other keyword value extraction unit is used for extracting keyword values from information other than the text body output by the text conversion unit. 5. The device according to claim 4, further comprising: a time information parsing unit for converting the time words extracted by the time extracting unit into time information. 6. The device as claimed in claim 4, further comprising: a synonym generation unit for generating corresponding synonyms for the important words extracted by the important word extraction unit. 7. The device according to claim 1, wherein the database building unit builds the multimedia content database based on keywords, keyword values and related attribute descriptions. 8. The device according to claim 1, wherein the database establishing unit further establishes a contact database, and the contact records of the contact database are updated along with the multimedia content database. 9. The device of claim 1, wherein the query input unit comprises: The text recognition unit is used to convert the query information input by the user through the keyboard or tablet into text data; A text capture unit and an optical character recognition OCR unit are used to capture and recognize character data on the queried photo; A facial recognition unit, configured to recognize the facial features of each person in the query photo, and record the result of the facial recognition as text data; and The speech recognition unit is used to convert the speech query input by the user into text data. 10. The apparatus of claim 1, wherein the search result output unit outputs the search result of the search unit to the user in a visualized manner. 11. The device according to claim 8, wherein the search unit searches the multimedia content related to the search term from the multimedia content database stored in the storage unit with reference to the contact database based on the search term extracted by the query input unit . 12. The apparatus of claim 8, wherein the search unit searches for a mutual relationship of the contacts in the multimedia content based on the multimedia content database and the contact database. 13. The device of claim 1, wherein if the query input unit extracts a plurality of search terms, the search unit searches for the plurality of search terms according to a predetermined logical relationship or a logical relationship set by a user. 14. A method for managing multimedia content in a mobile terminal, comprising: converting the multimedia content into text data, and extracting corresponding key values from the converted text data according to predetermined keywords reflecting the key attributes of the multimedia content; building a multimedia content database based on the key value; storing the established multimedia content database; Recognize the query information entered by the user as text data, and extract key search terms from the recognized text data; searching a stored multimedia content database for multimedia content related to the search term based on the extracted search term; and Output search results to the user. 15. The method of claim 14, wherein the step of converting the multimedia content into text data comprises at least one of the following steps: Convert short messages to text data; Convert emails to text data; convert voice calls into text data; Classify the photos according to the predetermined category and obtain photos similar to the photo through clustering processing, and record the classification and clustering results of the photos as text data; Recognize the facial features and total number of people in the photos, and record the recognition results as text data; and The position information of the positioning system is converted into text data with reference to the electronic map. 16. The method as claimed in claim 14, wherein, according to predetermined keywords reflecting multimedia content key attributes, the step of extracting corresponding keyword values from converted text data comprises: Divide the text body into terms; determining the part of speech of the divided plurality of words; Extract the names of people in the text body according to the parts of speech of the divided words; According to the parts of speech of the divided multiple words, the time words in the text body are extracted; extracting important words in the text body according to the parts of speech of the divided plurality of words; and Key values are extracted from information other than the body of the text. 17. The method of claim 16, further comprising converting the extracted temporal words into temporal information. 18. The method according to claim 16, further comprising: generating corresponding synonyms for the extracted important words. 19. The method of claim 14, wherein the multimedia content records in the multimedia content database include keywords, keyword values and related attribute descriptions. 20. The method of claim 14, further comprising: establishing a contact database, and the contact records of the contact database are updated along with the multimedia content database. 21. The method of claim 14, wherein the step of recognizing the user-entered query information as text data comprises at least one of the following steps: Convert the query information entered by the user through the keyboard or tablet into text data; capture and identify character data on the queried photo; Recognize the facial features of each person in the query photo, and record the recognition result as text data; and Convert voice queries entered by users into text data. 22. The method according to claim 14, wherein, in the step of outputting the search result to the user, the search result is output to the user in a visualized manner. 23. The method of claim 20, further comprising searching a stored multimedia content database for multimedia content related to the search term with reference to a contact database based on the extracted search term. 24. The method of claim 20, further comprising searching for mutual relationships of the contacts in the multimedia content based on the multimedia content database and the contact database. 25. The method according to claim 14, further comprising: if a plurality of search terms are extracted, searching for the plurality of search terms according to a predetermined logical relationship or a logical relationship set by a user.

Images