JP2010529542A - Time-series templates for text-to-animation systems - Google Patents

Abstract

A method of converting input text into input for an animation generator, the method comprising: receiving the input text; and a first set of data representing information associated with an action identified in the input text Is extracted from the input text, and the first set of data is used to complete the semantically annotated action template, and the first step represents information related to the description of any participants included in the action. Extracting two sets of data from the input text and using the second set of data to complete a semantically annotated description template; the semantically annotated action template and the semantic The description template annotated to the animation generator is sent to the animation generator. To provide a method comprising the flop.

Description

  The present invention relates to text-to-animation systems, and more particularly to extracting temporal information, semantic information, and general knowledge from text to generate animation sequences.

  Animations are being created for various applications not only to rapidly evolve the tools available, but also to gain the expertise to apply them. Some of its applications, such as text-to-scene (TTS), text-to-animation (TTA), or text-to-movie (TTM) systems, for understanding the natural language of action concepts. Resulting from the performance of the animation system.

  In some early systems, presentation (representation) was performed using only still images. Some systems also identify key concepts (words) or words in the text and then employ images to represent them, but provide a clear sequence that accurately captures the meaning of the text. It is not a thing. Other systems that perform animation conversion of text are built by editing animation sequences with each other, thus requiring a large static database of mini-animation clips that are restricted in use. Therefore, there is a need for a system that analyzes natural language text and places it into a form that can be automatically animated.

  According to a first general aspect of the invention, a method of converting input text into input for an animation generator, the method being identified in the input text and receiving the input text Extracting from the input text a first set of data representing information associated with the action, and completing an action template semantically annotated with at least the first set of data; A second set of data representing information related to the description of any included participants is extracted from the input text, and at least the second set of data is used to complete a semantically annotated description template. Steps, the semantically annotated action template and the semantically annotated description text. A method comprising the steps of: transmitting a plate to the animation generator.

  According to a second general aspect of the present invention, a system for converting input text into input for an animation generator, the system receiving said input text and outputting a semantic structure A module, a conceptual background database that stores semantic information, a predicate interaction database that stores common sense knowledge about action predicates, and a three-dimensional mapping that stores action predicate definitions and associated parameters A database, a first template that receives the semantic structure from the natural language processing module and represents information related to actions, and a second template that represents information related to descriptions of all participants included in the actions. Included in the database Automatically completed using multicast, it provides a system comprising a template generator which is adapted to send these first template and second template to the animation generator.

According to a third general aspect of the invention, a method for rendering information extracted from text for use in creating an animation, the method comprising:
Complete the semantically annotated action template by generating information related to the action predicate, semantic information for this action predicate, time information extracted from the fluent / event, and estimated information for common sense inference Step to
Participant information related to the participant in action, spatial information related to the position of the participant in the scene, and at least one of the emotional state, physical state, and behavior of the participant. Completing at least one semantically annotated descriptive template by generating dynamic information that influences one another and a link that joins the participant to the action;
A method is provided in which the template includes all the syntactic and semantic parameters that need to be used for the animation.

According to a fourth general aspect of the present invention, a system for analyzing a natural language text describing an action and creating an ordered action structure to be used to create an animation, the system But,
A natural language processing module that receives the natural language text as input and outputs a semantic structure;
A conceptual background database for storing semantic information;
A predicate interaction database that stores common sense knowledge about action predicates;
A three-dimensional mapping database storing action predicate definitions and associated parameters;
Automatically generate semantically annotated action templates that generate conceptual structure and represent information related to actions, and semantically annotated description templates that are related to the description of any participants involved in these actions The ordering action structure generator is completed, and the template provides a system adapted to be completed using the semantic structure and information contained in the database.

  Some of the information stored in the various databases can be generic information, and other information is context specific.

  As used herein, the term “event” is intended to mean an animation unit that occurs at a specific point on a time line. The event / fluid distinction is a design result, not an inherent property of the action. The term “fluent” refers to an animation unit that is held for a period of time. A fluent is a predicate whose truth value can be changed over a period of time. An event can occur simultaneously with any number of fluents.

  In the context of linguistics, predicates are understood to be a characteristic of words that can be used to make statements about something in the animation world. The predicate is an “animation word”, which means that the predicate has the meaning of being understood by the animation generator. For example, the predicate “table” is understood by an animation generator that displays a graphical representation of the table. The predicate “on” associated with this predicate “table” is understood as representing a position by the animation generator. The animation generator understands that something is on the graphical representation of the table. The entire set of predicates forms an animation language. Action predicates come with an actant slot in the context of linguistics and are understood to be a special kind of predicate that specifically refers to the action concept. An actor is a parameter (variable) of a predicate. Formally, give (X, Y, Z) (giving (X, Y, Z)) is an example of an action predicate having three actors (X, Y, Z). We often mention the action predicate give, even if give is the name of the action predicate give (X, Y, Z). Semantic information is understood as information relating to the meaning of either or both of words and sentences. Syntactic information is understood as information related to the form of a sentence. The modulator information is information that can affect the action predicate. For example, quickly affects walking. The animation end information is information for ending the ongoing animation. For example, killed terminates walking. The time graph information is information that temporally orders the action predicates according to the semantic structure. Existence and constraint information relates to certain preconditions and later conditions of the action predicate. For example, the action predicate take (x) (take (x)) has exists (x) (exist (x)) as a precondition, which requires that the object to be taken exist in the actual environment. It means that there is. For example, a non-animated object (table) is a predicate that is true if the concept table is a non-animated concept. An animation channel refers to an entity (such as a body, eyes, or head) to which an action can be applied. The semantic structure is a graph that characterizes the predicate-actant association. The arcs in these graphs are labeled with the semantic relationship of the actor to the predicate. These relationships are often referred to as term semantic roles.

  Further features and advantages of the present invention will become apparent from the following detailed description taken in conjunction with the accompanying drawings.

FIG. 1 is a block diagram illustrating a system that parses natural language text describing actions to produce an ordered action structure. FIG. 2A is a block diagram illustrating an example of a first ontology that represents the relationship between different concepts of an automobile. FIG. 2B is a block diagram illustrating an example ontology that associates color concepts. FIG. 2C is a block diagram illustrating an example ontology that associates the concept “run” with the instance “sprint”. FIG. 3 is a diagram showing a typical structure of input to the APIR database. FIG. 4 is a diagram illustrating an example of the first semantic structure extracted from the first sentence. FIG. 5 is an explanatory diagram showing an example of a SAAT template. FIG. 6 is an explanatory diagram showing an example of a SADT template. FIG. 7 is a diagram illustrating an example of the second semantic structure extracted from the second sentence. FIG. 8 is a diagram illustrating an example of a third semantic structure extracted from the third sentence. FIG. 9 is a diagram illustrating an example of a fourth semantic structure extracted from the fourth sentence. FIG. 10 is a block diagram showing an example of a conceptual structure.

  FIG. 1 illustrates a system 10 that parses natural language text that describes actions and generates an ordered action structure that should be used to generate action animations by an animation generator. In this system, a natural language processing (NLP) module 12 exists. This module takes text 14 as input and outputs semantic structure 16 using information contained in conceptual background (CB) database 18, glossary 20 and 3D mapping database 22. Next, the SAAT and SADT generator 24 includes information included in the semantic structure 16, the CB database 18, the 3D mapping database 22, and the action database predicate interaction relation (APIR) matrix database 30. Are used to generate a semantically annotated action template (SAAT) 26 and a semantically annotated description template (SADT) 28.

  The glossary 20 is part of the NLP module 12, is language dependent, and has morphological information, syntactic (syntax) information and semantic information for all contained vocabulary (lexical) units. Yes. Each vocabulary unit is given a vocabulary definition similar to that found in a printed dictionary. In the NLP module 12, a vocabulary unit is assigned to each word of the input text. Other parts of the NLP module 12 may include, but are not limited to, text segmentation, word semantic clarification, syntactic ambiguity, and tools for addressing speech act. Since NLP modules currently known in the art can be used in the system of the present invention, further description thereof is omitted. It should be noted that the glossary can be separated from the NLP module.

  Another database, called the 3D mapping database, is a bridge between NLP and animation. The 3D mapping database 22 is a bridge between the NLP module 12 and the SAAT and SADT generator 24. This 3D mapping database 22 contains the vocabulary of animation that the animation generator recognizes and understands. Is assigned a graphic representation or function in the animation. A unit entity or word in the vocabulary of animation is called a predicate. This database is language independent in the sense that the predicates are written in a pivot language. The input text 14 is written in a predetermined language such as English, French or German, and the predicates included in the 3D mapping database 22 are written in a pivot language.

  In one embodiment, the 3D mapping database includes six predicates: an action predicate (AP), a modulator predicate (MLP), a modifier predicate (MFP), and a conceptual asset (CA). And a spatial predicate (SRP) and a change predicate (ALP). An action predicate is an animation word or predicate that is used to describe an action to be performed on an animation. Usually, action predicates are verbs. The predicates “walk”, “take” and “kiss” are examples of action predicates. A conceptual asset is a predicate that adds to an action, usually a noun. The predicates “table”, “car”, “box”, “human” and “hand” are examples of conceptual assets. A modulator predicate is a parameter associated with an action predicate or conceptual asset. The modulator predicate can be used to specify a method of executing an action, a person's emotion, a feature of an object, and the like. Adverbs and adjectives are usually modulator predicates. Also, predicates such as “speed”, “emotion”, “color” and “side” are examples of modulator predicates. Modifier predicates are used to modify modulator predicates. A modifier predicate such as “plus” increases the modulator predicate to which it applies, and a modifier predicate such as “minus” decreases the modulator predicate to which it applies. Spatial relationship predicates are used to identify an initial, final or invariant spatial relationship. For example, in the entered text “Paul takes the book from the table”, the token “from” is the first time the book is on the table. Mean (initial spatial relationship), while in the input text “Paul puts the book on the table”, the token “on” It means that the book is on the table after the action (final spatial relationship). Examples of spatial predicates are "on (up)", "under (down)", "above" (up). The change predicate changes the action predicate. Predicates such as “stop”, “continue” and “pause” are examples of change predicates.

  The system 10 also has a CB database 18 that is a multiple inheritance concept structure intended to cover the semantic universe of applications. The 3D mapping database 22 includes semantic information regarding an animation world (animation word or predicate), while the CB database includes semantic information regarding a real world (real world). The CB database has the highest level concepts, middle level concepts, and ontology of instances and acts as a representation of the application's semantic world. An ontology represents a relationship between elements of the CB database 18. Since the ontology of the CB database 18 is written in the pivot language, it does not depend on the language.

  The ontology takes the form of a directed graph in which nodes are connected (connected) as shown in FIG. 2A. Nodes can be either concepts or instances. An instance can be a special case of a concept or provide additional information about the concept. Referring to FIG. 2A, the instances “estate car” and “sedan car” are special cases of the concept “family car”. Another type of additional information obtained by the CB database 18 relates to the possibility of animating a concept or instance. For example, the concept “table” can be classified as an animated concept in certain applications, which means that the table can be walked in 3D animation. In other animations, the concept “table” is considered non-animated and the table becomes a fixed object in 3D animation. An ontology determines whether a concept or instance is animated or not. The relationship that connects concepts to each other and the relationship that connects concepts to instances can be of any kind, and these relationships are universal in the sense that they are always correct regardless of language.

  Referring to FIG. 2A, the concept “vehicle” is a top-level concept that includes medium concepts such as the concepts “car”, “boat” and “motorbike”. It is. The medium concept “car” also includes medium concepts such as “sports car” and “family car”. A directed graph depicts a hierarchy between concepts and instances, starting with the concept with the broadest meaning and ending with the instance with the narrowest definition. All concepts and instances are expressed in a pivot language so that the CB database is language independent. FIG. 2B is another example of an ontology included in the CB database 18. The top-level concept “visual attribute” is linked to the middle-level concept “color”, which is “blue” It is connected to several instances such as “white” and “black”. FIG. 2 c shows another example of an ontology contained within the CB database 18, in which the instance “sprint” is connected to the concept “run”. In this case, the ontology represents a gradation of the speed of movement. Ontologies can represent intensity gradients such as “talk”, “shout”, “yell”.

  Each vocabulary unit of the glossary 20 is connected to a single concept or instance of the CB database 18 regardless of the language of the glossary 20. For example, the NLP module 12 receives the token “run”, the glossary 20 has two vocabulary entries “run 1” and “run 2”, and the first vocabulary entry is “operate”. Synonymous and the second vocabulary entry represents a type of displacement. The vocabulary entry “run 1” is associated with the concept “operate” and the second vocabulary entry, “run 2”, is associated with the concept “run”.

  In another embodiment, each lexical unit of glossary 20 is connected to a single concept or instance of CB database 18 regardless of the language of glossary 20. It should be understood that a vocabulary unit can be connected to more than one concept or instance. For example, the lexical unit “run” connects to at least two concepts “run 1” and “run 2”. The concept “run 1” is synonymous with “operate”, and the concept “run 2” represents a kind of displacement. For example, if the input text is written in French and the pivot language is English, each vocabulary unit expressed in French is connected to at least one concept or instance expressed in English and translated Is unnecessary. In this case, the vocabulary unit “courir” is connected only to the concept “run 2”. It should be understood that the connection between lexical units and concepts or instances depends on the language of the glossary 20.

  It should be understood that two or more vocabulary units may be referred to as the same concept or instance. For example, the lexical units “kiss” and “give a kiss” are both associated with the concept “kissing”.

The APIR matrix database 30 provides information about how two (or more) action predicates interact with each other when an action involves an action that already exists. In a sense, this information includes common sense knowledge about action predicates. The basic structure executed through the matrix database 30 is a graph. A node is an action predicate supported by an application. An edge (relation) is a semantic relationship (relation) that is effective in an animation environment. If node A is coupled to node B by semantic relationship S, it can be said to be ASB. For example, A: = KISS, B: = SPEAK, S: = CAN_CLIPED. In this case, the APIR graph includes the one shown in FIG. Thus, if the kiss action needs to be triggered while the talk action is occurring, the system knows that the talk needs to be clipped during the kiss. The information in the APIR matrix database 30 deals with synchronicity issues. Some examples of the main semantic relationships implemented in the APIR matrix database are shown in the following table.

  Referring again to FIG. 1, text 14 is entered by the user of the application. This text 14 can be written in any language. The NLP module 12 analyzes both the syntactic and meaning of the input text 14 and determines what it means. To perform this step, the NLP module 12 uses the information contained in the glossary 20 and the ontology of the CB database 18 in addition to the semantic and reasoning algorithms. For example, these algorithms are used to execute steps such as metonymy analysis, identical instruction analysis, semantic role labeling, word meaning clarification, and implicit action planning. It should be understood that some of the steps described above performed by the NLP module 12 may be performed by a module different from the NLP module 12. For example, implicit action planning may be performed by an action planner module that is independent of the NLP module 12. The role of the NLP module 12 is to translate the input text 14 into a semantic structure 16 and a syntactic structure. The first step is tokenization, which is a process of dividing a series of input characters. The NLP module 12 splits the text into words or tokens. The next process is a tagging process that marks the words of the input text 14 to correspond to specific parts of the speech. Each part of the speech falls into a variety of different grammatical categories that are used to classify words as nouns, verbs, prepositions, etc. Next, the NLP module 12 performs a syntactic analysis process, which is to analyze this series of words, in order to determine the grammatical structure of the series of words for a given grammar. Searching for semantic information is the last step performed by the semantic role labeling module included in the NLP module 12. This step consists in labeling the relationship between the participants of a given action against their respective semantic roles. A semantic role is obtained by using a combination of information and static rules included in the ontology of the CB database 18.

  In order to form a semantic structure, the NLP module needs to determine what the input word means. For example, the NLP module 12 associates the word “runs” of the input text with the lexical unit “run” connected to the concepts “run 1” and “run 2”. The NLP module then determines which of these two concepts represents the meaning of the input word associated with the lexical unit “run” by using a CB database ontology and word semantic clarification algorithm. For example, the NLP module 12 determines that the concept “run 1” is a suitable concept and that this concept “run 1” is associated with the input word “runs”. Next, the input word “runs” is mapped to the action predicate “operate” included in the 3D mapping database. Semantic structure 16 is a schematic interconnection of nodes. Each node has a token written in the language of the input text 14 with associated concepts written in the pivot language.

  Referring to the example shown in FIG. 2A, when the expression “grand tourer” is input, the NLP module 12 uses the vocabulary unit “grand” connected to the instance “grand tourer” in the CB database. Associated with “tourer”. When the predicate corresponding to “grand tourer” exists in the 3D mapping database 22, the NLP module 12 maps the input expression “grand tourer” to the predicate “grand tourer”. If the predicate “grand tourer” does not exist in the 3D mapping database 22 but the predicate “sport car” exists, the NLP module 12 maps the expression “grand tourer” to the predicate “sport car”. This example refers to text written in English, and the CB database 18 and 3D mapping database 22 that use English as a pivot language, but the same processing occurs when the text is written in another language. Should be understood. For example, the input text 14 can be written in French, in which case the expression “voiture grand tourism” (in English, grand tourer) is input to the NLP module in which a French glossary is provided. In this case, the expression “voiture grand tourisme” is combined with the vocabulary unit “voiture grand tourisme” connected to the instance “grand tourer” of the CB database 18. If the predicate “grand tourer” exists in the 3D mapping database 22, the NLP module 12 maps the expression “voiture grand tourisme” to the predicate “grand tourer”. Otherwise, the expression “voiture grand tourisme” is mapped to the predicate “sport car”.

  Referring to FIG. 2C, the term “sprints” is entered into the NLP module 12 by the user. The NLP module 12 connects the term “sprints” to the lexical unit “sprint”. If an action predicate corresponding to the instance “sprint” exists in the 3D mapping database 22, the NLP module maps the term “sprints” to the action predicate “sprint” and associates the term “sprints” with the instant “sprint”. Let However, if the action predicate “sprint” does not exist, the NLP module 12 maps the term “sprint” to the action predicate “run”. Alternatively, the NLP module converts the verb “sprint” into the expression “run fast” using the ontology shown in FIG. 2C. In this case, the term “sprints” is mapped to an action predicate “run” and a modulator predicate “fast”.

Once the entered word is associated with the corresponding instance or concept and mapped to the corresponding predicate, the semantic role labeling module of the NLP module 12 produces the semantic structure 16. FIG. 4 shows an example of the semantic structure 16. The sentence “Fred walks very quickly to the table” is input to the NLP module 12, except for the words “the” and “to” that the NLP module 12 ignores. Assign a concept to each word in this sentence. The words “Fred”, “walks”, “very”, “quickly” and “table” are the lexical units “Fred”, “walk”, “very”. , “Quickly” and “table”, respectively. These lexical units are the concepts “human”, “walking”, “intensifier”, “speed of event” Speed)) and “table” respectively. These concepts “human”, “walking”, “intensifier”, “speed of event”, and “table” are also mapped to the 3D mapping database 22. Each word and its associated concept occupies a node in the semantic structure. Related words and concepts that represent the behavior of the sentence are located at the top of the semantic structure. Other nodes are located below the action node as a function of importance in the text. These nodes are also connected by arrows that represent the syntactic functions they perform relative to the nodes on which they depend. For example, the node “Fred: human” depends on the node “walks: walking”, which depends on the role of the agent (operator) for the node “walks: walking”. The node “very: intensifier” is connected to the node “quickly: speed of event”. The reason is that the word “very” modifies the word “quickly” and the arrows represent the degree to which the node “very: intensifier” modifies the node “quickly: speed of event”. The semantic role of a node generated by the semantic role labeling algorithm is attached to an arrow pointing to the node.

  The database and NLP module described above are used to fill the template by the template generator 24 to represent knowledge extracted from the input text. The template generator 24 generates one SAAT for each action included in the input text and one SADT for each participant included in each action. The SAAT and SADT generator 24 receives semantic structures from the NLP module 12 and has access to the CB database 18, 3D mapping database 22, and APIR matrix database 30. SAAT and SADT generator 24 also uses the syntactic structure of input text 14 generated by NLP module 12. The SAAT and SADT generator 24 generates a conceptual structure (language independent). This conceptual structure is a kind of semantic structure, but is labeled with a predicate or a semantic role supported by the animation system. The conceptual structure can be represented as a time-ordered graph of SAATs, with each SAAT associated with a corresponding SADT. In a sense, the conceptual structure acts as a logical representation of the input text 14.

  In one embodiment, when using the semantic structure generated by the NLP module 12, the SAAT and SADT generator 24 identifies all actions. These actions are detected by looping where to parse and query the vocabulary database for links to action predicates in the 3D mapping database 22 via tokens of any meaning. Any semantic token found to be linked to an action predicate is considered to introduce an action. Next, the SAAT and SADT generator 24 classifies each node of the semantic structure into two parameter families: Actant vs. Modulator. The distinction is based on the semantic roles of the nodes, and the static classification of all semantic roles is defined in the 3D mapping database 22. The SAAT and SADT generator 24 generates a time graph that positions each previously identified action in time. The SAAT and SADT generator 24 considers the conceptual background information of each meaning token, the grammatical relationship, the information from the APIR matrix database 30 and the actions of each action to order the actions. This time graph is used as a stencil in step 4. Each node of this time graph includes a type link (previous, next time, simultaneous) to other nodes forming time information of the time graph. The SAAT and SADT generator 24 also generates one SAAT per action and places this SAAT in a previously generated time graph in response to the action associated with this SAAT. The SAAT and SADT generator 24 generates a SADT for each SAAT and associates the SADT with the corresponding SAAT. The time graph filled with SAAT and SADT represents the conceptual structure output by SAAT and SADT generator 24.

In one embodiment, the template generator 24 generates attribute predicates such as preconditions, invariants and later conditions used for common sense inferences about actions in a first step. These attribute predicates allow the action planner module to trigger essential actions that are not clear to the action flow. The domain of predicate parameters used for these attributes is equal to the predicate itself. In fact, a domain can be either a semantic role or a conceptual background, depending on the predicate. Table 2 below shows the actually supported predicates where X and Y are semantic roles, W is the type position, P is the type action predicate, and Z is the number of conceptual backgrounds.

In one embodiment, the SAAT is composed of six parts. Each part has information extracted from any of the semantic structures, databases 18, 22, and 30 and user interaction with the system. The six main categories of SAAT are shown in Table 3 below.

FIG. 5 shows a typical format of the SAAT template. In one embodiment of the present invention, the SADT consists of four parts. Each part has information extracted from either a semantic structure, a non-action predicate, or a user interaction with the system. The four main sections of SADT are shown in the table below.

  FIG. 6 shows a typical format of the SADT template. The templates (SAAT and SADT) are filled by the template generator 24 as follows. In the case of SAAT, general information is obtained directly from the information present in the 3D mapping database for a given action predicate, along with event / fluent information. This general information includes logical information and preconditions and later conditions generated by the template generator 24 by processing action predicates in the semantic structure for existing information. This general information also indicates whether the action predicate is fluent or an event. The concepts associated with action predicates are also part of this general information. The category of the verb template includes the semantic structure output by the NLP module 12 in addition to the modulator information. Modulator information is generated from filters assigned to some special semantic roles in the semantic structure such as "Manner" etc., and this information is digitized for use in animation. The modulator information includes all modulator predicates that are applied to the action predicate. The time division information includes animation end information, time graph information, and time information extracted from the event / fluent of the action predicate by the generator 24. The time graph information includes links to previous and next SAATs associated with other action predicates. The time graph gives the execution order of the SAAT, and this time graph is generated sequentially from the text. The information section of the APIR matrix is added by considering the current action unit with prior fluency at a particular time point. This information is also used to add animation end information in the time module in addition to the information resulting from the semantic structure. The impact of common sense inference is derived by resolving predicates in the APIR matrix database 30 and the CB database 18. For example, when using the CB database 18, the generator 24 determines that the action predicate “kiss” has an implicit action such as “look” and “walk”. . Barack sees Hillary, walks in the direction of Hillary, and finally kisses her before kissing Hillary. The template generator generates action predicates and corresponding templates for “look” and “walk” actions. Any dynamic information obtained from the user's interaction is stored in the user's interaction module (a special placeholder for information supplied by the user as a result of the user interface module).

  In the case of SADT, the participant information classification includes all static information related to predicates related to SADT. This category also has information such as the semantic role of the non-action predicate with which SADT is related, its emotional (emotional) state, and the like. Semantic roles associated with the participants, modulator predicates applied to the participants, and emotional states (such as happy, sad, etc.) are obtained from the semantic structure 16 output from the NLP module 12. Animated / non-animated information obtained from the CB database 18 or through user interaction is also included in the SADT participant information category. Any other information obtained from the user interface is placed in this section. The spatial information section includes information regarding the position of the element related to the SADT in the scene. This position is extracted from the semantic structure 16. Spatial information also includes spatial constraints that apply to SADT elements for other non-action predicates. These spatial constraints are determined by using information contained in the 3D mapping database. Animation channels are generated from a 3D knowledge map database. The animation channel is the part of this participant that is animated when an action is applied to the participant. For example, when the action “walking” is applied to a human, the human foot is the animation channel. The reason is that the legs are animated and humans perform the walking action. If the action “walking” is applied to a snake, the animation channel is the entire body of the snake. The classification of dynamic information has modifier predicates that affect the emotional state, physical state, behavior, and the like. Dynamic information is extracted from the semantic structure 16. The graph information section represents the SAAT associated with the SADT.

According to one embodiment, the temporal order in the system is addressed by using the following assumptions (1) and (2).
(1) The execution of the event is terminated when it is introduced into the text.
(2) The execution of a fluent begins when it is introduced into the text, and this execution is terminated as a result of a clear end occurring in the text or as some other fluent / event is introduced. Continue until you mark. The APIR matrix provides information about how events or fluent introductions affect existing fluents. The predicate is tagged as an event or fluent, and this information is retrieved from the 3D mapping database.

  The above description refers to a SAAT with six sections and a SADT with four sections, but the number of these sections has all the information necessary to produce an animation in the template. It should be understood that as long as it can be changed.

  An example of a conceptual structure generated by the SAAT and SADT generator 24 is shown below.

The following text is entered into the NLP module:
Sentence 1: “Fred walks very quickly to the table.”
Sentence 2: “Sally looks at him and talks to Mary.”
Sentence 3: “Mary slowly gives a red book to Paul while he sits on a chair.”
Sentence 4: “Peter tiptoes happily to the table and runs to the door.”

  4, 7, 8 and 9 show the semantic structures output by the NLP module 12 for these sentences 1, 2, 3 and 4, respectively. The adjective “red” associated with the word “book” in sentence 3 does not generate a node in its corresponding semantic structure in FIG. The reason is that the template generator 24 has access to the syntactic structure of this sentence. The template generator 24 determines the book color using the syntactic structure output by the NLP module 12.

  FIG. 10 shows an example of a conceptual structure 50 output from the template generator 24. SAAT 52 is generated for sentence 1, and two SADTs 54 and 56 are associated with SAAT 52. By using the semantic structure of FIG. 7, two SAATs 58 and 60 are generated. SAATs 58 and 60 are located below SAAT 52 to indicate that the actions described by these SAATs 58 and 60 occur after the actions described by SAAT 52. Also, these SAATs 58 and 60 are located on the same line to indicate that the actions described by these SAATs 58 and 60 occur simultaneously. The template generator 24 generates SAATs 62 and 64 using the semantic structure shown in FIG. SAATs 66 and 68 corresponding to the semantic structure shown in FIG. 9 are located one below the other because the actions associated with these SAATs 66 and 68 occur sequentially.

  The conceptual structure 50 contains all the information required to generate an animation and is sent to the animation generator. This animation generator reads out this conceptual structure and extracts all information. The animation generator uses this information to generate an animation corresponding to the input text.

  A group of individual elements connected to each other via individual data signal connections is shown in block diagram form, but, as will be understood by those skilled in the art, a preferred implementation by combining hardware and software elements. An example is obtained, where many of the illustrated data paths are achieved by data communication within a computer application or operating system. Accordingly, the depicted structure is intended to disclose a preferred embodiment of the invention.

  The present invention can be implemented as a method and can be performed on a system, a computer readable medium, or an electrical or electro-magnetic signal. The above-described embodiments of the present invention are illustrative only, and the scope of the present invention is limited only by the claims.

Claims (23)

A method of converting input text to input for an animation generator,
Receiving the input text;
A first set of data representing information associated with the action identified in the input text is extracted from the input text, and at least the first set of data is used to complete a semantically annotated action template. Steps,
A second set of data representing information relating to the description of any participant included in the action is extracted from the input text and is semantically annotated with at least the second set of data Steps to complete
Transmitting the semantically annotated action template and the semantically annotated description template to the animation generator. The method of claim 1, wherein extracting the first set of data from the input text comprises:
Information related to the action predicate, and
Semantic information for this action predicate,
Time information extracted from fluent / event,
A method comprising the step of extracting estimated information for common sense inference. 3. The method of claim 1 or 2, wherein extracting the second set of data from the input text comprises:
Participant information related to the participant in action,
Spatial information related to the position of the participant in the scene,
Dynamic information affecting at least one of the participant's emotional state, physical state and behavior;
Extracting a link that joins a participant to the action predicate.   3. The method according to claim 2, wherein the information related to the action predicate includes at least one of event / fluent information related to the action predicate and information on a precondition using a semantic structure / information on a subsequent condition. Method.   The method according to claim 2, wherein the semantic information for the action predicate includes at least one of a semantic structure for the action predicate and modulator information.   The method according to claim 2, wherein the time information includes at least one of a time link between an action predicate and another action predicate, and animation end information. 7. The method according to any one of claims 1-6, further comprising:
Receiving user information;
Using the user information to complete at least one of the semantically annotated action template and the semantically annotated description template.   3. The method according to claim 2, wherein the estimation information includes information related to bidirectionality of coexisting events.   4. The method of claim 3, wherein the participant information includes at least one of information related to an animated / non-animated state of the participant, a semantic role of the participant, an emotional state, and an animation channel. How to have one.   4. The method of claim 3, wherein the spatial information comprises at least one of information related to a participant's spatial location and information related to a spatial constraint applied to the participant. The method according to any one of claims 1 to 10, wherein the method further comprises:
Determining a plurality of actions from the input text;
Extracting the first set of data for each of these actions;
Completing the semantically annotated action template for each of these actions;
Extracting the second set of data for each participant associated with each of these actions;
Completing at least one said semantically annotated description template for each said participant;
Linking at least one said semantically annotated description template to a corresponding semantically annotated action template;
Generating a conceptual structure by temporally ordering the completed semantically annotated action templates.   2. The method of claim 1, wherein the step of extracting the first set of data from the input text and the step of extracting the second set of data from the input text are stored as semantic information. Executed using information contained in at least one of a database of a dynamic background, a three-dimensional mapping database storing action predicate definitions, and a predicate interaction database storing common sense knowledge about the action predicates how to.   The method of claim 1, wherein the step of completing the semantically annotated action template and the step of completing the semantically annotated description template include conceptual background for storing semantic information. A method of executing using information contained in at least one of a database of the above, a three-dimensional mapping database storing action predicate definitions, and a predicate interaction database storing common knowledge about the action predicates . A system that converts input text into input for an animation generator,
A natural language processing module that receives the input text and outputs a semantic structure;
A conceptual background database for storing semantic information;
A predicate interaction database that stores common sense knowledge about action predicates;
A three-dimensional mapping database storing action predicate definitions and associated parameters;
A first template that receives the semantic structure from the natural language processing module and represents information related to actions, and a second template that represents information related to descriptions of all participants included in the actions, A system comprising: a template generator that is automatically completed using information contained in the database and that sends the first template and the second template to the animation generator.   15. The system of claim 14, wherein the template generator temporally converts the plurality of first templates such that each of the plurality of first templates corresponds to one of the actions identified from the input text. A system designed to generate a conceptual structure by ordering.   15. The system according to claim 14, wherein the natural language processing module performs at least a tokenization step, a tagging step, a parsing step, and a semantic role labeling step.   15. The system of claim 14, wherein the first template comprises a first field for information related to an action predicate, a second field for semantic information for the action predicate, and a fluent / event. A system having a third field for extracted time information and a fourth field for estimation information for common sense inference.   15. The system of claim 14, wherein the second template is associated with a first field for participant information associated with a corresponding participant in action and a position of the participant in the scene. A second field for spatial information to play, a third field related to dynamic information affecting at least one of the participant's emotional state, physical state and behavior, and the party And a fourth field associated with a link that couples a shippant to the corresponding action.   16. The system of claim 15, wherein the conceptual structure includes a graph for temporally ordering the plurality of first templates.   15. The system of claim 14, wherein the natural language processing module is further adapted to generate a syntactic structure for the input text.   21. The system of claim 20, wherein the template generator is further configured to complete at least one of the first template and the second template using the syntactic structure. A method for representing information extracted from text for use in creating an animation, the method comprising:
Complete the semantically annotated action template by generating information related to the action predicate, semantic information for this action predicate, time information extracted from the fluent / event, and estimated information for common sense inference Step to
Participant information related to the participant in action, spatial information related to the position of the participant in the scene, and at least one of the emotional state, physical state, and behavior of the participant. Completing at least one semantically annotated descriptive template by generating dynamic information that influences one another and a link that joins the participant to the action;
A method in which the template contains all the syntactic and semantic parameters that need to be used for the animation. A system that parses natural language text describing an action and creates an ordered action structure that should be used to create an animation,
A natural language processing module that receives the natural language text as input and outputs a semantic structure;
A conceptual background database for storing semantic information;
A predicate interaction database that stores common sense knowledge about action predicates;
A three-dimensional mapping database storing action predicate definitions and associated parameters;
Automatically generate semantically annotated action templates that generate conceptual structure and represent information related to actions, and semantically annotated description templates that are related to the description of any participants involved in these actions A sequencing action structure generator for completion, wherein the template is completed using the semantic structure and information contained in the database.

Images