CN120407821A

CN120407821A - Information processing method, device, apparatus, and computer-readable storage medium

Info

Publication number: CN120407821A
Application number: CN202510560901.4A
Authority: CN
Inventors: 范硕; 耿凯; 吴峥
Original assignee: BOE Technology Group Co Ltd; Beijing BOE Technology Development Co Ltd
Current assignee: BOE Technology Group Co Ltd; Beijing BOE Technology Development Co Ltd
Priority date: 2025-04-29
Filing date: 2025-04-29
Publication date: 2025-08-01

Abstract

An information processing method, apparatus, device, and computer-readable storage medium are disclosed. The information processing method includes: in response to obtaining memory information for a target object, obtaining memory material based on the memory information; and generating record information for the memory information based on the memory material. The method of the disclosed embodiments can automatically generate record information without requiring the user to provide complete and rich material, thereby reducing the requirements for the user's expressive and writing skills, saving the user time in writing record information, and better meeting the user's spiritual needs.

Description

Information processing method, apparatus, device, and computer-readable storage medium

Technical Field

Embodiments of the present disclosure relate to an information processing method, apparatus, electronic device, and computer-readable storage medium.

Background

With the improvement of living standard, the mental demands of people are growing. For example, people may have a need to record interesting things that have happened or recall things that often cannot be met due to a fast pace of life, lack of time, etc.

Disclosure of Invention

At least one embodiment of the present disclosure provides an information processing method including, in response to acquiring memory information for a target object, acquiring memory material based on the memory information, and generating record information for the memory information based on the memory material.

For example, the information processing method provided by at least one embodiment of the present disclosure further includes providing guidance information, wherein the memory information is acquired based on the guidance information.

For example, in the information processing method provided in at least one embodiment of the present disclosure, the guidance information includes at least one of a plurality of field types of memory information, a plurality of element types of memory information, a plurality of modality types of memory information, a plurality of memory blocks.

For example, in the information processing method provided in at least one embodiment of the present disclosure, the providing guidance information includes providing the plurality of domain types of memory information, and providing a plurality of recommendation information of a target domain corresponding to a target domain type in response to a selection operation of the target domain type in the plurality of domain types, wherein the guidance information further includes the plurality of recommendation information.

For example, in the information processing method provided in at least one embodiment of the present disclosure, the providing guidance information further includes providing the plurality of element types of the memory information in response to a selection operation of a target recommendation information of the plurality of recommendation information, and providing the plurality of modality types of the memory information in response to a selection operation of a target element type of the plurality of element types.

For example, in an information processing method provided in at least one embodiment of the present disclosure, providing a plurality of recommendation information of a target domain corresponding to the target domain type includes querying a hypergraph for the target object based on the target domain corresponding to the target domain type to obtain a query result, and determining and providing the plurality of recommendation information based on the query result, wherein the hypergraph for the target object is generated based on image information of the target object and is updated according to history memory information for the target object acquired at a history time.

For example, in an information processing method provided in at least one embodiment of the present disclosure, the providing of the guidance information includes determining question information of a guidance dialog for the target object based on image information for the target object, and providing the question information of the guidance dialog.

For example, in the information processing method provided in at least one embodiment of the present disclosure, the determining problem information of a guiding dialogue for the target object based on image information for the target user includes determining topic information for the target object based on a hypergraph for the target object, and generating problem information of the guiding dialogue for the target object based on the topic information, wherein the hypergraph for the target object is generated based on the image information of the target object, and is updated according to history memory information for the target object acquired at a history time.

For example, in the information processing method provided in at least one embodiment of the present disclosure, the providing the question information of the guiding dialogue includes providing the question information of the guiding dialogue by a virtual character for the target object.

For example, in the information processing method provided in at least one embodiment of the present disclosure, the information processing method further includes performing statistical analysis on reply information provided by the target object for the question information of the guiding dialogue in response to obtaining the reply information, and providing the statistical analysis result to an associated object associated with the target object, wherein the statistical analysis result is at least used for representing the expression capability of the target object.

For example, in the information processing method provided in at least one embodiment of the present disclosure, the information processing method further includes determining that the acquired memory information includes the input information in response to detecting the input information after any one of the guidance information is selected.

For example, the information processing method provided by at least one embodiment of the present disclosure further includes transmitting the input information in response to a sharing operation of the input information, and determining that the acquired memory information further includes information obtained based on the supplemental information in response to receiving the supplemental information filled in for the input information.

For example, in an information processing method provided in at least one embodiment of the present disclosure, acquiring a memory material based on the memory information includes extracting key information of the memory information, and acquiring the memory material based on the key information.

For example, the information processing method provided by at least one embodiment of the present disclosure further includes determining, in response to obtaining the security requirement information, an entity for which the security requirement information is aimed, and performing desensitization processing on the entity for which the security requirement information is aimed in the memory information to obtain desensitized memory information, where the extracting key information of the memory information includes identifying the desensitized memory information to obtain key information of the memory information.

For example, in the information processing method provided in at least one embodiment of the present disclosure, the memory information includes at least two pieces of information belonging to at least two modalities, the key information includes information extracted for each of the at least two pieces of information, the acquiring the memory material based on the key information includes integrating the key information to obtain integrated information, and obtaining the memory material based on the integrated information.

For example, in the information processing method provided in at least one embodiment of the present disclosure, the obtaining the memory material based on the integrated information includes providing the integrated information, and obtaining adjusted information in response to an adjustment operation on the integrated information, where the memory material includes the adjusted information.

For example, in an information processing method provided in at least one embodiment of the present disclosure, the memory information includes a video, the extracting key information of the memory information includes extracting a key video frame in the video and an audio clip corresponding to the key video frame in the video, fusing visual features of the key video frame and audio features of the audio clip corresponding to the key video frame to obtain a fused feature, and determining the key information of the memory information based on the fused feature.

For example, in the information processing method provided in at least one embodiment of the present disclosure, the determining key information of the memory information based on the fused features includes determining key content included in the audio clip, generating description text for the key video frame based on the fused features and the key content, and extracting the key information from the description text.

For example, in the information processing method provided in at least one embodiment of the present disclosure, the obtaining the memory material based on the key information includes generating image information corresponding to the memory information based on the key information, where the memory material includes the image information and the key information.

For example, in the information processing method provided in at least one embodiment of the present disclosure, the generating image information corresponding to the memory information based on the key information includes extracting an image element from existing image information for the target object based on the key information, and generating image information corresponding to the memory information based on the image element.

For example, in an information processing method provided in at least one embodiment of the present disclosure, the memory information includes a video, the key information includes at least two key video frames extracted from the video and at least two audio clips corresponding to the at least two key video frames, the generating image information corresponding to the memory information based on the key information includes dividing the at least two key video frames into at least one video frame group based on play time information of the at least two key video frames, and generating a video clip based on the key video frames included in the video frame group and the audio clips corresponding to the key video frames for each video frame group.

For example, in an information processing method provided in at least one embodiment of the present disclosure, the extracting key information of the memory information includes extracting a plurality of key video frames in the video and a plurality of audio clips in the video corresponding to the plurality of key video frames, respectively, determining importance of content expressed by each of the key video frames and the audio clips corresponding to each of the key video frames, and determining at least two of the key video frames and at least two of the audio clips corresponding to the at least two of the key video frames, respectively, in which the importance of the expressed content satisfies a preset condition.

For example, in the information processing method provided in at least one embodiment of the present disclosure, the generating image information corresponding to the memory information based on the key information includes generating initial image information based on the key information, providing the initial image information, and adjusting the initial image information based on the adjustment information in response to receiving adjustment information fed back for the initial image information, so as to obtain image information corresponding to the memory information.

For example, in an information processing method provided in at least one embodiment of the present disclosure, the generating record information for the memory information based on the memory material includes querying a knowledge graph based on the memory material, obtaining knowledge information for the target object, and generating record information for the memory information based on the knowledge information and the memory material, wherein the knowledge graph is constructed based on a knowledge base matched with the target object.

For example, in the information processing method provided in at least one embodiment of the present disclosure, the step of querying a knowledge graph based on the memory material to obtain knowledge information for the target object includes querying the knowledge graph based on the memory material to obtain a plurality of candidate information, and filtering the plurality of candidate information based on a recommendation algorithm to obtain knowledge information for the target object.

For example, the information processing method provided by at least one embodiment of the present disclosure further includes performing emotion analysis on the memory information to determine emotion tendency information, wherein the generating of record information for the memory information based on the memory material includes generating record information for the memory information based on the emotion tendency information and the memory material.

For example, the information processing method provided by at least one embodiment of the present disclosure further includes updating a hypergraph for the target object based on the memory material, the hypergraph for the target object being generated based on image information of the target object.

For example, the information processing method provided by at least one embodiment of the present disclosure further includes recording update information of the hypergraph in response to update of the hypergraph, and outputting the hypergraph and highlighting the update information in the hypergraph in response to a first viewing request for the update information, wherein the first viewing request is generated in response to an operation of a first object having a hypergraph viewing authority.

For example, in an information processing method provided in at least one embodiment of the present disclosure, the update information includes a plurality of update sub-information corresponding to a plurality of update time periods, respectively, the outputting the hypergraph and highlighting the update information in the hypergraph in response to a first view request for the update information includes determining at least one update sub-information corresponding to a time period indicated by the first view request among the plurality of update sub-information in response to the first view request, and outputting the hypergraph and highlighting at least one update sub-information in the hypergraph.

For example, in an information processing method provided in at least one embodiment of the present disclosure, the generating record information for the memory information based on the memory material includes querying a hypergraph for the target object based on the memory material to obtain an associated material of the memory material, and generating record information for the memory information based on the memory material and the associated material, wherein the hypergraph for the target object is generated based on image information of the target object, and is updated according to history memory information for the target object acquired at a history time.

For example, in the information processing method provided in at least one embodiment of the present disclosure, the step of querying the hypergraph for the target object based on the memory material to obtain the associated material of the memory material includes determining a query policy for the hypergraph based on the requirement information for the record information, and querying the hypergraph based on the memory material and the query policy to obtain the associated material.

For example, the information processing method provided in at least one embodiment of the present disclosure further includes determining layout information of a plurality of the record information for a plurality of the memory information, respectively, based on a predetermined design format, and providing at least a part of the record information based on the layout information in response to a second viewing request for at least a part of the record information among the plurality of the record information, wherein the second viewing request is generated in response to an operation of a second object having a record viewing authority.

For example, the information processing method provided by at least one embodiment of the present disclosure further includes outputting the record information based on a predetermined output format in response to an output request for the record information, and transmitting the record information in response to a sharing operation for the record information, wherein the output request is generated in response to an operation of a third object having an output right.

For example, the information processing method provided by at least one embodiment of the present disclosure further includes performing encryption processing on the memory information and the recording information to obtain encrypted information, and storing the encrypted information.

The information processing apparatus includes a material acquisition module configured to acquire memory material based on memory information for a target object in response to acquiring the memory information, and an information generation module configured to generate record information for the memory information based on the memory material.

The at least one embodiment of the present disclosure further provides an electronic device, including a processing device, and a storage device including one or more computer program instructions, where the one or more computer program instructions, when executed by the processing device, implement the information processing method provided by the at least one embodiment of the present disclosure.

At least one embodiment of the present disclosure also provides a computer-readable storage medium that non-transitory stores computer-readable instructions, wherein the computer-readable instructions, when executed by a processor, implement the information processing method provided by at least one embodiment of the present disclosure.

Drawings

In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following brief description of the drawings of the embodiments will make it apparent that the drawings described below relate only to some embodiments of the present disclosure and are not limiting of the present disclosure.

FIG. 1 schematically illustrates an application scenario diagram of an information processing method and apparatus provided by at least one embodiment of the present disclosure;

FIG. 2 schematically illustrates a flow diagram of an information processing method provided by at least one embodiment of the present disclosure;

FIG. 3 schematically illustrates a flow diagram of an information processing method provided by at least one further embodiment of the present disclosure;

FIG. 4 schematically illustrates a schematic diagram of guidance information provided in at least one embodiment of the present disclosure;

FIG. 5 schematically illustrates a schematic diagram of guidance information provided in at least one other embodiment of the present disclosure;

FIG. 6 schematically illustrates a schematic diagram of generation of recommendation information in at least one embodiment of the present disclosure;

FIG. 7 schematically illustrates a schematic diagram of extracting key information in at least one embodiment of the present disclosure;

FIG. 8 schematically illustrates a schematic diagram of generating record information in at least one embodiment of the present disclosure;

FIG. 9 schematically illustrates a schematic diagram of generating recorded information in at least one other embodiment of the present disclosure;

FIG. 10 schematically illustrates a schematic diagram of an output hypergraph in at least one embodiment of the present disclosure;

FIG. 11 schematically illustrates a block diagram of an information processing apparatus provided by at least one embodiment of the present disclosure;

FIG. 12 schematically illustrates a structural diagram of an electronic device provided in accordance with at least one embodiment of the present disclosure, and

Fig. 13 schematically illustrates a schematic diagram of a computer-readable storage medium provided by at least one embodiment of the present disclosure.

Detailed Description

For the purpose of making the objects, technical solutions and advantages of the embodiments of the present disclosure more apparent, the technical solutions of the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings. It will be apparent that the described embodiments are some, but not all, of the embodiments of the present disclosure. All other embodiments, which can be made by one of ordinary skill in the art without the need for inventive faculty, are within the scope of the present disclosure, based on the described embodiments of the present disclosure.

Unless defined otherwise, technical or scientific terms used in this disclosure should be given the ordinary meaning as understood by one of ordinary skill in the art to which this disclosure belongs. The terms "first," "second," and the like, as used in this disclosure, do not denote any order, quantity, or importance, but rather are used to distinguish one element from another. Likewise, the terms "a," "an," or "the" and similar terms do not denote a limitation of quantity, but rather denote the presence of at least one. The word "comprising" or "comprises", and the like, means that elements or items preceding the word are included in the element or item listed after the word and equivalents thereof, but does not exclude other elements or items. The terms "connected" or "connected," and the like, are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "upper", "lower", "left", "right", etc. are used merely to indicate relative positional relationships, which may also be changed when the absolute position of the object to be described is changed.

It will be appreciated that prior to using the technical solutions disclosed in the embodiments of the present disclosure, the user should be informed and authorized of the type, usage range, usage scenario, etc. of the personal information related to the present disclosure in an appropriate manner according to the relevant legal regulations.

For example, in response to receiving an active request from a user, a prompt is sent to the user to explicitly prompt the user that the operation it is requesting to perform will require personal information to be obtained and used with the user. Thus, the user can autonomously select whether to provide personal information to software or hardware such as an electronic device, an application program, a server or a storage medium for executing the operation of the technical scheme of the present disclosure according to the prompt information.

As an alternative but non-limiting implementation, in response to receiving an active request from a user, the manner in which the prompt information is sent to the user may be, for example, a popup, in which the prompt information may be presented in a text manner. In addition, a selection control for the user to select to provide personal information to the electronic device in a 'consent' or 'disagreement' manner can be carried in the popup window.

It will be appreciated that the above-described notification and user authorization process is merely illustrative and not limiting of the implementations of the present disclosure, and that other ways of satisfying relevant legal regulations may be applied to the implementations of the present disclosure.

It will be appreciated that the data (including but not limited to the data itself, the acquisition or use of the data) involved in the present technical solution should comply with the corresponding legal regulations and the requirements of the relevant regulations.

With the improvement of living standard, the mental demands of people are growing. For example, people may have a need to record interesting things or recall things that have happened, but often the need is not met due to a fast pace of life, lack of time for people, etc., or due to a limited expressive power.

For example, as the degree of aging increases, the proportion of empty-nest elderly people increases. The writing and recall of the memory can help the old to share the own energy, emotion and story, help the old to express emotion of depth of heart and relieve the feeling of autism. And the writing and recall of the memory can also enable the old to feel self-value, strengthen the contact with families and friends and promote the understanding and recognition among family members. However, the recall records have higher requirements on the word expression capability of the author, and the recall persons are required to provide materials as abundant as possible, and all the factors can increase the writing difficulty of the recall records, so that the recall records are not beneficial to the elders to get emotion care.

For example, current child growth is of home and social concern. Especially for children who first go into school, parents can hardly feel curiosity about activities of children in collective life such as school, and also want to record spot drips in children' life. However, the expression capability and writing capability of the children are limited, parents are usually busy with work and have no time to record, and often interesting events in the growth process of the children cannot be recorded in time, so that families lose a lot of precious spirit and wealth.

In order to at least partially solve the above-described problems, an embodiment of the present disclosure provides an information processing method including, in response to acquiring memory information for a target object, acquiring memory material based on the memory information, and generating recording information for the memory information based on the memory material.

Some embodiments of the present disclosure also provide an information processing apparatus, an electronic device, and a computer-readable storage medium corresponding to the above information processing method.

The information processing method provided by at least one embodiment of the present disclosure may process the memory information provided by the user for the target object, and automatically generate the record information for the memory information. Therefore, the automatic generation of the recorded information can be realized without providing complete and rich materials for the user, the requirements on the expression capability and the writing capability of the user can be reduced, the time for writing the recorded information by the user is saved, and the mental requirements of the user are better met.

Embodiments of the present disclosure and some examples thereof are described in detail below with reference to the attached drawings. An application scenario of the information processing method and apparatus provided in the embodiments of the present disclosure is described schematically in the following with reference to fig. 1.

Fig. 1 schematically illustrates an application scenario of an information processing method and apparatus provided in at least one embodiment of the present disclosure.

As shown in fig. 1, in an exemplary application scenario 100, a terminal device 120 and a user of the terminal device are included. The user of the terminal device may be any user, for example, the elderly 111, the children 112, any legal guardian, relative, friend, etc. of the elderly 111 or the children 112, any person having a close relationship or being identified with the elderly 111 or the children 112, etc., or any user having a requirement for recording information, which is not limited in the embodiments of the present disclosure.

For example, the terminal device 120 may be a smart phone, tablet, laptop portable computer, desktop computer, wearable device, smart appliance, or the like. The terminal device 120 may be provided with a man-machine interaction interface, for example, to acquire memory information in response to a user operation and generate recording information based on the memory information. For example, the terminal device 120 may be installed with an instant messaging class application, a video play class application, a diary generation class application, a recall generation class application, a handwriting class application, etc., to which embodiments of the present disclosure are not limited.

For example, the terminal device 120 may generate the record information based on a generative artificial intelligence model or the like. The generated artificial intelligence model may be, for example, any large model such as a large language model or a multi-modal large model, which is not limited by the embodiments of the present disclosure. For example, a large model refers to a machine learning model with large scale parameters and complex computational structures, typically built from deep neural networks with billions or even billions of parameters. The generated artificial intelligence model may also be, for example, a distillation model of a large model, etc., as embodiments of the present disclosure are not limited in this regard.

For example, the application scenario 100 may further include a server 130, and the terminal device 120 may be communicatively connected to the server 130 through a network. The server 130 may be a background management server that provides support for the running of client applications installed in the terminal device 120, a blockchain server, etc., which is not limited by the embodiments of the present disclosure.

It should be noted that, the information processing method provided in at least one embodiment of the present disclosure may be performed by the terminal device 120, or may be performed by the server 130, or may be performed by a part of steps performed by the terminal device 120 and another part of steps performed by the server 130. Accordingly, the information processing apparatus provided in at least one embodiment of the present disclosure may be provided in the terminal device 120, may be provided in the server 130, may be provided in part in the terminal device 120 and another part in the server 130,

The implementation principle of the information processing method provided in at least one embodiment of the present disclosure will be explained and described below with reference to fig. 2 to 10.

Fig. 2 schematically illustrates a flowchart of an information processing method according to at least one embodiment of the present disclosure.

As shown in fig. 2, the information processing method 200 of this embodiment includes steps S210 to S220.

In step S210, in response to the acquisition of the memory information for the target object, the memory material is acquired based on the memory information.

Step S220, record information for the memory information is generated based on the memory material.

According to embodiments of the present disclosure, the memory information may be a short description of a memory segment entered by a user, may be an image and/or video segment taken at a historic time, may be an audio segment recorded at a historic time, or the like. For example, the embodiment may extract key information from the memory information and use the key information as the memory material. The key information may include, for example, entities, actions, etc. involved in memorizing the information. It should be noted that the user and the target object may be the same object, or the user may be any object allowed through the target object other than the target object.

For example, the memory information may include information of at least one of a plurality of modalities. The plurality of modalities may include, for example, a text modality, an image module, an audio modality, a video modality, and the like. For the memory information of different modes, the mode of extracting the key information is different to a certain extent.

For example, the memory information of the text modality may include text related to recall of the target object entered by the user through the input device. The text may express the perception of the target object, the target object's interest, or an important event associated with the target object, etc. The embodiment can perform semantic analysis on the memory information by using a natural language processing technology to extract characters, time, places, emotion keywords, topics and the like from the memory information, and takes the extracted information as key information.

Illustratively, the memory information of the image modality may include images uploaded by the user in relation to recall of the target object, such as old photos, taken mementos, etc. The embodiment can process the memory information by adopting an image recognition model to extract information such as characters, places, article categories, theme categories and the like in the memory information, and takes the extracted information as key information.

For example, the memory information of the audio modality may include speech entered by the user in real-time through a microphone. The embodiment can adopt natural language processing technology to convert the memory information into text information, and perform semantic analysis on the text information so as to extract characters, time, places, emotion keywords, subjects and the like from the text information, and take the extracted information as key information.

For example, speech recognition (Automatic Speech Recognition, ASR) techniques may be employed to convert the entered speech into text information and to perform semantic understanding and analysis of the text information based on a deep-learned semantic analysis model. For example, the semantic analysis model may correspond to an age bracket to which the age of the target object belongs. For example, taking a target object as a child as an example, the semantic analysis model can be trained by a large number of language samples of the child, so that the spoken and simplified expression intention of the child can be accurately understood, and key information such as activity content, characters, time, places and the like can be extracted. For example, when the recorded voice is a voice corresponding to "playing football at playground and colleagues today," the analyzed key information may include, for example, "playground", "playing football", "colleagues" and the like.

Illustratively, the memory information of the video modality may include pre-recorded video material uploaded by the user, such as videos of family parties, travel recorded videos, and the like. The embodiment can extract the key frames and the audio clips corresponding to the key frames from the videos included in the memory information, synthesize a short video which can completely reflect the contents contained in the key frames and the audio clips corresponding to the key frames, and take the short video as the key information. By the method, redundant content and flat and odorless content in the video can be removed, the memory occupied by the generated recorded information is reduced, and more efficient and accurate video recall is provided for users. Or the embodiment may also adopt the processing mode of the memory information of the image mode described in the above embodiment to process the extracted key frame, adopts the processing mode of the memory information of the audio mode described in the above embodiment to process the audio segment corresponding to the key frame, and collates and merges all the extracted information, thereby obtaining the key information. It should be understood that the manner of acquiring the memory material based on the memory information of the video modality described above is merely an example, and specific implementation may also be referred to in the following description, which is not limited by the embodiments of the present disclosure.

According to embodiments of the present disclosure, memory material may be input into a generative artificial intelligence model from which logging information is generated. For example, the generative artificial intelligence model may be a transducer (transducer) architecture based model or other similar high performance language model, etc., as embodiments of the present disclosure are not limited in this regard.

For example, the record information may include a diary generated based on the memory material or a recall generated based on the memory material, and the record information may be text information or multi-modal information, which is not limited in the embodiment of the present disclosure.

According to the information processing method provided by at least one embodiment of the disclosure, through processing the memory information provided by the user aiming at the target object to obtain the memory material and generating the record information aiming at the memory information based on the memory material, the automatic generation of the diary or the recall record can be realized without providing complete and rich materials by the user, the requirements on the expression capability and the writing capability of the user can be reduced, the time for the user to write the diary or recall record is saved, and the mental requirements of the user are better met. By the information processing method provided by the embodiment of the disclosure, experience of the target object can be tangible, and the generated record information can be used as a record of the past and can also be used as a profound dislike and summary of self-growth and life journey of the target object. The design concept of the information processing method is to help a target object to create a unique recall record through personalized content and flexible editing functions.

Fig. 3 schematically illustrates a flowchart of an information processing method provided in at least another embodiment of the present disclosure.

In at least one embodiment of the present disclosure, the memory information may be acquired, for example, based on guidance information provided by the terminal device. For example, the terminal device may first provide the guidance information, and in response to the user providing information for the guidance information, the provided information is taken as the memory information.

As shown in fig. 3, the information processing method 300 of this embodiment may include steps S310 to S330, and the step S330 may be performed before the step S310, for example.

In step S310, in response to the acquisition of the memory information for the target object, the memory material is acquired based on the memory information.

Step S320, record information for the memory information is generated based on the memory material.

Step S330, providing guidance information.

According to the embodiment of the present disclosure, the implementation principles of step S310 and step S320 are similar to the implementation principles of the foregoing step S210 and step S220, respectively, and are not repeated here.

According to embodiments of the present disclosure, the guidance information may be any information that facilitates the memory of the motivating target object. For example, the guidance information may include at least one of a plurality of field types of memory information, a plurality of element types of memory information, a plurality of modality types of memory information, a plurality of memory tiles.

Illustratively, the plurality of field types of the memory information may include any field type set according to actual demands, such as sports type, movie type, song type, television type, social news type, work experience type, interpersonal relationship type, daily life type, etc., which may relate to various aspects in daily activities, for example.

Illustratively, the plurality of element types of the memory information may include a person type, a place type (may also be an environment type), an action type (may also be a story type), and the like. The plurality of element types may, for example, encompass all elements in a discipline-like text composition (e.g., may also be a novel composition).

The plurality of modality types of the memory information may include, for example, the aforementioned text type, audio type, image type, video type, and the like, by way of example. The plurality of modality types may, for example, encompass a plurality of manifestations of the information.

Illustratively, the plurality of memory slabs may include, for example, "childhood recall," "student age," "work achievement," "retirement life," "family life," and the like. The plurality of memory tiles may, for example, be capable of covering various stages in a user's lifetime on a timeline, and/or may cover various major scenarios of a user's lifetime, as embodiments of the present disclosure are not limited in this regard.

According to an embodiment of the present disclosure, the guidance information may also be question information of a guidance dialog for the target object determined based on portrait information of the target object, for example. The issue information may include, for example, one or more issues.

For example, the representation information of the target object may include, for example, static and dynamic properties of the target object. The static attribute may include, for example, information of age, gender, academic, etc. Dynamic properties may include, for example, interactive objects, events, travel patterns, purchasing preferences, and the like. Taking the target object as a child as an example, the interaction object can comprise various campus characters such as teacher, principal staff, classmate A and the like of various disciplines, and various relatives such as table brothers, aunt, mother's brother and the like. The activity occasions may include, for example, various campus occasions such as common classrooms, playgrounds, libraries, reading corners, and the like, and various campus occasions such as parks, amusement parks, families, and the like, and the embodiments of the present disclosure are not limited thereto. The representation information may be entered in advance by the target object or a guardian of the target object, for example. For example, a terminal device executing the information processing method according to the embodiment of the present disclosure may be provided with a visual interactive interface, where the visual interactive interface may be implemented by using a graphical user interface design technology, and is easy and intuitive to operate, and for example, portrait information of a target object may be entered by a preset drop-down menu selection in the visual interactive interface or a manner of filling in a text input box. For example, the terminal device may store the image information in a structured database, and may employ a relational database management system to perform efficient data storage, retrieval and management to ensure data integrity and accuracy.

For example, a dialog generation algorithm in natural language processing technology may be used to generate a series of targeted and heuristic questions based on the image information of the target object, and the questions may be used as guiding information to guide the target object to call the relevant memory. For example, where the target object is a child, the questions may include "what is interesting today in the classroom," what you do with classmate a, and the like, and embodiments of the present disclosure are not limited in this regard.

In some embodiments, issue information of a guided dialog for a target object may also be determined based on the portrait information and/or a library of information maintained for the target object (e.g., may be a hypergraph for the target object) to ingeniously throw out issues, guide the target object to recall scene details, express a mental feeling. For example, when the target object is the elderly, the question information includes questions that may include "what you do with friends on that family party" and so on.

According to embodiments of the present disclosure, guidance information may be provided to a user or a target object so that the user or the target object provides memory information for the target object based on the guidance information. For example, the guidance information may be presented in an interactive interface or may be output in the form of voice. For example, in the case of outputting in the form of voice, the prompt information may be output in voices of different styles according to the age of the target object. For example, in the case where the target object is a child, the prompt information may be output in a personally relevant style. For example, in the case of outputting the prompt information in the form of voice, the prompt information can be synchronously displayed in the interactive interface at the same time so as to adapt to learning and cognition styles of target objects in different age periods.

For example, issue information for guiding a dialog may also be provided by a virtual character for a target object. The language style, emotion expression mode, image and the like of the virtual character of the object can be finely adjusted according to the actual scene, the preference of the target object and the like. Therefore, the provision of the problem information can be made more anthropomorphic, so that a more realistic communication scene is created for the target object, the memory of the target object can be better stimulated, and the accuracy and the integrity of the acquired memory information are improved.

According to the embodiment of the present disclosure, it is possible to determine that the acquired memory information includes the input information by detecting the input information after any one of the guidance information is selected. For example, in the case of presenting the guidance information, information input in a page presented after the guidance information is selected may be taken as detected input information, and in the case of outputting the guidance information in the form of voice, detected voice information may be taken as detected input information.

In at least one embodiment of the present disclosure, the input information detected after any one of the guiding information is selected may be further used as part of the memory information, and a function of sharing the input information may be provided for the user, so that the sharee may assist the user in providing more abundant and complete memory information for the target object. For example, the terminal device may transmit the input information in response to a sharing operation of the input information. For example, the input information may be sent to a background management server that provides support for the operation of a client application that is operated in the terminal device and is used to generate the record information, or may be sent to a background management server that provides support for the operation of an instant messaging application, where the background management server may, for example, push the received input information to the terminal device used by the sharee, so that the sharee may view the input information and fill in the supplemental information for the input information. The filled-in supplementary information may be transmitted, for example, to a background management server that provides support for the running of the client application for generating the record information, and pushed by the background management server to the terminal device used by the sharer. For example, a terminal device used by a sharer or a background management server providing support for the running of a client application for generating record information may determine that the acquired memory information further includes the supplemental information in response to receiving the supplemental information filled in for the input information. Through the technical scheme of the embodiment, the integrity and the richness of the content expressed by the obtained memory information can be improved, the integrity and the richness of the finally generated record information can be improved, the solitary feeling of the target object can be reduced through a sharing mode, and the user experience can be improved.

For example, after the user uploads or inputs information, the sharing link may be clicked to send the input information to a parent, who may enter the remote entry interface, for example, by clicking on the sharing link, through which supplemental information for the input information may be filled in. The client application for generating the record information may, for example, obtain the memory information based on the supplemental information. The supplemental information may be converted from a third person-based representation to a first person perspective representation based on natural language processing techniques or large language models, for example, or the converted information may be used as part of the memory information.

According to at least one embodiment of the present disclosure, by providing the guiding information, the target object may be stimulated to gradually evoke the memory, so that the accuracy and the integrity of the obtained memory information are improved, the accuracy and the validity of the finally generated recording information are improved, and the user experience is improved.

At least some embodiments of providing guidance information are described in detail below in conjunction with fig. 4-5. Fig. 4 schematically illustrates a schematic diagram of guidance information provided in at least one embodiment of the present disclosure.

In an embodiment, as shown in fig. 4, when a user needs to generate recording information using a terminal device, a client application for generating recording information in the terminal device may display an interactive interface 410, and in the interactive interface 410, a plurality of field types 411 for memorizing information are provided first. The plurality of domain types 411 may include "song channels", "movie channels", "sports events", "work experiences", "interpersonal relationship", and the like. It is to be understood that the plurality of domain types 411 shown in fig. 4 are merely examples to facilitate understanding of the present disclosure, and embodiments of the present disclosure are not limited in this respect.

For example, the terminal device may provide a plurality of element types 412 of the memory information in response to a selection operation of a first target domain type among the plurality of domain types 411. For example, in response to a user selection of "song channel," a plurality of element types 412 are added to the interactive interface 410, which plurality of element types 412 may include "persona," "environment," "story," to direct the target object to motivate a persona, environment, story, etc. associated with the song. For example, the first target domain type may be any of a plurality of domain types 411, and the plurality of element types shown to trigger different domain types may be the same or different, for example, and embodiments of the present disclosure are not limited thereto.

For example, the terminal device may provide a plurality of modality types 423 for memorizing information in response to a selection operation of a target element type among the plurality of element types 412. For example, the target element type may be any element type of a plurality of element types, e.g., the terminal device may switch the interactive interface 410 to the interactive interface 420 in response to a selection operation of the "person" element type, where the "text" modality type, the "image" modality type, the "video" modality type, and the "audio" modality type are presented in the interactive interface 420. It is to be appreciated that where the target element types are different element types, the plurality of modality types provided may be the same or different, and embodiments of the present disclosure are not limited in this regard.

According to the technical scheme, multi-level guiding information (such as a field level, an element type level and a modal level) can be provided for the target object, so that the target object is stimulated to continuously evoke memory, and the accuracy and the integrity of the obtained memory information are improved. It will be appreciated that the above-described hierarchical levels of guidance information are merely examples to facilitate an understanding of the present disclosure, and embodiments of the present disclosure are not limited thereto.

Fig. 5 schematically illustrates a schematic diagram of guidance information provided in at least another embodiment of the present disclosure.

In an embodiment, after providing the plurality of domain types of the memory information, for example, in response to a selection operation of a target domain type among the plurality of domain types, a plurality of recommended information of the target domain corresponding to the target domain type may be provided, and the provided guidance information includes the plurality of recommended information.

According to embodiments of the present disclosure, a plurality of recommendation information may be determined from portrait information of a target object. For example, a plurality of pieces of information of which the user of the age group to which the target subject's age belongs is of high interest in the target area may be provided to the target subject according to the age of the target subject. For example, if the target area is "song channel", a plurality of songs having the highest playing amount among songs played by the user of the age group may be recommended, if the target area is "movie channel", a plurality of movies having the highest scores among movies evaluated by the user of the age group may be recommended, and if the target area is "work experience", a plurality of professions having a large number of people in the industry in which the user of the age group is engaged in may be recommended, or a plurality of work occasions, such as occasions of meeting, occasions of business trip, or the like, may be recommended. It is to be understood that the above-described recommendation information is merely an example to facilitate understanding of the present disclosure, and embodiments of the present disclosure are not limited thereto.

For example, a plurality of recommendation information may be displayed in parallel in the interactive interface, or may be displayed in a scrolling manner in the interactive interface for selection by a user.

In an embodiment, as shown in fig. 5, a plurality of recommended information may be displayed in an alternative manner, and the target object may be better stimulated to evoke memory than in a manner in which a plurality of recommended information are displayed simultaneously. After "song channel" in the plurality of domain types 511 presented in the interactive interface 510 is selected, the interactive interface 510 may switch to the interactive interface 520, in which "song a"521 in the plurality of recommended songs is presented in the interactive interface 520. For example, the terminal device may update the interactive interface 520 to the interactive interface 530 in response to a sliding operation of the user in the interactive interface 520 along the target direction, in which interactive interface 530 "song B"531 among the plurality of recommended songs is presented. For example, the target direction may be a certain direction of the width direction of the interactive interface 530, for example, may be an arrow direction as shown in fig. 5. For example, "song a"521 and "song B"531 may be any two of the plurality of recommended information, or the recommended order of the plurality of recommended information may be an order in which the degree of recommendation is highest to low, and "song a"521 and "song B"531 may be songs ranked first and second by degree of recommendation. The degree of recommendation of the recommendation information can be determined, for example, based on the degree of matching of the recommendation information with the image information of the target object. It will be appreciated that the sliding operation described above is merely one exemplary manner of triggering the switching of recommendation information, and the recommendation order is also merely an example to facilitate understanding of the present disclosure, and embodiments of the present disclosure are not limited thereto.

According to the embodiment of the disclosure, the interactive interface can be switched to the interactive interface of a plurality of element types for displaying the memory information in response to the selection operation of the target recommendation information in the plurality of recommendation information. For example, the interactive interface may also be switched to an interactive interface that displays a plurality of modality types in response to a selection operation of a target element type of the plurality of element types, which is not limited by the embodiments of the present disclosure. For example, according to the actual requirement, the interactive interface may also be switched to an interactive interface displaying a plurality of modality types in response to a selection operation of the target recommendation information in the plurality of recommendation information.

In at least one embodiment of the present disclosure, in response to a selection operation of the target domain type, a drop-down menu or an input box may be further provided in the interactive interface, so that a user selects or inputs a time period corresponding to the recommendation information. The determined plurality of recommendation information may be information generated at a time period selected or entered by the user.

In one embodiment, for the target domain type, the content of the target domain may be recommended for the target object according to the portrait information of the target object and the selected (or input) period. For example, if the target field is "song channel", and the selected or input time period is 1 st 2012 to 12 nd 2012, then a plurality of songs whose release time or flow time is within 1 st 2012 to 12 nd 2012 may be pushed to the target object, if the target object has a relevant memory for a target song in the plurality of songs, the target song may be clicked, the terminal device may pop up the type of the associated memory (which may also be understood as the element type) in response to the operation, and then may enter the memory information input interface. For example, the acquired memory information may be audio entered by voice, and the audio may be, for example, audio of a target song humming a target object together with friends.

It will be appreciated that the foregoing embodiments of providing guidance information are merely examples to facilitate an understanding of the disclosure, and that the embodiments of the disclosure are not limited in this respect. The technical scheme of the guide information provided by the embodiment of the disclosure aims at providing a flexible and convenient material input mode for a user, supporting input of various modal types, enhancing recall awakening of a target object and improving experience of generating recorded information.

Fig. 6 schematically illustrates a schematic diagram of generating recommendation information in at least one embodiment of the present disclosure.

In at least one embodiment of the present disclosure, a unique knowledge graph may be maintained in advance for each user to represent an association relationship between various information of each user. When providing recommendation information of a target field to a target object, a knowledge graph for the target object may be queried first, and a plurality of recommendation information provided to the target object may be determined based on the query result. For example, a knowledge graph for a target object may be queried with the target domain as a query condition. The knowledge graph can better organize, manage and utilize mass information and can reflect the association relation between the information, so that the information associated with the target field in various information of the target object can be queried through querying the knowledge graph, the probability that the recommendation information generated based on the queried information has the association relation with the target object is high, the probability that the recommendation information is selected can be improved, and the guiding effect on the target object is improved.

In an embodiment, the knowledge-graph for the target object may be a hypergraph (HYPER GRAPH) structure, i.e., a hypergraph for the target object. The hypergraph for the target object can express the multiple relations among various information of the target object so as to improve the richness and the accuracy of the expressed information. Accordingly, as shown in fig. 6, when determining recommendation information of a target domain, a hypergraph 610 for a target object may be queried based on the target domain 601 to obtain a query result 602. The provided plurality of recommendation information 603 is then determined based on the query results 602.

For example, hypergraph g= (V, E) consists of two sets. V is a set of nodes, E is a set of superedges, each node in the set of nodes representing an information element, each superedge representing a relationship between a set of nodes (also referred to as superedge nodes). The information elements may be entities, events, concepts, etc.

For example, the target domain 601 may be used as a query condition to query the hypergraph 610, and an entity or event belonging to the target domain in information represented by a node connected by a hyperedge to the node corresponding to the target domain may be determined and used as the query result 602. After obtaining the query result, the query result may be used as recommendation information 603, for example. For example, the determined entity or event may be used as the candidate information, and then a preset number of information may be selected from the candidate information as the recommendation information 603 according to the weight of the superside connected between the node representing the entity or event and the node representing the target field.

For example, the embodiment can also adopt a hypergraph learning enhanced recommendation algorithm to realize the query of the hypergraph aiming at the target object, and obtain the query result. For example, when the hypergraph learning enhanced recommendation algorithm is adopted, query search can be performed on the global hypergraph so that the determined recommendation information is more accurate. For example, a global hypergraph may refer to all hypergraphs for all users.

According to embodiments of the present disclosure, a hypergraph 610 for a target object may be generated, for example, based on the representation information 606 of the target object. For example, an initial hypergraph frame may be constructed according to basic information of the target object, such as name, age, sex, living place, educational experience, work experience, living experience, etc., provided in advance by the user, to obtain a hypergraph for the target object. For example, the initial hypergraph frame comprises a plurality of general node categories and preliminary association relations, and lays a foundation for the subsequent integration and expansion of recall contents. For example, for a user of a particular age group, nodes related to popular culture, educational background, professional development, etc. may also be predefined in the hypergraph according to common interests and experiences of the age group, and preliminary connections established to reflect common lifestyle and potential recall associations. For example, if the target object is young and a fan is fierce, then the classic sporting event, the well-known star dynamics, and the social activity nodes associated with the target object may be preferentially associated during the young period of the target object when the initial hypergraph frame is constructed.

According to the embodiment of the disclosure, the hypergraph for the target object may be updated according to the history memory information for the target object acquired at the history time, for example. For example, after the history material is acquired according to the history information, the hypergraph for the target object may be updated according to the history material. For example, after the memory material 605 is acquired based on the memory information 604 acquired in real time, the hypergraph 610 for the target object may be updated based on the memory material 605 in addition to the memory information generated based on the memory material.

For example, as described above, when the memory material includes extracted entities, actions, etc., in updating the hypergraph based on the memory material, the embodiment may first add nodes to each entity in the memory material, and then add the hyperedges between the nodes according to the internal association between the information expressed by the newly added nodes and the information expressed by the existing nodes in the hypergraph, thereby implementing the updating of the hypergraph.

Illustratively, hypergraph techniques may organize and associate recall information of users through complex data structures. For example, the information represented by the nodes in the hypergraph may include a variety of different entities including, for example, people, places, events, hobbies, songs, movies, and the like. For example, when a piece of recall is entered for a target object regarding participation in an event in conjunction with certain people at a particular place, the place, people, nodes corresponding to the event may be created separately and superb established based on the inherent association therebetween.

For example, the hypergraph may be dynamically expanded and updated as new recall fragments for the target object are entered. Taking the example of the user selecting "song channel" and entering recall for song a, one can first find the node category "song" in the hypergraph and create a node therein that represents song a. Then, based on the described recall details such as events for learning away from K songs with classmates, interpersonal relationships to be appreciated by classmates, and a specific life stage (place and time information) of high school, nodes corresponding to classmates, place and time information are created, respectively, and closely connected with the nodes of song a by passing edges. In this way, one superside in the supergraph aiming at the target object can be simultaneously connected with the node of the song A, a plurality of classmates nodes participating in the K song, the site node of the K song and the time period node of the high school, so that the interaction between the complex situation and the multiple elements of the recall is comprehensively reflected.

According to at least one embodiment of the present disclosure, through dynamic updating and expanding of the hypergraph, the hypergraph can be continuously optimized along with the continuously recorded recall content for the target object, and rich details and complex network relations of life experiences of the target object can be accurately expressed through the hyperedge.

In at least one embodiment of the present disclosure, the determined recommendation information may be, for example, a recall cue. For example, when determining recommendation information based on hypergraph, the hidden link between different memory segments can be found by analyzing node connection mode and weight of superedge in hypergraph (the weight can be dynamically adjusted according to factors such as recall frequency and emotion intensity), so as to provide more coherent and deep recall clues for target objects. For example, hypergraphs may be used to mine potential associations between memories and provide personalized recommended content for users.

For example, if it is found by querying the hypergraph that a plurality of song nodes related to a period of high school are all frequently connected to the same group of classmates nodes and specific social activity nodes, it may be inferred that the music activity of the period of high school is of great importance in the life of the target object, the target object may be recommended to further recall and supplement more details about the interleaving of music hobbies and interpersonal relationships of this period, or to provide some background information and topics related to the popular music culture at that time (since the hypergraph is updated continuously based on the memory information for the target object, updated hypergraph is taken into account in the subsequent recommendation, so that more accurate, more interesting content may be recommended to the target object) to enrich the recall content and emotion hierarchy.

Illustratively, when updating the hypergraph, with the memory information (i.e. recall content) provided by the user for the target object, the consistency and the difference between the newly input information and the existing hypergraph structure can be automatically analyzed, and the hypergraph is further optimized and adjusted through a machine learning algorithm (such as a clustering algorithm in unsupervised learning, an association rule mining algorithm, and the like).

For example, if a plurality of memory segments which are newly input and aimed at a target object are found to surround a hobbies or social circles which are not fully paid attention to before, nodes and superedges related to the field can be automatically enhanced in a hypergraph, and the organization and association mode of recall of the field are continuously perfected and refined through comparison and reference with the hypergraph modes of other similar users, so that the understanding capability and the presentation effect of the system on recall content of the user are improved, and more accurate, personalized and insight-enriched recall creation support is provided for the user, so that the user can better review and collect own unique life stories.

In at least one embodiment of the present disclosure, in the case that a hypergraph for the target object is constructed, for example, the problem information of the guiding dialogue for the target user described above may also be generated by relying on the hypergraph. For example, determining question information for a guided dialog for a target user based on portrait information for the target object includes determining topic information for the target user based on a hypergraph for the target user, and generating question information for the guided dialog for the target user based on the topic information. By generating the problem information by depending on the hypergraph, the interest degree of the target object on the problem information can be improved, the target object is stimulated to arouse memory, and the user experience is improved.

For example, according to the memory association in the hypergraph, the character relationship, event venation and scene in the hypergraph can be analyzed, and various life scenes and social occasions of the deep memory of the target object, such as a warm family party, a campus class full of book fragrance, a old street bazaar with the longevity of the user, and the like, can be estimated. The embodiment can take the estimated life scene, social occasion and the like as topic information of the target user. Question information for the guided dialog for the target user can then be generated based on the topic information, for example by means of a large language model. For example, the process of analyzing and supposing topic information can be implemented by adopting a recommendation algorithm with enhanced hypergraph learning, and the result supposedly obtained by the recommendation algorithm is topic information of a target user. It will be appreciated that the principles of generating topic information described above are merely examples to facilitate an understanding of the present disclosure and embodiments of the present disclosure are not limited in this respect.

In at least one embodiment of the present disclosure, in addition to providing problem information of a guiding dialogue in a scenario where a guiding target object provides memory information, in a memory recovery scenario where the target object loses part of memory due to physical reasons, the problem information of the guiding dialogue is provided to the target object to excite the target object to evoke memory, which is beneficial to improving rehabilitation effect of the target object.

For example, in the memory recovery scenario, in response to obtaining the answer information provided by the target object for the question information of the guidance dialog, statistical analysis may be performed on the answer information to obtain a statistical analysis result at least for characterizing the expression ability of the target object. The statistical analysis result may be provided to an associated object associated with the target object, for example. For example, the associated object may be a guardian of the target object, an attending physician, or the like, a parent of the target object, or the like, as embodiments of the present disclosure are not limited in this regard. By providing the statistical analysis result to the associated object, for example, the associated object can assist in accurately evaluating the memory recovery condition of the target object, so as to facilitate timely adjustment of the treatment strategy for the target object.

For example, the integrity, diversity, etc. of the information expressed by the reply information may be analyzed. For example, a pre-trained text quality assessment model or the like can be used for processing the reply information to obtain a statistical analysis result. For example, statistical analysis results may also be used to characterize the mental activity of a target object, etc., as embodiments of the present disclosure are not limited in this regard.

In one embodiment, the statistical analysis result may include a vocabulary, emotion change, etc. included in the reply information. For example, the answer information may be subjected to word segmentation processing, and the number of words obtained after the invalid words and the stop words are removed may be used as the vocabulary. For example, the reply information may be divided into a plurality of sentences arranged in time series, emotion categories of the plurality of sentences may be determined by using a text emotion classification model, and emotion category sequences of the plurality of sentences arranged in time series may be used as statistical analysis results expressing emotion changes.

According to the embodiments of the present disclosure, after the memory information is acquired, for example, key information in the memory information may be extracted first, and then memory material may be acquired based on the key information. For example, as described above, the extracted key information may be used directly as memory material.

In an embodiment, the memory information may include at least two pieces of information belonging to at least two modalities, respectively, and the memory information may include one or at least two pieces of information belonging to each modality. When extracting the key information in the memory information, the key information may be extracted for each information included in the memory information, and the extracted key information includes the key information extracted for each information of at least two pieces of information, which may be understood as extracting at least two pieces of key information corresponding to the at least two pieces of information, respectively. In this embodiment, when the memory material is obtained based on the key information, the key information may be integrated to obtain integrated information, and then the memory material may be obtained based on the integrated information.

For example, the modalities of memorizing the information including the information may include at least two of text, image, video, audio, and the like.

For example, if there may be a coincidence in the key information extracted from at least two pieces of information, only one key information may be retained for the coincidence, and the rest may be removed, thereby achieving integration of the key information. The embodiment can use the integrated information as memory material. In an embodiment, a weight may be further set for each key information, and for the overlapping key information, the set weight may be higher to obtain integrated information.

For example, the integration of the key information may include, for example, classification and organization of the key information to ensure that the key information of each type complements each other to form a complete recall statement.

In at least one embodiment of the present disclosure, after the integrated information is obtained, for example, the integrated information may also be provided to the user, and a function of adjusting the integrated information may be provided to the user. For example, the adjusted information may be obtained in response to an adjustment operation on the integrated information. The adjusted information may be used as the memory material or as part of the memory material.

For example, a user-friendly editing interface can be provided, and a user can modify and supplement the integrated information, for example, text editing, image adjustment and video editing can be supported, so that the user can customize the memory materials according to personal preference, the generated recording information can more meet the requirements of the user, and the user experience is improved.

For example, the integrated information may be arranged in a time sequence, and provided in a time axis, so that a user can review and understand the context of past experiences, and the setting of the time axis can help the user to better understand the cause and effect relationship and the background of the event.

The principle of extracting key information of the memory information of the video modality will be described in detail with reference to fig. 7. Fig. 7 schematically illustrates a schematic diagram of extracting key information in at least one embodiment of the present disclosure.

In at least one embodiment of the present disclosure, for example, for a video in the memory information, a key video frame and an audio clip corresponding to the key video frame in the video may be extracted first. Then, the visual features of the key video frames and the audio features of the audio clips can be fused to obtain fused features. And finally, determining key information for the video included in the memory information based on the fused features. For example, when the memory information includes only video, the key information for the video is the key information of the extracted memory information.

For example, the video may be decoded and preprocessed, for example, prior to extracting the key video frames and audio clips. For example, an input video as memory information may be decoded using a video codec library (e.g., a multimedia framework FFmpeg), decomposed into a sequence of successive video frames, and simultaneously an audio stream extracted, with the audio converted to a standard format (e.g., WAV) for subsequent processing. The method can also perform preliminary processing on the video frame sequence, such as unifying parameters of resolution, color space and the like, ensures the consistency and compatibility of each video frame, and is convenient for subsequent feature extraction and analysis operation.

For example, inter-frame differences may be calculated, and key video frames determined based on a metric of inter-frame differences and a shot boundary detection algorithm. A key video frame may be understood as a video frame that has a large change in visual content from neighboring frames, possibly containing important visual information. For example, feature vectors for each video frame may be calculated based on visual features of the video frame, with inter-frame differences being measured as Euclidean or cosine distances between feature vectors of adjacent video frames. For example, visual features of a video frame may include local features and/or global features. For example, the local features may include features extracted using image recognition and description algorithms such as Scale-invariant feature transforms (Scale-INVARIANT FEATURE TRANSFORM, SIFT) and/or accelerated robust features (Speeded Up Robust Features, SURF). For example, the global features may include color histograms and/or gradient direction histograms, etc. For example, visual features of video frames may also be extracted by employing a deep learning model, as embodiments of the present disclosure are not limited in this regard.

For example, high quality video frames may also be screened out of video frames determined based on the inter-frame difference metric and the shot boundary algorithm as key video frames in combination with the results of the image quality assessment. It will be appreciated that key video frames may represent, for example, key points of change and important scenes of video in visual content.

For example, short-time Fourier transforms (Short-Time Fourier Transform, STFT) and/or Mel frequency cepstral coefficients (Mel Frequency Cepstrum Coefficient, MFCC) may be employed to extract spectral features and/or semantic features of the audio segment as the audio features of the audio segment.

For example, visual features of the key video frames may be fused with corresponding audio features to construct multi-modal feature vectors. For example, a vector representing the visual feature of the key video frame may be spliced with a feature vector representing a mel-frequency cepstrum coefficient or the like of an audio clip near the moment of the key video frame to form a new fusion feature representation, so as to comprehensively consider the visual and auditory information of the video and more comprehensively capture important content in the video.

For example, the embodiment may input the fused features to an image description generation model, which generates text describing the picture content of the key video frame as key information. Or the fused features can be input into the image recognition model, and the information output by the image recognition model is used as key information.

For example, as shown in fig. 7, in an embodiment, after extracting a key video frame 702 included in a video 701 and an audio segment 703 corresponding to the key video frame 702, and fusing visual features 704 of the key video frame with audio features 705 of the corresponding audio segment to obtain a fused feature 706, an image description generation model may be used to generate description text 707 describing the key video frame, for example. Subsequently, the key information describing the text 707 may be extracted as the key information 708 for the video included in the memory information, using the principle of extracting the key information of the memory information of the text modality described above. The embodiment of the disclosure determines the key information by adopting the mode of generating the description text describing the video frame and then extracting the key information from the description text, so that the consistency of formats and the like of the key information extracted from the memory information of different modes is facilitated, and the key information extracted for the information of different modes is conveniently integrated.

For example, the image description generation model may employ a deep learning-based encoder-decoder architecture, for example, and may be composed of a convolutional neural network and a long-short-term memory network. For example, the image description generation model may be trained on data using a large number of image-text to learn the mapping relationship from image features to natural language descriptions. For example, the descriptive text generated by the image description generation model may describe the main elements, scenes and action information in the key video frames as detailed as possible, for example, the descriptive text may be "a person in a picture is looking at sunset at sea, and sea waves are beating the beach".

In at least one embodiment of the present disclosure, the category and location information of a particular object in a key video frame, etc., may also be considered and incorporated into the descriptive text, for example, when generating descriptive text 707. For example, a target detection algorithm may be used to identify the category and location of the object in the key video frame, the descriptive text generated by the image description generation model may be used as the initial text, and the descriptive text 707 may be obtained by merging the identified category and location of the object into the initial text. For example, the resulting descriptive text 707 may be "a yellow dog running on a green lawn to play at a cheerful time" in the center of the screen. By integrating the category and position information of the object in the key video frame, the accuracy and the richness of the obtained description text can be improved, and more visual and specific content text of the key video frame can be provided for the user.

In at least one embodiment of the present disclosure, key content included in the audio clip may also be considered, for example, in generating descriptive text 707. For example, in determining key information of the memory information based on the post-fusion features, the key content 709 included in the audio clip 703 may be determined first. Descriptive text 707 for the key video frames is then generated based on the fused features 706 and the key content 709. The description text obtained by the embodiment can more completely reflect the key content contained in the key video frames and the corresponding audio clips.

For example, for an audio clip, an audio classification model may be employed to identify and label different types of sounds in the audio clip, such as speech, music, environmental sound effects, etc., from which a starting timestamp of the different types of sounds in the audio clip may be determined, for example. For example, the audio classification model may be a deep learning based convolutional neural network, and the input to the audio classification model may be an audio segment or the aforementioned audio feature. This embodiment may use the identified sound type, mood description, and/or prominent ambient sound effects as key content 709 of the audio clip. For example, the embodiment may take the descriptive text generated by the image description generation model as the initial text, then convert the key content 709 into text form, and associate and integrate with the initial text, thereby obtaining the descriptive text 707.

For example, if a voice dialogue is identified in an audio segment corresponding to a certain key video frame, the voice dialogue is taken as a section of dialogue about a travel plan, the dialogue content can be transcribed and added into an initial text to obtain a description text, wherein a figure in a picture is enclosed at a desk, a map and a travel manual are placed on the desk, the figure in the audio is discussing the formation of a tomorrow, and the plan is to visit a museum first and then to visit a local special restaurant.

In at least one embodiment of the present disclosure, in the process of acquiring the memory material based on the key information, for example, image information corresponding to the memory information may be generated based on the key information, and both the image information and the key information may be used as the memory material. For example, the image information may be an image or a video, which is not limited in the embodiments of the present disclosure. By the embodiment, the richness of the memory materials can be improved, and the vividness and the infectivity of the generated record information can be improved. For example, both elderly people reviewing the past and the next years and children recording campus life often face the dilemma of lack of image data. By the technical scheme of the embodiment of the disclosure, diversified image data can be generated for users according to text description and the existing small amount of image resources.

For example, a generator may be employed to generate image information based on the key information. For example, the embodiment may convert the key information into text feature vectors, which are then input into a generator, which generates the visual information. For example, the generator may be a generator included in a pretrained Generative Antagonism Network (GAN) large model. For example, a GAN large model consists of a generator and a arbiter. In the training stage, the generator is responsible for generating a preliminary image according to the input text feature vector, and the discriminator is used for generating the feature difference of the image and the real image through comparison, so that parameters of the generator are continuously optimized, and the image generated by the generator is more vivid and accords with the text description. In the training process, massive image data comprising various scenes, characters and activities are used, so that a generator obtained through training can have strong generalization capability and can generate diversified images according to different text descriptions.

For example, taking a descriptive text of a college period riding activity interesting object, which aims at memory information of a target object as a section of memory of the target object, a semantic analysis model in a natural language processing technology can be adopted to deeply analyze the descriptive text. The model is based on a deep learning architecture, and key elements in the descriptive text can be accurately identified by learning a large number of daily activity descriptive text, including information such as time (such as a certain year), places (specific places of riding activities), people (college colleges), actions (riding), event details (specific situations of interesting things) and the like. The information may be, for example, key information.

In an embodiment, for example, a geographic information system (Geographic Information System, GIS) data and a historical image database may be combined to further locate an activity site and extract scene characteristics, obtain information such as topography, relief, surrounding environment, etc. of the site, and use the information as a part of key information to provide geographic and environmental background information for subsequent image generation.

For example, for the descriptive text of the active interest of the college periodic riding, the generator can generate a picture conforming to the current scene according to the extracted character features, the place features and the activity features so as to show a joy picture that the colleagues play on the specific place, and the vivid visual elements are added to the recall of the user. For example, character features may include facial features, stature scale, etc., learned from existing classmates. The location features may include, for example, road conditions, scenery elements, and the like. The activity features may include, for example, riding posture, vehicle type, etc., to which embodiments of the present disclosure are not limited.

In at least one embodiment of the present disclosure, the image information may be, for example, image information obtained based on expansion and fusion of existing image resources. For example, when generating the image information, the image element may be extracted from the existing image information for the target object based on the key information, and then the image information corresponding to the memory information may be generated based on the image element.

For example, the extracted image elements may be in one-to-one correspondence with entities included in the key information. For example, if the key information includes a target object, a friend a, and a playground, the extracted image element may include an image of the target object, an image of the friend a, and an image of the operation, which are extracted from the existing image information. After obtaining the image elements, for example, image fusion and video editing techniques in computer graphics may be applied to combine and recreate different image elements, thereby obtaining image information corresponding to the memory information. By generating the image information based on the image elements in the existing image information, the fidelity of the generated image information can be improved, the authenticity of the finally obtained recorded information can be improved, and the user experience can be improved.

For example, for a child recall descriptive text or voice information provided by a school event, further visual data generation may be performed using previously entered scene photos, videos of the school event, and visual data of the child interaction object. For example, feature extraction and labeling can be performed on existing images through image recognition and video analysis technologies, and the feature extraction and labeling comprises information such as scene layout, object types, person identities and the like. Then, according to the activity details of the recall of the children, different image elements extracted from the existing images are combined and re-authored by using the image fusion and video editing technology in computer graphics. For example, if a child recalls a scene of a sports meeting in a school playground, the system can select a proper background from panoramic photos of the existing playground, and draw and fuse images of people of the child and their classmates in other activities to generate a photo containing the specific classmates in the sports meeting scene, or combine a plurality of short segments into a complete simulated sports meeting video through video editing and special effect addition, and display various wonderful moments of the child in the sports meeting, such as running, rope skipping and other projects, so as to obtain image information corresponding to the memorized information.

For example, in an embodiment, image information of different styles can be generated according to the needs of the user, for example, cartoon-style cartoon can be generated to present the campus life of the child, so that the child is more vivid and interesting, and the aesthetic and cognitive characteristics of the child are met.

In at least one embodiment of the present disclosure, where the memory information provided comprises video, there may be instances where the video is longer and the expressed information comprises some redundant, unimportant information. In this embodiment, for example, a key video frame extracted from a video and an audio clip corresponding to the key video frame may be used as key information, and a video clip serving as image information may be generated based on the key information, so that the finally obtained image information may be more efficient and more accurate, which is beneficial to improving the conciseness and accuracy of the finally generated recording information, and is beneficial to improving the user experience.

For example, when generating the image information based on the key information, the embodiment may divide at least two key video frames in the key information into at least one video frame group based on the play time information of the at least two key video frames. Then, for each video frame group, a video clip may be generated based on the key video frame included in the video frame group and the audio clip corresponding to the key video frame. For example, the generated image information includes a video clip generated for each of the at least one video frame group.

For example, a merging operation may be performed on video frames within a group of video frames to obtain a continuous short video segment of significant importance. For example, in the video merging process, a video generation technology based on a transform architecture can be used for generating video clips so as to solve the problem of continuity and consistency between video frames, ensure smooth transition of video pictures and continuity of audio in the generated video clips, and avoid the condition of blocking or incoherence. It will be appreciated that the techniques employed in the video merging process described above are merely examples to facilitate an understanding of the present disclosure, and embodiments of the present disclosure are not limited in this respect.

For example, when dividing at least two key video frames into at least one video frame group, for example, in addition to the play time information, the content correlation between key video frames may be considered, for example, video frames having high correlation may be divided into the same video frame group.

In at least one embodiment of the present disclosure, when extracting key information from video, for example, video frames having a large change in visual content from neighboring video frames may be extracted as key video frames in the manner described above, and then only video frames, which are selected from the key video frames and whose importance of the expressed content satisfies a preset condition, may be used as key information. Therefore, the effectiveness and the accuracy of the key video frames which can be used as key information can be improved, and the conciseness and the accuracy of finally generated record information are further improved.

For example, for each key video frame, a trained multimodal classification model may be used to analyze the fused features obtained by fusing the visual features of the key video frame and the audio features corresponding to the key video frame, so as to determine whether the content expressed by each key video frame and the audio segment corresponding to the key video frame has a significant meaning. For example, the output of the multimodal classification model may be a level of importance. For example, importance levels may include levels that are very important, general, unimportant, very unimportant, etc., as set by actual demand. The embodiment can use a key video frame whose importance level is very important and important as a video frame whose importance of the expressed content satisfies a preset condition. Or the output of the multimodal classification model may indicate that the video frame is important or not, and the video frame indicating importance may be used as a video frame whose importance of the expressed content satisfies a preset condition, which is not limited by the embodiments of the present disclosure.

For example, the multimodal classification model may be a recurrent neural network model based on an attention mechanism or a model based on a transducer architecture, training data of the model may be constructed by a manual annotation mode, for example, the model may include a large number of video segments with significant content such as important events, character dialogues, key scene changes, and the like, and corresponding multimodal features and tag information indicating importance, and the model may identify feature modes of the significant content by learning the data.

In an embodiment, the generated video clips may be at least two, for example. After obtaining at least two video clips, the embodiment may further sort and order the at least two video clips, for example, the at least two video clips may be ordered according to the time sequence of the video frames corresponding to the video clips in the video serving as the memory information, or reorganized according to the relevance and importance of the content between the video clips, so as to obtain a short video set which is refined and contains the video key information.

In at least one embodiment of the present disclosure, image information generated based on the key information may be provided to a user as initial image information. The initial image information may then be adjusted based on the adjustment information in response to receiving the adjustment information fed back for the image information, and the resulting adjusted information may be used as the image information corresponding to the memory information. By the embodiment, the finally obtained image information can be more in line with the unique aesthetic and requirements of the user, and personalized and high-quality image information can be generated for the user.

For example, for the image data generating function, a friendly and convenient user interaction interface can be provided for a user, and the user can intuitively check the generated image information through the interface, so that the user can evaluate and feed back the image information conveniently. For example, the interface may provide a simple and easily understood operation button, which may be provided with options such as "satisfactory", "dissatisfied", "parts to be modified", etc. Or the user can also detail the adjustment information for the image information by means of text description or hand drawing sketch, for example, the adjustment information can be to adjust the expression of the person, change the background color or add specific objects, etc. After receiving the adjustment information, for example, an adjustment operation corresponding to the adjustment information may be performed on the initial image information based on the adjustment information, for example, an expression of a person in the initial image information may be adjusted, a background color of each video frame in the initial image information may be replaced, a specific object may be added to each video frame in the initial image information, and the embodiment of the present disclosure is not limited thereto.

For example, in an embodiment, based on the adjustment information, the parameters of the model for generating the image information may be adjusted and optimized according to the preference and the modification opinion of the user by using a reinforcement learning algorithm in machine learning, so that the image information generated next time more accords with the expectations of the user. Therefore, through continuous user feedback and model optimization iteration, the model can gradually learn the unique aesthetic and demands of the user, so that more personalized and high-quality image data are generated, richer and more accurate visual materials are provided for the creation of recorded information, and the manufacturing quality and user experience of the recorded information are greatly improved.

In at least one embodiment of the present disclosure, the generated recording information is not limited to a text form, for example, when the recording information is generated, image data such as images and/or videos included in the memory material may be embedded into a proper position of the text by adopting typesetting and image fusion technologies, and the insertion point of the image data is determined according to the text content and semantic structure of the text. For example, if the text in the recorded information includes "we see sunset at sea", an image of sunset at sea or a related video clip may be inserted at the corresponding location of the text to enhance the visual effect and the infectivity of the recorded information.

In at least one embodiment of the present disclosure, the acquired memory material may be stored in a system database, which may be accessed and managed by a user at any time, for example. For example, a client application for generating recorded information may be provided with a friendly user interaction interface to facilitate user browsing, modifying and supplementing memory materials and memory information.

For example, a user may view memory material derived based on key information and feedback and modify the memory material. For example, a client application for generating recorded information allows editing of incomplete or inaccurate information to ensure that the resulting recorded information is more authentic and personalized.

Fig. 8 schematically illustrates a schematic diagram of generating recorded information in at least one embodiment of the present disclosure.

According to embodiments of the present disclosure, a model matching a target object may be selected from generated artificial intelligence models that are fine-tuned for different types of users to generate recording information based on memory materials. For example, the model that matches the child may be a generative artificial intelligence model that is fine-tuned for the child writing characteristics. For example, a large number of children diary samples can be adopted to finely tune the model, so that the model can master the characteristics of vocabulary, grammar, sentence patterns, expression habits and the like of the children language, and diary contents conforming to the cognition level and language style of the children can be generated.

For example, when generating the record information, a preset style preference may also be considered to generate personalized record information for the target object using the conditional generation capability of the generated artificial intelligence model. The predetermined style preference types may include, for example, a conciseness type, a lively and lively type, a story-narrative type, and the like. For example, in the process of generating the recorded information, the generated artificial intelligent model not only can skillfully integrate the acquired key information into diary content, but also can combine context and language logic, and add proper connective words, adjectives, verbs and the like, so that the recorded information is more coherent, rich and vivid. For example, for the memory information of "playing football at playgrounds and classmates today," the generated vivid and lively recorded information may be, for example, "playing football at great popularity on wide playgrounds by me and classmates, with the sunlight being particularly bright today. We run full of sweat but are particularly happy, i have also entered a ball woolen-.

In at least one embodiment of the present disclosure, in the case that the target object is a child or any user who needs to learn knowledge, when the record information is generated based on the memory material, for example, knowledge points that need to be learned may also be fused into the record information, so as to facilitate the tight combination of the knowledge that needs to be learned by the target object and life, so that the process of learning the knowledge is more natural and relaxed, and is helpful to improve learning enthusiasm and knowledge absorptivity of the target object.

For example, after obtaining the memory material, the embodiment of the disclosure may first query a knowledge graph constructed based on a knowledge base matched with the target object based on the memory material, obtain knowledge information for the target object, and then generate record information for the memory information based on the knowledge information and the memory material.

For example, according to actual needs, the knowledge graph may be a knowledge graph covering knowledge of each discipline, or may be a knowledge graph for a specific field (e.g., a financial field, an artificial intelligence field, etc.). Knowledge maps can be stored and managed, for example, based on a map database (e.g., neo4 j). Taking a target object as an example of a child, the nodes in the knowledge graph can comprise various knowledge points, such as English words, ancient poetry, mathematical calculation formula, scientific concepts, historical events and the like, and the nodes are connected through semantic relations, wherein the semantic relations can comprise relations such as 'belonging subject', 'semantic relation', 'application scene'. For example, in the process of constructing the knowledge graph, data mining and semantic analysis can be performed on authoritative teaching materials, academic documents and online education resources in a large number of education fields, knowledge points and correlations thereof are extracted, and the knowledge graph is constructed based on the knowledge points and correlations thereof, so that the comprehensiveness and accuracy of the knowledge graph are ensured.

For example, after the memory material is obtained, knowledge points closely related to each element in the memory material can be found out through the query and reasoning functions of the knowledge graph to serve as potential learning content. And taking the found knowledge points as knowledge information for the target object. In the query, the key information included in the memory material is used as a query condition, and the query depth can be determined by limiting the number of knowledge points, for example. For example, when the target object is a child, if the target object refers to "art class in classroom", the memory material may include classroom, art class, and the like. For example, english words (e.g., paint, draw, color, etc.), ancient poems (e.g., poems describing painting will), mathematical formulas (e.g., mathematical formulas for calculating the number of painting tools and canvas area), etc. related to art may be queried in the knowledge map as knowledge information for the target object.

In at least one embodiment of the present disclosure, as shown in fig. 8, knowledge points obtained by querying a knowledge graph 810 based on memory materials 801 may be used as candidate information 802, for example, a plurality of candidate information may be queried. Subsequently, the embodiment may use the recommendation algorithm 803 to filter the plurality of candidate information, that is, filter the plurality of candidate information 802, and use the knowledge points obtained by filtering as knowledge information 804 for the target object.

For example, after the information related to the memory material is queried, the queried information can be related to a corresponding element (such as an art class) in the memory material, and then a collaborative filtering algorithm, a content-based recommendation algorithm or a mixed recommendation algorithm combining the collaborative filtering algorithm and the content-based recommendation algorithm is adopted to generate a personalized learning content recommendation list for the target object based on the related result, and learning content in the learning content recommendation list is used as knowledge information for the target object. For example, collaborative filtering algorithm discovers knowledge points learned by other users with similar activity experiences and learning behaviors by analyzing learning data of the other users in similar scenes, and uses the knowledge points as references for recommending knowledge information for the current target object. For example, the content-based recommendation algorithm can accurately recommend according to semantic similarity between the active content of the current target object and the knowledge points, and mainly considers factors such as correlation between the knowledge points and the active scene, learning stage of the target object and cognition level. The knowledge information of the target object obtained by the embodiment is more accurate and effective, and is beneficial to further improving the learning enthusiasm and the knowledge absorptivity of the target object.

For example, if the activity related to the provided memory information is "playground play", for the case that the target object is a child of a lesser age, the filtered knowledge information may include some simple english words, such as "ball", "run", "jump", and so on, and simple addition and subtraction formulas related thereto, such as "3 balls on playground, 2 balls in total. For the situation that the target object is older, the filtered knowledge information can comprise English sentences describing playground activities, ancient poems containing motion elements, mathematical knowledge related to geometric figure area calculation (such as perimeter and area calculation of playground runway shape) and the like, so that the recommended content is challenging and practical, and knowledge vision of children can be effectively expanded.

According to embodiments of the present disclosure, after knowledge information 804 is obtained, for example, knowledge information 804 and memory material 801 may be used together as input to a generated artificial intelligence large model from which record information 805 for the memory information is generated.

According to the embodiment of the disclosure, the recommended learning content can be skillfully embedded into the activity experience description of the target object by generating the record information based on the knowledge information obtained by the query, and presented to the target object in a natural and vivid mode, so that the interest and the contextuality of learning can be enhanced. For example, a large model of artificial intelligence can be generated, and learning content can be integrated into an active narrative according to the writing style and language habit of the diary of a target object to form a coherent text paragraph with educational significance.

For example, if the knowledge information obtained by filtering includes the english word "flower", the ancient poem "temple of Dalin" and the mathematical calculation "2+3=? the text generated embedded with learning content may be, for example," how beautiful flower is seen in garden today, "which reminds me of the poem of the ' four months of people's own, the temple flowers are beginning to bloom ', as if the flowers in our garden were struggling. For the mother, we ask me again, if 2 flowers are first put in the garden, then 3 flowers are put in the garden, and 2+3=5 (flowers) tweed is put in the garden, mathematics can really be interesting |).

In at least one embodiment of the present disclosure, when generating the record information, for example, the emotion tendencies of the content expressed by the memory information may also be considered, so as to increase the emotion depth of the generated record information and increase the infectivity of the record information.

For example, the information processing method may further include performing emotion analysis on the memory information to determine emotion tendency information. Subsequently, record information for the memory information may be generated based on the emotion tendencies information and the memory material. For example, emotion tendencies in the memory information may be identified using emotion analysis techniques, which may include, for example, emotion dictionary-based methods, conventional machine learning methods, deep learning methods, and the like, as embodiments of the present disclosure are not limited in this regard.

For example, emotion trend information and memory material may be used as inputs to a generative artificial intelligence model to generate record information from the generative artificial intelligence model. For example, information corresponding to important emotion moments in information generated by the generated artificial intelligence model may be highlighted based on emotion tendency information, and for example, an expression scheme of the corresponding information may be adjusted to an exaggerated expression scheme.

Fig. 9 schematically illustrates a schematic diagram of generating recorded information in at least another embodiment of the present disclosure.

In at least one embodiment of the present disclosure, when a hypergraph is generated based on image information of a target object and is continuously updated according to memory information, for example, the hypergraph for the target object may be queried based on the acquired memory material to obtain associated material of the memory material, and then record information for the memory information is generated based on the memory material and the associated material. The related materials are obtained by inquiring the hypergraph, and the recorded information is generated by combining the related materials, so that the richness of the generated recorded information can be improved, and the user experience can be improved.

For example, the hypergraph aiming at the target object can construct multi-layer complex network structures such as a hobby net, an interpersonal net, a working net and the like based on the portrait information and the provided memory information, for example, the target object is taken as a core, the hobby net can organically weave the memory nodes related to various hobby from small to large, and the germination, development, transition process and the forgetting experience of the hobby are recorded in detail. The personal network can comprehensively integrate various character relation nodes such as friends and relatives, colleagues and neighbors and the corresponding interactive memory segments to construct a huge and warm personal memory map. The work network can faithfully record each key node of careers of target objects, and all achievements, frustrations and growth are reserved in the form of nodes and connections from the initial job place to the achievement.

After the memory material is obtained, dynamic and accurate optimization screening can be developed on the memory data contained in the hypergraph and the multi-layer network structure by taking the memory material as a query condition, so as to obtain the associated material. After the associated material is obtained, the associated material and the memory material may be input together into a generative artificial intelligence model, and the record information for the memory information may be generated by the generative artificial intelligence model.

In at least one embodiment of the present disclosure, as shown in fig. 9, when a hypergraph is queried, a query policy 902 for the hypergraph may be determined first, for example, based on requirement information for memory information. Then, the hypergraph 910 for the target object is queried based on the memory material 901 and the query policy 902, thereby obtaining the associated material 903 of the memory material 901. After the associated material 903 is obtained, record information 904 for the memory information can be generated based on the memory material 901 and the associated material 903.

For example, if the requirement indicated by the requirement information for the record information is to generate the record information of a long length, when the hypergraph is queried, the graph searching algorithm can be adopted to deeply mine the memory rich ore, continuously expand details along the superside connected with the node corresponding to the memory material, extract rich information from a plurality of levels of associated nodes, and comprehensively carving story details to obtain a plurality of and all associated materials, so that the record information generated based on the associated materials and the memory materials is plump and detailed. If the requirement indicated by the requirement information aiming at the record information is short record information, when the hypergraph is inquired, a graph searching algorithm can be adopted to accurately lock the core node, and redundant information is removed according to indexes such as importance and relevance of the node, so that the associated material is obtained, and the record information generated based on the associated material and the memory material is short, exquisite and direct-hit.

For example, a centrality algorithm (CENTRALITY ALGORITHMS) in graph theory can be used to identify and pick out nodes and supersides which are at the core position in the whole supergraph, have higher influence and are representative according to the importance of the nodes and the connection tightness degree in the supergraph, so as to be used as key materials for generating short version record information. The selected materials not only can cover main venation and bright spots in life experience of the target object, but also can keep certain continuity and integrity through the associated information reserved by the hypergraph, so that the recorded information shows unique charm and core value of life of the target object within limited space, and personalized requirements of different users on the amount of recall records and topics are met. For example, the centrality algorithm may be a variation of a web page ranking algorithm, such as a centrality-of-degree (DEGREE CENTRALITY) algorithm or a centrality-of-proximity (Closeness Centrality) algorithm, etc., as embodiments of the present disclosure are not limited in this regard.

For example, the generated record information can be fused with multimedia elements such as graphics, audio, video and the like, and the embedding of files in various formats is supported.

For example, for the scene that the target object is the old man and needs to generate the recall, the client application operated by the terminal equipment for generating the record information can integrate the memory materials regularly, and generate the recall with the synchronization of the picture and the sound and the picture according to the preset theme (for example, my life) by applying the intelligent typesetting and multimedia fusion technology, and present the recall with the synchronization of the picture and the sound and the picture to the target object in a proper display form, such as a page with large fonts and high contrast on a tablet computer, so as to assist the target object to review life and strengthen memory, and simultaneously provide a window for medical staff and family members to know the inner world of the target object.

In at least one embodiment of the present disclosure, the client application that is executed by the terminal device and used for generating the record information may further provide a function of exporting the record information to the user, so as to facilitate the user or the target object to view at any time. For example, the information processing method may also output the memory information based on a predetermined output format, for example, in response to an output request for the recorded information.

In an embodiment, the information processing method may further send the record information, for example, in response to a sharing operation for the record information. For example, the record information may be sent to a background management server that provides support for the running of the instant messaging application, and the background management server pushes the record information to the instant messaging application running in the terminal device used by the sharees in a link form or the like, which is not limited in the embodiments of the present disclosure.

For example, the sharing operation may include an operation of sharing the recorded information to social media or to friends and/or family, for example. Through the sharing function, the transmissibility of the recorded information can be increased, and emotion communication between the target object and other people can be promoted.

For example, when the information is exported, provided and/or recorded, the typesetting of the recorded information can be optimized through a text typesetting engine, for example, proper fonts, font sizes, line spacing and paragraph formats can be set, so that the provided or exported recorded information is tidier and more attractive, and is easy for users to read and appreciate.

For example, the method can support the conversion of the recorded information into a format which is convenient to store and share, such as a picture format, a PDF format, a word document format or an electronic book format, or support printout, so that the user can conveniently store and share the recorded information, and further arouse the target object to wake up memory or record daily emotion and enthusiasm.

In at least one embodiment of the present disclosure, in case that a user provides a plurality of pieces of memory information describing multi-section memory of a target object through operations of different time periods, recording information may be generated for the plurality of pieces of memory information, respectively, resulting in a plurality of pieces of recording information. For example, the information processing method of the embodiment may further determine layout information of a plurality of pieces of recording information based on a predetermined design format for the plurality of pieces of recording information obtained. Upon subsequent detection of a request for viewing at least a portion of the plurality of recorded information, at least a portion of the recorded information requested for viewing may be provided based on the determined layout information.

For example, the predetermined design format may include a selected format of a selectable plurality of formats. The selectable plurality of formats may include, for example, diary format, story format, yearbook format, etc. Each of the plurality of formats corresponds to a typesetting mode. The embodiment may take a typesetting manner corresponding to a predetermined design format as the determined typesetting information such that the recording information provided based on the typesetting information is a style corresponding to the predetermined design format, such as a diary style, a story description style, a yearbook style, etc., to which the embodiment of the present disclosure is not limited. The record information requested to be checked is provided based on the typesetting information, so that the display effect of the record information is improved in appearance and easy to read by a user.

For example, the user may implement screening of a plurality of record information by selecting or inputting a time period so that the terminal device provides only the screened record information.

Fig. 10 schematically illustrates a schematic diagram of an output hypergraph in at least one embodiment of the present disclosure.

In at least one embodiment of the present disclosure, in the case where a hypergraph for a target object is generated based on image information of the target object, and the hypergraph for the target object is updated in real time based on the provided memory information for the target object, the information processing method of the embodiment of the present disclosure may further record update information of the hypergraph in response to the update of the hypergraph. And responding to the viewing request of the updated information, outputting the hypergraph and prominently reflecting the updated information in the hypergraph so as to more directly reflect the updated condition of the hypergraph. Based on the output updated information, the user initiating the viewing request can learn in real time the condition of the target object arousing memory or the recent activity direction of the target object, so as to facilitate setting a better treatment or cultivation strategy for the target object.

For example, as shown in fig. 10, if the hypergraph 1010 is a hypergraph before update, that is, before update of the hypergraph corresponding to update information for which a view request is made, the network structure of the hypergraph is provided. Then in response to the view request, a hypergraph 1020 is output. In the hypergraph 1020, the dotted line portion corresponds to a node and a hyperedge recorded by the update information.

For example, when the target object is a patient with dysmnesia, a guardian or an attending doctor of the target object can learn how frequently and how deeply the target object evokes memory changes based on the updated hypergraph information, so as to assign a personalized cognitive training scheme to the target object.

For example, when the target object is a child, the guardian of the target object can learn about social activities and learning progress of the target object according to the updated information of the hypergraph, so as to timely give correction or tutoring when social behaviors beyond the cognitive range of the target object or learning progress is slower, thereby being beneficial to healthy growth of the target object.

For example, the update information may be recorded with information such as an added node, a newly added superside, and the like. When the hypergraph is output in response to the viewing request, for example, nodes and hyperedges of the updated information record can be output in a highlighted manner such as continuous flashing, color change or dynamic extension. It is to be understood that the above-described protruding manner is merely an example to facilitate understanding of the present disclosure, and embodiments of the present disclosure are not limited thereto.

For example, for each update of the hypergraph, update information may be recorded. For example, the recorded update information includes a plurality of update sub-information corresponding to a plurality of update time periods, respectively. For example, the update condition of each day or any time span may be recorded as one update sub-information. When a viewing request is initiated, a time period corresponding to the updated sub-information that needs to be viewed may be selected. And takes the selected time period as a portion of the information indicated in the view request. Then, in response to the viewing request, at least one of the plurality of updated sub-information corresponding to the time period indicated by the first viewing request may be first determined. The hypergraph is then output and the at least one updated sub-information is highlighted in the hypergraph.

For example, a client application for generating recorded information may be provided with a timeline screening function through an interactive interface to facilitate a user to accurately see changes in hypergraphs over different time periods. For example, if the selected middle segment is the past week, only the memory nodes newly added in the week and hypergraph structure adjustment caused by the nodes can be displayed, so that the user can conveniently focus the influence of the recent memory stimulus on the target object.

In at least one embodiment of the present disclosure, the client application for generating the record information may further provide a node detail popup function, and the user clicks any interested node in the output hypergraph, so that a window containing information such as detailed description, input source, associated frequency and the like of the node can be popped up, so as to conveniently and deeply understand the background of each memory segment. For example, if a doctor finds that the target object is hot in recall reaction to the working scene, the doctor can adjust the intelligent interaction scene, increase the interaction frequency of the working theme, optimize the topic guiding direction and comprehensively advance the rehabilitation process of the patient by the collaboration system.

According to the embodiment of the disclosure, in consideration of the continuous enhancement of personal privacy protection consciousness of people, and in order to avoid malicious use of user information, a client application for generating recorded information can be provided with a quick desensitization and confidentiality function of sensitive information, for example, so as to effectively solve privacy concerns of users in the content input and sharing process.

In at least one embodiment of the present disclosure, the information processing method provided by the embodiments of the present disclosure may further include determining, in response to obtaining the security requirement information, an entity for which the security requirement information is directed, and performing desensitization processing on the entity for which the security requirement information is directed in the memory information, to obtain the desensitized memory information.

For example, security requirement information may be entered via an input interface, which may indicate, for example, a need to desensitize a particular person name, a particular location, etc. Taking the requirement of desensitizing a specific person name as an example, the method of the embodiment can adopt natural language processing technology to comprehensively scan memory information, record information and the like and quickly locate all text fragments related to the person. Then, the position and the context of the character name in the text are accurately identified by utilizing a pre-trained entity identification model. The importance and the occurrence frequency of the person in the whole recall story line and the tightness degree of the person with other people and events are intelligently judged based on the association information of the hypergraph, and for example, the name of the person can be replaced by a more accurate expression based on the importance, the occurrence frequency and the tightness degree so as to realize desensitization treatment.

For example, for a person name that needs to be desensitized, a variety of desensitization modes may be provided for selection by the user. The plurality of desensitization modes include, for example, a simple anonymization process, replacing the real name with a common designation of "mr", "lady", and the like. The various desensitization modes may include, for example, generating a specific pseudonym based on character features and roles, such as "optimistic elder Zhang", "good plum teacher", etc., to maximize the hiding of the true identity information while ensuring that the identities of the characters are distinguishable. In the replacement process, the system can use natural language generation technology to finely adjust the memory information and the recorded information, so that the statement after replacement is smooth and natural, and the whole readability and logic property of the memory information and the recorded information are not affected. For example, the original memory information is "i am yesterday and Zhang San go to park together, zhang San also carried his puppy", and after desensitization treatment it may become "i am yesterday goes to park together with friends who take the walk, he also carried his puppy).

For example, in the case where the memory information is desensitized, the key information of the memory information is extracted, and the desensitized memory information can be identified to obtain the key information of the memory information.

In at least one embodiment of the present disclosure, in addition to the desensitization processing, an advanced encryption algorithm may be used to encrypt all the information input by the user and the generated information in the data storage layer, so as to ensure the security of the data in the database. Whether the memory information in text form, the uploaded picture video data, the personal account information of the user, the preference setting and the like are stored in a database after being encrypted. For example, an advanced encryption standard (Advanced Encryption Standard, AES) algorithm is used to encrypt text data, convert the data into a ciphertext form for storage, and only when the user accesses the data legally and passes authentication, the data is restored into a plaintext form by using a corresponding decryption key for viewing and editing by the user. For example, the information input by the user may include memory information or the like, and the generated information may include recording information or the like.

In at least one embodiment of the present disclosure, a strict access control mechanism may also be provided to limit access to recorded information, hypergraph and memory information, memory material, etc. according to the role and rights of the user. For example, when a user wants to create a memory for a target object, different sharing levels may be set, such as visible only by himself, visible by family, visible by a particular friend, etc., each level corresponding to different access rights and operation rights. Through user authentication and authorization technology, such as multi-factor authentication modes based on passwords, fingerprint recognition, facial recognition and the like, only authorized users can access corresponding memory contents.

For example, in accessing various information, the client application for generating recorded information may also record in detail the operation log of each user, including access time, access content, operations performed (e.g., viewing, editing, downloading, etc.), so as to enable quick tracing and troubleshooting of the root cause when an information leakage event occurs.

For example, a user initiating a view request to view update information of a hypergraph is a user having hypergraph view rights, and the request to view update information of a hypergraph is generated only in response to an operation of the user having hypergraph view rights (e.g., may be the first object). The user who initiates the viewing request to view the record information is the user who has the record viewing authority, and the request to view the record information is generated only in response to an operation of the user who has the record viewing authority (for example, may be the second object). For example, the user who initiates the output record information request is a user who has output authority, and the output request to output record information is generated only in response to an operation of the user who has output authority (for example, may be the third object). It is to be understood that the above rights are merely examples to facilitate an understanding of the present disclosure, and embodiments of the present disclosure are not limited thereto.

In at least one embodiment of the present disclosure, to prevent external network attacks and malicious theft of user information, a series of network security safeguards may also be deployed for client applications used to generate recorded information, including firewalls, intrusion detection systems (Intrusion Detection System, IDS), intrusion prevention systems (Intrusion Prevention System, IPS), and the like. The firewall blocks external unauthorized network traffic from entering the system by setting access rules. The intrusion detection system monitors network activities in real time and gives an early warning to abnormal traffic and attack behaviors. The intrusion protection system can actively protect common network attacks such as hacking attack, malicious software intrusion, data leakage attack and the like, ensure the security and stability of the system in a network environment, provide omnibearing protection for sensitive information of users, and enable the users to record and share personal stories by using client applications for generating record information without worrying about personal privacy leakage.

Fig. 11 schematically shows a block diagram of an information processing apparatus provided in at least one embodiment of the present disclosure.

As shown in fig. 11, the information processing apparatus 1100 of this embodiment includes a material acquisition module 1110 and an information generation module 1120.

The material acquisition module 1110 is configured to acquire memory material based on the memory information in response to acquiring the memory information for the target object. For example, the material obtaining module 1110 may implement step S210, and a specific implementation method thereof may refer to a description related to step S210, which is not described herein.

The information generation module 1120 is configured to generate recording information for the memory information based on the memory material. For example, the information generating module 1120 may implement step S220, and a specific implementation method thereof may refer to a description related to step S220, which is not described herein.

In at least one embodiment of the present disclosure, the above-described information processing apparatus 1100 may further include a guidance information providing module configured to provide guidance information. For example, the memory information is acquired based on the guidance information. For example, the guidance information providing module may implement step S330, and the specific implementation method thereof may refer to the related description of step S330, which is not described herein.

In at least one embodiment of the present disclosure, the guiding information includes at least one of a plurality of field types of the memory information, a plurality of element types of the memory information, a plurality of modality types of the memory information, and a plurality of memory blocks, for example.

In at least one embodiment of the present disclosure, the guidance information providing module may include, for example, a field type providing sub-module and a recommendation information providing sub-module. The domain-type providing sub-module is configured to provide a plurality of domain types of the memory information. The recommended information providing sub-module is configured to provide a plurality of recommended information of a target domain corresponding to a target domain type in response to a selection operation of the target domain type among the plurality of domain types. For example, the guidance information further includes a plurality of recommendation information.

In at least one embodiment of the present disclosure, the guidance information providing module may further include an element type providing sub-module and a modality type providing sub-module, for example. The element type providing sub-module is configured to provide a plurality of element types of the memory information in response to a selection operation of the target recommendation information among the plurality of recommendation information. The modality type providing sub-module is configured to provide a plurality of modality types of the memory information in response to a selection operation of a target element type among the plurality of element types.

In at least one embodiment of the present disclosure, the recommendation information providing sub-module may include, for example, a first query unit and a recommendation information determining unit. The first query unit is configured to query the hypergraph aiming at the target object based on the target domain corresponding to the target domain type, and obtain a query result. The recommendation information determining unit is configured to determine and provide a plurality of recommendation information based on the query result. For example, the hypergraph for the target object is generated based on the image information of the target object, and is updated according to the history memory information for the target object acquired at the history time.

In at least one embodiment of the present disclosure, the above-described guidance information providing module may include, for example, a problem information determining sub-module and a problem information providing sub-module. The question information determination sub-module is configured to determine question information of a guided dialog for a target object based on portrait information for the target object. The question information providing sub-module is configured to provide question information of the guided dialog.

In at least one embodiment of the present disclosure, the upper question information determination submodule may include, for example, a topic determination unit and a question generation unit. The topic determination unit is configured to determine topic information for a target object based on a hypergraph for the target object. The question generation unit is configured to generate question information of a guidance dialog for the target object based on the topic information. For example, a hypergraph for a target object is generated based on image information of the target object, and is updated based on history memory information for the target object acquired at a history time.

In at least one embodiment of the present disclosure, the question information providing sub-module may be specifically configured to provide question information of the guided dialog by the virtual character for the target object.

In at least one embodiment of the present disclosure, the information processing apparatus 1100 may further include, for example, a statistical analysis module and an analysis result providing module. The statistical analysis module is configured to perform statistical analysis on the reply information in response to obtaining the reply information provided by the target object aiming at the problem information of the guiding dialogue, so as to obtain a statistical analysis result. The analysis result providing module is configured to provide statistical analysis results to an associated object associated with the target object. For example, the statistical analysis results are used at least to characterize the expressive power of the target object.

In at least one embodiment of the present disclosure, the above-described information processing apparatus may further include a memory information determining module configured to determine that the acquired memory information includes the input information in response to detecting the input information after any one of the guidance information is selected.

In at least one embodiment of the present disclosure, the above information processing apparatus may further include an information transmitting module configured to transmit the input information in response to a sharing operation of the input information. The above-described memory information determination module may be further configured to determine that the acquired memory information further includes information derived based on the supplemental information in response to receiving the supplemental information filled in for the input information.

In at least one embodiment of the present disclosure, the material acquisition module 1110 may include an information extraction sub-module and a material acquisition sub-module, for example. The information extraction sub-module is configured to extract key information of the memory information. The material acquisition sub-module is configured to acquire the memory material based on the key information.

In at least one embodiment of the present disclosure, the information processing apparatus 1100 may further include an entity determination module and a desensitization module, for example. The entity determination module is configured to determine an entity for which the security requirement information is directed in response to acquiring the security requirement information. The desensitization module is configured to desensitize the entity aiming at the security requirement information in the memory information to obtain the desensitized memory information. Accordingly, the information extraction sub-module may be configured to identify the desensitized memory information to obtain key information of the memory information.

In at least one embodiment of the present disclosure, the memory information includes at least two pieces of information belonging to at least two modalities, and the key information includes information extracted for each of the at least two pieces of information. The material acquisition sub-module may include an integration unit and a material acquisition unit. The integration unit is configured to integrate the key information to obtain integrated information. The material obtaining unit is configured to obtain the memory material based on the integrated information.

In at least one embodiment of the present disclosure, the material obtaining unit may include an information providing sub-unit and an adjusting sub-unit. The information providing subunit is configured to provide the integrated information. The adjustment subunit is configured to obtain adjusted information in response to an adjustment operation of the integrated information, the memory material including the adjusted information.

In at least one embodiment of the present disclosure, the memory information includes video. The information extraction sub-module may include a key frame extraction unit, a feature fusion unit, and a key information determination unit. The key frame extraction unit is configured to extract key video frames in the video and audio clips in the video corresponding to the key video frames. The feature fusion unit is configured to fuse the visual features of the key video frames with the audio features of the audio clips corresponding to the key video frames to obtain fused features. The key information determination unit is configured to determine key information of the memory information based on the fused features.

In at least one embodiment of the present disclosure, the key information determination unit may include a content determination subunit, a text generation subunit, and an information extraction subunit. The content determination subunit is configured to determine key content included in the audio clip. The text generation subunit is configured to generate descriptive text for the key video frames based on the fused features and the key content. The information extraction subunit is configured to extract key information from the descriptive text.

In at least one embodiment of the present disclosure, the material acquisition sub-module may be configured to generate image information corresponding to the memory information based on the key information, wherein the memory material includes the image information and the key information.

In at least one embodiment of the present disclosure, the material acquisition sub-module may include an element extraction unit and a first image generation unit. The element extraction unit is configured to extract an image element from existing image information for the target object based on the key information. The first image generation unit is configured to generate image information corresponding to the memory information based on the image elements.

In at least one embodiment of the present disclosure, the memory information includes a video, and the key information includes at least two key video frames extracted from the video and at least two audio clips respectively corresponding to the at least two key video frames. The material acquisition sub-module may include a frame dividing unit and a fragment generating unit. The frame dividing unit is configured to divide the at least two key video frames into at least one video frame group based on play time information of the at least two key video frames. The clip generation unit is configured to generate, for each video frame group, a video clip based on a key video frame included in the video frame group and an audio clip corresponding to the key video frame, wherein the image information includes the video clip generated for each video frame group.

In at least one embodiment of the present disclosure, the information extraction sub-module may include an information extraction unit, an importance determination unit, and a filtering unit. The information extraction unit is configured to extract a plurality of key video frames in the video and a plurality of audio clips in the video corresponding to the plurality of key video frames, respectively. The importance determination unit is configured to determine an importance of the content expressed by each key video frame and the audio clip corresponding to each key video frame. The filtering unit is configured to determine at least two key video frames whose importance degree of the expressed content satisfies a preset condition and at least two audio clips respectively corresponding to the at least two key video frames.

In at least one embodiment of the present disclosure, the material acquisition sub-module may include a second image generation unit, an information providing unit, and an information adjustment unit. The second image generation unit is configured to generate initial image information based on the key information. The information providing unit is configured to provide initial image information. The information adjustment unit is configured to adjust the initial image information based on the adjustment information in response to receiving the adjustment information fed back for the initial image information, resulting in image information corresponding to the memory information.

In at least one embodiment of the present disclosure, the information generation module 1120 may include, for example, a spectrum query sub-module and a generation sub-module. The map query sub-module is configured to query a knowledge map based on the memory material, obtaining knowledge information for the target object. The generation sub-module is configured to generate record information for the memory information based on the knowledge information and the memory material. The knowledge graph is constructed based on a knowledge base matched with the target object.

In at least one embodiment of the present disclosure, the profile query sub-module may include a second query unit and a filtering unit. The second query unit is configured to query the knowledge graph based on the memory material to obtain a plurality of candidate information. The filtering unit is configured to filter the plurality of candidate information based on the recommendation algorithm to obtain knowledge information for the target object.

In at least one embodiment of the present disclosure, the information processing apparatus 1100 may further include an emotion information determination module configured to perform emotion analysis on the memory information and determine emotion trend information. The information generation module may be specifically configured to generate record information for the memory information based on the emotion tendencies information and the memory material.

In at least one embodiment of the present disclosure, the information processing apparatus 1100 may further include a hypergraph update module configured to update a hypergraph for a target object based on the memory material, the hypergraph for the target object being generated based on image information of the target object.

In at least one embodiment of the present disclosure, the above-described information processing apparatus 1100 may further include an information recording module and an information output module. The information recording module is configured to record updated information of the hypergraph in response to the updating of the hypergraph. The information output module is configured to output a hypergraph and highlight the updated information in the hypergraph in response to a first view request for the updated information. Wherein the first view request is generated in response to an operation of the first object having hypergraph view rights.

In at least one embodiment of the present disclosure, the update information includes a plurality of update sub-information corresponding to a plurality of update time periods, respectively. The information output module may include a sub-information determination sub-module and an output sub-module. The sub-information determination sub-module is configured to determine, in response to the first viewing request, at least one of the plurality of updated sub-information corresponding to a time period indicated by the first viewing request. The output sub-module is configured to output the hypergraph and prominently embody at least one updated sub-information in the hypergraph.

In at least one embodiment of the present disclosure, the information generation module 1120 may include a hypergraph query sub-module and an information generation sub-module. The hypergraph query sub-module is configured to query the hypergraph for the target object based on the memory material to obtain the associated material of the memory material. The information generation sub-module is configured to generate record information for the memory information based on the memory material and the associated material. The hypergraph for the target object is generated based on the portrait information of the target object, and is updated according to the history memory information of the target object acquired at the history moment.

In at least one embodiment of the present disclosure, the hypergraph query sub-module may include a policy determination unit and a third query unit. The policy determination unit is configured to determine a query policy for the hypergraph based on the requirement information for the record information. The third query unit is configured to query the hypergraph based on the memory material and the query policy to obtain the associated material.

In at least one embodiment of the present disclosure, the above-described information processing apparatus 1100 may further include a composition information determining module and a recording information providing module. The layout information determining module is configured to determine layout information of a plurality of pieces of record information for a plurality of pieces of memory information, respectively, based on a predetermined design format. The recording information providing module is configured to provide at least part of the recording information based on the layout information in response to a second viewing request for at least part of the recording information. Wherein the second view request is generated in response to an operation of the second object having record viewing rights.

In at least one embodiment of the present disclosure, the above-described information processing apparatus 1100 may further include a recording information output module and a recording information transmitting module. The recording information output module is configured to output the recording information based on a predetermined output format in response to an output request for the recording information. The record information transmitting module is configured to transmit record information in response to a sharing operation for the record information. Wherein the output request is generated in response to an operation of the third object having the output right.

In at least one embodiment of the present disclosure, the information processing apparatus 1100 may further include an information encryption module and an information storage module. The information encryption module is configured to encrypt the memory information and the record information to obtain encrypted information. The information storage module is configured to store the encrypted information.

For example, each unit included in the information processing apparatus may be implemented by a hardware (e.g., circuit) module or a software module, and the like, which will not be described in detail. For example, these elements may be implemented by a Central Processing Unit (CPU), an image processor (GPU), a Tensor Processor (TPU), a Field Programmable Gate Array (FPGA), or other form of processing unit having data processing and/or instruction execution capabilities, and corresponding computer instructions.

It should be noted that, in the embodiment of the present disclosure, the information processing apparatus may include more or less circuits or units, and the connection relationship between the respective circuits or units is not limited, and may be determined according to actual requirements. The specific configuration of each circuit is not limited, and may be constituted by an analog device, a digital chip, or other suitable means according to the circuit principle.

At least one embodiment of the present disclosure also provides an electronic device comprising processing means, and storage means comprising one or more computer program instructions, which when executed by the processing means, for example, perform the information processing method provided by any of the embodiments of the present disclosure.

Fig. 12 is a schematic structural diagram of an electronic device according to at least one embodiment of the present disclosure. The terminal devices in the embodiments of the present disclosure may include, but are not limited to, mobile terminals such as mobile phones, notebook computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and the like, and stationary terminals such as digital TVs, desktop computers, and the like. The electronic device shown in fig. 12 is merely an example and should not be construed to limit the functionality and scope of use of the disclosed embodiments.

For example, in some examples, the electronic device includes an information processing apparatus (e.g., implemented by the processing apparatus 1201 and the output apparatus 1207 shown in fig. 12, etc.) provided in any of the embodiments of the present disclosure to obtain memory information for a target object in response to obtaining the memory information, obtain memory material based on the memory information, and generate record information for the memory information based on the memory material.

For example, as shown in fig. 12, in some examples, the electronic device 1200 includes a processing means (e.g., a central processor, a graphics processor, etc.) 1201, which may perform various appropriate actions and processes according to a program stored in a Read Only Memory (ROM) 1202 or a program loaded from a storage means 1208 into a Random Access Memory (RAM) 1203. In the RAM 1203, various programs and data required for the operation of the computer system are also stored. The processing device 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An input/output (I/O) interface 1205 is also connected to the bus 1204.

For example, components may be connected to I/O interface 1205 including input devices 1206, including for example, touch screens, touch pads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc., output devices 1207, including for example, liquid Crystal Displays (LCDs), speakers, vibrators, etc., storage devices 1208, including for example, magnetic tape, hard disk, etc., and communication devices 1209, including network interface cards, such as LAN cards, modems, etc. The communication device 1209 may allow the electronic apparatus 1200 to perform wireless or wired communication with other apparatuses to exchange data, performing communication processing via a network such as the internet. The drive 1210 is also connected to the I/O interface 1205 as needed. Removable media 1211, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, or the like, is mounted on drive 1210 as needed so that a computer program read therefrom is installed into storage 1208 as needed. While fig. 12 illustrates an electronic device 1200 including various means, it is to be understood that not all illustrated means are required to be implemented or included. More or fewer devices may be implemented or included instead.

For example, the electronic device 1200 may further include a peripheral interface (not shown), and the like. The peripheral interface may be various types of interfaces, such as a USB interface, a lightning (lighting) interface, etc. The communication device 1209 may communicate with networks and other equipment via wireless communications, such as the internet, intranets, and/or wireless networks such as cellular telephone networks, wireless Local Area Networks (LANs), and/or Metropolitan Area Networks (MANs). The wireless communication may use any of a variety of communication standards, protocols, and technologies including, but not limited to, global System for Mobile communications (GSM), enhanced Data GSM Environment (EDGE), wideband code division multiple Access (W-CDMA), code Division Multiple Access (CDMA), time Division Multiple Access (TDMA), bluetooth, wi-Fi (e.g., based on the IEEE 802.11 a, IEEE 802.11 b, IEEE 802.11 g, and/or IEEE 802.11 n standards), voice over Internet protocol (VoIP), wi-MAX, protocols for email, instant messaging, and/or Short Message Service (SMS), or any other suitable communication protocol.

For example, the electronic device may be any device such as a mobile phone, a tablet computer, a notebook computer, an electronic book, a game console, a television, a digital photo frame, a navigator, or any combination of electronic devices and hardware, which is not limited in the embodiments of the present disclosure.

For example, according to embodiments of the present disclosure, the processes described above with reference to flowcharts may be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product comprising a computer program embodied on a non-transitory computer readable medium, the computer program comprising program code for performing the method shown in the flow chart. In such an embodiment, the computer program may be downloaded and installed from a network via the communication device 1209, or installed from the storage device 1208, or installed from the ROM 1202. When executed by the processing device 1201, the computer program performs the above-described information processing method defined in the method of the embodiment of the present disclosure.

It should be noted that the computer readable medium described in the present disclosure may be a computer readable signal medium or a computer readable storage medium, or any combination of the two. The computer readable storage medium can be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or a combination of any of the foregoing. More specific examples of a computer-readable storage medium may include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. In an embodiment of the present disclosure, a computer readable storage medium may be any tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. Whereas in embodiments of the present disclosure, the computer-readable signal medium may comprise a data signal propagated in baseband or as part of a carrier wave, with computer-readable program code embodied therein. Such a propagated data signal may take any of a variety of forms, including, but not limited to, electro-magnetic, optical, or any suitable combination of the foregoing. A computer readable signal medium may also be any computer readable medium that is not a computer readable storage medium and that can communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. Program code embodied on a computer readable medium may be transmitted using any appropriate medium, including but not limited to electrical wiring, fiber optic cable, RF (radio frequency), and the like, or any suitable combination of the foregoing.

In some embodiments, the clients, servers may communicate using any currently known or future developed network protocol, such as HTTP (Hyper Text Transfer Protocol ), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), the internet (e.g., the internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed networks.

The computer readable medium may be included in the electronic device or may exist alone without being incorporated into the electronic device.

The computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to acquire memory material based on the memory information in response to acquiring the memory information for the target object, and generate record information for the memory information based on the memory material.

Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, including, but not limited to, an object oriented programming language such as Java, smalltalk, C ++ and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any kind of network, including a Local Area Network (LAN) or a Wide Area Network (WAN), or may be connected to an external computer (for example, through the Internet using an Internet service provider).

The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic that may be used include Field Programmable Gate Arrays (FPGAs), application Specific Integrated Circuits (ASICs), application Specific Standard Products (ASSPs), systems-on-a-chip (SOCs), complex Programmable Logic Devices (CPLDs), and the like.

In various embodiments of the present disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. The machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a Random Access Memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

At least one embodiment of the present disclosure also provides a storage medium. Fig. 13 is a schematic diagram of a storage medium according to at least one embodiment of the present disclosure. For example, as shown in fig. 13, the storage medium 1300 non-transitory stores computer readable instructions 1301, which when executed by a computer (including a processor) can perform the information processing method provided by any of the embodiments of the present disclosure.

For example, the storage medium may be any combination of one or more computer-readable storage media, such as one computer-readable storage medium containing computer-readable program code for an information question-answering method and another computer-readable storage medium containing computer-readable program code for an information processing method. For example, when the program code is read by a computer, the computer may execute the program code stored in the computer storage medium, performing, for example, the information processing method provided by any of the embodiments of the present disclosure.

For example, the storage medium may include a memory card of a smart phone, a memory component of a tablet computer, a hard disk of a personal computer, random Access Memory (RAM), read Only Memory (ROM), erasable Programmable Read Only Memory (EPROM), portable compact disc read only memory (CD-ROM), flash memory, or any combination of the foregoing, as well as other suitable storage media.

The following points need to be described:

(1) The drawings of the embodiments of the present disclosure relate only to the structures related to the embodiments of the present disclosure, and other structures may refer to the general design.

(2) The embodiments of the present disclosure and features in the embodiments may be combined with each other to arrive at a new embodiment without conflict.

The foregoing is merely exemplary embodiments of the present disclosure and is not intended to limit the scope of the disclosure, which is defined by the appended claims.

Claims

1. An information processing method, comprising:

Acquiring memory material based on the memory information in response to acquiring the memory information for the target object, and

Record information for the memory information is generated based on the memory material.

2. The method of claim 1, further comprising:

providing guide information;

wherein the memory information is acquired based on the guidance information.

3. The method of claim 2, wherein the guidance information comprises at least one of a plurality of domain types of memory information, a plurality of element types of memory information, a plurality of modality types of memory information, a plurality of memory patches.

4. A method according to claim 3, wherein said providing guidance information comprises:

providing the plurality of domain types of memory information;

Providing a plurality of recommended information of a target domain corresponding to a target domain type among the plurality of domain types in response to a selection operation of the target domain type,

Wherein the guidance information further includes the plurality of recommendation information.

5. The method of claim 4, wherein the providing guidance information further comprises:

Providing the plurality of element types of the memory information in response to a selection operation of the target recommendation information among the plurality of recommendation information, and

Providing the plurality of modality types of the memory information in response to a selection operation of a target element type of the plurality of element types.

6. The method of claim 4, wherein providing the plurality of recommendation information for the target area corresponding to the target area type comprises:

inquiring hypergraph aiming at the target object based on the target domain corresponding to the target domain type to obtain inquiry result, and

Determining and providing the plurality of recommendation information based on the query results,

The hypergraph for the target object is generated based on the image information of the target object, and is updated according to the history memory information of the target object acquired at the history moment.

7. The method of claim 2, wherein the providing guidance information comprises:

Determining question information of a guiding dialogue for the target object based on the portrait information for the target object, and

Providing question information of the guiding dialogue.

8. The method of claim 7, wherein the determining question information for the guided dialog for the target user based on the portrait information for the target object includes:

determining topic information for the target object based on the hypergraph for the target object, and

Generating question information of a guide dialog for the target object based on the topic information,

9. The method of claim 7 or 8, wherein the providing question information of the guiding dialog comprises:

question information of the guided dialog is provided by means of a virtual character for the target object.

10. The method of claim 7, further comprising:

In response to obtaining the reply information provided by the target object aiming at the question information of the guiding dialogue, carrying out statistical analysis on the reply information to obtain statistical analysis results, and

Providing the statistical analysis result to an associated object associated with the target object,

Wherein the statistical analysis result is at least used for characterizing the expression capacity of the target object.

11. The method of claim 2, further comprising:

In response to detecting input information after any one of the guidance information is selected, it is determined that the acquired memory information includes the input information.

12. The method of claim 11, further comprising:

Transmitting the input information in response to a sharing operation of the input information, and

In response to receiving the supplemental information filled in for the input information, determining that the acquired memory information further includes information derived based on the supplemental information.

13. The method of claim 1, wherein obtaining memory material based on the memory information comprises:

extracting key information of the memory information, and

And acquiring the memory material based on the key information.

14. The method of claim 13, further comprising:

determining an entity for which the security requirement information is directed in response to obtaining the security requirement information, and

Desensitizing the entity aimed by the security requirement information in the memory information to obtain desensitized memory information,

The key information of the memory information is extracted, wherein the key information of the memory information is obtained by identifying the desensitized memory information.

15. The method of claim 13 or 14, wherein the memory information comprises at least two information belonging to at least two modalities, the key information comprising information extracted for each of the at least two information;

The obtaining the memory material based on the key information comprises the following steps:

Integrating the key information to obtain integrated information, and

And obtaining the memory material based on the integrated information.

16. The method of claim 15, wherein the deriving the memory material based on the integrated information comprises:

Providing the integrated information, and

And responding to the adjustment operation of the integrated information to obtain the adjusted information, wherein the memory material comprises the adjusted information.

17. The method of claim 13, wherein the memory information comprises video, and wherein the extracting key information of the memory information comprises:

extracting a key video frame in the video and an audio fragment corresponding to the key video frame in the video;

fusing the visual features of the key video frames and the audio features of the audio clips corresponding to the key video frames to obtain fused features, and

And determining key information of the memory information based on the fused features.

18. The method of claim 17, wherein the determining key information for the memory information based on the post-fusion features comprises:

determining key content included in the audio clip;

generating descriptive text for the key video frames based on the fused features and the key content, and

And extracting the key information from the descriptive text.

19. The method of claim 13, wherein the obtaining the memory material based on the key information comprises:

Generating image information corresponding to the memory information based on the key information,

The memory material comprises the image information and the key information.

20. The method of claim 19, wherein the generating image information corresponding to the memory information based on the key information comprises:

Extracting image elements from existing image information for the target object based on the key information, and

And generating image information corresponding to the memory information based on the image element.

21. The method of claim 19, wherein the memory information comprises a video, the key information comprises at least two key video frames extracted from the video and at least two audio clips corresponding to at least two of the key video frames, respectively;

the generating image information corresponding to the memory information based on the key information includes:

dividing at least two key video frames into at least one video frame group based on play time information of the at least two key video frames;

For each of the video frame groups, generating video clips based on key video frames included in the video frame group and audio clips corresponding to the key video frames,

Wherein the image information includes video clips generated for each of the video frame groups.

22. The method of claim 21, wherein the extracting key information of the memory information comprises:

extracting a plurality of key video frames in the video and a plurality of audio clips in the video, which correspond to the key video frames respectively;

determining importance of the content expressed by each key video frame and the audio clip corresponding to each key video frame, and

And determining at least two key video frames with importance degrees meeting preset conditions and at least two audio fragments respectively corresponding to the at least two key video frames.

23. The method of claim 19, wherein the generating image information corresponding to the memory information based on the key information comprises:

generating initial image information based on the key information;

providing the initial image information, and

And in response to receiving the adjustment information fed back for the initial image information, adjusting the initial image information based on the adjustment information to obtain image information corresponding to the memory information.

24. The method of claim 1, wherein the generating record information for the memory information based on the memory material comprises:

inquiring knowledge graph based on the memory material to obtain knowledge information aiming at the target object, and

Generating record information for the memory information based on the knowledge information and the memory material,

Wherein the knowledge graph is constructed based on a knowledge base that matches the target object.

25. The method of claim 24, wherein the querying a knowledge graph based on the memory material to obtain knowledge information for the target object comprises:

Inquiring the knowledge graph based on the memory material to obtain a plurality of alternative information, and

And filtering the plurality of candidate information based on a recommendation algorithm to obtain knowledge information aiming at the target object.

26. The method of claim 1, further comprising:

Carrying out emotion analysis on the memory information to determine emotion tendency information;

wherein the generating record information for the memory information based on the memory material includes:

and generating record information for the memory information based on the emotion tendency information and the memory material.

27. The method of claim 1, further comprising:

And updating a hypergraph aiming at the target object based on the memory material, wherein the hypergraph aiming at the target object is generated based on the image information of the target object.

28. The method of claim 27, further comprising:

recording update information of the hypergraph in response to the update of the hypergraph, and

Outputting the hypergraph and highlighting the updated information in the hypergraph in response to a first viewing request for the updated information,

Wherein the first view request is generated in response to an operation of a first object having hypergraph view rights.

29. The method of claim 28, wherein the update information includes a plurality of update sub-information corresponding to a plurality of update time periods, respectively;

The outputting the hypergraph and highlighting the updated information in the hypergraph in response to a first viewing request for the updated information comprises:

Determining at least one of the plurality of updated sub-information corresponding to a time period indicated by the first viewing request in response to the first viewing request, and

Outputting the hypergraph and highlighting at least one piece of updated sub-information in the hypergraph.

30. The method of claim 1, wherein the generating record information for the memory information based on the memory material comprises:

Inquiring hypergraph aiming at the target object based on the memory material to obtain related material of the memory material, and

Generating record information for the memory information based on the memory material and the associated material,

31. The method of claim 30, wherein the querying the hypergraph for the target object based on the memory material results in associated material of the memory material, comprising:

Determining a query policy for the hypergraph based on the requirement information for the record information, and

And inquiring the hypergraph based on the memory material and the inquiry strategy to obtain the associated material.

32. The method of claim 1, further comprising:

Determining layout information of a plurality of the record information for a plurality of the memory information, respectively, based on a predetermined design format, and

Providing at least part of the recorded information based on the layout information in response to a second viewing request for at least part of the recorded information among the plurality of the recorded information,

Wherein the second view request is generated in response to an operation of a second object having record view rights.

33. The method of claim 1, further comprising:

outputting the recording information based on a predetermined output format in response to an output request for the recording information;

in response to a sharing operation for the record information, transmitting the record information,

Wherein the output request is generated in response to an operation of the third object having the output right.

34. The method of claim 1, further comprising:

Encrypting the memory information and the recording information to obtain encrypted information, and

And storing the encrypted information.

35. An information processing apparatus comprising:

A material acquisition module configured to acquire memory material based on the memory information in response to acquisition of the memory information for the target object, and

An information generation module configured to generate record information for the memory information based on the memory material.

36. An electronic device, comprising:

Processing device, and

A storage device storing one or more computer program instructions;

Wherein the one or more computer program instructions, when executed by the processing device, perform the method of any one of claims 1-34.

37. A computer readable storage medium, non-transitory storing computer readable instructions, wherein the computer readable instructions when executed by a processor implement the method of any one of claims 1-34.