CN108803880A - Control device based on brain signal and method - Google Patents

Abstract

The present invention provides a brain signal-based control device and method. The control device includes: a detector configured to collect and send the user's brain signal in response to the user's brain activity; a processor configured to communicate with the user's brain activity. communicate with the detector to receive the user's brain signal collected by the detector, perform image reconstruction processing on the received brain signal, and generate information for controlling the target device based on information obtained by identifying the reconstructed image data A control instruction; a communication unit configured to send a control signal including the control instruction to a target device. The invention collects and processes the user's brain signal based on the wearable miniaturized magnetic resonance imaging device and the image reconstruction algorithm, realizes the control of the target device through the brain signal, and satisfies various terminal devices and massive types of brain signals The need for control provides a better user experience.

Description

Control device and method based on brain signal

technical field

The present invention relates to the technical field of brain-computer interaction, and more specifically, to a control device and method based on brain signals.

Background technique

In recent years, various smart terminals such as smart home appliances, smart phones, computers, and wearable devices have been widely used in people's lives. operation control. However, when the user's hands are occupied by other tasks, such as housework, childcare, driving, etc., the smart terminal cannot be operated. Therefore, the user has a strong demand to realize the operation control of the smart terminal in all situations. In response to such needs, brain-computer interaction technology is emerging day by day.

The existing brain-computer interaction methods or brain signal control methods are roughly divided into three types. One relies on extracting the frequency band data of the brain signal to analyze the state of the human brain, including attention, sleep depth, etc., and then achieve feedback training to help users enter a certain level. One is to make the user face a series of options whether to choose or not by extracting the relevant error potential of the brain signal, and the brain signal analysis can automatically determine whether the user chooses a certain option; The brain signal during a certain control (such as imagining a water drop moving upward in a relaxed state) is saved as a standard brain signal, and when the user implements brain signal control, the extracted brain signal is compared with the standard signal to determine whether to implement the corresponding command (such as controlling the target to move up).

With the explosive growth of smart terminals and IoT devices (Internet of Things), no matter the type or quantity, their requirements for control methods are more diverse and intelligent, and at the same time, the control methods need to be as universal as possible while satisfying individualization. sex. The brain-signal-based control method in the prior art has relatively large limitations, and cannot meet the brain-signal control needs of various intelligent terminals for a large number of types.

Contents of the invention

Aiming at the deficiencies of the prior art, the present invention provides a control device and method based on brain signals to realize the control of various intelligent terminals and massive types of brain signals.

According to an aspect of the present invention, there is provided a control device based on brain signals, including: a detector configured to collect and send the user's brain signals in response to the user's brain activity; a processor configured to communicate with the The detector communicates to receive the user's brain signal collected by the detector, performs image reconstruction processing on the received brain signal, and generates a control for controlling the target device based on information obtained by identifying the reconstructed image data an instruction; a communication unit configured to send a control signal including the control instruction to a target device.

Preferably, the processor is configured to: reconstruct the image data from the received brain signal, and analyze the reconstructed image data using text and/or graphic recognition algorithms to identify at least one item of information; determining a target device according to the identified information, and generating a control instruction for controlling the target device.

Preferably, the control signal includes at least one of an infrared control signal, a wifi network message signal, a mobile network message signal and a bluetooth signal.

Preferably, the control device further includes: a memory configured to store the user's brain activity data and preliminary instructions corresponding to the user's brain activity data.

Preferably, the user's brain activity data includes at least one of text data, icon data, and identification data, as well as user-defined password data, number data, and picture data.

Preferably, the processor is further configured to: in the simulation training mode, conduct simulation training for the user according to the data stored in the memory, and judge and give feedback on the user's simulation training results.

Preferably, the judging and giving feedback on the simulation training results of the user includes: judging whether the control instructions analyzed by the simulation training are correct instructions according to the preliminary instructions stored in the memory, wherein, when the control instructions analyzed by the simulation training When it matches the preliminary instruction, the control instruction is judged to be correct, and the simulation training of the user is marked as passed. Pass, and optimize image reconstruction algorithms and text and/or graphic recognition algorithms.

Preferably, the detector is a wearable miniaturized magnetic resonance imaging device.

According to another aspect of the present invention, a brain signal-based control method is provided, the control method comprising the following steps: collecting and sending the user's brain signal in response to the user's brain activity; receiving the collected user's brain signal, performing image reconstruction processing on the received brain signal, and generating control instructions for controlling the target device based on information obtained by identifying the reconstructed image data; sending the control signal including the control instructions to the target device.

Preferably, the step of receiving the collected user's brain signal, performing image reconstruction processing on the received brain signal, and generating a control instruction for controlling the target device based on information obtained by identifying the reconstructed image data includes: Image data is reconstructed from the received brain signal, and the reconstructed image data is analyzed by text and/or graphic recognition algorithms to identify information including at least one of text, symbols, and graphics; determine the target device based on the identified information , and generate a control instruction for controlling the target device.

Preferably, the control signal includes at least one of an infrared control signal, a wifi network message signal, a mobile network message signal and a bluetooth signal.

Preferably, the control method further includes: in the simulation training mode, performing simulation training on the user according to the stored data, and judging and giving feedback on the user's simulation training results, wherein the stored data includes the user's brain activity data and preliminary instructions corresponding to the user's brain activity data.

Preferably, the user's brain activity data includes at least one of text data, icon data, and identification data, as well as user-defined password data, number data, and picture data.

Preferably, the step of judging and giving feedback on the simulation training results of the user includes: judging whether the control instructions analyzed by the simulation training are correct instructions according to the stored preliminary instructions, wherein, when the control instructions analyzed by the simulation training are the same as the preliminary instructions When the instructions match, the control instruction is judged to be correct, and the simulation training of the user is marked as passed. When the control instruction analyzed by the simulation training does not match the preliminary instruction, the control instruction is judged to be an error instruction, and the simulation training result of the user is marked as failed. And optimize the image reconstruction algorithm and text and/or graphic recognition algorithm.

Preferably, the step of collecting and sending the user's brain signal in response to the user's brain activity is realized by a wearable miniaturized magnetic resonance imaging device.

According to another aspect of the present invention, there is provided a terminal device capable of brain signal control, the terminal device comprising: a communication unit configured to receive a user's brain signal from an external brain signal acquisition device; a processor configured to In order to communicate with the communication unit to receive the brain signal of the user, perform image reconstruction processing on the received brain signal to obtain a control instruction for controlling the terminal device, and generate a corresponding control signal according to the control instruction to execute a corresponding operation.

Preferably, the terminal device further includes: a memory configured to store the user's brain activity data and preliminary instructions corresponding to the user's brain activity data, and the processor, in the simulated training mode, according to the The data simulates training for the user, and judges and gives feedback on the simulation training results of the user, wherein the brain activity data of the user includes text data, icon data and identification data, as well as user-defined password data, number data and picture data at least one of the

According to another aspect of the present invention, there is provided a brain signal acquisition device for brain signal control, the brain signal acquisition device includes: a detector, which collects the user's brain signal in response to the user's brain activity; a processor , configured to communicate with the detector to receive the user's brain signal, and perform image reconstruction processing on the received brain signal to obtain a control instruction for controlling the terminal device, and generate a corresponding control signal according to the control instruction to control the terminal device to perform corresponding operations, and

In the simulated training mode, the user is simulated and trained according to the stored data, and the results of the simulated training of the user are judged and fed back, wherein the stored data includes the user's brain activity data and preliminary instructions corresponding to the user's brain activity data .

According to yet another aspect of the present invention, there is provided a brain signal processing device for brain signal control, the brain signal acquisition device includes: a communication unit configured to receive a user's brain signal from an external brain signal acquisition device; The processor is configured to communicate with the communication unit to receive the user's brain signal, perform image reconstruction processing on the received brain signal to obtain a control instruction for controlling the terminal device, and generate a corresponding control signals to control the terminal equipment to perform corresponding operations, and

In the simulated training mode, the user is simulated and trained according to the stored data, and the results of the simulated training of the user are judged and fed back, wherein the stored data includes the user's brain activity data and preliminary instructions corresponding to the user's brain activity data .

Description of drawings

The detailed description of the present invention will be carried out below in conjunction with accompanying drawing, and above-mentioned feature of the present invention and other object, characteristic and advantage will become clearer, wherein:

FIG. 1 shows a block diagram of a control device based on brain signals according to an embodiment of the present invention;

Fig. 2 shows a schematic diagram of a user directly controlling a mobile phone to make a call through a brain signal according to an exemplary embodiment of the present invention;

FIG. 3 shows a flowchart of a brain signal-based control method according to an embodiment of the present invention;

Fig. 4 shows a block diagram of a terminal device capable of brain signal control according to an embodiment of the present invention;

Fig. 5 shows a block diagram of a brain signal acquisition device for brain signal control according to an embodiment of the present invention;

Fig. 6 shows a block diagram of a brain signal processing device for brain signal control according to an embodiment of the present invention.

Detailed ways

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. Wherein, the same reference numerals always represent the same components.

In the following, the definitions of relevant technical terms are described:

1. Deep neural network

Deep Neural Networks (DNN) is the basis of deep learning. Deep Neural Networks (DNN) specifically refer to fully connected neuron structures, and do not include convolutional units or temporal associations.

2. Deep generator network

A Deep Generator Network (DGN) is a pre-trained algorithm that creates realistic images from raw input. Essentially, a DGN places finished details on an image to make it look more natural.

3. KNN algorithm

The K-Nearest Neighbor (KNN) algorithm is one of the simplest machine learning algorithms. The idea of the algorithm is: if most of the K most similar samples in the feature space (that is, the nearest neighbors in the feature space) of a sample belong to a certain category, then the sample also belongs to this category, and the KNN algorithm selected The neighbors of the algorithm are all objects that have been correctly classified, and the algorithm only determines the category of the sample to be classified according to the category of the nearest one or several samples in the classification decision.

Fig. 1 shows a block diagram of a brain signal-based control device according to an embodiment of the present invention.

As shown in FIG. 1 , the brain signal-based control device 100 includes a detector 101 , a processor 102 , a communication unit 103 and a memory 104 . Wherein, the detector 101 is configured to collect and send the user's brain signal in response to the user's brain activity. The processor 102 is configured to communicate with the detector to receive the user's brain signal collected by the detector, perform image reconstruction processing on the received brain signal, and generate information for Control commands to control the target device. The communication unit 103 is configured to transmit a control signal including a control instruction to the target device. The memory 104 is configured to store the user's brain activity data and preliminary instructions corresponding to the user's brain activity data. According to an embodiment of the present invention, specifically, the detector 101 is configured as a wearable miniaturized magnetic resonance imaging device MRI, the MRI device detects the user's brain activity operation, and the collected user's brain signal is an MRI image. The processor 102 reconstructs the brain signal received from the detector 101 to obtain image data, and uses text and/or graphic recognition algorithms to analyze the reconstructed image data to identify at least one of text, symbols and graphics. information, and then determine the target device according to the identified information, and generate control instructions for controlling the target device. The communication unit 103 sends a control signal including a control instruction corresponding to the target device to the target device.

According to an embodiment of the present invention, the processor 102 extracts brain signals of the user collected by the MRI equipment, that is, MRI image data, and performs data processing on the extracted MRI images. Assuming that the processor 102 performs image processing on the MRI image based on the depth image reconstruction algorithm, the specific process is as follows: first decode the collected brain signal of the user to obtain the deep neural network (DNN) feature, and then perform an image processing on any input image. The pixel value is continuously iteratively optimized, so that the DNN features of the image are constantly approaching the DNN features obtained by decoding the brain signal to complete the reconstruction of the MRI image. Among them, the process of continuous iterative optimization is performed in the input space of the introduced deep generator network (DNN). Then, the processor 102 denoises the reconstructed image, and uses text and/or graphic recognition algorithms to analyze the denoised image to identify information including at least one of text, symbols, and graphics, and then according to The identified information identifies the target device, and generates control instructions for controlling the target device. Assuming that the KNN algorithm is used to analyze the denoised image to identify graphic information, or the text recognition algorithm is used to analyze the denoised image to identify text and/or symbol information, the control command obtained after image analysis can be Number commands, letter commands, punctuation commands, icon commands, logo commands or user-defined commands, etc. It should be understood that the above examples of text and/or graphic recognition algorithms are only exemplary examples, and the text and/or graphic recognition algorithms that can be used in the present invention are not limited thereto. Finally, the target device is determined according to the obtained control instruction and the communication unit 103 is controlled to send a control signal corresponding to the target device to the target device. Wherein, the control signal includes at least one of an infrared control signal, a wifi network message signal, a mobile network message signal and a bluetooth signal. The way in which the communication unit 103 sends the control signal is a way of wireless transmission. According to an embodiment of the present invention, assuming that the determined target device is a smart terminal device, such as a smart phone or a smart TV, the communication unit 103 sends a control signal including a control instruction to the smart target device, and the smart target device has Receive and process the message of the control command, and perform command parsing and command execution on the received control command. Assuming that the determined target device is a non-smart terminal device, such as an air conditioner or a chandelier, the processor 102 needs to convert the control instruction into an instruction applicable to the control interface of the target device, and the communication unit 103 sends a control signal including the instruction to to the target device. It should be understood that the foregoing examples of the target device are only exemplary examples, and the applicable target devices in the present invention are not limited thereto.

According to an embodiment of the present invention, the communication unit 103 sends a control signal including a control command to the target device, and the target device performs related operations according to the received control signal. For example, the target device performs the operation of outputting text symbols, wherein the text symbols include but not limited to alphabetic typing, direct output of various languages, or more recently, the content of the text output is used as a control command, and the client either receives The target device of the control signal uses itself or the remote semantic server to decompose the text control instruction into instructions that the target device can execute through semantic analysis, and then the target device performs related operations. Another example is to control the target device to perform the operation of moving the target, wherein the moving target includes but is not limited to moving the mouse or the focus position, moving the direction and position of the remote control toy, moving the direction and position of the virtual target in the game, and moving the direction and position of the sweeping robot. location etc. For another example, the target device is controlled to perform call operations, open or close operations, open and adjust air conditioners, and the like. The brain signal-based control apparatus 100 according to an embodiment of the present invention will be described in detail below with reference to specific embodiments of different target devices.

According to the embodiment of the present invention, assuming that the determined target device is a smart phone, the brain signal-based control device 100 realizes the process of the user directly controlling the smart phone to make a call through the brain signal, as shown in FIG. 2 . Specifically, assuming that the user imagines a text command "Call 10010", the MRI device detects the user's brain signal to obtain corresponding MRI image data. The processor 102 is a cloud server, and the cloud server extracts the MRI image data and reconstructs, denoises and analyzes the extracted MRI image based on the depth image reconstruction algorithm, and obtains the control instruction "Call 10010", which can be determined by the obtained control instruction The target device is a smartphone, and a corresponding control signal is generated. The communication unit 103 sends the control signal to the smart phone. At this time, the communication unit 103 may be an integrated device inside the cloud server. The smart phone receives the control signal containing the control command and analyzes the control command to control the smart phone to automatically execute the operation of dialing "10010". According to the embodiment of the present invention, the brain signal-based control device 100 realizes the process that the user directly controls the mobile phone to send messages through the brain signal: Suppose the user imagines the text instructions of "send WeChat to Xiao Wang" and "see you at 9 o'clock", then The MRI device detects the user's brain signal to obtain MRI image data corresponding to the text. Processor 102 extracts the MRI image data and reconstructs, denoises and analyzes the extracted MRI image based on the depth image reconstruction algorithm, obtains the control instructions as "send WeChat to Xiao Wang" and "see you at 9 points", and according to the obtained The control instruction determines that the target device is the smart phone and WeChat in the smart phone, and the communication unit 103 sends the control signal including the control instruction to the smart phone. The smart phone receives the control signal including the control command, analyzes the control command and executes the corresponding operation. The corresponding operation includes opening the WeChat software in the smart phone, finding Xiao Wang in the contacts, and opening the WeChat account with Xiao Wang. The dialog box between sends a message "see you at 9 o'clock" to Xiao Wang. It should be understood that the above examples of the user directly controlling the operation of the smart terminal device through the brain signal are only exemplary examples, and the operations performed by the smart terminal that can be used in the present invention are not limited thereto.

According to the embodiment of the present invention, assuming that the determined target device is a smart TV, the brain signal-based control device 100 realizes the process of the user directly controlling the smart TV to play a TV program with a custom logo through the brain signal as follows: the user imagines or Recalling the custom logo, the MRI equipment detects the user's brain signal to obtain the MRI image data corresponding to the custom logo. The processor 102 extracts the MRI image data and reconstructs the extracted MRI image based on the depth image reconstruction algorithm, and performs denoising and KNN algorithm processing on the reconstructed image data of the self-defined station logo to resolve the image corresponding to the self-defined station logo. control instructions, and determine that the target device is a smart TV according to the control instructions. The communication unit 103 sends the control signal including the control command to the smart TV. The smart TV performs command analysis on the control command and executes corresponding operations, wherein performing the corresponding operations specifically includes turning on the smart TV and selecting to play a TV program with a user-defined logo.

According to the embodiment of the present invention, assuming that the determined target device is a player, the brain signal-based control device 100 realizes the process of the user directly controlling the player through the brain signal as follows: the user imagines or recalls the control signs related to the player, For example, switch, play, pause, fast forward, fast rewind, mute, volume up, volume down, off, recording and other control signs, the MRI equipment detects the user's brain signal to obtain the corresponding MRI image data. The processor 102 extracts the MRI image data and reconstructs the extracted MRI image based on the depth image reconstruction algorithm to obtain image data corresponding to the control sign imagined or recalled by the user, and the processor 102 performs reconstruction on the obtained reconstructed image data. Denoise and KNN algorithm processing, analyze the corresponding control instructions, and determine the target device as the player according to the control instructions. At this time, the processor 102 also needs to convert the control instructions into control signals suitable for the control interface of the player, for example Control the player's infrared control signal or wifi network message signal, etc. The communication unit 103 sends an infrared control signal or a wifi network message signal to the player to control the player. The player receives the control signal and performs specific operations according to the received control signal, such as controlling the player to perform a playback operation or a fast-forward operation.

According to an embodiment of the present invention, assuming that the determined target device is a movable target, the process of the brain signal-based control device 100 realizing the user directly controlling the target to move through the brain signal is as follows: Direction symbols, such as "↑", "↓", "←", and "→" arrows, respectively represent moving up, down, left, and right, for example, the focus of the main interface of the TV moves, and the cursor moves during text input , the movement of objects in games or virtual environments, the direction movement of remote control toys, etc., the MRI equipment detects the user's brain signal to obtain the corresponding MRI image data. The processor 102 extracts the MRI image data and reconstructs the extracted MRI image based on the depth image reconstruction algorithm, and performs denoising and KNN algorithm processing on the reconstructed direction symbol image data, and analyzes the control instruction corresponding to the direction symbol, and according to The control instruction determines the target device. At this time, the processor 102 also needs to convert the control instruction into a control signal suitable for the control interface of the device where the target is located, such as an infrared control signal for controlling a remote control toy or a Bluetooth signal for controlling a game running device, etc. , wherein, the game running device includes a mobile phone, a computer, a Pad, a PSP, and the like. It should be understood that the above-mentioned examples of game running devices are only exemplary examples, and the game running devices that can be used in the present invention are not limited thereto. The communication unit 103 sends the control signal including the control instruction to the device where the target is located. The device where the target is located receives the control signal and performs a specific control movement operation according to the received control signal, such as controlling the target character in the game to perform an upward movement operation, that is, a forward operation. It should be understood that the above-mentioned example of the user directly controlling the movable object to perform the moving operation through the brain signal is only an exemplary example, and the movable object that can be used in the present invention is not limited thereto.

According to an embodiment of the present invention, the memory 104 stores the user's brain activity data and preliminary instructions corresponding to the user's brain activity data, wherein the user's brain activity data includes text data, icon data and identification data, and user-defined password data , at least one of number data and picture data. For example, text data can select a language familiar to the user. Common application scenarios include text input and all data that can be described in text. As the icon data, a user-familiar directional icon can be used. The logo data can be selected from currently more common identifiers, or text data, such as play, tentative, fast forward, fast rewind, etc., and can also customize the logo settings for commonly used control objects. For example, to intelligently control the set-top box, you can Set the TV station number or station logo familiar to the user as the identification data for opening the TV station. In addition, for some confidential special scenarios, users can use self-defined password data, number data and/or picture data, for example, the password data of the smart door lock switch and the picture data of the screen lock and unlock of the smart terminal. Memory 104 may be any suitable type of mass memory that provides any type of information that may need to be manipulated, memory 104 may be volatile or non-volatile, magnetic, semiconductor, tape, optical, Removable, non-removable, or other types of storage devices or tangible (ie, non-transitory) computer-readable media, including but not limited to ROM, flash memory, dynamic RAM, and static RAM.

According to the embodiment of the present invention, the processor 102 also conducts simulation training for the user according to the data stored in the memory 104, and judges and gives feedback on the simulation training result of the user. Specifically, in the simulation training mode, the user recalls and/or imagines the brain activity data stored in the memory 104, and the processor 102 performs image reconstruction processing on the acquired brain signals and processing of text and/or graphic recognition algorithms to obtain control instructions . The processor 102 matches the obtained control instruction with the preliminary instruction stored in the memory 104. When the control instruction matches the preliminary instruction, it judges that the control instruction is correct, and marks the user's simulation training as passed. Conversely, when the control instruction does not match the preliminary instruction, it is judged that the control instruction is an error instruction, and the simulation training result of the user is marked as failed, and the image reconstruction algorithm and text and/or graphic recognition algorithm are optimized. According to an embodiment of the present invention, assuming that in the simulated training mode, the user selects a certain instruction for training, for example, selects commonly used instructions such as "turn on the air conditioner" or "turn on the TV", and the user memorizes the image data related to the instruction And choose to enter the simulation training mode. When the user recalls the instruction, the detector 101 operates in response to the user's brain activity and collects the user's brain signal, and the processor 102 extracts and processes the brain signal to obtain a control instruction, and determines the target device and the corresponding device according to the control instruction. control signal. The communication unit 103 sends the control signal to the target device, so as to complete the user's training of the instruction. The processor 102 extracts the preliminary instruction in the memory 104 , matches the obtained control instruction with the stored preliminary instruction, judges whether the user's training result is correct, and feeds back the judgment result to the processor 102 . Among them, when the control instruction analyzed by the simulation training matches the preliminary instruction, it is judged that the control instruction is a correct instruction, and the simulation training of the user is marked as passed; when the control instruction analyzed by the simulation training does not match the preliminary instruction, it is judged that the control instruction is Incorrect instructions, mark the user's simulated training results as failed, and optimize the image reconstruction algorithm and text and/or graphic recognition algorithm. In addition, the processor 102 also feeds back wrong training results to the memory 104 and visually displays the wrong training results and related instructions, so as to enhance the user's memory of related preliminary instructions.

Fig. 3 shows a flowchart of a brain signal-based control method according to an embodiment of the present invention.

As shown in FIG. 3 , in step S301 , in response to the user's brain activity, the user's brain signal is collected and sent. According to an embodiment of the present invention, specifically, the brain signal generated by the user's brain activity is collected by a detector, wherein the detector is a wearable miniaturized magnetic resonance imaging device (MRI device), and the MRI device detects the user's brain activity The operation detects and collects the user's brain signal, that is, the MRI image.

Next, in step S302, the collected user's brain signal is received, image reconstruction processing is performed on the received brain signal, and a control instruction for controlling the target device is generated based on information obtained by identifying the reconstructed image data. Specifically, image data is reconstructed from the received brain signal, and text and/or graphic recognition algorithms are used to perform data analysis on the reconstructed image data to identify information including at least one of text, symbols and graphics. Here, image reconstruction may be performed on the received brain signal through a depth image reconstruction algorithm. Then, determine the target device according to the identified information, and generate a control instruction for controlling the target device. Wherein, performing data analysis on the reconstructed image data includes performing denoising processing on the image data, and using text and/or graphic recognition algorithms to analyze the denoised image to obtain corresponding control instructions. Finally, the target device to be controlled by the user's brain signal and the control signal corresponding to the target device are determined according to the control instruction. Here, the control signal includes at least one of an infrared control signal, a wifi network message signal, a mobile network message signal and a bluetooth signal.

Step S303, sending the control signal including the control instruction to the target device. Specifically, a corresponding control signal is generated according to the control instruction obtained in step S302 and the target device, and the control signal including the control instruction is sent to the target device. Wherein, when the determined target device is an intelligent terminal device, the control signal including the control instruction is sent to the intelligent terminal device, and when the determined target device is a non-smart terminal device, the control instruction is converted into a The control command of the control interface of the device, and the control signal including the control command is sent to the non-smart terminal device.

According to the embodiment of the present invention, the brain signal-based control method can also perform simulated training on the user according to the stored data in the simulated training mode, and judge and give feedback on the simulated training results of the user. Wherein, the stored data includes the user's brain activity data and preliminary instructions corresponding to the user's brain activity data. Specifically, in the simulated training mode, the user recalls and/or imagines certain stored brain activity data and performs operations as described above from step S301 to step S303, and then judges the control analyzed by the simulated training according to the stored preliminary instructions. Whether the instruction is correct instruction. Among them, when the control instruction analyzed by the simulation training matches the preliminary instruction, it is judged that the control instruction is a correct instruction, and the simulation training of the user is marked as passed; when the control instruction analyzed by the simulation training does not match the preliminary instruction, it is judged that the control instruction is Incorrect instructions, mark the user's simulated training results as failed, and optimize the image reconstruction algorithm and text and/or graphic recognition algorithm.

Fig. 4 shows a block diagram of a terminal device capable of brain signal control according to an embodiment of the present invention.

As shown in FIG. 4 , a terminal device 400 capable of brain signal control includes a communication unit 401 , a processor 402 and a memory 403 . Wherein, the communication unit 401 is configured to receive the user's brain signal from an external brain signal acquisition device. The processor 402 is configured to communicate with the communication unit 401 to receive the user's brain signal, perform image reconstruction processing on the received brain signal to obtain a control instruction for controlling the terminal device, and generate a corresponding control signal according to the control instruction to perform the corresponding operation. The memory 403 is configured to store the user's brain activity data and preliminary instructions corresponding to the user's brain activity data. Specifically, in the simulation training mode, the processor 402 performs simulation training for the user according to the data stored in the memory 403, and judges and gives feedback on the simulation training results of the user, wherein the stored brain activity data of the user includes text data, At least one of icon data, identification data, and user-defined password data, number data, and picture data.

Fig. 5 shows a block diagram of a brain signal acquisition device for brain signal control according to an embodiment of the present invention.

As shown in FIG. 5 , a brain signal acquisition device 500 for brain signal control includes a detector 501 and a processor 502 . Wherein, the detector 501 collects the user's brain signal in response to the user's brain activity. The processor 502 communicates with the detector 501 to receive the user's brain signal, and performs image reconstruction processing on the received brain signal to obtain a control instruction for controlling the terminal device, and then generates a corresponding control signal according to the control instruction to control the terminal device Execute the corresponding operation, and the processor 502 performs simulated training on the user according to the stored data in the simulated training mode, and judges and gives feedback on the simulated training results of the user, wherein the stored data includes the user's brain activity data and Preliminary instructions corresponding to the user's brain activity data.

Fig. 6 shows a block diagram of a brain signal processing device for brain signal control according to an embodiment of the present invention.

As shown in FIG. 6 , a brain signal processing device 600 for brain signal control includes a communication unit 601 and a processor 602 . Wherein, the communication unit 601 is configured to receive the user's brain signal from an external brain signal acquisition device. The processor 602 is configured to communicate with the communication unit 601 to receive the user's brain signal, perform image reconstruction processing on the received brain signal to obtain a control instruction for controlling the terminal device, and generate a corresponding control signal according to the control instruction to The terminal device is controlled to perform corresponding operations, and the processor 602 is still in the simulated training mode to perform simulated training on the user according to the stored data, and to judge and give feedback on the simulated training results of the user, wherein the stored data includes the user's brain Activity data and preliminary instructions corresponding to the user's brain activity data.

The control device and method based on brain signals according to the embodiments of the present invention can realize various control operations of brain signals through image reconstruction algorithms and wearable miniaturized magnetic resonance imaging equipment, and realize control based on brain signals The wide range meets the needs of various terminal devices and massive types of brain signal control, and provides a better user experience.

While the invention has been shown and described with reference to certain exemplary embodiments thereof, it will be understood by those skilled in the art that changes may be made without departing from the spirit and scope of the invention as defined by the claims and their equivalents. Various changes in form and detail.

Claims (19)

1. a kind of control device based on brain signal, which is characterized in that the control device includes：

Detector is configured to respond to the brain activity of user, acquires and send the brain signal of user；

Processor is configured as being communicated the brain signal of the user to receive the detector acquisition with the detector, right The brain signal of reception carries out image reconstruction process, and the information obtained is identified based on the image data to reconstruction, to generate Control instruction for control targe equipment；

Communication unit is configured as the control signal including the control instruction being sent to target device.

2. control device as described in claim 1, which is characterized in that the processor is configured as：

It is rebuild from the brain signal of reception and obtains image data, and utilize the image of word and/or algorithm for pattern recognition to reconstruction Data are parsed to identify including the information of at least one in word, symbol and figure；

Target device is determined according to the information of identification, and generates the control instruction for controlling the target device.

3. control device as described in claim 1, which is characterized in that the control signal includes infrared control signal, wifi At least one of network message signal, mobile network's message signals and Bluetooth signal.

4. control device as claimed in claim 2, which is characterized in that the control device further includes：

Memory is configured as the brain activity data of storage user and preliminary finger corresponding with the brain activity data of user It enables.

5. control device as claimed in claim 4, which is characterized in that the brain activity data of the user include word number According at least one of, icon data and mark data and user-defined code data, number data and image data.

6. control device as claimed in claim 4, which is characterized in that the processor is additionally configured to：

Under simulated training pattern, simulated training is carried out to user according to the data stored in the memory, and to user's Simulated training result is judged and is fed back.

7. control device as claimed in claim 6, which is characterized in that the simulated training result to user carry out judge and Feedback includes：

Judge whether the control instruction that simulated training parses is correctly to refer to according to the preliminary instruction stored in the memory It enables, wherein when control instruction that simulated training parses matches with preliminary instruction, judge control instruction for right instructions, The simulated training of label user passes through, and when the control instruction that simulated training parses is mismatched with preliminary instruction, judges to control Instruction is false command, and the simulated training result of user is marked not pass through, and is known to image reconstruction algorithm and word and/or figure Other algorithm optimizes.

8. control device as described in claim 1, which is characterized in that the detector be wearable micro magnetic resonance at As equipment.

9. a kind of control method based on brain signal, which is characterized in that the control method includes the following steps：

In response to the brain activity of user, the brain signal of user is acquired and sent；

The brain signal for receiving the user of acquisition carries out image reconstruction process to the brain signal of reception, and based on the image to reconstruction The information obtained is identified in data, to generate the control instruction for control targe equipment；

Control signal including the control instruction is sent to target device.

10. control method as claimed in claim 9, which is characterized in that the brain signal of the user for receiving acquisition, to receiving Brain signal carry out image reconstruction process, and the information that obtains is identified based on the image data to reconstruction and generates for controlling The step of control instruction of target device includes：

It is rebuild from the brain signal of reception and obtains image data, and utilize the image of word and/or algorithm for pattern recognition to reconstruction Data are parsed to identify including the information of at least one in word, symbol and figure；

Target device is determined according to the information of identification, and generates the control instruction for controlling the target device.

11. control method as claimed in claim 9, which is characterized in that the control signal includes infrared control signal, wifi At least one of network message signal, mobile network's message signals and Bluetooth signal.

12. control method as claimed in claim 9, which is characterized in that the control method further includes：

Under simulated training pattern, the data according to storage carry out simulated training to user, and to the simulated training result of user Judged and fed back, wherein the data of the storage include the brain activity data of user and the brain activity with user The corresponding preliminary instruction of data.

13. control method as claimed in claim 12, which is characterized in that the brain activity data of the user include word number According at least one of, icon data and mark data and user-defined code data, number data and image data.

14. control method as claimed in claim 13, which is characterized in that the simulated training result to user judges Include with the step of feedback：

Judge whether the control instruction that simulated training parses is right instructions according to the preliminary instruction of storage, wherein work as mould When the control instruction that quasi- training parses matches with preliminary instruction, judge that control instruction for right instructions, marks the mould of user Quasi- training passes through, and when the control instruction that simulated training parses is mismatched with preliminary instruction, judges that control instruction refers to for mistake It enables, the simulated training result of user is marked not pass through, and image reconstruction algorithm and word and/or algorithm for pattern recognition are carried out excellent Change.

15. control method as claimed in claim 9, which is characterized in that the brain activity in response to user, acquisition are concurrent The step of brain signal for sending user, is realized by wearable micro MR imaging apparatus.

16. a kind of terminal device that can carry out brain signal control, which is characterized in that the terminal device includes：

Communication unit is configured as receiving the brain signal of user from external brain signal acquisition device；

Processor is configured as being communicated with the communication unit to receive the brain signal of user, and to the brain signal of reception Image reconstruction process is carried out to obtain the control instruction for control terminal equipment, and generate accordingly according to the control instruction Signal is controlled to execute corresponding operation.

17. terminal device as claimed in claim 16, which is characterized in that the terminal device further includes：

Memory is configured as the brain activity data of storage user and preliminary finger corresponding with the brain activity data of user It enabling, the processor carries out simulated training under simulated training pattern, according to the data stored in the memory to user, and The simulated training result of user is judged and fed back, wherein the brain activity data of the user include lteral data, figure Mark at least one of data and mark data and user-defined code data, number data and image data.

18. a kind of brain signal harvester for carrying out brain signal control, which is characterized in that the brain signal harvester packet It includes：

Detector acquires the brain signal of user in response to the brain activity of user；

Processor is configured as being communicated with the detector to receive the brain signal of user, and to the brain signal of reception into Row image reconstruction process generates corresponding control to obtain the control instruction for control terminal equipment, according to the control instruction Signal executes corresponding operation with control terminal equipment, and

Under simulated training pattern, the data according to storage carry out simulated training to user, and to the simulated training result of user Judged and fed back, wherein the data of storage include the brain activity data of user and the brain activity data with user Corresponding preliminary instruction.

19. a kind of brain signal processing unit for carrying out brain signal control, which is characterized in that the brain signal acquisition device packet It includes：

Communication unit is configured as receiving the brain signal of user from external brain signal acquisition device；

Processor is configured as being communicated with the communication unit to receive the brain signal of user, and to the brain signal of reception Image reconstruction process is carried out to obtain the control instruction for control terminal equipment, and generate accordingly according to the control instruction It controls signal and corresponding operation is executed with control terminal equipment, and

Under simulated training pattern, the data according to storage carry out simulated training to user, and to the simulated training result of user Judged and fed back, wherein the data of storage include the brain activity data of user and the brain activity data with user Corresponding preliminary instruction.