CN116301345A - VR equipment control method based on visual brain-computer interface - Google Patents

Abstract

The application relates to a VR equipment control method based on a visual brain-computer interface. The method comprises the following steps: displaying an input panel within a target area in response to a panel display request; receiving a first target indication signal sent by control equipment; and determining a first target visual stimulus region corresponding to the first target visual stimulus coding sequence according to the first target visual stimulus coding sequence indicated by the first target indication signal, and executing target operation corresponding to the first target visual stimulus region. By adopting the method, the convenience of the user in the VR space for operation can be improved.

Description

VR device control method based on visual brain-computer interface

technical field

The present application relates to the technical field of human-computer interaction, in particular to a VR device control method based on a visual brain-computer interface.

Background technique

With the advancement of VR (Virtual Reality, virtual reality) technology, VR devices are also gradually developing towards a more detailed simulation and a more realistic experience for users.

In the prior art, a user often needs to control a VR device through a VR handle. However, the VR controller has problems such as unnatural operation and simple interaction mode in the VR space. For example, when implementing keyboard input, the user needs to sequentially select the letters on the virtual keyboard through the direction keys of the handle to input. The operation process is complicated and time-consuming, which causes inconvenience for the user.

Contents of the invention

Based on this, it is necessary to provide a VR device control method based on a visual brain-computer interface to address the above technical problems.

In a first aspect, the present application provides a VR device control method based on a visual brain-computer interface. The methods include:

In response to the panel display request, an input panel is displayed in the target area, the input panel includes at least one first visual stimulation area, each of the first visual stimulation areas corresponds to a different operation, and each of the first visual stimulation areas Blinking according to the visual stimulation coding sequence corresponding to each of the first visual stimulation areas, wherein the target area is the area between the VR device display area and the user, and for any two visual stimulation coding sequences, two The correlation of the visual stimulus coding sequence is less than a correlation threshold;

receiving the first target indication signal sent by the control device, the first target indication signal is an indication signal generated according to the first target electroencephalogram signal, and the first target indication signal is used to indicate the first target electroencephalogram signal Corresponding target visual stimulation coding sequence, wherein, the first target EEG signal is the EEG signal generated when the user gazes at the target visual stimulation area, and the target visual stimulation area is the at least one first visual stimulation area any one of the first visual stimulation areas;

According to the target visual stimulation coding sequence indicated by the first target indication signal, determine the target visual stimulation area corresponding to the target visual stimulation coding sequence, and execute the target operation corresponding to the target visual stimulation area.

In one embodiment, the input panel includes a mouse input panel, the mouse input panel includes a mouse indicator block, each of the first visual stimulation areas is radially arranged around the mouse indicator block, each of the first The visual stimulation areas correspond to different moving directions respectively.

In one embodiment, the performing the target operation corresponding to the target visual stimulation area includes:

moving the mouse to a specified position in the display area according to the moving direction corresponding to the first target visual stimulation area;

According to the movement direction corresponding to the first target visual stimulation area, move the mouse indicator block to a designated position in the mouse input panel;

The first visual stimulation area for each moving direction is redrawn with the mouse indicator block as the center.

In one embodiment, the moving the mouse to a specified position in the display area according to the moving direction corresponding to the first target visual stimulation area includes:

Acquiring the number of consecutive movements corresponding to the movement direction;

Determine the length of movement of the mouse according to the number of continuous movements, and the length of movement is positively correlated with the number of continuous movements;

Moving the mouse to a designated position along the moving direction according to the moving length.

In one embodiment, the displaying the input panel in the target area in response to the panel display request includes:

In response to a panel display request, an input panel control area is displayed in the display area, the input panel control area includes at least one second visual stimulation area, each of the second visual stimulation areas corresponds to a different input panel, and each The second visual stimulation areas flash according to the visual stimulation code sequences corresponding to each of the second visual stimulation areas;

In the case of receiving the second target indication signal sent by the control device, according to the second target visual stimulation coding sequence indicated by the second target indication signal, determine the second target visual stimulus corresponding to the second target visual stimulation coding sequence. stimulation area, and generate an input panel corresponding to the second target visual stimulation area in the target area, the second target indication signal is an indication signal generated according to the second target EEG signal, and the second target EEG signal is an EEG signal generated when the user gazes at a second target visual stimulation area, and the second target visual stimulation area is any one of the at least one second visual stimulation area.

In one embodiment, the input panel is any one of a vertical input panel or a parallel input panel, wherein the vertical input panel is perpendicular to the display area, and the parallel input panel is perpendicular to the display area. The display areas are parallel to each other, and the color of the parallel input panel is translucent.

In the second aspect, the present application also provides a VR device control method based on a visual brain-computer interface. The methods include:

receiving the target EEG signal sent by the EEG device, the target EEG signal is generated when the user looks at the target visual stimulation area, the target visual stimulation area is any one of at least one visual stimulation area in the input panel, the The input panel is generated in the target area by the VR device in response to the panel display request, each of the visual stimulation areas corresponds to a different operation, and each of the visual stimulation areas is respectively in accordance with the visual stimulation code sequence corresponding to each of the visual stimulation areas performing flashing, wherein the target area is the area between the display area of the VR device and the user, and for any two visual stimulation coding sequences, the correlation of the two visual stimulation coding sequences is less than a correlation threshold;

From the reference EEG signals corresponding to each visual stimulation coding sequence, determine the target reference EEG signal matching the target EEG signal, and determine according to the preset correspondence between the reference EEG signal and the visual stimulation coding sequence A target visual stimulation coding sequence corresponding to the target reference EEG signal, and generating a target indicating signal corresponding to the target visual stimulation coding sequence;

sending the target indication signal to the VR device, so that the VR device determines the target visual stimulation area corresponding to the target visual stimulation coding sequence according to the target visual stimulation coding sequence indicated by the target indication signal, And execute the target operation corresponding to the target visual stimulation area.

In the third aspect, the present application also provides a VR device control system based on a visual brain-computer interface, including a VR device, an EEG device, and a control device, wherein,

The VR device is configured to, in response to a panel display request, display an input panel corresponding to the panel display request in a target area, the input panel includes at least one visual stimulation area, each of the visual stimulation areas corresponds to a different operation , and each of the visual stimulation areas flashes according to the visual stimulation code sequence corresponding to each of the visual stimulation areas, wherein the target area is the area between the VR device display area and the user, for any two of the visual stimulation areas stimulus coding sequences, two of said visual stimulus coding sequences are less than a correlation threshold;

The EEG device is configured to detect a target EEG signal generated when the user gazes at a target visual stimulation area, and send the target EEG signal to the control device, the target visual stimulation area being the at least one any one of the first visually stimulating regions;

The control device is configured to receive the target EEG signal sent by the EEG device, determine the target reference EEG signal corresponding to the target EEG signal, and according to the preset correspondence between the reference EEG signal and the visual stimulation coding sequence , determining the target visual stimulus coding sequence corresponding to the target reference EEG signal, and generating a target indication signal corresponding to the target visual stimulation coding sequence, and sending the target indication signal to the VR device;

The VR device is further configured to receive a target indication signal sent by the control device, determine a target visual stimulation area corresponding to the target visual stimulation coding sequence according to the target visual stimulation coding sequence indicated by the target indication signal, and execute The target operation corresponding to the target visual stimulation area.

In the fourth aspect, the present application also provides a VR device control device based on a visual brain-computer interface, the device comprising:

A display module, configured to display an input panel in a target area in response to a panel display request, the input panel includes at least one first visual stimulation area, each of the first visual stimulation areas corresponds to a different operation, and each of the The first visual stimulation areas flash respectively according to the visual stimulation coding sequence corresponding to each of the first visual stimulation areas, wherein the target area is the area between the display area of the VR device and the user, for any two visual stimulation coding sequences, the correlation of two said coding sequences of visual stimuli is less than a correlation threshold;

The receiving module is configured to receive a first target indication signal sent by the control device, the first target indication signal is an indication signal generated according to the first target EEG signal, and the first target indication signal is used to indicate the first target indication signal A first target visual stimulation coding sequence corresponding to a target EEG signal, wherein the first target EEG signal is an EEG signal generated when the user gazes at the first target visual stimulation area, and the first target visual stimulation an area is any one of said at least one first visually stimulating area;

An execution module, configured to determine the first target visual stimulation area corresponding to the first target visual stimulation coding sequence according to the first target visual stimulation coding sequence indicated by the first target indication signal, and execute the first target visual stimulation Stimulus regions correspond to target manipulations.

In one of the embodiments, the input panel includes a mouse input panel, the mouse input panel includes a mouse indicator block, each of the first visual stimulation areas is radially arranged around the mouse indicator block, and each of the first visual stimulation areas A visual stimulation area corresponds to different moving directions respectively.

In one of the embodiments, the execution module is also used for:

moving the mouse to a specified position in the display area according to the moving direction corresponding to the first target visual stimulation area;

According to the movement direction corresponding to the first target visual stimulation area, move the mouse indicator block to a designated position in the mouse input panel;

The first visual stimulation area for each moving direction is redrawn with the mouse indicator block as the center.

In one of the embodiments, the execution module is also used for:

Acquiring the number of consecutive movements corresponding to the movement direction;

Determine the length of movement of the mouse according to the number of continuous movements, and the length of movement is positively correlated with the number of continuous movements;

Moving the mouse to a designated position along the moving direction according to the moving length.

In one of the embodiments, the display module is also used for:

In response to a panel display request, an input panel control area is displayed in the display area, the input panel control area includes at least one second visual stimulation area, each of the second visual stimulation areas corresponds to a different input panel, and each The second visual stimulation areas flash according to the visual stimulation code sequences corresponding to each of the second visual stimulation areas;

In the case of receiving the second target indication signal sent by the control device, according to the second target visual stimulation coding sequence indicated by the second target indication signal, determine the second target visual stimulus corresponding to the second target visual stimulation coding sequence. stimulation area, and generate an input panel corresponding to the second target visual stimulation area in the target area, the second target indication signal is an indication signal generated according to the second target EEG signal, and the second target EEG signal is an EEG signal generated when the user gazes at a second target visual stimulation area, and the second target visual stimulation area is any one of the at least one second visual stimulation area.

In one embodiment, the input panel is any one of a vertical input panel or a parallel input panel, wherein the vertical input panel is perpendicular to the display area, and the parallel input panel is perpendicular to the display area. The display areas are parallel, and the color of the parallel input panel is translucent.

In the fifth aspect, the present application also provides a VR device control device based on a visual brain-computer interface, the device comprising:

The receiving module is used to receive the target EEG signal sent by the EEG device, the target EEG signal is generated when the user looks at the target visual stimulation area, and the target visual stimulation area is at least one of the visual stimulation areas in the input panel In any one, the input panel is generated in the target area by the VR device in response to the panel display request, each of the visual stimulation areas corresponds to a different operation, and each of the visual stimulation areas corresponds to each of the visual stimulation areas. The visual stimulation coding sequence is flashed, wherein the target area is the area between the VR device display area and the user, and for any two visual stimulation coding sequences, the correlation between the two visual stimulation coding sequences is less than the correlation sexual threshold;

The determination module is used to determine the target reference EEG signal matching the target EEG signal from the reference EEG signals corresponding to each visual stimulation coding sequence, and determine the target reference EEG signal according to the preset reference EEG signal and the visual stimulation coding sequence. Correspondence, determining the target visual stimulus coding sequence corresponding to the target reference EEG signal, and generating a target indication signal corresponding to the target visual stimulation coding sequence;

A sending module, configured to send the target indication signal to the VR device, so that the VR device determines the target visual stimulus code sequence corresponding to the target visual stimulus code sequence according to the target visual stimulus code sequence indicated by the target indication signal target visual stimulation area, and execute the target operation corresponding to the target visual stimulation area.

In a sixth aspect, the present application also provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements any one of the above methods when executing the computer program.

In a seventh aspect, the present application also provides a computer-readable storage medium. The computer-readable storage medium has a computer program stored thereon, and when the computer program is executed by a processor, any one of the above methods is realized.

In an eighth aspect, the present application also provides a computer program product. The computer program product includes a computer program that implements any one of the above methods when the computer program is executed by a processor.

The aforementioned VR device control method, device, system, computer device, and storage medium based on a visual brain-computer interface display an input panel in the area between the VR device display area and the user according to the user's panel display request, and the input panel includes at least one The first visual stimulation area, and each first visual stimulation area flashes according to different visual stimulation coding sequences. The correlation between any two coding sequences of visual stimuli is less than the correlation threshold. When the user gazes at any first target visual stimulation area in the first visual stimulation area, the first target EEG signal will be generated. The VR device receives the first target indication signal generated by the control device according to the first target EEG signal, and judges the target operation to be performed according to the first target indication signal. The embodiment of the present application uses the space between the display area of the VR device and the user to display the input panel, and according to the EEG signal generated when the user looks at the first visual stimulation area in the input panel, the VR device executes the target corresponding to the visual stimulation area Therefore, the user can realize the control of the VR device in the VR space without the need of a VR handle, which can improve the convenience of the user's operation. The use of visual stimulus coding sequences with low mutual correlation can also improve the accuracy of differentiating EEG signals induced by different visual stimulus coding sequences.

Description of drawings

Fig. 1 is an application environment diagram of a VR device control method based on a visual brain-computer interface in an embodiment;

FIG. 2 is a schematic flow diagram of a VR device control method based on a visual brain-computer interface in an embodiment;

Fig. 3 is a schematic diagram of a target area in an embodiment;

Figure 4 is a schematic diagram of a target area in one embodiment;

Fig. 5 is a comparison diagram of multiple visual stimulation coding sequences in one embodiment;

Figure 6 is a schematic diagram of a mouse input panel in one embodiment;

FIG. 7 is a schematic flow chart of step 206 in an embodiment;

Fig. 8 is a schematic diagram of the movement of the mouse pointer block and each first visual stimulation area in one embodiment;

FIG. 9 is a schematic flow chart of step 704 in an embodiment;

FIG. 10 is a schematic flow chart of step 202 in an embodiment;

Fig. 11 is a schematic diagram of the positions of each input panel relative to the display area in an embodiment;

Figure 12 is a schematic diagram of a vertical input panel in one embodiment;

Figure 13 is a schematic diagram of a parallel input panel in an embodiment;

Figure 14 is a schematic diagram of an embedded input panel in one embodiment;

FIG. 15 is a schematic flowchart of a VR device control method based on a visual brain-computer interface in an embodiment;

Fig. 16 is a structural block diagram of a VR device control device based on a visual brain-computer interface in an embodiment;

Fig. 17 is a structural block diagram of a VR device control device based on a visual brain-computer interface in an embodiment;

Figure 18 is a diagram of the internal structure of a computer device in one embodiment.

Detailed ways

In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

The VR device control method based on the visual brain-computer interface provided in the embodiment of the present application can be applied but not limited to the application environment shown in FIG. 1 . Wherein, the VR device 102 communicates with the control device 104 , and the control device 104 communicates with the EEG device 106 . When the VR device 102 receives the panel display request, it displays the input panel corresponding to the panel display request in the target area, and the user generates corresponding EEG signals by gazing at the visual stimulation area on the input panel, and the EEG device 106 collects the user's EEG signals. signal, and send the EEG signal to the control device 104. The control device 104 analyzes the EEG signal, determines the indication signal corresponding to the EEG signal, and sends the indication signal to the VR device 102 . The VR device 102 performs corresponding target operations according to the instruction signal. Among them, the control device 104 can be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, Internet of Things devices and portable wearable devices, and the Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart vehicle-mounted devices wait. Portable wearable devices can be smart watches, smart bracelets, head-mounted devices, and the like.

In one embodiment, as shown in FIG. 2 , a VR device control method based on a visual brain-computer interface is provided. In this embodiment, the method is applied to the VR device 102 in FIG. 1 as an example for illustration, including the following steps :

Step 202, in response to the panel display request, display the input panel in the target area, the input panel includes at least one first visual stimulation area, and each first visual stimulation area corresponds to a different operation, and each first visual stimulation area is in accordance with each The visual stimulation coding sequence corresponding to the first visual stimulation area flashes, wherein the target area is the area between the VR device display area and the user, and for any two visual stimulation coding sequences, the correlation between the two visual stimulation coding sequences is less than the correlation sex threshold.

In the embodiment of the present application, the user sends a panel display request to the VR device, so that the VR device generates and displays an input panel in a target area. The panel display request can be sent through a VR handle or other control methods, and the input panel can be a keyboard input panel (for example, a keyboard input panel composed of 40 target stimulus blocks), a mouse input panel (for example, a mouse input panel composed of multiple target stimulus blocks) input panel) or a user-defined input panel, etc., which are not specifically limited in this embodiment of the present application.

The target area refers to the area between the display area of the VR device and the user. The display area refers to the display screen generated in the VR virtual space when the VR device only displays two-dimensional images (for example, when watching ordinary videos, texts, images or browsing web pages through the VR device). Referring to FIG. 3 , after the user enters the VR space, he has a viewpoint in the virtual space, and at the same time, the display area of the VR device is displayed parallel to the user's field of view at a certain distance from the user. In this case, the target area refers to a three-dimensional space between the display area of the VR device and the user's viewpoint.

When there is no display area in the VR device (such as when the user enters a VR game or other VR applications), the target area can refer to a spherical area surrounding the user's viewpoint (as shown in Figure 4). When the user's viewpoint moves, the target area follows The user viewpoint moves together. The input panel can be displayed in any area of the target area, such as displaying the input panel on a plane parallel to the user's field of view, or displaying the input panel on a plane perpendicular to the user's field of view, etc., which is not specifically limited in this embodiment of the present application .

The embodiments of the present application may also be applied to AR devices. When applied to an AR device, a target area may refer to a three-dimensional space between a real object and a user.

The input panel includes at least one first visual stimulation area, each first visual stimulation area corresponds to a different operation, and different visual stimulation areas flash according to their respective visual stimulation code sequences, wherein any two visual stimulation codes The correlation between the sequences is smaller than the correlation threshold to improve the accuracy of judging the area the user is looking at. The correlation threshold can be set by those skilled in the art according to actual needs.

In addition, the visual stimulus coding sequence selected in the embodiment of the present application can also have the characteristics that the bandwidth is greater than the bandwidth threshold, and the phase change is not periodic, and the bandwidth threshold can also be set by those skilled in the art according to actual needs, so as to achieve transmission The information efficiency is higher, and the effect of reducing the user's visual fatigue is achieved. Moreover, by using a visual stimulus code sequence with non-periodic phase change, when generating a visual stimulus code sequence, a large number of sequences whose mutual correlation is less than the correlation threshold can also be generated to support the need to simultaneously display dozens of different Input panel for coding sequences of visual stimuli. Fig. 5 is a comparison of phase changes between the visual stimulus coding sequence in the embodiment of the present application and the M-Sequence (M-Sequence) and Steady-State Visual Evoked Potentials sequence (SSVEP, Steady-State Visual Evoked Potentials) in the prior art.

The operation corresponding to the first visual stimulation area can be a simple operation consisting of one action, such as opening an application, adjusting the volume of the device, inputting a certain letter, etc., or it can be a user-defined operation consisting of multiple actions, such as Open a certain video in the video application, and adjust the volume to a fixed value, etc., which is not specifically limited in this embodiment of the present application.

The visual stimulation coding sequence corresponding to each visual stimulation area can be preset, or randomly selected from the visual stimulation coding sequence library when generating the control panel. Each visual stimulation coding sequence in the visual stimulation coding sequence library has its corresponding reference EEG signal, and the reference EEG signal refers to the most likely EEG signal generated by the user when watching the area flickering according to the visual stimulation coding sequence. Each visual stimulus coding sequence in the visual stimulation coding sequence library may be shown to the user in advance (for example, when the user uses the VR device for the first time) (for example, by making a plurality of regions blink and display according to each visual stimulation coding sequence), and the user may be collected. Based on the reference EEG signals generated by different visual stimulation coding sequences, and recording the corresponding relationship between the reference EEG signals and the visual stimulation coding sequences. In practical applications, the control device can determine which visual stimulus code sequence the user is looking at that flashes according to the user's EEG signal (generally, the control device can complete the accurate judgment after the user gazes at the area for 0.2-0.3s) ), and then send the determined target visual stimulation coding sequence to the VR device, so that the VR device judges which first target visual stimulation region corresponds to the target visual stimulation coding sequence, and executes the target operation corresponding to the first target visual stimulation region .

For example, if the generated input panel includes two first visual stimulation regions (region A and region B), region A corresponds to visual stimulus code sequence A, and corresponds to the operation of turning pages up, and region B corresponds to visual stimulus code sequence B, and corresponds to the operation of turning the page down. When the area flickers according to the visual stimulus code sequence A, the induced user EEG signal is the reference EEG signal A, and when the area is flickered according to the visual stimulus code sequence B, the evoked user EEG signal is the reference EEG signal B, then the actual The process of controlling the VR device is as follows: if the user needs to turn down the page, then focus on area B. The EEG device collects the first target EEG signal of the user at this time, and sends the first target EEG signal to the control device. The control device compares the similarity of the first target EEG signal with the reference EEG signal A and the reference EEG signal B respectively. Since the used visual stimulation coding sequences have little correlation with each other, the probability of similarity between the first target EEG signal and the reference EEG signal B is higher. Therefore, the control device determines that what the user is looking at is the area that blinks according to the visual stimulus code sequence B, and based on this, sends the first target indication signal to the VR device, indicating to the VR device that the user is looking at the area that blinks according to the visual stimulus code sequence B Area. The VR device judges that the area corresponding to the visual stimulus coding sequence B is area B, and the operation corresponding to area B is page down, and then performs the page down operation.

Step 204, receiving the first target indication signal sent by the control device, the first target indication signal is an indication signal generated according to the first target EEG signal, and the first target indication signal is used to indicate the first target EEG signal corresponding to A target visual stimulation coding sequence, wherein the first target EEG signal is the EEG signal generated when the user gazes at the first target visual stimulation area, and the first target visual stimulation area is any one of at least one first visual stimulation area A first visual stimulation area.

In the embodiment of the present application, when the user determines the target operation to be performed and gazes at the first target visual stimulation area corresponding to the target operation, the EEG device collects the first target EEG signal generated by the user, and sends the first target EEG signal The signal is sent to the control device. The control device further determines the reference EEG signal corresponding to the first target EEG signal, and further determines the first target visual stimulus coding sequence corresponding to the first target EEG signal according to the correspondence between the reference EEG signal and the visual stimulation coding sequence. After the control device generates the first target indication signal for indicating that the first target EEG signal of the VR device corresponds to the first target visual stimulation coding sequence, it sends the first target indication signal to the VR device, and the VR device then receives the first target visual stimulation code sequence. Target indication signal.

Step 206, according to the first target visual stimulation coding sequence indicated by the first target indication signal, determine the first target visual stimulation area corresponding to the first target visual stimulation coding sequence, and execute the target operation corresponding to the first target visual stimulation area.

In the embodiment of the present application, after receiving the first target indication signal, the VR device judges the first target visual stimulation area corresponding to the first target visual stimulation code according to the first target visual stimulation code indicated by the first target indication signal, And execute the target operation corresponding to the first target visual stimulation area, so as to complete the process of the user controlling the VR device through the EEG signal.

According to the VR device control method based on the visual brain-computer interface provided by the embodiment of the present application, according to the user's panel display request, the input panel is displayed in the area between the VR device display area and the user, and the input panel includes at least one first visual stimulation area. , and each first visual stimulation region flashes according to a different visual stimulation coding sequence. The correlation between any two coding sequences of visual stimuli is less than the correlation threshold. When the user gazes at any first target visual stimulation area in the first visual stimulation area, the first target EEG signal will be generated. The VR device receives the first target indication signal generated by the control device according to the first target EEG signal, and judges the target operation to be performed according to the first target indication signal. The embodiment of the present application uses the space between the display area of the VR device and the user to display the input panel, and according to the EEG signal generated when the user looks at the first visual stimulation area in the input panel, the VR device executes the target corresponding to the visual stimulation area Therefore, the user can control the VR device in the VR space without a VR handle, which can improve the user's operation convenience. The use of visual stimulus coding sequences with low mutual correlation can also improve the accuracy of differentiating EEG signals induced by different visual stimulus coding sequences.

In one embodiment, the input panel includes a mouse input panel, the mouse input panel includes a mouse indicator block, each first visual stimulation area is radially arranged around the mouse indicator block, and each first visual stimulation area corresponds to a different moving direction.

In the embodiment of the present application, as shown in FIG. 6, when the input panel is a mouse input panel, the mouse input panel may include a mouse indicating block for indicating the position of the mouse, and a plurality of first blocks for controlling the mouse to move in different directions. A visually stimulating area. Wherein, the mouse indicator block may be marked with a more eye-catching color without blinking. When generating the mouse input panel, the mouse pointer block can be generated at a random position, or a mouse pointer block can be generated at a fixed preset position (such as the center of the mouse input panel). Each first visual stimulation area is radially arranged around the mouse indicator block, and the position of each first visual stimulation area on the mouse input panel can be related to its corresponding moving direction. For example, the movement direction corresponding to the first visual stimulation area located above the mouse indication block can be upward movement, and the movement direction corresponding to the first visual stimulation area located at the bottom right of the mouse indication block can be lower right movement, etc., to improve the user's Operating experience.

Both the mouse indicator block and each first visual stimulation area can be composed of one or more display blocks, and the display blocks belonging to the same first visual stimulation area flash according to the same visual stimulation coding sequence. Composing the first visual stimulation area by a plurality of display blocks is beneficial to realize that when the mouse pointer block moves, each first visual stimulation area dynamically moves together with the mouse pointer block, so that the user only needs to focus on the point where the mouse pointer block needs to be moved. position, the operation of the mouse can be realized without judging which first visual stimulation area needs to be watched according to the real-time position of the mouse indicator block: the reason is that the user’s gaze position must be a certain display block on the mouse input panel, And when each first visual stimulation area and the mouse pointer block move dynamically together, the relative orientations of each display block and mouse pointer block forming each first visual stimulation area are in fact the corresponding movement of the first visual stimulation area where it is located. direction. Referring to Figure 6, if the initial state of the mouse input panel is A, the user needs to move the mouse indicator block filled with oblique lines to the position of the display block filled with horizontal lines, then the mouse input panel will pass through B and C. A state: It can be seen that the user only needs to keep looking at the position where the mouse indicator block needs to be moved to realize the operation of the mouse and the mouse indicator block, without moving the line of sight during the operation.

It should be noted that the mouse input panel in Figure 6 includes 8 first visual stimulation areas, and the visual stimulation areas A-H correspond to moving up to the left, moving up, moving up to the right, moving to the right, moving down to the right, and moving down , moving to sit down, and moving to the left as examples, but in fact, the number of first visual stimulation areas can also be increased or decreased according to actual needs, which is not specifically limited in this embodiment of the present application.

In the VR device control method based on the visual brain-computer interface provided by the embodiment of the present application, the first visual stimulation area corresponding to the mouse pointer block and each moving direction is set on the mouse input panel, and each first visual stimulation area is represented by the mouse pointer block The center is arranged in a radial shape, enabling the user to see the real-time position of the mouse on the mouse input panel when operating the mouse, thereby improving the convenience of the user's operation.

In one embodiment, as shown in FIG. 7, in step 206, the target operation corresponding to the first target visual stimulation area is executed, including:

Step 702, move the mouse to a designated position in the display area according to the moving direction corresponding to the first target visual stimulation area.

Step 704, move the mouse pointing block to a designated position in the mouse input panel according to the moving direction corresponding to the first target visual stimulation area.

Step 706, centering on the mouse indication block, redrawing the first visual stimulation area for each mouse movement direction.

In the embodiment of the present application, after receiving the first target indication signal, the VR device can simultaneously move the mouse on the display area and move the mouse indication block on the mouse input panel. Exemplarily, the preset distance for moving the mouse and the mouse pointing block during each operation can be preset, and when the first target pointing signal is received, the mouse and the mouse pointing block will be pointed in the direction indicated by the first target pointing signal The block moves the preset distance to the specified position; or, according to the user's habit of operating the mouse in the real space, when the mouse moves continuously in the same direction, the moving distance of the mouse and the mouse pointing block increases each time the operation is performed, etc. This embodiment of the present application does not specifically limit it.

Since the mouse indicator block on the mouse input panel should reflect the real-time position of the mouse on the display area, it is also possible to set the distance between the moving distance of the mouse and the moving distance of the mouse indicator block according to the proportional relationship between the display area and the mouse input panel. ratio. For example, when the size ratio of the display area to the mouse input panel is 8:1, the proportional relationship between the distance that the mouse moves each time and the distance that the mouse pointer moves each time can also be 8:1.

After the mouse pointing block is moved, the first visual stimulation area for each mouse moving direction can be redrawn according to the position of the mouse pointing block after moving. Referring to Figure 8, after the mouse moves to the upper right, the first visual stimulation areas for the eight moving directions can be regenerated around the mouse pointer block, so as to achieve the effect of making each first visual stimulation area and the mouse pointer block move together , so that the user only needs to focus on the position where the mouse needs to be moved when operating the mouse to move, and can operate the mouse without moving the line of sight, thereby improving the user's operating experience.

It should be noted that the regenerated visual stimulus code sequence corresponding to each first visual stimulus region may be the same as or different from the original visual stimulus code sequence, which is not specifically limited in this embodiment of the present application.

In the VR device control method based on the visual brain-computer interface provided by the embodiment of the present application, according to the mouse movement direction indicated by the first target indication signal, the mouse is moved on the display area at the same time, and the mouse indication block is moved on the mouse input panel, and the The moved mouse pointer block is the center, and the first visual stimulation area for each mouse movement direction is redrawn, so that each first visual stimulation area can be moved together with the mouse pointer block, so that the user's line of sight is following the movement of the mouse pointer block During the process, you only need to watch the position where you need to move the mouse, and you can operate the mouse without moving your sight, which can improve the convenience of user operation; at the same time, because the mouse indicator block moves continuously, it can also avoid the rapid jump of the mouse indicator block, which cannot stable phenomenon.

In one embodiment, as shown in FIG. 9, in step 704, moving the mouse to a specified position in the display area according to the moving direction corresponding to the first target visual stimulation area includes:

Step 902, acquiring the number of consecutive movements corresponding to the mouse movement direction.

Step 904: Determine the moving length of the mouse according to the number of consecutive moves, and the moving length is positively correlated with the number of consecutive moves.

Step 906, move the mouse to a designated position along the mouse moving direction according to the moving length.

In the embodiment of this application, there may be a counter in the VR device that records the number of times the mouse moves in the same direction continuously. The counter records the direction in which the mouse moved last time (hereinafter referred to as the historical direction) and the number of times the mouse moved continuously in this direction. (that is, the number of consecutive moves). When the current moving direction of the mouse is the same as the historical direction, the number of consecutive moves is increased by one, and when the current moving direction of the mouse is different from the historical direction, the number of continuous moving is reset. When moving the mouse, the current moving length of the mouse can be determined according to the number of continuous movements recorded in the counter. For example, the unit movement length can be preset, and the current movement length of the mouse can be determined according to the product of the number of continuous movements and the unit movement length. Alternatively, other methods may be used to determine the current movement length of the mouse, as long as the movement length of the mouse is positively correlated with the number of consecutive movements, which is not specifically limited in this embodiment of the present application.

For example, take the product of the number of continuous moves and the length of the unit movement as the current movement length of the mouse. After the VR device determines that the mouse should move to the right this time, the VR device can calculate the direction of the mouse movement and the counter Compare the historical direction in . If the historical direction is moving to the left, that is, the direction of the current mouse movement is different from the historical direction, the VR device will reset the historical direction to move to the right, and reset the number of consecutive movements to 1 (or other initial values) , and then determine that the moving length of the mouse is 1 unit moving length, and move the mouse to the right by 1 unit moving length to the specified position. When the mouse is moved next time, if the VR device still determines that the mouse should move to the right this time, that is, the direction of the mouse movement is the same as the historical direction, the VR device will add 1 to the number of continuous movements (that is, the current value is 2), and then Determine that the moving length of the mouse this time is 2 units of moving length, and move the mouse to the right by 2 units of moving length to the specified position...and so on until the user finishes inputting.

It should be noted that the process of moving the mouse indicator block on the mouse control panel may also refer to the above-mentioned process of moving the mouse in the display area, which will not be repeated in this embodiment of the present application.

The VR device control method based on the visual brain-computer interface provided by the embodiment of the present application determines the moving length of the mouse according to the number of consecutive moves corresponding to the moving direction of the mouse, and makes the moving length and the number of continuous moving positively correlated, and then controls the mouse according to the moving length Move, when the user continuously moves the mouse in the same direction, speeds up the moving speed of the mouse and improves the convenience of the user's operation.

In one embodiment, as shown in FIG. 10, in step 202, in response to the panel display request, displaying the input panel in the target area includes:

Step 1002, in response to the panel display request, display the input panel control area in the display area, the input panel control area includes at least one second visual stimulation area, each second visual stimulation area corresponds to a different input panel, and each second visual stimulation area The stimulation areas flash respectively according to the visual stimulation coding sequence corresponding to each second visual stimulation area.

Step 1004, in the case of receiving the second target indication signal sent by the control device, according to the second target visual stimulation coding sequence indicated by the second target indication signal, determine the second target visual stimulus corresponding to the second target visual stimulation coding sequence area, and generate an input panel corresponding to the second target visual stimulation area in the target area, the second target indication signal is an indication signal generated according to the second target EEG signal, and the second target EEG signal is the user looking at the second target The electroencephalogram signal generated when the visual stimulation area is used, the second target visual stimulation area is any second visual stimulation area in at least one second visual stimulation area.

In the embodiment of the present application, after receiving the panel display request sent by the user, the VR device can display the second visual stimulation area corresponding to each input panel generated by the user in the current interface of the VR device in the display area. For example, when there is a text input interface in the display area, the second visual stimulation area corresponding to the keyboard input panel and the mouse input panel can be displayed in the display area; when there is no text input interface in the display area, the mouse input can be displayed in the display area The panel and the second visual stimulation area corresponding to the user-defined shortcut key input panel are not specifically limited in this embodiment of the present application.

Each second visual stimulation area flashes according to a different visual stimulation coding sequence, and the control device generates a second target indication signal by judging which visual stimulation coding sequence induces the user's EEG signal, so that the VR device generates a corresponding input panel. The control device determines the second target indication signal, and the process of the control device interacting with the EEG device and the VR device can refer to the relevant description of the control device determining the first target indication signal in the foregoing embodiments, and the embodiment of the present application is not repeated here. repeat.

The VR device control method based on the visual brain-computer interface provided in the embodiment of the present application can generate each second visual stimulation area, so that the user can send a panel display request to the VR device by watching the second visual stimulation area, so that the user can further reduce the number of It is necessary to operate the VR device through other devices, so as to improve the convenience for users to operate in the VR space.

In one embodiment, the input panel is either a vertical input panel or a parallel input panel, wherein the vertical input panel is perpendicular to the display area, the parallel input panel is parallel to the display area, and the parallel input panel The color is translucent.

In the embodiment of the present application, the input panel may be a vertical input panel or a parallel input panel. Alternatively, the input panel can also be arranged inside the display area as an embedded input panel. Referring to Figure 11, if the plane where the display area is located is the yz plane, and a three-dimensional coordinate system is constructed in the VR space, then the vertical input panel can be set on the xy plane or xz plane, or other planes parallel to the xy plane or xz plane Inside, that is, perpendicular to the display area. Referring to FIG. 12 , it is an example of a vertical input panel arranged on the xy plane.

Parallel input panels may be parallel to the yz plane. Referring to FIG. 13 , when the input panel is a parallel input panel, the parallel input panel can be set to be translucent, so that the user can see the content displayed in the display area through the parallel input panel.

The recessed input panel can be arranged in the yz plane. Referring to FIG. 14, the size of the embedded input panel should be smaller than the size of the display area.

The VR device control method based on the visual brain-computer interface provided by the embodiment of the present application can generate a vertical input panel or a parallel input panel. The vertical input panel can use the free space between the display area and the user more efficiently, and the vertical input panel can The parallel input panel is more in line with the user's input habits in the real space; the parallel input panel can enable the user to see the content in the display area at the same time when inputting, which is convenient for the user to check the input content. Therefore, in the use of VR space While reducing the free space, it can also improve the convenience of user operation.

In one embodiment, as shown in FIG. 15 , a VR device control method based on a visual brain-computer interface is provided. In this embodiment, the application of the method to the control device 104 in FIG. 1 is used as an example for illustration, including the following steps :

Step 1502, receiving the target EEG signal sent by the EEG device, the target EEG signal is generated when the user gazes at the target visual stimulation area, the target visual stimulation area is any one of at least one visual stimulation area in the input panel, and the input panel consists of The VR device is generated in the target area in response to the panel display request. The input panel includes at least one visual stimulation area, and each visual stimulation area corresponds to a different operation, and each visual stimulation area is coded according to the visual stimulation code sequence corresponding to each visual stimulation area. Flashing is performed, wherein the target area is the area between the display area of the VR device and the user, and for any two visual stimulus coding sequences, the correlation between the two visual stimulation coding sequences is less than a correlation threshold.

Step 1504, from the reference EEG signals corresponding to each visual stimulation coding sequence, determine the target reference EEG signal matching the target EEG signal, and according to the preset correspondence between the reference EEG signal and the visual stimulation coding sequence, A target visual stimulus coding sequence corresponding to the target reference EEG signal is determined, and a target indication signal corresponding to the target visual stimulation coding sequence is generated.

Step 1506: Send the target indication signal to the VR device, so that the VR device determines the target visual stimulation area corresponding to the target visual stimulation coding sequence according to the target visual stimulation coding sequence indicated by the target indication signal, and executes the target visual stimulation area corresponding to operate.

In the embodiment of the present application, the EEG device collects the target EEG signal generated when the user gazes at the target visual stimulation area in the input panel of the VR device, and sends the target EEG signal to the control device. Reference EEG signals corresponding to each visual stimulation coding sequence are pre-recorded in the control device. For the acquisition method of the reference EEG signals, refer to the relevant descriptions of the foregoing embodiments, and the embodiments of the present application will not repeat them here. The control device determines the target reference EEG signal most similar to the target EEG signal from each reference EEG signal by matching the target EEG signal with each reference EEG signal, and determines the target visual stimulus corresponding to the target reference EEG signal coding sequence, and then generate a target indication signal for informing the VR device of the target visual stimulation coding sequence, and send the target indication signal to the VR device. For the interaction process among the control device, the EEG device, and the VR device, and the respective operations performed by the control device, the EEG device, and the VR device, reference may be made to the relevant descriptions of the foregoing embodiments, and details will not be repeated in this embodiment of the present application.

According to the VR device control method based on the visual brain-computer interface provided by the embodiment of the present application, according to the user's panel display request, the input panel is displayed in the area between the VR device display area and the user, and the input panel includes at least one first visual stimulation area. , and each first visual stimulation region flashes according to a different visual stimulation coding sequence. The correlation between any two coding sequences of visual stimuli is less than the correlation threshold. When the user gazes at any first target visual stimulation area in the first visual stimulation area, the first target EEG signal will be generated. The control device receives the target EEG signal, determines the target reference EEG signal corresponding to the target EEG signal, and determines the target visual stimulation code sequence corresponding to the target reference EEG signal, and then notifies the VR device of the target visual stimulus code through the first target indication signal sequence, so that the VR device can determine the target operation to be performed according to the target visual stimulation area corresponding to the target visual stimulation coding sequence. The embodiment of the present application uses the space between the display area of the VR device and the user to display the input panel, and according to the EEG signal generated when the user looks at the first visual stimulation area in the input panel, the VR device executes the target corresponding to the visual stimulation area Therefore, the user can realize the control of the VR device in the VR space without the need of a VR handle, which can improve the convenience of the user's operation. The use of visual stimulus coding sequences with low mutual correlation can also improve the accuracy of differentiating EEG signals induced by different visual stimulus coding sequences.

It should be understood that although the steps in the flow charts involved in the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in the flow charts involved in the above-mentioned embodiments may include multiple steps or stages, and these steps or stages are not necessarily executed at the same time, but may be performed at different times For execution, the execution order of these steps or stages is not necessarily performed sequentially, but may be executed in turn or alternately with other steps or at least a part of steps or stages in other steps.

In one embodiment, a VR device control system based on a visual brain-computer interface is provided, the system includes a VR device, an EEG device, and a control device, wherein:

The VR device is configured to, in response to a panel display request, display an input panel corresponding to the panel display request in a target area, the input panel includes at least one visual stimulation area, each of the visual stimulation areas corresponds to a different operation , and each of the visual stimulation areas flashes according to the visual stimulation code sequence corresponding to each of the visual stimulation areas, wherein the target area is the area between the VR device display area and the user, for any two of the visual stimulation areas stimulus coding sequences, two of said visual stimulus coding sequences are less than a correlation threshold;

The EEG device is configured to detect a target EEG signal generated when the user gazes at a target visual stimulation area, and send the target EEG signal to the control device, the target visual stimulation area being the at least one any one of the first visually stimulating regions;

The control device is configured to receive the target EEG signal sent by the EEG device, determine the target reference EEG signal corresponding to the target EEG signal, and according to the preset correspondence between the reference EEG signal and the visual stimulation coding sequence , determining the target visual stimulus coding sequence corresponding to the target reference EEG signal, and generating a target indication signal corresponding to the target visual stimulation coding sequence, and sending the target indication signal to the VR device;

The VR device is further configured to receive a target indication signal sent by the control device, determine a target visual stimulation area corresponding to the target visual stimulation coding sequence according to the target visual stimulation coding sequence indicated by the target indication signal, and execute The target operation corresponding to the target visual stimulation area.

In the embodiment of the present application, the interaction process between the VR device, the control device and the EEG device, and the specific operations performed by the VR device, the control device and the EEG device during the VR device control process based on the visual brain-computer interface refer to the previous embodiments. The relevant description is enough, and the embodiment of the present application will not repeat it here.

In the VR device control system based on the visual brain-computer interface provided in the embodiment of the present application, according to the user's panel display request, the input panel is displayed in the area between the VR device display area and the user, and the input panel includes at least one first visual stimulation area. , and each first visual stimulation region flashes according to a different visual stimulation coding sequence. The correlation between any two coding sequences of visual stimuli is less than the correlation threshold. When the user gazes at any first target visual stimulation area in the first visual stimulation area, the first target EEG signal will be generated. The VR device receives the first target indication signal generated by the control device according to the first target EEG signal, and judges the target operation to be performed according to the first target indication signal. The embodiment of the present application uses the space between the display area of the VR device and the user to display the input panel, and according to the EEG signal generated when the user looks at the first visual stimulation area in the input panel, the VR device executes the target corresponding to the visual stimulation area Therefore, the user can realize the control of the VR device in the VR space without the need of a VR handle, which can improve the convenience of the user's operation. The use of visual stimulus coding sequences with low mutual correlation can also improve the accuracy of differentiating EEG signals induced by different visual stimulus coding sequences.

Based on the same inventive concept, the embodiment of the present application also provides a VR device control device based on a visual brain-computer interface for implementing the above-mentioned method for controlling a VR device based on a visual brain-computer interface. The solution to the problem provided by the device is similar to the implementation described in the above method, so the specific limitations in one or more embodiments of the VR device control device based on the visual brain-computer interface provided below can be referred to above The limitation of the VR device control method based on the visual brain-computer interface will not be repeated here.

In one embodiment, as shown in FIG. 16 , a VR device control device based on a visual brain-computer interface is provided, including: a presentation module 1602, a receiving module 1604, and an execution module 1606, wherein:

The display module 1602 is configured to display the input panel in the target area in response to the panel display request, the input panel includes at least one first visual stimulation area, each of the first visual stimulation areas corresponds to a different operation, and each The first visual stimulation areas flash according to the visual stimulation coding sequence corresponding to each of the first visual stimulation areas, wherein the target area is the area between the VR device display area and the user, and for any two visual stimulation areas stimulus coding sequences, two of said visual stimulus coding sequences are less than a correlation threshold;

The receiving module 1604 is configured to receive a first target indication signal sent by the control device, the first target indication signal is an indication signal generated according to the first target EEG signal, and the first target indication signal is used to indicate the The first target visual stimulation coding sequence corresponding to the first target EEG signal, wherein the first target EEG signal is the EEG signal generated when the user gazes at the first target visual stimulation area, and the first target visual stimulation The stimulation area is any one of the at least one first visual stimulation area;

Execution module 1606, configured to determine the first target visual stimulation region corresponding to the first target visual stimulation coding sequence according to the first target visual stimulation coding sequence indicated by the first target indication signal, and execute the first target The target manipulation corresponding to the visual stimulus area.

The VR device control device based on the visual brain-computer interface provided by the embodiment of the present application displays the input panel in the area between the VR device display area and the user according to the user's panel display request, and the input panel includes at least one first visual stimulation area , and each first visual stimulation region flashes according to a different visual stimulation coding sequence. The correlation between any two coding sequences of visual stimuli is less than the correlation threshold. When the user gazes at any first target visual stimulation area in the first visual stimulation area, the first target EEG signal will be generated. The VR device receives the first target indication signal generated by the control device according to the first target EEG signal, and judges the target operation to be performed according to the first target indication signal. The embodiment of the present application uses the space between the display area of the VR device and the user to display the input panel, and according to the EEG signal generated when the user looks at the first visual stimulation area in the input panel, the VR device executes the target corresponding to the visual stimulation area Therefore, the user can realize the control of the VR device in the VR space without the need of a VR handle, which can improve the convenience of the user's operation. The use of visual stimulus coding sequences with low mutual correlation can also improve the accuracy of differentiating EEG signals induced by different visual stimulus coding sequences.

In one of the embodiments, the input panel includes a mouse input panel, the mouse input panel includes a mouse indicator block, each of the first visual stimulation areas is radially arranged around the mouse indicator block, and each of the first visual stimulation areas A visual stimulation area corresponds to different moving directions respectively.

In one of the embodiments, the executing module 1606 is further configured to:

moving the mouse to a specified position in the display area according to the moving direction corresponding to the first target visual stimulation area;

According to the movement direction corresponding to the first target visual stimulation area, move the mouse indicator block to a designated position in the mouse input panel;

The first visual stimulation area for each moving direction is redrawn with the mouse indicator block as the center.

In one of the embodiments, the executing module 1606 is further configured to:

Acquiring the number of consecutive movements corresponding to the movement direction;

Determine the length of movement of the mouse according to the number of continuous movements, and the length of movement is positively correlated with the number of continuous movements;

Moving the mouse to a designated position along the moving direction according to the moving length.

In one of the embodiments, the display module 1602 is also used to:

In response to a panel display request, an input panel control area is displayed in the display area, the input panel control area includes at least one second visual stimulation area, each of the second visual stimulation areas corresponds to a different input panel, and each The second visual stimulation areas flash according to the visual stimulation code sequences corresponding to each of the second visual stimulation areas;

In the case of receiving the second target indication signal sent by the control device, according to the second target visual stimulation coding sequence indicated by the second target indication signal, determine the second target visual stimulus corresponding to the second target visual stimulation coding sequence. stimulation area, and generate an input panel corresponding to the second target visual stimulation area in the target area, the second target indication signal is an indication signal generated according to the second target EEG signal, and the second target EEG signal is an EEG signal generated when the user gazes at a second target visual stimulation area, and the second target visual stimulation area is any one of the at least one second visual stimulation area.

In one embodiment, the input panel is any one of a vertical input panel or a parallel input panel, wherein the vertical input panel is perpendicular to the display area, and the parallel input panel is perpendicular to the display area. The display areas are parallel, and the color of the parallel input panel is translucent.

In one embodiment, as shown in FIG. 17 , a VR device control device based on a visual brain-computer interface is provided, including: a receiving module 1702, a determining module 1704, and a sending module 1706, wherein:

The receiving module 1702 is configured to receive the target EEG signal sent by the EEG device, the target EEG signal is generated when the user gazes at the target visual stimulation area, and the target visual stimulation area is at least one visual stimulation area in the input panel Any one of the above, the input panel is generated in the target area by the VR device in response to the panel display request, each of the visual stimulation areas corresponds to a different operation, and each of the visual stimulation areas is in accordance with each of the visual stimulation areas The corresponding visual stimulation coding sequence flashes, wherein the target area is the area between the VR device display area and the user, and for any two visual stimulation coding sequences, the correlation between the two visual stimulation coding sequences is less than Relevance Threshold;

Determining module 1704, configured to determine the target reference EEG signal matching the target EEG signal from the reference EEG signals corresponding to each visual stimulation coding sequence, and determine the target reference EEG signal according to the preset reference EEG signal and the visual stimulation coding sequence Corresponding relationship, determine the target visual stimulus code sequence corresponding to the target reference EEG signal, and generate the target indication signal corresponding to the target visual stimulus code sequence;

A sending module 1706, configured to send the target indication signal to the VR device, so that the VR device determines the target visual stimulus code sequence corresponding to the target visual stimulus code sequence according to the target visual stimulus code sequence indicated by the target indication signal. the target visual stimulation area, and execute the target operation corresponding to the target visual stimulation area.

The VR device control device based on the visual brain-computer interface provided by the embodiment of the present application displays the input panel in the area between the VR device display area and the user according to the user's panel display request, and the input panel includes at least one first visual stimulation area , and each first visual stimulation region flashes according to a different visual stimulation coding sequence. The correlation between any two coding sequences of visual stimuli is less than the correlation threshold. When the user gazes at any first target visual stimulation area in the first visual stimulation area, the first target EEG signal will be generated. The control device receives the target EEG signal, determines the target reference EEG signal corresponding to the target EEG signal, and determines the target visual stimulation code sequence corresponding to the target reference EEG signal, and then notifies the VR device of the target visual stimulus code through the first target indication signal sequence, so that the VR device can determine the target operation to be performed according to the target visual stimulation area corresponding to the target visual stimulation coding sequence. The embodiment of the present application uses the space between the display area of the VR device and the user to display the input panel, and according to the EEG signal generated when the user looks at the first visual stimulation area in the input panel, the VR device executes the target corresponding to the visual stimulation area Therefore, the user can realize the control of the VR device in the VR space without the need of a VR handle, which can improve the convenience of the user's operation. The use of visual stimulus coding sequences with low mutual correlation can also improve the accuracy of differentiating EEG signals induced by different visual stimulus coding sequences.

Each module in the above-mentioned VR equipment control device based on visual brain-computer interface can be fully or partially realized by software, hardware and combinations thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, and can also be stored in the memory of the computer device in the form of software, so that the processor can invoke and execute the corresponding operations of the above-mentioned modules.

In one embodiment, a computer device is provided. The computer device may be a server, and its internal structure may be as shown in FIG. 18 . The computer device includes a processor, memory and a network interface connected by a system bus. Wherein, the processor of the computer device is used to provide calculation and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The network interface of the computer device is used to communicate with an external terminal via a network connection. When the computer program is executed by a processor, a VR device control method based on a visual brain-computer interface is realized.

Those skilled in the art can understand that the structure shown in Figure 18 is only a block diagram of a partial structure related to the solution of this application, and does not constitute a limitation to the computer equipment on which the solution of this application is applied. The specific computer equipment can be More or fewer components than shown in the figures may be included, or some components may be combined, or have a different arrangement of components.

In one embodiment, a computer device is provided, including a memory and a processor, where a computer program is stored in the memory, and the processor implements the steps in the foregoing method embodiments when executing the computer program.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the steps in the foregoing method embodiments are implemented.

In one embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the steps in the foregoing method embodiments are implemented.

It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all Information and data authorized by the user or fully authorized by all parties.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above-mentioned embodiments can be completed by instructing related hardware through computer programs, and the computer programs can be stored in a non-volatile computer-readable memory In the medium, when the computer program is executed, it may include the processes of the embodiments of the above-mentioned methods. Wherein, any reference to storage, database or other media used in the various embodiments provided in the present application may include at least one of non-volatile and volatile storage. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive variable memory (ReRAM), magnetic variable memory (Magnetoresistive Random Access Memory, MRAM), Ferroelectric Random Access Memory (FRAM), Phase Change Memory (Phase Change Memory, PCM), graphene memory, etc. The volatile memory may include random access memory (Random Access Memory, RAM) or external cache memory. As an illustration and not a limitation, RAM can be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (Dynamic Random Access Memory, DRAM). The databases involved in the various embodiments provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a blockchain-based distributed database, etc., but is not limited thereto. The processors involved in the various embodiments provided by this application can be general-purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, data processing logic devices based on quantum computing, etc., and are not limited to this.

The technical features of the above embodiments can be combined arbitrarily. To make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be It is considered to be within the range described in this specification.

The above-mentioned embodiments only express several implementation modes of the present application, and the description thereof is relatively specific and detailed, but should not be construed as limiting the patent scope of the present application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be determined by the appended claims.

Claims (13)

1. A VR device control method based on a visual brain-computer interface, characterized in that the method comprises: In response to the panel display request, an input panel is displayed in the target area, the input panel includes at least one first visual stimulation area, each of the first visual stimulation areas corresponds to a different operation, and each of the first visual stimulation areas Blinking according to the visual stimulation coding sequence corresponding to each of the first visual stimulation areas, wherein the target area is the area between the VR device display area and the user, and for any two visual stimulation coding sequences, two The correlation of the visual stimulus coding sequence is less than a correlation threshold; receiving the first target indication signal sent by the control device, the first target indication signal is an indication signal generated according to the first target electroencephalogram signal, and the first target indication signal is used to indicate the first target electroencephalogram signal The corresponding first target visual stimulation coding sequence, wherein the first target EEG signal is the EEG signal generated when the user gazes at the first target visual stimulation area, and the first target visual stimulation area is the at least any one of the first visually stimulating regions; According to the first target visual stimulation coding sequence indicated by the first target indication signal, determine the first target visual stimulation area corresponding to the first target visual stimulation coding sequence, and execute the target corresponding to the first target visual stimulation area operate. 2. The method according to claim 1, wherein the input panel includes a mouse input panel, the mouse input panel includes a mouse pointer block, and each of the first visual stimulation areas is centered on the mouse pointer block Arranged radially, each of the first visual stimulation regions corresponds to a different moving direction. 3. The method according to claim 2, wherein the performing the target operation corresponding to the first target visual stimulation area comprises: moving the mouse to a specified position in the display area according to the moving direction corresponding to the first target visual stimulation area; According to the movement direction corresponding to the first target visual stimulation area, move the mouse indicator block to a designated position in the mouse input panel; The first visual stimulation area for each moving direction is redrawn with the mouse indicator block as the center. 4. The method according to claim 3, wherein the moving the mouse to a specified position in the display area according to the moving direction corresponding to the first target visual stimulation area comprises: Acquiring the number of consecutive movements corresponding to the movement direction; Determine the length of movement of the mouse according to the number of continuous movements, and the length of movement is positively correlated with the number of continuous movements; Moving the mouse to a designated position along the moving direction according to the moving length. 5. The method according to any one of claims 1 to 4, wherein the displaying the input panel in the target area in response to the panel display request comprises: In response to a panel display request, an input panel control area is displayed in the display area, the input panel control area includes at least one second visual stimulation area, each of the second visual stimulation areas corresponds to a different input panel, and each The second visual stimulation areas flash according to the visual stimulation code sequences corresponding to each of the second visual stimulation areas; In the case of receiving the second target indication signal sent by the control device, according to the second target visual stimulation coding sequence indicated by the second target indication signal, determine the second target visual stimulus corresponding to the second target visual stimulation coding sequence. stimulation area, and generate an input panel corresponding to the second target visual stimulation area in the target area, the second target indication signal is an indication signal generated according to the second target EEG signal, and the second target EEG signal is an EEG signal generated when the user gazes at a second target visual stimulation area, and the second target visual stimulation area is any one of the at least one second visual stimulation area. 6. The method according to claim 5, wherein the input panel is any one of a vertical input panel or a parallel input panel, wherein the vertical input panel and the display area are perpendicular to each other , the parallel input panel is parallel to the display area, and the color of the parallel input panel is translucent. 7. A VR device control method based on a visual brain-computer interface, characterized in that the method comprises: receiving the target EEG signal sent by the EEG device, the target EEG signal is generated when the user looks at the target visual stimulation area, the target visual stimulation area is any one of at least one visual stimulation area in the input panel, the The input panel is generated in the target area by the VR device in response to the panel display request, each of the visual stimulation areas corresponds to a different operation, and each of the visual stimulation areas is respectively in accordance with the visual stimulation code sequence corresponding to each of the visual stimulation areas performing flashing, wherein the target area is the area between the display area of the VR device and the user, and for any two visual stimulation coding sequences, the correlation of the two visual stimulation coding sequences is less than a correlation threshold; From the reference EEG signals corresponding to each visual stimulation coding sequence, determine the target reference EEG signal matching the target EEG signal, and determine according to the preset correspondence between the reference EEG signal and the visual stimulation coding sequence A target visual stimulation coding sequence corresponding to the target reference EEG signal, and generating a target indicating signal corresponding to the target visual stimulation coding sequence; sending the target indication signal to the VR device, so that the VR device determines the target visual stimulation area corresponding to the target visual stimulation coding sequence according to the target visual stimulation coding sequence indicated by the target indication signal, And execute the target operation corresponding to the target visual stimulation area. 8. A VR device control system based on a visual brain-computer interface, including a VR device, an EEG device, and a control device, wherein, The VR device is configured to, in response to a panel display request, display an input panel corresponding to the panel display request in a target area, the input panel includes at least one visual stimulation area, each of the visual stimulation areas corresponds to a different operation , and each of the visual stimulation areas flashes according to the visual stimulation code sequence corresponding to each of the visual stimulation areas, wherein the target area is the area between the VR device display area and the user, for any two of the visual stimulation areas stimulus coding sequences, two of said visual stimulus coding sequences are less than a correlation threshold; The EEG device is configured to detect a target EEG signal generated when the user gazes at a target visual stimulation area, and send the target EEG signal to the control device, the target visual stimulation area being the at least one any one of the first visually stimulating regions; The control device is configured to receive the target EEG signal sent by the EEG device, determine the target reference EEG signal corresponding to the target EEG signal, and according to the preset correspondence between the reference EEG signal and the visual stimulation coding sequence , determining the target visual stimulus coding sequence corresponding to the target reference EEG signal, and generating a target indication signal corresponding to the target visual stimulation coding sequence, and sending the target indication signal to the VR device; The VR device is further configured to receive a target indication signal sent by the control device, determine a target visual stimulation area corresponding to the target visual stimulation coding sequence according to the target visual stimulation coding sequence indicated by the target indication signal, and execute The target operation corresponding to the target visual stimulation area. 9. A VR device control device based on a visual brain-computer interface, characterized in that the device comprises: A display module, configured to display an input panel in a target area in response to a panel display request, the input panel includes at least one first visual stimulation area, each of the first visual stimulation areas corresponds to a different operation, and each of the The first visual stimulation areas flash respectively according to the visual stimulation coding sequence corresponding to each of the first visual stimulation areas, wherein the target area is the area between the display area of the VR device and the user, for any two visual stimulation coding sequences, the correlation of two said coding sequences of visual stimuli is less than a correlation threshold; The receiving module is configured to receive a first target indication signal sent by the control device, the first target indication signal is an indication signal generated according to the first target EEG signal, and the first target indication signal is used to indicate the first target indication signal A first target visual stimulation coding sequence corresponding to a target EEG signal, wherein the first target EEG signal is an EEG signal generated when the user gazes at the first target visual stimulation area, and the first target visual stimulation an area is any one of said at least one first visually stimulating area; An execution module, configured to determine the first target visual stimulation area corresponding to the first target visual stimulation coding sequence according to the first target visual stimulation coding sequence indicated by the first target indication signal, and execute the first target visual stimulation Stimulus regions correspond to target manipulations. 10. A VR device control device based on a visual brain-computer interface, characterized in that the device comprises: The receiving module is used to receive the target EEG signal sent by the EEG device, the target EEG signal is generated when the user looks at the target visual stimulation area, and the target visual stimulation area is at least one of the visual stimulation areas in the input panel In any one, the input panel is generated in the target area by the VR device in response to the panel display request, each of the visual stimulation areas corresponds to a different operation, and each of the visual stimulation areas corresponds to each of the visual stimulation areas. The visual stimulation coding sequence is flashed, wherein the target area is the area between the VR device display area and the user, and for any two visual stimulation coding sequences, the correlation between the two visual stimulation coding sequences is less than the correlation sexual threshold; The determination module is used to determine the target reference EEG signal matching the target EEG signal from the reference EEG signals corresponding to each visual stimulation coding sequence, and determine the target reference EEG signal according to the preset reference EEG signal and the visual stimulation coding sequence. Correspondence, determining the target visual stimulus coding sequence corresponding to the target reference EEG signal, and generating a target indication signal corresponding to the target visual stimulation coding sequence; A sending module, configured to send the target indication signal to the VR device, so that the VR device determines the target visual stimulus code sequence corresponding to the target visual stimulus code sequence according to the target visual stimulus code sequence indicated by the target indication signal target visual stimulation area, and execute the target operation corresponding to the target visual stimulation area. 11. A computer device, comprising a memory and a processor, the memory stores a computer program, wherein the processor implements the method according to any one of claims 1 to 7 when executing the computer program step. 12. A computer-readable storage medium, on which a computer program is stored, wherein when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are realized. 13. A computer program product, comprising a computer program, characterized in that, when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are realized.

Images