TW201316951A - Control method and system of brain computer interface with stepping delay flickering sequence - Google Patents

Abstract

A control method of brain computer interface with stepping delay flickering sequence. First, a plurality of different flickering sequences are generated by an encoding process according to a static flashing segment and a plurality of stepping delay flashing segments divided in different time sequences. Next, a plurality of visual targets corresponding to the flickering sequences are displayed. Subsequently, a response signal generated by an organism evoked by the visual targets is acquired. After that, a mathematic manner is utilized to perform a signal process to the response signal to distinguish which one of the visual targets is gazed by the organism. Next, a controlling command corresponding to one of the visual targets is generated. A control system of brain computer interface with stepping delay flickering sequence is disclosed herein.

Description

Brain-computer interface control method and system for step delay flicker sequence

The present invention relates to a method and system for controlling a brain-computer interface, and more particularly to a method and system for controlling a brain-computer interface of a step-delayed scintillation sequence.

With the continuous innovation and advancement of medical engineering, in recent years, the Brain Computer Interface (BCI) system has been widely used in the drive control interface of various specific devices. For example, the action commands of the wheelchair, the hospital bed, and the entertainment multimedia device can generate corresponding visual stimulation signals through the display device of the brain-computer interface system. Next, the brain-computer interface system acquires an electroencephalogram signal correspondingly generated when the user looks at the display device, and determines an action command corresponding to the brain wave signal to perform an action control on the device.

The traditional brain-computer interface technology includes the Steady-state Visual Evoked Potential (SSVEP) system, the Flash Visual Evoked Potential (FVEP) system, and the Phase tagged flickering sequence. A potential system and a Biphasic stimulation visual evoked potential system for generating visual stimuli through disparate visual scintillation sequence coding.

However, the steady-state visual evoked potential system can provide limited visual options. If multi-frequency stimulation is used to increase visual options, it will cause different brainwave response energy and messy flash pictures. The scintillation visual evoked potential system produces a messy picture of the flash sequence and is susceptible to interference from other physiological signals. Phase-calibrated flash sequence The visual evoked potential system must first correct the initial flash phase under different physiological responses of different users, and the response phase shift due to the user's visual fatigue during long-term use, so that the flash phase must be recalibrated. . The bi-phase stimuli visual evoked potential system must measure the reference phase of the visual stimuli signal and the brain wave signal, so that the redundant data is increased.

Therefore, the conventional techniques still have the above drawbacks and deficiencies to be solved.

One aspect of the present disclosure is a brain-computer interface control system that provides a step-delayed scintillation sequence, including a scintillation sequence generating unit, a display unit, a measuring unit, and a signal processing unit. The scintillation sequence generating unit is configured to generate a plurality of distinct scintillation sequence signals formed by encoding a plurality of stepped delay flash segments divided by the stable flash segment and the distinct timing. The display unit is electrically coupled to the blinking sequence generating unit, and includes a plurality of display regions for displaying a plurality of target images corresponding to the blinking sequence signals. The measuring unit connects the living body through the electrode to obtain a response signal generated by the biological object induced by the target image. The signal processing unit is electrically coupled to the measuring unit and the scintillation sequence generating unit, and performs signal processing on the response signal through a mathematical method to identify which one of the target images the biological system looks at to generate a corresponding control command.

In accordance with an embodiment of the present disclosure, the plurality of delay periods corresponding to the stepped delay flash segments are different.

In accordance with an embodiment of the present disclosure, wherein the stepped flash segment corresponds to a delayed flash frequency and the delayed flash frequency is adjustable.

In accordance with an embodiment of the present disclosure, the duration of the stable flash segment is adjustable.

In accordance with an embodiment of the present disclosure, the stable flash segment of each of the scintillation sequence signals has a plurality of flash periods, and the flash period corresponds to a stable flash frequency.

In accordance with an embodiment of the present disclosure, wherein the scintillation sequence signals have the same or different stable flash frequencies.

In accordance with an embodiment of the present disclosure, the light-dark interval ratio of the flash period is adjustable.

In accordance with an embodiment of the present disclosure, the scintillation sequence generating unit includes a programmable wafer. The programmable wafer is at least one of a single chip, a programmable logic gate array, and a microcontroller, and generates a scintillation sequence signal by a combination of software, hardware, or hardware and software.

In accordance with an embodiment of the present disclosure, the display brightness and display pattern of the display area are adjustable.

Another aspect of the present disclosure is a brain-computer interface control method that provides a step-delayed blinking sequence, comprising the following operational steps. First, a plurality of distinct scintillation sequence signals are generated, wherein the scintillation sequence signal is formed by encoding a plurality of stepped delay flash segments divided by a stable flash segment and a distinct timing. Subsequently, a plurality of target images corresponding to the blinking sequence signals are displayed. Next, the response signal generated by the organism induced by the target image is acquired. Thereafter, the response signal is signal processed by a mathematical method to identify which of the target images the biological system is looking at. Subsequently, a control command is generated for one of the corresponding target images.

According to an embodiment of the present disclosure, the brain-computer interface control method further includes: eliminating at least one of a response signal by a digital filtering method, a time averaging technique, an independent component analysis method, an empirical mode decomposition method, and a wavelet analysis method. Noise outside the frequency range to improve the signal-to-noise ratio of the response signal.

In accordance with an embodiment of the present disclosure, wherein signal processing the response signal comprises the following operational steps. First, a superimposed average of the response signals is performed to produce an average response signal. Subsequently, a plurality of normalized energies of the average response signal are calculated based on the timing division of the stable flash segment and the stepped delay flash segment. Then, according to the maximum normalized energy in the normalized energy, it is determined which one of the response signal and the scintillation sequence signal has the greatest correlation to identify the gaze target of the organism. Thereafter, it is determined whether the maximum normalized energy is greater than a preset threshold to determine whether a control command corresponding to the gaze target of the living body is generated.

In accordance with an embodiment of the present disclosure, the mathematical method includes at least one of a Fourier transform, a superimposed average comparison template energy intensity, a neural network, a support vector machine, and a hidden Markov model.

According to the technical content of the above technical aspect of the disclosure, the brain-computer interface control system can achieve the purpose of multi-channel target image display with less flash frequency, and has a stable and fast target recognition time to improve the brain engine. The ease of use of the interface.

The spirit and scope of the present disclosure will be apparent from the following description of the preferred embodiments of the present disclosure. Modifications do not depart from the spirit and scope of the disclosure.

1 is a schematic diagram of a brain-computer interface control system 100 in accordance with an embodiment of the present disclosure. The brain-computer interface control system 100 may include a blinking sequence generating unit 110, a display unit 120, a measuring unit 130, and a signal processing unit 140.

Also refer to Figure 2. 2 is a schematic diagram showing the encoding of a scintillation sequence signal in the brain-computer interface control system 100 as shown in FIG. 1. Flash sequence generating unit 110 may be used to produce a plurality of different flashing sequences of the signal, wherein the signal lines from stable flash sequence flash zone _F t (e.g.: 468.75 ms) and a plurality of different timing division of the stepper delay flash zone t _d (for example: 0 ms, 5.2 ms, 10.4 ms, 15.6 ms, 20.8 ms, 26 ms) is coded to form.

The display unit 120 is electrically coupled to the blinking sequence generating unit 110, and may include a plurality of display regions, for example, the display region 121 to the display region 126, for displaying a plurality of target images corresponding to the blinking sequence signals. In this embodiment, the light-emitting elements of the display area 121 to the display area 126 may be any one of a bulb, an LED, and an LCD, and are not limited thereto.

The measuring unit 130 can connect the living body 1000 through the electrodes 131 to 133 to obtain a response signal generated by the target 1000 induced by the target image. In the present embodiment, the measuring unit 130 adheres the electrode 131 (brain wave electrode) to the Oz vision area of the living body 1000 according to the international 10-20 system, and the electrode 132 (reference electrode) is adhered to the ear milk of the living body 1000. Mastoid, the electrode 133 (ground electrode) is adhered to the forehead of the living body 1000. In addition, the measuring unit 130 may include a biomedical signal amplifier and an analog digital converter (not shown) to perform signal amplification on the analog brain wave signal obtained by the measuring unit 130 and convert the signal into a corresponding digital signal.

The signal processing unit 140 is electrically coupled to the measuring unit 130 and the flicker sequence generating unit 110, and can perform signal processing on the response signal by mathematical methods to identify which one of the target images of the living body 1000 is to be generated. The control command is given to the control unit 150 so that the control unit 150 can manipulate the peripheral device or the communication device.

As shown in Figure 2. Embodiment, a delay in one step embodiment of the present disclosure flash zone t _d corresponding to the plurality of different periods of delay lines. For example, the step-delay flash segments t _{d of the} first to sixth blinking sequence signals can be configured to be 0 ms, 5.2 ms, 10.4 ms, 15.6 ms, 20.8 ms, 26 ms to form six asynchronous flashes with asynchronous in-phase delay. The scintillation sequence signal of segment t _d .

Embodiment, the stepping flash zone t _d delay corresponding to the delay based flash frequency, and flash frequency delay lines be adjustable in one embodiment of the present disclosure. For example, the delayed flash frequency corresponding to the step-delay flash segment t _{d of} the first to sixth blinking sequence signals can be adjusted to any one of 0 Hz to 200 Hz. That is, the stepping flash delay t _d may comprise segments lit state, the stepping flash signal is normally a dark state or a specific frequency.

In one embodiment of the present disclosure, the stabilizing section flash duration t _f the system are adjustable. For example, the duration of the stable flash segment t _f can be adjusted to 468.75 ms, and is not limited thereto.

In one embodiment of the present disclosure, each of the flash zone flash sequence stabilization signals t _f has a plurality of flash period, corresponding to the periodic system and flash flash frequency stability. For example, the first to sixth sections scintillator stable flashing signal sequence of t _f flash 15 may have a light-dark cycle of the flash. If the duration of the stable flash section t _f is 468.75 ms, the stable flash frequency corresponding to each flash period is 32 Hz.

In one embodiment of the present disclosure, the scintillation sequence signals may have the same or different stable flash frequencies. For example, the first to sixth scintillation sequence signals may have a stable flash frequency of 32 Hz, and the seventh to twelfth scintillation sequence signals (not shown) may have a stable flash frequency of 35 Hz. It should be noted that the number of the above-mentioned scintillation sequence signals and the corresponding stable flash frequency are determined according to actual operation requirements, and are not limited thereto.

In one embodiment of the present disclosure, the light-dark interval ratio of the flash period is adjustable. For example, a flash cycle corresponding to a stable flash frequency of 31.25 Hz is taken as an illustration. In this embodiment, the flash period is 32 ms, and the bright and dark intervals can be adjusted to 16 ms and 16 ms, respectively, so that the flash period has a duty cycle of 50%-50%. In addition, the bright and dark intervals can be adjusted to 24 ms and 8 ms, respectively, so that this flash cycle has a duty cycle of 75%-25%.

Figure 3 is a diagram showing the encoding of a plurality of scintillation sequence signals in the brain-computer interface control system as shown in Figure 2. For example, when six control actions are required to operate the peripheral device and the communication device, the blink sequence generation unit 110 may generate first to sixth blink sequence signals corresponding to the control actions. The first to sixth blinking signal sequence contains a duration of 468.75 ms flash stabilizing segment t _f, and t _f each segment having stable flash 15 flashes of the flash light and dark cycle.

In the present embodiment, the signal sequence may comprise the first scintillator 0 ms delay of the stepping flash zone t _d. That is, the first sequence of flashes the timing signal based on a stable flashing zone after next and then a steady t _f t _f of the flashing segment encoding mode. The second scintillator may comprise 5.2 ms sequence signal delay of the stepping flash zone t _d. That is, the second scintillator signal sequence based on the timing of the flash zone in order to stabilize and then 5.2 ms after t _f of the stepping flash segment delay t _d, then followed by the next stable manner flash segment encoding the t _f. Similarly, the third to sixth flicker sequence signals may respectively include a step delay flash segment t _{d of} 10.4 ms, 15.6 ms, 20.8 ms, and 26 ms. Then, according to the timing division manner of the above-mentioned stable flash segment t _f and the plurality of step-delay flash segments t _d to generate six flicker sequence signals having asynchronous asynchronous delay segments t _d , and then respectively driven The display area 121 to the display area 126 of the display unit 120 are used to generate corresponding target images.

In one embodiment of the present disclosure, the blink sequence generation unit 110 includes a programmable wafer. The programmable wafer can be at least one of a single chip, a field programmable gate array (FPGA), and a microcontroller, and generates a scintillation sequence signal by a combination of software, hardware, or hardware and software.

In an embodiment of the present disclosure, the display brightness of the display area 121 to the display area 126 and the display pattern are adjustable. That is, the display brightness can be adjusted according to the visual comfort, and the display pattern can be any one of a geometric pattern, a static pattern, and a dynamic pattern, and is not limited thereto.

4 is a flow chart showing a method of controlling a brain-computer interface of a step-delayed blinking sequence according to an embodiment of the present disclosure. The above control method can be applied to the brain-computer interface control system 100 of the step-delay scintillation sequence as shown in FIG. 1, and can include the following steps. First, in step 410, generating a plurality of different sequences of blinking signal, wherein the signal sequence-based scintillation step _f t and a plurality of different division of the timing delay from stable flash zone flash zone is formed by encoding _D t. It should be noted that the scintillation sequence signal in this embodiment is the same as or similar to the coding mode shown in FIG. 2 and FIG. 3, and details are not described herein again. Subsequently, in step 420, a plurality of target images corresponding to the blinking sequence signals are displayed. Next, in step 430, the response signal generated by the target 1000 induced by the target image is acquired. In addition, the brain-computer interface control method includes digital filtering method, Temporal ensemble averaging, Independent component analysis, Empirical mode decomposition, and Wavelet analysis. At least one of the noise signals other than the preset frequency range (eg, 29 Hz to 35 Hz) in the response signal is cancelled to improve the signal to noise ratio of the response signal, as shown in step 440.

Thereafter, in step 450, the response signal can be signal processed mathematically to identify which of the biological 1000 gaze target images. In an embodiment. The above mathematical method may include at least one of a Fourier transform, a superimposed average comparison template energy intensity, a neural network, a support vector machine, and a Hidden Markov model. Provide the algorithms needed for signal processing.

Subsequently, in step 460, a control command of one of the corresponding target images is generated to control the peripheral device to perform a corresponding action with the communication device.

In step 450, the signal processing of the response signal may include a number of operational steps. Here, an embodiment will be described. Please refer to FIGS. 5 to 6B. FIG. 5 is a flow chart showing a method for controlling a brain-computer interface of a step-delayed blinking sequence according to another embodiment of the present disclosure. Fig. 6A is a timing chart showing the response signal of the brain-computer interface control method as shown in Fig. 5. FIG. 6B is a timing diagram showing the calculation of the response signal according to the distinct cutting period in the brain-computer interface control method shown in FIG. 5. The above control method can be applied to the brain-computer interface control system 100 of the step-delay scintillation sequence as shown in FIG. 1, and can include the following steps. It should be noted that steps 510 to 540 are the same as or similar to steps 410 to 440 shown in FIG. 4, and details are not described herein again.

In step 550, can be based on a stable flashing section t _f t and the stepping timing delay flash zone dividing the average response _{d is} superimposed signals, to produce an average response signal. Subsequently, in step 560, a plurality of normalized powers of the average response signal are calculated.

For example, each scintillation sequence signal can be represented as a periodic sequence pattern of s _i (t)=s _i (t+T _i ), where T _i =t _f +(i-1)t _b , t _f is stable The duration of the flash segment (eg 468.75 ms), t _b is the basic delay time, and i is the number of target images.

In this embodiment, the flicker sequence signal can be converted using the Fourier transform method, and a mathematical pattern as shown in the following formula is obtained:

S _i ( jω )= e ^jωTi S _i ( jω )

In this embodiment, the scintillation sequence signal s _i (t) can be cut from 0 ms to 468.75 ms using different trigger events. For example, the scintillation sequence signal is subjected to a cutting process with a cutting period of T _k such that it becomes a segment s _i (t), s _i (t+T _k ), ..., s _i (t+5T _k ), and is calculated Average response signal of the scintillation sequence signal ( t ) is:

Then, using the Fourier transform method, the above average response signal is converted into The type of ( jω ) and can be expressed as follows:

When the kth cutting period is the same as the i-th gaze target (k=i), the average response signal is obtained as follows:

When the kth cutting period is different from the i-th gaze target (k≠i), the average response signal is obtained as follows:

In the present embodiment, since the flash frequency f ₀ can be set to 32 Hz, by setting the frequency ω = ω ₀ , where ω = 2π f , t _b =1/6 f , the kth cutting period can be Average response signal when it is different from the i-th gaze target ( jω ) is expressed by the following formula:

It can be seen that the average response signal when the kth cutting period is different from the i-th gaze target is 0.

In an embodiment of the present disclosure, the response signal may be composed of six scintillation sequence signals that are mathematically different weights, and the brain wave signal of the organism 1000 is added to the uncorrelated noise in the SSVEP system, and the response signal may be represented. For the following formula:

Where s _i (t) represents the brainwave response signal of the corresponding scintillation sequence signal, t ₀ is the SSVEP reaction delay time, and a _i is the weight of the i-th target image corresponding to the response signal. In this embodiment, the response signal x(t) can be cut into a plurality of epochs by different cutting periods T _k (k=1, . . . , M). Therefore, the above period can be expressed as E _k (j), where E _k (j) is the j-th period segment in which the response signal x(t) is cut with the trigger event of the kth flicker sequence signal.

Then, using the above-mentioned superposition averaging technique, And calculate the normalized energy .

When the living body 1000 looks at the display area 123, since the display area 123 is driven by the third blinking sequence signal, the response signal of the living body 1000 acquired by the measuring unit 130 can be expressed as shown in FIG. 6A. Schematic diagram of timing. In the signal processing program, the response signal may be cut according to the timing division manner of the first to sixth flicker sequence signals to obtain a timing diagram as shown in FIG. 6B. Therefore, the normalized energy obtained by cutting the response signal by the cutting period T ₁ corresponding to the first scintillation sequence signal is P ₁ =0.1725. Similarly, the normalized energies obtained by cutting the response signals by the cutting periods T ₃ to T ₆ corresponding to the second to sixth scintillation sequence signals are 0.0837, 0.4136, 0.1283, 0.1206, and 0.0813, respectively.

Since the response signal is synchronized with the gazing scintillation sequence signal, the target that the organism 1000 is looking at can be obtained by the maximum normalized power (MNP). That is, searching for the largest of the six normalized energies, and obtaining the maximum normalized energy is P ₃ =0.4136, as shown in step 570, and determining the response signal and the third scintillation sequence signal according to the maximum normalized energy. Maximum relevance. Therefore, the target that the identifiable organism 1000 is looking at is the display area 123.

Thereafter, in step 580, it is determined whether the maximum normalized energy is greater than a preset threshold to identify whether an effective brain wave signal is acquired, and whether a control command corresponding to the fixation target of the living body 1000 is generated.

Compared with the conventional method, in the above embodiment of the present disclosure, the step-delay flash coded scintillation sequence signal can be used for visual stimulation, and the time-averaging method is used to perform mathematical operations to identify the user's gaze target. In this way, it is possible to achieve a multi-channel target image display with less flash frequency, a display screen with no flashing sequence, no visual discomfort during use, and is less susceptible to interference from other physiological signals, and A more stable and fast target recognition time to increase the ease of use of the brain-computer interface.

The steps mentioned in the present disclosure may be adjusted according to actual needs, except for the order in which they are specifically described, and may be performed simultaneously or partially simultaneously, without being limited to the above.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention, and the present invention can be modified and modified without departing from the spirit and scope of the present invention. The scope is subject to the definition of the scope of the patent application attached.

100. . . Brain-computer interface control system for step-delay flashing sequence

110. . . Scintillation sequence generating unit

120. . . Display unit

121～126. . . Display area

130. . . Measuring unit

131～133. . . electrode

140. . . Signal processing unit

150. . . control unit

1000. . . organism

410～460. . . Steps

510 ~ 590. . . Steps

The above and other objects, features, advantages and embodiments of the present invention will become more apparent and understood.

1 is a schematic diagram of a brain-computer interface control system of a step-delayed blinking sequence in accordance with an embodiment of the present disclosure.

Fig. 2 is a diagram showing the coding of a scintillation sequence signal in the brain-computer interface control system as shown in Fig. 1.

Figure 3 is a diagram showing the encoding of a plurality of scintillation sequence signals in the brain-computer interface control system as shown in Figure 2.

4 is a flow chart showing a method of controlling a brain-computer interface of a step-delayed blinking sequence according to an embodiment of the present disclosure.

FIG. 5 is a flow chart showing a method for controlling a brain-computer interface of a step-delayed blinking sequence according to another embodiment of the present disclosure.

Fig. 6A is a timing chart showing the response signal of the brain-computer interface control method as shown in Fig. 5.

FIG. 6B is a timing diagram showing the calculation of the response signal according to the distinct cutting period in the brain-computer interface control method shown in FIG. 5.

510 ~ 590. . . Steps

Claims (13)

A brain-computer interface control system for step-delayed scintillation sequences, comprising a scintillation sequence generating unit for generating a plurality of different scintillation sequence signals, wherein the scintillation sequence signals are divided by a stable flash segment and a distinct timing a plurality of step-delay flash segments are encoded and formed; a display unit electrically coupled to the flicker sequence generating unit, the display unit includes a plurality of display regions for displaying a plurality of target images corresponding to the flicker sequence signals a measuring unit, connected to the living body through the electrode for acquiring a response signal generated by the target image induced by the target image; and a signal processing unit electrically coupled to the measuring unit and the scintillation sequence generated And modulating the response signal by a mathematical method to identify which of the target images the biological system looks at to generate a corresponding one of the control commands. The brain-computer interface control system of claim 1, wherein the plurality of delay periods corresponding to the step-delay flash segments are different. The brain-computer interface control system of claim 1, wherein the step-delay flash segments correspond to a delayed flash frequency, and the delayed flash frequency is adjustable. The brain-computer interface control system of claim 1, wherein the duration of the stable flash segment is adjustable. The brain-computer interface control system of claim 1, wherein the stable flash segment of each of the plurality of scintillation sequence signals has a plurality of flash periods, and the flash periods correspond to a stable flash frequency. The brain-computer interface control system of claim 5, wherein the scintillation sequence signals have the same or different such stable flash frequencies. The brain-computer interface control system of claim 5, wherein the light-dark interval ratio of the flash periods is adjustable. The brain-computer interface control system of claim 1, wherein the blinking sequence generating unit comprises a programmable chip, the programmable chip being at least one of a single chip, a programmable logic gate array, and a microcontroller. The scintillation sequence signals are generated by a combination of software, hardware or hardware and software. The brain-computer interface control system of claim 1, wherein display brightness and display patterns of the display areas are adjustable. A brain-computer interface control method for step-delayed scintillation sequence, comprising: generating a plurality of different scintillation sequence signals, wherein the scintillation sequence signals are a plurality of step-delay flash regions divided by a stable flash segment and a distinct timing Segments are encoded and formed; displaying a plurality of target images corresponding to the plurality of scintillation sequence signals; acquiring a response signal generated by the organisms induced by the target images; and performing signal processing on the response signals by a mathematical method to identify the The biological system looks at which of the target images; and generates a control command corresponding to one of the target images. The brain-computer interface control method according to claim 10, further comprising: eliminating at least one of the response signals by at least one of a digital filtering method, a time averaging technique, an independent component analysis method, an empirical mode decomposition method, and a wavelet analysis method; Noise outside the frequency range is set to increase the signal to noise ratio of the response signal. The brain-computer interface control method of claim 10, wherein the step of signal processing the response signal comprises: performing a superimposed average of the response signals to generate an average response signal; and according to the stable flash segment and the steps Calculating a plurality of normalized energies of the average response signal by timing division of the delayed flash segment; determining which of the response signals and the flicker sequence signals has the largest value according to one of the normalized energies Correlation, to identify the gaze target of the living body; and determine whether the maximum normalized energy is greater than a predetermined threshold to determine whether to generate the control command corresponding to the gaze target of the living body. The brain-computer interface control method of claim 10, wherein the mathematical method comprises at least one of a Fourier transform, a superimposed average comparison template energy intensity, a neural network, a support vector machine, and a hidden Markov model.