TWI879573B - Method and electronic device for dynamically adjusting cursor display - Google Patents

Abstract

A method and an electronic device for dynamically adjusting cursor display are provided. The method for dynamically adjusting cursor display is adapted to an electronic device coupled to a display and an input device. The method for dynamically adjusting cursor display includes: collecting movement traces of a cursor displayed on the display; filtering moving trajectories whose displacement is greater than a threshold, and taking the moving trajectories as cursor trajectory data; determining whether the cursor trajectory data includes a search cursor trajectory data; in response to the cursor trajectory data including the search cursor trajectory data, adjusting a display setting of the cursor according to display information of the electronic device.

Description

Method and electronic device for dynamically adjusting cursor display

The present disclosure relates to a method and an electronic device for automatically adjusting a cursor display based on display information and a cursor track, and more particularly to a method and an electronic device for dynamically adjusting a cursor display.

When using a computer, users often have trouble finding the cursor position, especially when using a smaller screen with a high display resolution. The existing cursor settings must be adjusted manually by the user and cannot immediately let the user know the cursor position and adjust it according to the browsing content. In addition, when the color of the desktop is similar to the color of the cursor, the user will also be unable to find the cursor position. Therefore, how to automatically detect whether the user is looking for the cursor and automatically adjust the cursor display settings according to the desktop information and window information has become a direction that people in this field are eager to develop.

In view of this, the present disclosure proposes a method and an electronic device for dynamically adjusting cursor display, which can solve the above technical problems.

The disclosed embodiment provides a method for dynamically adjusting cursor display, which is applicable to an electronic device coupled to a display and an input device, and the method includes the following steps. Collect the movement trajectory of the cursor displayed on the display. Filter the movement trajectory with a displacement greater than a threshold from the movement trajectory, and use the movement trajectory as cursor trajectory data. Determine whether the cursor trajectory data includes searching for cursor trajectory data. In response to the cursor trajectory data including searching for cursor trajectory data, adjust the display setting of the cursor accordingly according to the display information of the electronic device.

The disclosed embodiment provides an electronic device, which includes a memory and a processor. The processor is coupled to a display and an input device, and executes multiple modules stored in the memory. The cursor signal collection module collects the movement trajectory of the cursor displayed on the display. The trajectory monitoring module filters out the movement trajectory with a displacement greater than a threshold from the movement trajectory, and uses the movement trajectory as the cursor trajectory data. The trajectory judgment module judges whether the cursor trajectory data includes the search cursor trajectory data. Responding to the cursor track data includes searching for the cursor track data, and the cursor display adjustment module adjusts the display setting of the cursor according to the display information of the electronic device, wherein the display information is related to the desktop color, the window color, the window information, the window size, the desktop size and the window control information.

Based on the above, the method and electronic device for dynamically adjusting the cursor display of the disclosed embodiment can determine whether the user is looking for the cursor position on the display by analyzing the movement track of the cursor. And, based on the desktop information and the window information, the cursor setting is adjusted to a suitable size and/or a suitable color. In this way, the method and electronic device for dynamically adjusting the cursor display can automatically determine that the current movement track of the cursor is to find the cursor position, and then adjust the cursor display setting so that the user can quickly find the cursor position to improve the convenience of use.

In order to make the above features and advantages of the present disclosure more clearly understood, embodiments are specifically cited below and described in detail with reference to the accompanying drawings.

Some embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings. When the same element symbols appear in different drawings, they will be regarded as the same or similar elements. These embodiments are only part of the present disclosure and do not disclose all possible implementations of the present disclosure. More precisely, these embodiments are only examples of methods and devices within the scope of the patent application of the present disclosure.

FIG. 1 is a block diagram of an electronic device according to an embodiment of the present disclosure. The electronic device 100 may include an input device and an electronic device, wherein the electronic device may include a display 130, a processor 110, and a memory 120. In one embodiment, the electronic device 100 is a computer device having a basic input and output system (BIOS), such as a laptop or a desktop computer. In one embodiment, the processor 110 of the electronic device is coupled to the display 130 and the memory 120, and the electronic device is coupled to the input device. In another embodiment, the electronic device includes the processor 110 and the memory 120, and the electronic device is coupled to the input device and the display 130.

The memory 120 of the electronic device 100 may be a system memory, such as any type of volatile random access memory (RAM). The input device is used to receive user commands/instructions input by the user. The input device 130 may be a keyboard, a mouse, or a touch device, etc., and the present disclosure is not limited thereto. In one embodiment, the electronic device 100 is a computer system including a processor 110, a memory 120, and a display 130.

The processor 110 is, for example, a central processing unit (CPU), an application processor (AP), or other programmable general-purpose or special-purpose microprocessor, digital signal processor (DSP) or other similar devices, integrated circuits and combinations thereof. The processor 110 can access and execute program codes, codes or instructions recorded in the basic input and output system device 110 and the system memory to implement the method of dynamically adjusting the cursor display in the embodiment of the present disclosure.

The display 130 is used to display information, setting interface and operation interface. In different embodiments, the display 130 can be a liquid crystal display (LCD), a light emitting diode (LED) display, an organic light emitting diode (OLED) display, etc., and the present disclosure is not limited thereto.

FIG. 2 is a flow chart of a method for dynamically adjusting the cursor display according to an embodiment of the present disclosure, and the method flow of FIG. 2 can be implemented by the electronic device 100 of FIG. 1 . Please refer to FIG. 1 to FIG. 2 , and the steps of the method for dynamically adjusting the cursor display of the present embodiment are described below in conjunction with the various components of the electronic device 100 in FIG. 1 . In the present embodiment, the memory 120 stores a plurality of modules, and the processor 110 further executes the plurality of modules to perform various operations. As shown in FIG. 1 , the plurality of modules include a cursor signal collection module 121, a track monitoring module 122, a track judgment module 123, a cursor display adjustment module 124, a desktop detection module 125, and a window information collection module 126.

In step S210, the processor 110 executes the cursor signal collection module 121 to collect the movement track of the cursor displayed on the display 130. For example, the user controls the input device (such as a mouse) that is communicatively connected or electrically connected to the electronic device 100 to move the cursor displayed on the display 130. In this way, the cursor signal collection module 121 collects the movement track of the cursor displayed on the screen of the display 130. In one embodiment, the cursor signal collection module 121 collects at least one cursor event of the cursor. The cursor event is at least one of a movement event, a key event, a scroll wheel event, and an out-of-window event, and the cursor signal collection module 121 records the position coordinates of the cursor event to generate a movement track.

In step S220, the processor 110 executes the trajectory monitoring module 122, and further selects the moving trajectory whose displacement is greater than a threshold from the moving trajectory. The threshold corresponding to the displacement is between 1 inch and 10 inches. In another embodiment, the threshold corresponding to the displacement is between 5 pixels and 1000 pixels. For example, the threshold corresponding to the displacement is 2 inches or 200 pixels, but the present case should not be limited to this. In this way, the trajectory monitoring module 122 can select the moving trajectory whose displacement per time unit (e.g., 2 seconds, 5 seconds, etc.) is greater than the threshold.

In one embodiment, the displacement of the cursor movement trajectory can be calculated by the following formula (1): V _n = ((X _n - X _n-1 )^2 + (Y _n - Y _n-1 )^2)^0.5 ……(1)

_Vn : cursor displacement, _Xn : mouse X-axis position at sampling point n, _Yn : mouse Y-axis position at sampling point n, Xn _-1 : mouse X-axis position at time point n-1 before sampling point n, Yn _-1 : mouse Y-axis position at time point n-1 before sampling point n.

In step S230, the track monitoring module 122 uses the moving track with a displacement greater than the threshold as the cursor track data, so that the subsequent processor 110 can further calculate and judge. In step S240, the processor 110 executes the track judgment module 123 to judge whether the cursor track data includes the search cursor track data. In one embodiment, the track judgment module 123 compares the cursor track data obtained in step S230 with the sample data of the search cursor track stored in the memory 120, and further judges whether the cursor track data includes the search cursor track data. In other words, the track determination module 123 determines whether the current moving track is the data of the cursor track by comparing the pre-stored sample of the cursor track and the current moving track of the cursor.

In step S250, the processor 110 executes the cursor display adjustment module 124 to respond to the cursor track data, including searching for the cursor track data, and correspondingly adjusting the display settings of the cursor according to the display information of the electronic device 100. It is worth noting that the display information is related to at least one of the desktop color, window color, window information, window size, desktop size, and window control information. In one embodiment, the cursor display adjustment module 124 calculates the cursor suitable size according to the screen resolution default value, display width, preset cursor size upper limit value, display height, and control weight. The control weight is related to the preset weight value of the window control information corresponding to the current position of the cursor. In addition, the cursor suitable size is not greater than the cursor size upper limit value. In this way, the cursor display adjustment module 124 adjusts the size and/or color of the cursor according to the cursor suitable size. In one embodiment, the cursor display adjustment module 124 can calculate the cursor suitable size through the following formulas (2) to (4): R = (W _a * (W _st * H _st ) / _As + W _d ) / (W _a + W _d ) if the mouse is on a clickable control ……(2) R = (W _st * H _st ) / _As if the mouse is not on a clickable control ……(3) _Sp = Min (S _pth , R * S) ……(4)

_Sp : cursor fit size, _Wa : screen resolution weight, _Wd : clickable control weight, R: adjustment ratio value, _Wst : desktop width at sampling point t (i.e., width of display panel of display 130), _Hst : desktop height at sampling point t (i.e., height of display panel of display 130), _As : general screen resolution size default value (e.g., 1920x1080), _Spth : maximum value of mouse pointer size (i.e., upper limit value), S: default mouse pointer size. The weights mentioned in this case are preset numbers, such as positive numbers between 0 and 5. For example, _Wa is 0.3 and _Wd is 0.5, but this case should not be limited to this. The clickable control item is a clickable control item where the cursor is located when capturing the image, such as a button, an OK key, or a Save key.

With such a configuration, the method for dynamically adjusting the cursor display and the electronic device 100 can determine whether the user is looking for the cursor on the display 130 by analyzing the movement trajectory. When the electronic device 100 determines that the user is looking for the cursor, the processor 110 executes the cursor display adjustment module 124 to adjust the size and/or color of the cursor so that the user can quickly find the cursor position. In another embodiment, the display setting includes display effects. In this way, when it is determined that the user is looking for the cursor, the cursor display adjustment module 124 activates a preset cursor effect (such as a guide line, a sound effect, or a magnifying glass effect, etc.).

FIG3 is a flow chart of capturing a display screen to calculate a cursor color according to an embodiment of the present disclosure. FIG4 is a flow chart of multiple modules according to an embodiment of the present disclosure. As shown in FIG3, the processor 110 is further used to execute steps S310 to S350. In step S310, the processor 110 executes the desktop detection module 125 to detect whether the display 130 has a display switching event. The display switching event is one of a window switching event, a new window opening event, and a large-scale desktop change event. The window switching event is the current window being switched to another window. The new window opening event is the user opening a new window on the display 130 through an input device. A large-scale desktop change event refers to a desktop change that exceeds a certain proportion (such as 10%, 20%, or 30% of the entire desktop), such as a user dragging a window.

In one embodiment, the ratio value is related to the window width, window height and window display height of the event window, the window display width and the desktop height, the desktop width and the position coordinates of the window. In one embodiment, the ratio of the window to the desktop area can be calculated by the following formulas (5) to (7): W _kmt = Min(W _kt , W _st - X _kt ) if X _kt ＞= 0 W _kmt = Min(W _kt , W _st + X _kt ) if X _kt ＜ 0……(5) H _kmt = Min(H _kt , H _st - Y _kt ) if Y _kt ＞= 0 H _kmt = Min(H _kt , H _st + Y _kt ) if Y _kt ＜ 0……(6) A _kt = W _kmt * H _kmt ……(7)

A _kt : The ratio of the kth window to the desktop area at the tth sampling point, k and t are positive integers, W _kmt : The visible window width of window k at sampling point t, H _kmt : The visible window height of window k at sampling point t, W _kt : The window width of window k at sampling point t, H _kt : The window height of window k at sampling point t, W _st : The desktop width at sampling point t, H _st : The desktop height at sampling point t, X _kt : The X-axis coordinate of window k at sampling point t, Y _kt : The Y-axis coordinate of window k at sampling point t.

In step S320, in response to detecting a display switching event and the ratio value being greater than a threshold, the desktop detection module 125 captures the current display image. In other words, in response to detecting a display switching event, i.e., the desktop detection module 125 detects that the display 130 has a display switching event and the desktop area changes by more than a ratio value, the display 130 captures the current display image (i.e., the current desktop image).

In step S330, the desktop detection module 125 calculates the average color value of each pixel in the current display screen as the desktop reference color. For example, the desktop detection module 125 reads the buffer data in the memory 120 to obtain the average value of the color of each pixel or each area unit (e.g., 9 pixels, each square inch, etc.) of the current display screen of the display 130.

In step S340, the processor 110 executes the window information collection module 126 to calculate the window average color value according to the average color value and the window image corresponding to the current cursor position, and then the window information collection module 126 uses the window average color value as the window reference color.

In step S350, the cursor display adjustment module 124 performs a color complement operation on the desktop reference color and the window reference color to obtain the cursor color, and then adjusts the display setting of the cursor according to the cursor color. _Cp = Complement(( _Wd * _Cd + _Ww * _Cw + _Wc * _Cxyw ) / ( _Wd + _Ww + _Wc )) ... (8)

C _p : cursor color, W _d : weight of desktop reference color value, W _w : weight of window reference color value, W _c : weight of the color value at the current cursor position, Complement() : complement function, C _xyd : color of pixel (x, y), C _w : window reference color, C _d : desktop reference color.

As shown in FIG4 , whenever the desktop detection module 125 detects a display switching event, and the desktop switching ratio value of the display switching event is greater than a threshold value (e.g., 20 square inches, 16 square inches, or 64 square inches), the window information collection module 126 inputs the desktop reference color and the window reference color value to the cursor display adjustment module 124. In this way, when the track determination module 123 determines that the cursor has found a cursor track, the cursor display adjustment module 124 can directly calculate the cursor color according to the desktop reference color and the window reference color, and then adjust the display settings (e.g., color) of the cursor in real time. In other words, by setting the display switching event detection through the desktop detection module 125, it is possible to avoid the problem that the processor 110 calculates the cursor suitable color (i.e., cursor color) and the cursor suitable size when the user searches for the cursor, thereby causing excessive computing load and low computing efficiency in a short period of time. In other words, by respectively detecting/monitoring whether there is a display switching event and searching for the cursor track through the desktop detection module 125 and the track monitoring module 122, the computing load is dispersed to improve the efficiency of the processor 110 and the smooth operation of the overall electronic device 100.

FIG5 is a flow chart of obtaining a weighted window color value according to window information according to an embodiment of the present disclosure. Referring to FIG5, the processor 110 is further used to execute steps S410 to S430. In step S410, a weighted color value is obtained according to the average color value of each pixel and the desktop weighting coefficient. In one embodiment, the processor 110 obtains the color value (Rxy, Gxy, Bxy) of each pixel of the desktop, and calculates the weighted color value of the desktop through the following formula (9) and formula (10): ……(9) ……(10)

C _d : desktop weighted average color value, C _xy : original color value of the desktop pixel (x, y), x, y are the coordinates of each point on the display 130, R _xy , G _xy or B _xy , W _st : desktop width at sampling point t, H _st : desktop height at sampling point t, Z: desktop weighted adjustment coefficient. The larger the Z value, the larger the area classified as the edge part, usually Z = < 1, Sigmoid(): S function (sigmoid function). C _xyd : desktop weighted color value of pixel (x, y). The weighted color value is related to the average color value through the weighting coefficient to increase the color value of the central area of the desktop. In this way, it can be set according to the user's high probability that the cursor operation will be concentrated in the center of the desktop rather than the edge of the desktop. Therefore, the influence weight of some edge pixels is reduced, thereby improving the visibility of the cursor.

In step S420, the processor 110 executes the window information collection module 126 to obtain a weighted window color value according to the window average color value and the window title, window program and window control information of the window screen. The window information includes the window title and the window program. The weighted window color value is related to the color value obtained according to the window control information and the corresponding control weighting coefficient. The processor 110 can calculate the weighted window color value through the following formulas (11) to (13), where formula (11) is used to calculate the weighted window color value of the window without a window sub-control item at the measured coordinate (x, y), formula (12) is used to calculate the weighted window color value of the sub-control item of the window that does not support clickable function at the measured coordinate (x, y), and formula (13) is used to calculate the weighted window color value of the sub-control item of the window that supports clickable function at the measured coordinate (x, y). Formulas (11) to (13) are as follows: C _xyw = W _n * C _xy if there is no sub-control item on (x, y) ………(11) C _xyw = W _u * C _xy if there is the un-clickable sub-control item on (x, y) ………(12) C _xyw = W _c * C _xy if there is the clickable sub-control item on (x, y) ……(13)

_Cxyw : weighted window color value of the pixel (x, y) of the w window, _Cxy : original color value of the pixel (x, y) of the desktop, such as _Rxy , _Gxy or _Bxy , _Wn : color weight of no child control, _Wu : color weight of non-supported clickable function child control, _Wc : color weight of supported clickable function child control. It is additionally explained that the window information collection module 126 obtains the title (Title) and process (Process) name of the current active window through the Windows application programming interface (API). As shown in Table 1 below, the window information collection module 126 can obtain window information through the corresponding application programming interface: Function Application Programming Interface Get window title GetForegroundWindow() GetWindowTextLength() GetWindowText() Get the window process name. GetWindowThreadProcessId() GetModuleBaseName()

In one embodiment, the window information collection module 126 compares the control at the current cursor position based on the historical data of multiple control item images to determine the window control item information corresponding to the current cursor position. In step S430, the window information collection module 126 uses the weighted window color value as the window reference color.

In another embodiment, the window information collection module 126 sums up the weighted window color value of each point coordinate (x, y) obtained by formula (11) to formula (13) through the following formula (14).

As shown below, the weighted average color value C _w of the window is calculated using formula (14), and C _w is passed to the mouse pointer style adjustment module for calculation. ……(14)

C _w : Window weighted average color value. In this embodiment, the window information collection module 126 uses the window weighted average color value as the window reference color.

FIG6 is a schematic diagram of a process of monitoring the movement trajectory of a cursor according to an embodiment of the present disclosure. Referring to FIG6 , the trajectory monitoring module 122 is further used to execute steps S510 to S595 to generate mouse pointer trajectory information (i.e., trajectory record group). In step S510, the trajectory monitoring module 122 obtains the cursor position information. Specifically, the trajectory monitoring module 122 can obtain the cursor position coordinates of each sampling point by reading the cache data in the memory 120 or reading the data of other modules. In step S520, the track monitoring module 122 calculates the cursor displacement according to the cursor position information obtained in step S510. The cursor displacement can be calculated by the above formula (1).

In step S530, the trajectory monitoring module 122 calculates the displacement slope. In one embodiment, the trajectory monitoring module 122 can calculate the displacement slope of each time point in the plurality of moving trajectories by the following formula (15): _Sn = ( _Xn - _Xn-1 ) / ( _Yn - _Yn-1 ) ... (15)

S _n : displacement slope at time point n, X _n : X-axis coordinate at time point n, X _n-1 : X-axis coordinate at time point n-1, Y _n : Y-axis coordinate at time point n, Y _n-1 : Y-axis coordinate at time point n-1.

In step S540, the trajectory monitoring module 122 records the initial position. In step S550, the trajectory monitoring module 122 calculates the number of positive and negative changes in the displacement slope according to the initial position and the displacement slope. In step S560, the trajectory monitoring module 122 evaluates the number of changes in the displacement slope of the moving trajectory (i.e., sample) at each time point, and then executes the corresponding moving trajectory in the multiple moving trajectories in step S570 or the corresponding step in step S580. In other words, the trajectory monitoring module 122 obtains the number of movement returns according to the change of the cursor initial position and the displacement slope, and when the number of movement returns is 2, the trajectory monitoring module 122 then executes step S570. When the number of movement returns is 3, the trajectory monitoring module 122 then executes step S580.

In step S570, the trajectory monitoring module 122 records the turning position and the number of accumulated samples. In step S580, the trajectory monitoring module 122 records the end position and the number of accumulated samples. The trajectory monitoring module 122 takes the moving trajectory with a specific number of movement and return times as a trajectory record group, and records the current number of samples (i.e., the number of accumulated samples in step S570 and step S580).

After executing step S570 or step S580, the trajectory monitoring module 122 then executes step S590. In step S590, the trajectory monitoring module 122 calculates the number of samples to be collected. Specifically, the trajectory monitoring module 122 compares whether the cumulative number of samples reaches the number of samples to be collected. The number of samples to be collected is a preset number, such as 3, 6, 10, or 20. When the cumulative number of samples reaches the number of samples to be collected, the trajectory monitoring module 122 then executes step S595, otherwise it executes step S510. In another embodiment, the trajectory monitoring module 122 calculates the number of samples to be collected by the cumulative number of samples recorded in step S570, the cumulative number of samples recorded in step S580, the number of trajectories of a positive and negative value change, and the sampling ratio. Specifically, the number of samples to be collected is equal to the quotient of the sampling ratio and the average number of samples (numbers) passed by a positive and negative value change, and the average number of samples passed by a positive and negative value change can be calculated by the following formula: 0.5*(0.5*the cumulative number of samples recorded in step S570 plus the cumulative number of samples recorded in step S580).

In step S595 , the track monitoring module 122 outputs the mouse pointer track information. Specifically, when the track monitoring module 122 collects a certain number of track record sets, it outputs the track record sets (ie, the mouse pointer track information) to the track determination module 123 .

FIG. 7 is a flow chart of determining the movement trajectory of the cursor according to an embodiment of the present disclosure. FIG. 8 is a schematic diagram of the movement trajectory of the cursor according to an embodiment of the present disclosure. Referring to FIG. 7 , the trajectory determination module 123 is further used to execute steps S610 to S680 to determine whether the trajectory record group includes the search cursor trajectory data. In step S610, the trajectory determination module 123 obtains the mouse pointer trajectory information (i.e., the trajectory record group).

In step S620 and step S630, the trajectory determination module 123 performs graphic recognition to obtain a graphic category confidence score. In one embodiment, the trajectory determination module 123 performs graphic recognition on the trajectory record group through a convolutional neural network model to obtain a graphic confidence score. As shown in FIG8 , a trajectory record group within a unit time (e.g., 3 seconds, 5 seconds, 6 seconds) is compared with the moving trajectory of the trajectory record group by the convolutional neural network model through the signal based on searching for the cursor graphic (i.e., the preset training sample), and then obtains the graphic confidence score according to the graphic similarity. The graph confidence score is related to the confidence level of the output of the convolutional neural network model. The higher the comparison similarity, the higher the graph confidence score.

In step S640, the trajectory determination module 123 normalizes the mouse displacement. Specifically, the trajectory determination module 123 performs normalization processing on the displacement in the trajectory record set to obtain a two-dimensional array. The trajectory determination module 123 can perform normalization through the following formulas (16) to (19): X _a = ( ) / N……(16) Y _a = ( ) / N……(17) I _n = X _n – X _a ……(18) J _n = Y _n -Y _a ……(19)

N: total number of samples, X _a : average value of mouse X-axis position, Y _a : average value of mouse Y-axis position, _In : normalized mouse X-axis position of sampling point n, J _n : normalized mouse Y-axis position of sampling point n, n is a positive integer. Thus, the trajectory judgment module 123 normalizes the displacement slope of the trajectory record group from the original one-dimensional array to obtain a two-dimensional array arranged in (X _n , Y _n ).

In one embodiment, the convolution neural network model and the recurrent neural network model are neural network models trained by multiple search cursor graphs. The trajectory determination module 123 further obtains a three-dimensional array based on the two-dimensional array and the displacement slope of the trajectory record set. For example, the trajectory determination module 123 adds the corresponding displacement slope to the two-dimensional array obtained in step S640 based on the same sampling point to obtain a three-dimensional array (e.g. ( _Xn , _Yn , _Sn ).

In step S650, the trajectory determination module 123 performs time series analysis. That is, the trajectory determination module 123 performs pattern recognition on the trajectory record set through the recurrent neural network model to obtain a signal confidence score. Specifically, the recurrent neural network model performs time series analysis on the three-dimensional array to determine the signal confidence score of the trajectory record set based on the similarity between the time dependency and correlation between each position and the previous position in the three-dimensional array and the time dependency and correlation in multiple search cursor patterns (i.e., the preset comparison samples). In other words, the trajectory judgment module 123 compares the time and slope correlation of each data in the trajectory record group with the previous time point and the time and slope correlation in the comparison sample through the recurrent neural network model, thereby obtaining the signal flow category confidence score (i.e., signal confidence score) (step S660).

In step S670, the trajectory determination module 123 calculates the total confidence score F according to the signal confidence score and the image confidence score. F = (W _c * C _c + W _r * C _r ) / (W _c + W _r ) …… (16)

W _c : "Find Mouse Pointer" graph confidence score weight, W _c : "Find Mouse Pointer" signal confidence score weight, _Cr : signal confidence score, C _c : graph confidence score.

In other words, the trajectory determination module 123 performs weighted operations on the image confidence score and the signal confidence score based on the image weight and the signal weight to obtain a confidence score (ie, a total confidence score).

In the present device, the trajectory determination module 123 performs graphic recognition on the trajectory record group based on multiple seek cursor graphics to obtain a confidence score, wherein in response to the confidence score being greater than a threshold value ( _Fth ), the trajectory determination module 123 determines that the trajectory record group of the moving trajectory includes seek cursor trajectory data, that is, determines that the user is seeking the position of the cursor on the display 130 (step S680).

In summary, in the embodiments disclosed herein, the method and electronic device for dynamically adjusting the cursor display respectively detect/monitor whether there is a display switching event and find the cursor track, thereby achieving the effect of distributing the computing load and overall smooth operation. In addition, the method and electronic device for dynamically adjusting the cursor display adjust the cursor setting to a suitable size and/or a suitable color based on the desktop information and the window information. In this way, the method and electronic device for dynamically adjusting the cursor display can automatically determine that the current movement track of the cursor is to find the cursor position, and then adjust the cursor display setting so that the user can quickly find the cursor position to improve the convenience of use. Furthermore, the weighted color value is related to the average color value by using a weighted coefficient to increase the weight of the color value of the central area of the desktop, thereby reducing the weight of the pixels at the edge to improve the visibility of the cursor.

Although the present disclosure has been disclosed as above by way of embodiments, it is not intended to limit the present disclosure. Any person having ordinary knowledge in the relevant technical field may make some changes and modifications without departing from the spirit and scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be subject to the definition of the attached patent application scope.

100: electronic device 110: processor 120: memory 130: display 121: cursor signal collection module 122: track monitoring module 123: track judgment module 124: cursor display adjustment module 125: desktop detection module 126: window information collection module S210~S250, S310~350, S410~S430, S510~S595, S610~S680: steps

FIG. 1 is a block diagram of an electronic device according to an embodiment of the present disclosure. FIG. 2 is a flow chart of a method for dynamically adjusting cursor display according to an embodiment of the present disclosure. FIG. 3 is a flow chart of capturing a display screen to calculate a cursor color according to an embodiment of the present disclosure. FIG. 4 is a flow chart of multiple modules according to an embodiment of the present disclosure. FIG. 5 is a flow chart of obtaining a weighted window color value according to window information according to an embodiment of the present disclosure. FIG. 6 is a flow chart of monitoring a cursor movement trajectory according to an embodiment of the present disclosure. FIG. 7 is a flow chart of determining a cursor movement trajectory according to an embodiment of the present disclosure. FIG. 8 is a schematic diagram of a cursor movement trajectory according to an embodiment of the present disclosure.

S210~S250: Steps

Claims (18)

A method for dynamically adjusting cursor display is applicable to an electronic device coupled to a display and an input device, the method comprising: collecting a moving track of a cursor displayed on the display; filtering the moving track with a displacement greater than a threshold from the moving track, and using the moving track with a displacement greater than the threshold as cursor track data; determining whether the cursor track data includes search cursor track data; and in response to the cursor track data including the search cursor track data, correspondingly adjusting the display setting of the cursor according to display information of the electronic device, wherein the display information is related to desktop color, window color, window information, view Window size, desktop size and window control information, wherein the step of filtering out the movement trajectory with a displacement greater than the threshold from the movement trajectory and using the movement trajectory with a displacement greater than the threshold as the cursor trajectory data includes: calculating the displacement according to the movement trajectory, and filtering out the movement trajectory with a displacement greater than the threshold to calculate the displacement slope according to the displacement; obtaining the movement return times according to the cursor initial position and the change of the displacement slope, and using the movement trajectory with a specific number of movement return times as a trajectory record group, and outputting the trajectory record group when a collection quantity of the trajectory record group is collected. The method for dynamically adjusting the cursor display as described in claim 1 further includes: detecting a display switching event of the display, wherein in response to detecting the display switching event, calculating the average color value of each pixel in the current display screen as a desktop reference color; calculating the window average color value based on the average color value and the window screen corresponding to the current cursor position, and using the window average color value as the window reference color; performing a color complement operation on the desktop reference color and the window reference color to obtain a cursor color, and then adjusting the display setting of the cursor according to the cursor color. The method for dynamically adjusting the cursor display as described in claim 2, wherein before the step of performing a color infill operation on the desktop reference color and the window reference color to obtain the cursor color, the method further comprises: obtaining a weighted color value according to the average color value of each pixel and the desktop weighting coefficient, and using the weighted color value as the window reference color, wherein the weighted color value is related to the average color value by increasing the central area of the desktop through the weighting coefficient. The method further comprises: obtaining a weighted window color value according to the average color value of the window and the window title, window program and window control information of the window screen, and using the weighted window color value as the reference color of the window, wherein the window information includes the window title and the window program, wherein the weighted window color value is related to the color value obtained according to the window control information and the corresponding control weighting coefficient. The method for dynamically adjusting the cursor display as described in claim 2, wherein the display switching event is at least one of a window switching event, a new window opening event, and a large-scale desktop change event, wherein in response to detecting the display switching event, the step of calculating the average color value of each pixel in the current display screen includes: when it is detected that the display has the display switching event and the desktop area changes by more than a ratio value, the current display screen on the display is captured to calculate the average color value, wherein the ratio value is related to the window width, window height and window display height of the event window, the window display width and desktop height, the desktop width and the window position coordinates. The method for dynamically adjusting cursor display as described in claim 1, wherein the step of collecting the movement track of the cursor displayed on the display includes: collecting at least one cursor event of the cursor, and recording the position coordinates of the cursor event to generate the movement track, wherein the cursor event is at least one of a movement event, a key event, a scroll wheel event, and a move-out-of-window event. The method for dynamically adjusting the cursor display as described in claim 1, wherein the step of determining whether the cursor track data includes the seek cursor track data comprises: performing graphic recognition on the track record group based on multiple seek cursor graphics to obtain a confidence score; in response to the confidence score being greater than a threshold value, determining that the track record group of the moving track includes the seek cursor track data. The method for dynamically adjusting the cursor display as described in claim 6, wherein the step of performing the graphic recognition on the trajectory record group to obtain the confidence score includes: performing the graphic recognition on the trajectory record group through a convolutional neural network model to obtain a graphic confidence score; performing the graphic recognition on the trajectory record group through a recurrent neural network model to obtain a signal confidence score; based on the graphic weight and The signal weight performs a weighted operation on the graphic confidence score and the signal confidence score to obtain the confidence score, wherein the convolution neural network model and the recurrent neural network model are neural network models trained by the search cursor graphics, wherein the convolution neural network model compares the movement trajectory of the trajectory record group based on the search cursor graphics to obtain the graphic confidence score. The method for dynamically adjusting the cursor display as described in claim 7, wherein the step of performing the graphic recognition on the trajectory record set through the recurrent neural network model to obtain the signal confidence score includes: performing normalization processing on the displacement in the trajectory record set to obtain a two-dimensional array, and then obtaining a three-dimensional array based on the two-dimensional array and the displacement slope of the trajectory record set; performing time series analysis on the three-dimensional array to determine the signal confidence score of the trajectory record set based on the time dependency of each position in the three-dimensional array and the correlation with the previous position. A method for dynamically adjusting cursor display as described in claim 8, wherein the cursor fit size is calculated based on the screen resolution default value, display width, cursor size upper limit value, display height and control weight, wherein the control weight is related to the default weight value of the window control information corresponding to the current position of the cursor, and the cursor fit size is not greater than the cursor size upper limit value, wherein the display setting of the cursor is adjusted based on the cursor fit size. An electronic device includes: a memory storing a plurality of modules; and a processor coupled to a display and an input device, and loading and executing the modules stored in the memory, wherein the modules include: a cursor signal collection module collecting the movement track of the cursor displayed on the display; a track monitoring module filtering the movement track; The moving track with a displacement greater than a threshold is output, and the moving track with a displacement greater than the threshold is used as cursor track data; a track judgment module is used to judge whether the cursor track data includes the search cursor track data; and a cursor display adjustment module is used to adjust the cursor display according to the display information of the electronic device in response to the cursor track data including the search cursor track data. The display setting of the cursor is adjusted accordingly, wherein the display information is related to the desktop color, the window color, the window information, the window size, the desktop size and the window control information, wherein the track monitoring module calculates the displacement according to the moving track, and filters out the moving track whose displacement is greater than the threshold value to calculate the displacement slope according to the displacement, wherein The trajectory monitoring module obtains the number of movement returns according to the initial position of the cursor and the change of the displacement slope, and the trajectory monitoring module uses the movement trajectory with a specific number of movement returns as a trajectory record group, wherein when the trajectory monitoring module collects a collection quantity of the trajectory record group, the trajectory record group is output to the trajectory judgment module. The electronic device as claimed in claim 10, wherein the modules further include: a desktop detection module, detecting that the display has a display switching event, wherein the desktop detection module calculates the average color value of each pixel in the current display screen as a desktop reference color in response to detecting the display switching event; and a window information collection module, calculating the window average color value according to the average color value and the window screen corresponding to the current cursor position, and using the window average color value as the window reference color, wherein the cursor display adjustment module performs a color complement operation on the desktop reference color and the window reference color to obtain a cursor color, and then adjusts the display setting of the cursor according to the cursor color. The electronic device as claimed in claim 11, wherein the processor further executes the desktop detection module to: obtain a weighted color value according to the average color value of each pixel and the desktop weighting coefficient, and use the weighted color value as the window reference color, wherein the weighted color value is related to the average color value by increasing the weight of the central area of the desktop through the weighting coefficient, wherein the processor further executes the window information receiving module The set module is used to obtain a weighted window color value according to the window average color value and the window title, window program and window control information of the window screen, and use the weighted window color value as the reference color of the window, wherein the window information includes the window title and the window program, wherein the weighted window color value is related to the color value obtained according to the window control information and the corresponding control weighting coefficient. The electronic device as claimed in claim 11, wherein the display switching event is at least one of a window switching event, a new window opening event, and a large-scale desktop change event, wherein the desktop detection module detects that the display has the display switching event and the desktop area changes by more than a ratio value, then captures the current display image on the display to calculate the average color value, wherein the ratio value is related to the window width, window height, and window display height of the event window, the window display width and the desktop height, the desktop width, and the window position coordinates. The electronic device as claimed in claim 10, wherein the cursor signal collection module collects at least one cursor event of the cursor, wherein the cursor event is at least one of a move event, a key event, a scroll wheel event, and a move-out-of-window event, and the cursor signal collection module records the position coordinates of the cursor event to generate the movement trajectory. An electronic device as described in claim 10, wherein the trajectory determination module performs graphic recognition on the trajectory record set based on a plurality of seek cursor graphics to obtain a confidence score, wherein in response to the confidence score being greater than a threshold value, the trajectory determination module determines that the trajectory record set of the moving trajectory includes the seek cursor trajectory data. An electronic device as claimed in claim 15, wherein the trajectory determination module performs the graphic recognition on the trajectory record set through a convolutional neural network model to obtain a graphic confidence score, wherein the trajectory determination module performs the graphic recognition on the trajectory record set through a recurrent neural network model to obtain a signal confidence score, wherein the trajectory determination module performs the graphic recognition on the trajectory record set through a recurrent neural network model to obtain a signal confidence score, The graphic confidence score and the signal confidence score are weighted to obtain the confidence score, wherein the convolution neural network model and the recurrent neural network model are neural network models trained by the search cursor graphics, wherein the convolution neural network model compares the movement trajectory of the trajectory record set based on the search cursor graphics to obtain the graphic confidence score. An electronic device as described in claim 16, wherein the trajectory judgment module performs normalization processing on the displacement in the trajectory record group to obtain a two-dimensional array, and then obtains a three-dimensional array based on the two-dimensional array and the displacement slope of the trajectory record group, wherein the recurrent neural network model performs time series analysis on the three-dimensional array to judge the signal confidence score of the trajectory record group based on the time dependency of each position in the three-dimensional array and the correlation with the previous position. An electronic device as described in claim 10, wherein the cursor display adjustment module calculates the cursor fit size according to the screen resolution default value, the display width, the cursor size upper limit value, the display height, and the control weight, wherein the control weight is related to the default weight value of the window control information corresponding to the current position of the cursor, and the cursor fit size is not greater than the cursor size upper limit value, wherein the cursor display adjustment module adjusts the display setting of the cursor according to the cursor fit size.