TWI885935B - Display parameter adjustment method and electronic device - Google Patents

Abstract

A display parameter adjustment method and an electronic device are disclosed. The method includes: performing a first image classification training on a neural network model based on a first training mode; performing a second image classification training on the neural network model based on a second training mode, wherein a total number of the convolutional layer participating in the second image classification training is less than a total number of the convolutional layer participating in the first image classification training; obtaining first color distribution information corresponding to an image; providing the first color distribution information to the neural network model to identify a type of the image through the neural network model; performing a first-stage adjustment on a display parameter based on a classification result; obtaining second color distribution information corresponding to the image; calculating color compensation information based on the second color distribution information; and performing a second-stage adjustment on the display parameter based on the color compensation information.

Description

Display parameter adjustment method and electronic device

The present invention relates to a display parameter adjustment method and an electronic device.

Some types of monitors or display devices support users to manually switch between multiple display modes of the monitor. However, in practice, it is impractical to allow users to manually switch display modes. The reason is that even if most users initially set the display mode of the monitor to a certain display mode (such as document mode), after switching the display content of the monitor (such as switching from browsing a work report to watching a video), users often choose to use the previously set display mode, thereby viewing the image presented by the monitor in an inappropriate display mode. Furthermore, even if the user is willing to select a display mode manually or automatically by the system, when the image content displayed by the monitor changes significantly, the selected display mode cannot be dynamically adjusted according to the display parameters of the monitor in real time, resulting in the user never being able to obtain the best viewing experience of the image displayed by the monitor.

In view of this, the present invention provides a display parameter adjustment method and an electronic device, which can improve the above-mentioned problems and effectively enhance the user's viewing experience of the images presented by the display.

An embodiment of the present invention provides a display parameter adjustment method, which includes: setting a neural network model to a first training mode and performing a first image classification training on the neural network model based on the first training mode; after completing the first image classification training, setting the neural network model to a second training mode and performing a second image classification training on the neural network model based on the second training mode, wherein in the neural network model, the total number of convolution layers participating in the second image classification training is less than the total number of convolution layers participating in the first image classification training; presenting an image through a display ; obtaining first color distribution information corresponding to the image; providing the first color distribution information to the neural network model to classify the image through the neural network model; performing a first-stage adjustment on the display parameters used by the display according to the classification result of the neural network model on the image; obtaining second color distribution information corresponding to the image according to the result of the first-stage adjustment; calculating color compensation information according to the second color distribution information; and performing a second-stage adjustment on the display parameters used by the display according to the color compensation information.

The embodiment of the present invention further provides an electronic device, which includes a display, a storage circuit and a processor. The storage circuit is used to store a neural network model. The processor is coupled to the display and the storage circuit. The processor is used to: set the neural network model to a first training mode and perform a first image classification training on the neural network model based on the first training mode; after completing the first image classification training, set the neural network model to a second training mode and perform a second image classification training on the neural network model based on the second training mode, wherein in the neural network model, the total number of convolutional layers participating in the second image classification training is less than the total number of convolutional layers participating in the first image classification training; present an image through the display; obtain a corresponding The method comprises: providing the first color distribution information of the image; providing the first color distribution information to the neural network model to classify the image through the neural network model; performing a first-stage adjustment on the display parameters used by the display according to the classification result of the neural network model on the image; obtaining second color distribution information corresponding to the image according to the result of the first-stage adjustment; calculating color compensation information according to the second color distribution information; and performing a second-stage adjustment on the display parameters used by the display according to the color compensation information.

Based on the above, after completing the first image classification training in the first training mode, the total number of convolutional layers in the neural network model that participate in the second image classification training in the second training mode can be reduced to effectively lock the feature parameters that have been updated and further train the recognition of detailed features. In addition, after completing the training of the neural network model, the display parameters used by the display can be adjusted in two stages. Among them, the adjustment in the first stage is based on the classification results of the image by the neural network model, and the adjustment in the second stage is to further dynamically compensate the display parameters used by the display on the basis of the adjustment in the first stage. In this way, the user's viewing experience of the image presented by the display can be effectively improved.

FIG. 1 is a schematic diagram of an electronic device according to an embodiment of the present invention.

1 , the electronic device 10 may be a smart phone, a tablet computer, a laptop computer, a desktop computer, a game console, a server, or a car computer, etc., and the type of the electronic device 10 is not limited thereto.

The electronic device 10 includes a display 11, a storage circuit 12, and a processor 13. The display 11 is used to present images. For example, the display 11 may include a plasma display (Plasma Display), a liquid crystal display (LCD), a thin film transistor liquid crystal display (TFT-LCD), an organic light-emitting diode (OLED), and a light-emitting diode display (LED display), etc., and the type of the display 11 is not limited thereto.

In one embodiment, the display 11 is integrated in the electronic device 10. In one embodiment, the display 11 can also be disposed outside the electronic device 10 and can be coupled to the electronic device 10 via a wired or wireless method.

The storage circuit 12 is used to store data. For example, the storage circuit 12 may include a read-only memory (ROM), a solid state disk (SSD), a traditional hard disk (HDD), a flash memory module, an embedded MultiMedia Card (eMMC), a universal flash storage (UFS) device, or other types of non-volatile storage media.

The processor 13 is coupled to the display 11 and the storage circuit 12. The processor 13 can be used to be responsible for the overall or partial operation of the electronic device 10. For example, the processor 13 may include a central processing unit (CPU), or other programmable general-purpose or special-purpose microprocessors, digital signal processors (DSP), programmable controllers, application specific integrated circuits (ASIC), programmable logic devices (PLD), or other similar devices or combinations of these devices. In one embodiment, the processor 13 may also include a graphics processing unit (GPU), a vision processing unit (VPU), a neural network processor (NPU), or other processors dedicated to (or conducive to) performing image processing or neural network processing.

In one embodiment, the electronic device 10 may further include various peripheral devices and/or input/output devices such as a network interface card, a mouse, a keyboard, a touch panel, a speaker, a microphone, and/or a power management circuit. The present invention does not limit the types of the peripheral devices and/or the input/output devices.

In one embodiment, the storage circuit 12 may store a neural network model 101. The neural network model 101 may be used to classify the image presented by the display 11. For example, the neural network model 101 may be implemented using EfficientNet or other types of convolutional neural network (CNN) architectures, and the present invention is not limited thereto.

FIG2 is a schematic diagram of a neural network model according to an embodiment of the present invention.

2, the neural network model 101 includes an input terminal 21, an output terminal 22 and a computing core 23. The computing core 23 is coupled between the input terminal 21 and the output terminal 22. The input terminal 21 is used to receive input data and input the received input data to the computing core 23. The output terminal 22 can be used to generate output data according to the calculation result of the computing core 23. The computing core 23 includes convolutional layers 24(1)~24(m). During the training of the neural network model 101, the convolutional layers 24(1)~24(m) can execute predefined operations in sequence, and the feature parameters used by the convolutional layers 24(1)~24(m) can be updated. For example, the characteristic parameter may include a weight parameter or other types of parameters, which are not limited by the present invention.

In one embodiment, during the training of the neural network model 101, the convolutional layers 24(1)-24(m) may process the input data in sequence and generate output results through the output terminal 22. For example, the output results may reflect the type of image corresponding to the input data recognized or recognized by the neural network model 101.

In one embodiment, the processor 13 can use the verification data to verify the output of the neural network model 101, and update the feature parameters used by at least one of the convolutional layers 24(1)-24(m) according to the verification result. In this way, after using a large amount of training data to train the neural network model 101 for image classification, the accuracy of the neural network model 101 for image classification can be gradually improved.

In one embodiment, during the training of the neural network model 101, the processor 13 may set at least one of the convolutional layers 24(1)-24(m) to a locked state or an unlocked state. In particular, among the convolutional layers 24(1)-24(m), only the feature parameters used by the convolutional layers set to the unlocked state (or the convolutional layers not set to the locked state) can be updated, while the feature parameters used by the convolutional layers set to the locked state cannot be updated. For example, during the training of the neural network model 101, assuming that the convolution layer 24(i) is set to an unlocked state and the convolution layer 24(j) is set to a locked state, only the feature parameters used by the convolution layer 24(i) can be updated, while the feature parameters used by the convolution layer 24(j) cannot be updated. In addition, the processor 13 can dynamically set or switch each of the convolution layers 24(1) to 24(m) to be in a locked state or an unlocked state.

In one embodiment, in the early stage of training the neural network model 101, the processor 13 may set the neural network model 101 to a specific training mode (also referred to as the first training mode). The processor 13 may perform image classification training on the neural network model 101 based on the first training mode (also referred to as the first image classification training). For example, in the first training mode, the processor 13 may input training data into the neural network model 101, verify the output of the neural network model 101 based on the verification data, and update the feature parameters used by at least one convolution layer in the convolution layers 24(1)~24(m) that is not currently in a locked state based on the verification result.

In one embodiment, after completing the first image classification training, the processor 13 may further set the neural network model 101 to another training mode (also referred to as the second training mode). The processor 13 may perform image classification training on the neural network model 101 based on the second training mode (also referred to as the second image classification training). For example, in the second training mode, the processor 13 may also input training data into the neural network model 101, verify the output of the neural network model 101 according to the verification data, and update the feature parameters used by at least one convolution layer in the convolution layers 24(1)~24(m) that is not currently in a locked state according to the verification result. It should be noted that, in the neural network model 101, the total number of convolutional layers participating in the second image classification training may be less than the total number of convolutional layers participating in the first image classification training.

In one embodiment, assuming that in the first training mode, N(1) of the convolutional layers 24(1)-24(m) are not set to a locked state (or N(1) of the convolutional layers are set to an unlocked state), then the N(1) convolutional layers that are not set to a locked state can participate in the first image classification training. In particular, the feature parameters used by the N(1) convolutional layers that are not set to a locked state can be updated in the first image classification training to reflect the training results of the first image classification training.

On the other hand, assuming that in the second training mode, N(2) of the convolutional layers 24(1)~24(m) are not set to a locked state (or N(2) convolutional layers are set to an unlocked state), then these N(2) convolutional layers that are not set to a locked state can participate in the second image classification training. N(1) can be greater than N(2). In particular, the feature parameters used by these N(2) convolutional layers that are not set to a locked state can be updated in the second image classification training to reflect the training results of the second image classification training.

In one embodiment, it is assumed that at least one convolution layer (also referred to as the first convolution layer) in the neural network model 101 is not set to a locked state in the first training mode. Therefore, in the first training mode, the first convolution layer can participate in the first image classification training of the neural network model 101, and the feature parameters used by the first convolution layer can be updated in the first training mode to reflect the training results of the first image classification training.

However, after leaving the first training mode and entering the second training mode, in the second training mode, the processor 13 may set the first convolution layer to a locked state (i.e., switching the first convolution layer from an unlocked state in the first training mode to a locked state). Therefore, in the second training mode, the first convolution layer will not be able to participate in the second image classification training of the neural network model 101, and the feature parameters used by the first convolution layer will not be updated in the second training mode. Therefore, in the second training mode, the processor 13 may perform the second image classification training only on at least one convolution layer (also referred to as the second convolution layer) in the neural network model 101 that is not set to a locked state, and may update the feature parameters used by the second convolution layer accordingly to reflect the training results of the second image classification training.

In one embodiment, compared with the second convolution layer, the first convolution layer is farther away from the output end of the neural network model 101 in the computing core of the neural network model 101. Or, from another perspective, compared with the second convolution layer, the first convolution layer is closer to the input end of the neural network model 101 in the computing core of the neural network model 101.

FIG3 is a schematic diagram of a neural network model in a first training mode according to an embodiment of the present invention.

3 , in the first training mode, the processor 13 may set the convolution layers 24(1)-24(m) to an unlocked state (marked as T in FIG. 3 ). In the first training mode, by performing a first image classification training on the convolution layers 24(1)-24(m) in the unlocked state, feature parameters used by at least a portion of the convolution layers 24(1)-24(m) may be updated to reflect the results of the first image classification training.

FIG4 is a schematic diagram of a neural network model in a second training mode according to an embodiment of the present invention.

4 , in the second training mode, the processor 13 may switch the convolutional layers 24(1)-24(n) (i.e., the first convolutional layers) to a locked state (marked as F in FIG. 4 ), and maintain the convolutional layers 24(n+1)-24(m) (i.e., the second convolutional layers) in an unlocked state (marked as T in FIG. 4 ). In particular, compared to at least one of the convolutional layers 24(n+1)-24(m), at least one of the convolutional layers 24(1)-24(n) is farther away from the output terminal 22 of the neural network model 101 and/or closer to the input terminal 21 of the neural network model 101.

In the second training mode, by performing the second image classification training on the convolutional layers 24(n+1)~24(m) in the unlocked state, the feature parameters used by at least part of the convolutional layers 24(n+1)~24(m) can be updated to reflect the execution result of the second image classification training. At the same time, in the second training mode, the convolutional layers 24(1)~24(n) in the locked state do not participate in the second image classification training. Therefore, the feature parameters used by the convolutional layers 24(1)~24(n) will not be updated in the second training mode.

It should be noted that in the embodiments of FIG. 3 and FIG. 4 , the total number and/or distribution of convolutional layers set to a locked state and/or an unlocked state can be adjusted according to practical needs, and the present invention is not limited thereto.

In one embodiment, in the early stage of training the neural network model 101 (i.e., in the first training mode), the first convolutional layer is included in the first image classification training, and the feature parameters used by the first convolutional layer can be preliminarily updated to provide the neural network model 101 with a more general or larger range of recognition capabilities for the images corresponding to the input data. However, after the first image classification training of the neural network model 101 is completed, in the later stage of training the neural network model 101 (i.e., in the second training mode), the first convolution layer is excluded from the second image classification training (i.e., the first convolution layer is set to a locked state), so that the feature parameters established by the first convolution layer in the first training mode can be effectively retained, and the feature parameters used by the second convolution layer closer to the output end of the neural network model 101 are further updated. In this way, the recognition ability of the neural network model 101 for more detailed or smaller range images can be improved. In one embodiment, by using different training modes (such as a first training mode and a second training mode) at different times to train the neural network model 101, the training efficiency of the neural network model 101 can be effectively improved.

In one embodiment, in response to the usage level of the training data of the neural network model 101 reaching a preset level, the processor 13 may switch the neural network model 101 to a second training mode. For example, assume that the training data includes 5,000 pieces of data. In the first training mode, if the processor 13 detects that a specific proportion of the training data (e.g., 4/5, i.e., 4,000 pieces of data) has been used to train the neural network model 101, the processor 13 may determine that the usage level of the training data has reached a preset level and switch the neural network model 101 from the first training mode to the second training mode. In the second training mode, the processor 13 may continue to use the remaining training data (e.g., 1,000 pieces of data) to train the neural network model 101. However, if the usage level of the training data of the neural network model 101 does not reach the preset level (for example, the above-mentioned specific proportion of training data has not been used to train the neural network model 101), the processor 13 may maintain the neural network model 101 in the first training mode.

It should be noted that the processor 13 can also switch the neural network model 101 from the first training mode to the second training mode according to other decision-making methods. For example, in one embodiment, the processor 13 can determine whether the image classification accuracy of the neural network model 101 for the input data has reached a preset accuracy (e.g., 80% or other ratios). If the image classification accuracy of the neural network model 101 for the input data has reached the preset accuracy, the processor 13 can determine that the usage level of the training data has reached the preset level and switch the neural network model 101 from the first training mode to the second training mode. However, if the image classification accuracy of the neural network model 101 for the input data does not reach the preset accuracy, the processor 13 may determine that the usage level of the training data does not reach the preset level and maintain the neural network model 101 in the first training mode.

In one embodiment, after completing the training of the neural network model 101 (i.e., completing the training in the first training mode and the second training mode), the processor 13 may present an image (also referred to as a target image) through the display 11. For example, the processor 13 may instruct the display 11 to present the target image. At the same time, the processor 13 may obtain color distribution information corresponding to the target image (also referred to as first color distribution information). For example, the first color distribution information may reflect the color information of each pixel position in the display screen of the display 11 during the period when the display 11 presents the target image. For example, the color information of each pixel position may be represented by a color vector. For example, each color vector may be represented as (V(R), V(G), V(B)). V(R), V(G) and V(B) are all values between 0 and 255. For example, V(R), V(G) and V(B) may be "124", "23" and "64" respectively, depending on the image currently displayed by the display 11, and the present invention is not limited thereto. In addition, the color vector may also be represented by elements conforming to other types of color spaces, and the present invention is not limited thereto.

In one embodiment, the processor 13 may provide the first color distribution information to the neural network model 101 so as to classify the image currently presented by the display 11 (i.e., the target image) through the neural network model 101. For example, the neural network model 101 after the above training may classify the target image as an image belonging to a specific type (also referred to as a target type) among a plurality of candidate types according to the first color distribution information. For example, the plurality of candidate types may include a plurality of customized image types such as general, landscape, sports, games, theaters and/or eye protection, and the plurality of candidate types are not limited thereto. For example, assuming that the first color distribution information reflects that there is a large area of blue sky, mountains, sea, forests and/or lakes in the target image, the neural network model 101 may recognize that the type of the target image belongs to "landscape" according to the first color distribution information. By analogy, the neural network model 101 can identify the target image as belonging to different types according to different first color distribution information.

In one embodiment, the processor 13 may perform preliminary adjustment (also referred to as first-stage adjustment) on at least one display parameter used by the display 11 according to the classification result of the target image by the neural network model 101. For example, in the first-stage adjustment, the processor 13 may query a data table according to the classification result of the target image by the neural network model 101 to obtain parameter adjustment information. For example, this data table may record parameter adjustment information corresponding to or matching multiple types of images. If the classification result of the target image by the neural network model 101 reflects that the target image belongs to a certain type (i.e., the target type), the processor 13 may obtain parameter adjustment information corresponding to or matching the target type from the data table (also referred to as target parameter adjustment information). Then, the processor 13 may perform a first-stage adjustment on at least one display parameter used by the display 11 according to the target parameter adjustment information. For example, the first-stage adjustment may include adjusting one or more display parameters such as contrast, brightness, saturation and/or color temperature of the display 11 according to the target parameter adjustment information to apply the display setting related to the target type to the display 11.

For example, assuming that after applying the display settings related to "general" to the display 11, the first stage adjustment may include setting the contrast, brightness, saturation and/or color temperature of the display 11 to default values to meet the display requirements of general types of images (such as general word processing programs or browsing web pages). Alternatively, assuming that after applying the display settings related to "landscape" to the display 11, the first stage adjustment may include increasing the contrast and/or brightness of the display 11 to improve the presentation effect of landscape-type images (such as landscape photos). Alternatively, assuming that after applying the display settings related to "theater" to the display 11, the first stage adjustment may include lowering the brightness of the display 11 to present a sense of presence similar to watching a movie in a theater. Similarly, through the first stage adjustment, the processor 13 can preliminarily improve the display quality of the display 11 for different types of images.

In one embodiment, according to the result of the first stage adjustment, the processor 13 may further obtain another color distribution information corresponding to the target image (also referred to as the second color distribution information). For example, similar to the first color distribution information, the second color distribution information may also reflect the color information of each pixel position in the display screen of the display 11 during the period when the display 11 presents the target image. However, the difference from the first color distribution information is that the second color distribution information may further reflect the result of the adjustment of at least one display parameter of the display 11 by the processor 13 in the first stage adjustment. For example, assuming that after the display setting related to "landscape" is applied to the display 11, compared with the first color distribution information, the second color distribution information may reflect that the contrast and/or brightness of the image currently presented by the display 11 (i.e., the target image) is improved. Alternatively, assuming that after applying the display setting related to "theater" to the display 11, the second color distribution information may reflect that the brightness of the image (i.e., the target image) currently displayed by the display 11 is reduced compared to the first color distribution information. Similarly, after applying different types of display settings, the second color distribution information may reflect different adjustment results of the display parameters of the display 11 by the processor 13 in the first stage of adjustment.

In one embodiment, the processor 13 may calculate color compensation information based on the second color distribution information. For example, the color compensation information may be used to improve the image presentation quality defects of the display 11 caused by or omitted from the first stage adjustment of the display parameters of the display 11. The processor 13 may perform advanced adjustment (also referred to as second stage adjustment) on at least one display parameter used by the display 11 based on the color compensation information.

For example, assume that in the first stage of adjustment, the processor 13 applies the display settings related to "theater" (i.e., the target type) to the display 11 to reduce the brightness of the display screen of the display 11. In this case, if the user suddenly switches the display screen of the display 11 to present a landscape photo (i.e., an image that does not belong to the target type), the second color distribution information can reflect that the color distribution of the target image currently presented by the display 11 has changed significantly. At this time, the processor 13 can detect based on the second color distribution information that the currently used display settings related to "theater" are not conducive to presenting this type of "landscape" image. Therefore, the processor 13 can perform a second stage adjustment on one or more display parameters such as contrast, brightness, saturation and/or color temperature used by the display 11 based on the adjustment result of the first stage according to the second color distribution information, so as to try to adjust the display setting of the display 11 to be more suitable for presenting the target image currently presented by the display 11.

In one embodiment, the processor 13 may obtain grayscale vector information corresponding to a plurality of pixel positions in the target image according to the second color distribution information. For example, the processor 13 may obtain the grayscale vector information through the following equations (1.1) to (1.5).

(1.1)

R   (1.2)

G   (1.3)

B   (1.4)

(1.5)

In equations (1.1) to (1.5), matrices W, R, G, and B are used to represent gray vector information, red vector information, green vector information, and blue vector information corresponding to multiple pixel positions in the target image, respectively. m is used to represent the length of the display screen of the display 11, n is used to represent the width of the display screen of the display 11, and m×n can be used to represent the size (or resolution) of the display screen of the display 11. R(ij), G(ij), and B(ij) can be used to represent the color information of the pixel at position (i, j) in the display screen of the display 11, where R(ij) is the color information of the red component, G(ij) is the color information of the green component, and B(ij) is the color information of the blue component. For example, the color information of the pixel at position (i, j) in the display screen of the display 11 can be represented by a color vector (R(ij), G(ij), B(ij)). W(ij) is used to represent the grayscale information (such as grayscale value) of the pixel at position (i, j) in the display screen of the display 11. In addition, a, b and c can be constants. For example, a, b and c can be "0.2126", "0.7152" and "0.0722" respectively, and the values of a, b and c can be adjusted according to practical needs. It should be noted that the calculation methods of matrix multiplication and matrix addition are known techniques and will not be elaborated here.

In one embodiment, the processor 13 may calculate the grayscale expected value corresponding to the target image according to the grayscale vector information. For example, the processor 13 may obtain the grayscale expected value through the following equation (2.1).

E[W]=a×E[R]+b×E[G]+c×E[B] (2.1)

In equation (2.1), E[W] can be used to represent the grayscale expected value corresponding to the target image. It should be noted that the calculation method of the expected value belongs to the known technology and will not be elaborated here. Then, the processor 13 can obtain the color compensation information according to the grayscale expected value.

In one embodiment, the processor 13 may determine whether the grayscale expected value is greater than a critical value (also referred to as a first critical value). For example, the first critical value may be "231" or other values. If the grayscale expected value is greater than the first critical value, the processor 13 may instruct the display 11 to reduce the brightness of the display screen of the display 11. For example, if the grayscale expected value is greater than the first critical value, the processor 13 may instruct the display 11 to subtract an adjustment parameter (also referred to as a first adjustment parameter) from the brightness value in the display parameter to reduce the brightness of the display screen of the display 11. Both the first critical value and the first adjustment parameter may be set and adjusted according to practical needs, and the present invention is not limited thereto. For example, in one embodiment, the color compensation information may include this first adjustment parameter.

In one embodiment, the processor 13 may determine whether the grayscale expected value is less than a critical value (also referred to as a second critical value). For example, the second critical value may be "39" or other values, and the second critical value is less than the first critical value. If the grayscale expected value is less than the second critical value, the processor 13 may instruct the display 11 to increase the brightness of the display screen of the display 11. For example, if the grayscale expected value is less than the second critical value, the processor 13 may instruct the display 11 to add an adjustment parameter (also referred to as a second adjustment parameter) to the brightness value in the display parameter to increase the brightness of the display screen of the display 11. Both the second critical value and the second adjustment parameter may be set and adjusted according to practical needs, and the present invention is not limited thereto. For example, in one embodiment, the color compensation information may include the second adjustment parameter.

In one embodiment, the processor 13 may obtain the variance corresponding to the red channel, the variance corresponding to the green channel, and the variance corresponding to the blue channel in the target image according to the red vector information (R), the green vector information (G), and the blue vector information (B) in the above equations (1.2) to (1.4), respectively. For example, the variance corresponding to the red channel may reflect the measure of the (average) distance between R(11)-R(mn) and the average value of R(11)-R(mn), the variance corresponding to the green channel may reflect the measure of the (average) distance between G(11)-G(mn) and the average value of G(11)-G(mn), and the variance corresponding to the blue channel may reflect the measure of the (average) distance between B(11)-B(mn) and the average value of B(11)-B(mn). The processor 13 may bring at least one of these variances into a specific algorithm or equation to calculate the required color compensation information. Then, the processor 13 may use the color compensation information to perform a second stage adjustment on at least one display parameter of the display 11.

In one embodiment, the processor 13 may also configure other color detection and adjustment rules to perform a second stage adjustment on at least one display parameter of the display 11 according to the second color distribution information. The configured color detection and adjustment rules may be set and adjusted according to practical needs, and the present invention is not limited thereto.

In one embodiment, the processor 13 can determine whether the difference between the display content of the previous image and the display content of the next image in the plurality of continuous images presented by the display 11 is less than a threshold value (also referred to as a third threshold value). If the difference between the display content of the previous image and the display content of the next image is less than the third threshold value, the processor 13 can continue to use the color compensation information for the previous image without recalculating the color compensation information for the next image. In this way, the computational burden of the processor 13 can be effectively reduced. However, if the difference between the display content of the previous image and the display content of the subsequent image is not less than (for example, greater than or equal to) the third critical value, the processor 13 can recalculate the color compensation information for the subsequent image and perform a second stage adjustment on at least one display parameter of the display 11 according to the color compensation information.

In one embodiment, the processor 13 may determine whether the adjustment range of at least part of the display parameters of the display 11 indicated by the color compensation information is greater than a critical value (also referred to as a fourth critical value). If the adjustment range of at least part of the display parameters of the display 11 indicated by the color compensation information is greater than the fourth critical value, in the second stage adjustment, the processor 13 may switch to performing segmented adjustment on the display parameters. For example, assuming that in the second stage adjustment, the default is to increase the brightness value of the display screen of the display 11 by 30 units at one time. After determining that the adjustment amplitude for the brightness value is greater than the fourth critical value, in the second stage adjustment, the processor 13 can switch to performing segmented adjustment on the brightness value, for example, the preset single increase of the brightness value of the display 11 by 30 units is adjusted to three consecutive increases of the brightness value of the display 11, and each increase is only 10 units. In addition, each segmented adjustment can be configured according to the preset segmentation rule, and the present invention is not limited. In this way, excessive stimulation to the user's eyes can be avoided due to a single adjustment of the display parameters of the display 11 being too large.

In one embodiment, if there are multiple types of sub-images in a target image, the classification of the target image by the neural network model 101 may be inaccurate or there may be multiple reliable classification results. In this case, if the classification result with the highest probability is selected to perform the first stage adjustment on the display 11, the applied display setting may seriously affect the image presentation quality of the remaining different types of sub-images in the target image.

In one embodiment, if there are multiple types of sub-images in a target image, the processor 13 can obtain an average adjustment parameter based on multiple classification results of the target image by the neural network model 101. Then, the processor 13 can perform the first stage adjustment on the display parameters used by the display 11 based on the average adjustment parameter. In this way, a more average display quality optimization can be performed for multiple types of sub-images that exist simultaneously in the target image.

FIG. 5 is a schematic diagram showing a target image having multiple types of sub-images according to an embodiment of the present invention.

Please refer to FIG. 5 , assuming that image 51 is the target image, and image 51 contains both image 501 of type A (also referred to as the first sub-image) and image 502 of type B (also referred to as the second sub-image). In this case, if the first color component information corresponding to image 51 is provided to the neural network model 101 for analysis, the neural network model 101 may determine that the type of image 51 belongs to either type A or type B. However, if the display setting corresponding to type A is simply applied to the display 11, although the display quality of image 501 of type A can be effectively improved, the display quality of image 502 of type B may be seriously affected at the same time. Alternatively, if the display setting corresponding to type B is simply applied to the display 11, although the display quality of the image 502 belonging to type B can be effectively improved, it may also seriously affect the display quality of the image 501 belonging to type A.

In one embodiment, when there are both an image 501 (i.e., the first sub-image) belonging to type A and an image 502 (i.e., the second sub-image) belonging to type B in the image 51, the processor 13 may average (or weighted average) the adjustment parameter corresponding to the image 501 (also referred to as the first sub-adjustment parameter) and the adjustment parameter corresponding to the image 502 (also referred to as the second sub-adjustment parameter) to obtain an average adjustment parameter. Then, the processor 13 may perform the first stage adjustment on the display parameter used by the display 11 according to the average adjustment parameter. In this way, although the aforementioned display quality optimization effect for a single type of image cannot be perfectly achieved, there is still a chance to optimize the display quality of multiple types of sub-images in a single target image on an average basis.

In one embodiment, the processor 13 can determine whether there are multiple reliable classification results for the target image in the classification results of the neural network model 101. For example, assume that the classification results of the target image by the neural network model 101 include multiple probability values that are higher than the decision value (e.g., 0.5). For example, these probability values higher than the decision value respectively reflect the classification results of the target image by the neural network model 101. At this time, the processor 13 can determine that there are multiple types of sub-images in the target image at the same time. At the same time, the processor 13 can determine the types of the multiple sub-images in the target image according to these probability values higher than the decision value.

In one embodiment, if there is only one reliable classification result (e.g., a probability value higher than the decision value) in the classification result of the target image by the neural network model 101, the processor 13 may determine that there are no sub-images of multiple types in the target image at the same time. For example, this single probability value higher than the decision value may reflect the classification result of the target image by the neural network model 101. In addition, the processor 13 may determine the type of the target image based on the probability value higher than the decision value.

Thus, regardless of the decision result of the neural network model 101, the processor 13 can use a corresponding adjustment strategy to adjust at least part of the display parameters of the display 11 to optimize the image display quality of the display 11. The relevant operation details have been described above and will not be repeated here.

FIG. 6 is a flow chart of a display parameter adjustment method according to an embodiment of the present invention.

Referring to FIG. 6 , in step S601, the neural network model is set to the first training mode and the first image classification training is performed on the neural network model based on the first training mode. After the first image classification training is completed, in step S602, the neural network model is set to the second training mode and the second image classification training is performed on the neural network model based on the second training mode. In particular, in the neural network model, the total number of convolutional layers participating in the second image classification training may be less than the total number of convolutional layers participating in the first image classification training.

FIG. 7 is a flow chart of a display parameter adjustment method according to an embodiment of the present invention.

Please refer to FIG. 7 . In step S701, an image is presented through a display. In step S702, first color distribution information corresponding to the image is obtained. In step S703, the first color distribution information is provided to a neural network model so as to classify the image through the neural network model. In step S704, a first stage adjustment is performed on display parameters used by the display according to the classification result of the neural network model for the image. In step S705, second color distribution information corresponding to the image is obtained according to the result of the first stage adjustment. In step S706, color compensation information is calculated according to the second color distribution information. In step S707, the display parameters used by the display are adjusted in the second stage according to the color compensation information.

However, each step in FIG. 6 and FIG. 7 has been described in detail above, and will not be repeated here. It is worth noting that each step in FIG. 6 and FIG. 7 can be implemented as multiple program codes or circuits, and the present invention is not limited thereto. In addition, the methods in FIG. 6 and FIG. 7 can be used in conjunction with the above exemplary embodiments, or can be used alone, and the present invention is not limited thereto.

In summary, the display parameter adjustment method and electronic device proposed in the embodiment of the present invention can train the neural network model by adopting different training modes (such as the first training mode and the second training mode) at different times to effectively improve the training efficiency of the neural network model. In addition, the display parameter adjustment method and electronic device proposed in the embodiment of the present invention can also use a two-stage adjustment mechanism to adjust the display parameters used by the display according to the type of image currently displayed by the display. In particular, in the first stage of adjustment, the display parameters used by the display are adjusted accordingly, mainly referring to the classification results of the neural network model on the image presented by the display. In the second stage of adjustment, the display parameters used by the monitor are further fine-tuned based on the results of the first stage of adjustment, thereby improving the defects in the image presentation quality of the monitor caused or omitted by the adjustment of the display parameters of the monitor in the first stage of adjustment. In this way, the image presentation quality of the monitor can be effectively improved, thereby improving the user's viewing experience of the images presented by the monitor.

Although the present invention has been disclosed as above by the embodiments, they are not intended to limit the present invention. Any person with ordinary knowledge in the relevant technical field can make some changes and modifications without departing from the spirit and scope of the present invention. Therefore, the protection scope of the present invention shall be defined by the scope of the attached patent application.

10: electronic device 11: display 12: storage circuit 13: processor 101: neural network model 21: input port 22: output port 23: computing core 24(1)~24(m): convolutional layer 51: image 501: image (first sub-image) 502: image (second sub-image) S601, S602, S701~S707: steps

FIG. 1 is a schematic diagram of an electronic device according to an embodiment of the present invention. FIG. 2 is a schematic diagram of a neural network model according to an embodiment of the present invention. FIG. 3 is a schematic diagram of a neural network model in a first training mode according to an embodiment of the present invention. FIG. 4 is a schematic diagram of a neural network model in a second training mode according to an embodiment of the present invention. FIG. 5 is a schematic diagram of a target image according to an embodiment of the present invention in which there are multiple types of sub-images. FIG. 6 is a flow chart of a display parameter adjustment method according to an embodiment of the present invention. FIG. 7 is a flow chart of a display parameter adjustment method according to an embodiment of the present invention.

S701~S707: Steps

Claims (16)

A display parameter adjustment method, comprising: Setting a neural network model to a first training mode and performing a first image classification training on the neural network model based on the first training mode; After completing the first image classification training, setting the neural network model to a second training mode and performing a second image classification training on the neural network model based on the second training mode, wherein in the neural network model, the total number of convolutional layers participating in the second image classification training is less than the total number of convolutional layers participating in the first image classification training; Presenting an image through a display; Obtaining first color distribution information corresponding to the image; Providing the first color distribution information to the neural network model to classify the image through the neural network model; According to the classification result of the image by the neural network model, the display parameters used by the display are adjusted in the first stage; According to the result of the first stage adjustment, the second color distribution information corresponding to the image is obtained; According to the second color distribution information, the color compensation information is calculated; and According to the color compensation information, the display parameters used by the display are adjusted in the second stage. The display parameter adjustment method as described in claim 1, wherein the steps of setting the neural network model to the second training mode and performing the second image classification training on the neural network model based on the second training mode include: Setting the first convolution layer in the neural network model to a locked state, wherein the first convolution layer participates in the first image classification training in the first training mode; and In the second training mode, only the second convolution layer in the neural network model that is not set to the locked state is subjected to the second image classification training. A display parameter adjustment method as described in claim 2, wherein the first convolution layer is farther away from the output end of the neural network model in the computing core of the neural network model than the second convolution layer. The display parameter adjustment method as described in claim 1, wherein the step of setting the neural network model to the second training mode includes: In response to the neural network model's usage of training data reaching a preset level, switching the neural network model to the second training mode. The display parameter adjustment method as described in claim 1, wherein the step of calculating the color compensation information according to the second color distribution information includes: According to the second color distribution information, grayscale vector information corresponding to multiple pixel positions in the image is obtained; According to the grayscale vector information, a grayscale expected value corresponding to the image is calculated; and According to the grayscale expected value, the color compensation information is obtained. The display parameter adjustment method as described in claim 1, wherein the step of calculating the color compensation information according to the second color distribution information includes: If the difference between the display content of the previous image in the image and the display content of the next image in the image is less than a critical value, the color compensation information for the previous image is used. The display parameter adjustment method as described in claim 1, wherein the step of performing the second-stage adjustment on the display parameter used by the display according to the color compensation information includes: If the adjustment range for the display parameter indicated by the color compensation information is greater than a critical value, in the second-stage adjustment, performing segmented adjustment on the display parameter. The display parameter adjustment method as described in claim 1, wherein the step of performing the first stage adjustment on the display parameter used by the display according to the classification result of the image by the neural network model includes: If there are multiple types of sub-images in the image, obtain an average adjustment parameter according to multiple classification results of the image by the neural network model; and Perform the first stage adjustment on the display parameter used by the display according to the average adjustment parameter. An electronic device, comprising: a display; a storage circuit for storing a neural network model; and a processor, coupled to the display and the storage circuit, wherein the processor is used to: set the neural network model to a first training mode and perform a first image classification training on the neural network model based on the first training mode; after completing the first image classification training, set the neural network model to a second training mode and perform a second image classification training on the neural network model based on the second training mode, wherein in the neural network model, the total number of convolutional layers participating in the second image classification training is less than the total number of convolutional layers participating in the first image classification training; present an image through the display; Obtaining first color distribution information corresponding to the image; Providing the first color distribution information to the neural network model to classify the image through the neural network model; Performing a first-stage adjustment on the display parameters used by the display according to the classification result of the neural network model on the image; Obtaining second color distribution information corresponding to the image according to the result of the first-stage adjustment; Calculating color compensation information according to the second color distribution information; and Performing a second-stage adjustment on the display parameters used by the display according to the color compensation information. The electronic device as described in claim 9, wherein the processor sets the neural network model to the second training mode and performs the second image classification training on the neural network model based on the second training mode, including: Setting the first convolution layer in the neural network model to a locked state, wherein the first convolution layer participates in the first image classification training in the first training mode; and In the second training mode, only the second convolution layer in the neural network model that is not set to the locked state is subjected to the second image classification training. An electronic device as described in claim 10, wherein the first convolution layer is farther away from the output end of the neural network model in the computing core of the neural network model than the second convolution layer. The electronic device as described in claim 9, wherein the operation of the processor setting the neural network model to the second training mode includes: In response to the neural network model's usage of training data reaching a preset level, switching the neural network model to the second training mode. The electronic device as described in claim 9, wherein the operation of the processor calculating the color compensation information according to the second color distribution information includes: According to the second color distribution information, grayscale vector information corresponding to multiple pixel positions in the image is obtained; According to the grayscale vector information, a grayscale expected value corresponding to the image is calculated; and According to the grayscale expected value, the color compensation information is obtained. The electronic device as described in claim 9, wherein the operation of the processor calculating the color compensation information according to the second color distribution information includes: If the difference between the display content of the previous image in the image and the display content of the next image in the image is less than a critical value, the color compensation information for the previous image is used. The electronic device as described in claim 9, wherein the processor performs the second-stage adjustment of the display parameter used by the display according to the color compensation information, including: If the adjustment range for the display parameter indicated by the color compensation information is greater than a critical value, in the second-stage adjustment, the display parameter is adjusted in stages. The electronic device as described in claim 9, wherein the processor performs the first-stage adjustment on the display parameters used by the display according to the classification result of the image by the neural network model, including: If there are multiple types of sub-images in the image, obtain an average adjustment parameter according to multiple classification results of the image by the neural network model; and Perform the first-stage adjustment on the display parameters used by the display according to the average adjustment parameter.

Images