CN116128827A - Intelligent evaluation method, device, equipment and computer readable storage medium - Google Patents

Abstract

The present invention provides a high dynamic range image evaluation method based on a deep neural network. The evaluation method includes: identifying a scene in the high dynamic range image and dividing an area of interest through a deep convolutional neural network; combining the scene identification and The high dynamic range images after the regions of interest are divided form a training data set; the training data set is shared by constructing a data analysis model and a scoring model; the data analysis is completed based on the efficient convolutional neural network of the mobile vision application Model training; the scoring model completes the scoring of the high dynamic range image according to the scene recognition information output by the data analysis model and the region of interest, and sends the collected verification set images into the trained analysis model and In the scoring model, the division of the region of interest and the conformity of the final scoring to the expectation are verified to improve the evaluation effect of the non-reference HDR image quality.

Description

An intelligent evaluation method, device, equipment and computer-readable storage medium

technical field

The present invention relates to the technical field of image quality evaluation, in particular to an evaluation method, equipment, device and computer-readable storage medium for a reference-free high dynamic range image based on a deep convolutional neural network.

Background technique

The development of high dynamic range (High Dynamic Range, HDR) imaging technology has changed the traditional way of image display, which can bring people a more realistic visual experience. However, in the process of image acquisition, compression, storage and transmission, degradation will inevitably be introduced. Image quality directly reflects the quality of user experience, and it is the common desire of image consumers to reduce or even completely eliminate degradation. Research on the quality evaluation of high dynamic range images can effectively help solve some degradation problems.

Image quality evaluation can be divided into subjective quality evaluation method and objective quality evaluation method in terms of methods. The subjective quality evaluation method is time-consuming, expensive, and difficult to operate. Therefore, it is necessary to establish a suitable objective quality evaluation model to predict image quality. The objective quality evaluation methods of high dynamic range images can be divided into high dynamic range image quality evaluation methods based on low dynamic range (Low Dynamic Range, LDR) image quality evaluation and quality evaluation methods designed for high dynamic range images. Traditional low dynamic range image quality evaluation methods, such as MSE (PSNR), SSIM, MSSIM, VIF, VSNR, etc., these methods cannot be directly used for high dynamic range image quality evaluation, because these methods are based on the assumption that the pixel value of the image It is designed under the condition that the pixel value perceived by the human eye satisfies a linear relationship, which is not true in high dynamic range images. The high dynamic range image quality evaluation method based on low dynamic range image quality evaluation needs to perform log operation or PU encoding preprocessing on the image first, so that the pixel value of the image and the pixel value perceived by the human eye roughly satisfy the linear relationship, and then use the low dynamic range Image quality evaluation method, after log operation or PU coding preprocessing, although the evaluation effect has been greatly improved, it needs to be further improved, because they cannot reflect the high dynamic range image after the dynamic range and contrast are improved. Different visual attention mechanisms for low dynamic range images.

At present, there are only several quality evaluation methods designed for high dynamic range images, typically the visual difference prediction method HDR-VDP-2 proposed by Mantiuk et al. HDR-VQM method for quality assessment design. These methods have well simulated the perception of the human eye on the high brightness range of the high dynamic range image, and have been widely used. However, these methods are full-reference high dynamic range image quality evaluation methods, which require reference images and distorted images. However, in practical applications, reference images are often unavailable or non-existent. At present, the research on the objective quality evaluation method of non-reference high dynamic range images is still relatively lacking. Therefore, accurate evaluation of the quality of reference-free high dynamic range images is an urgent problem to be solved.

An excellent objective quality evaluation method for high dynamic range images should be able to well reflect the characteristics of human visual perception. The reference objective quality evaluation methods only consider brightness distortion and ignore chromaticity distortion, which is inconsistent with human visual perception, especially for high dynamic range images with vivid colors.

The invention relates to an image quality evaluation method, aiming to solve the problem of quality evaluation of HDR images without reference to high dynamic range;

With the rapid development of mobile phone imaging technology, people have higher and higher requirements for image quality, and higher and higher requirements for image dynamic range. Of course, there are more and more methods for image quality evaluation, but how to Evaluating images is still a challenge;

HDR images can display high-brightness areas and low-brightness areas in the same image at the same time, and the exposure display range is wider, which is more in line with human visual characteristics. HDR can solve the display problem very well, making the image details richer. Therefore, to ensure that the system can provide a good visual experience, the evaluation of HDR image quality is very important.

Although the subjective evaluation of the human eye is the ultimate standard of image quality, it is too time-consuming and tedious to evaluate a large amount of image data, and there is no way to become a mainstream rapid evaluation method;

According to the dependence degree of the image objective evaluation method on the reference image, the objective quality evaluation method can be divided into three image quality evaluation methods: full reference, semi-reference and no reference. With the deepening of research, the accuracy of the full-reference quality assessment method is getting higher and higher, but this method requires a large number of undistorted reference images, which is difficult to quickly evaluate the quality of a group of images. Therefore, the no-reference image quality assessment method does not require any information of the undistorted reference image, and can evaluate the quality of the distorted image only based on the distorted image, so it has become a research hotspot in the field of machine vision and image processing.

Contents of the invention

The object of the present invention is to provide a high dynamic range image evaluation method, equipment, device, and computer-readable storage medium based on a deep neural network to improve the evaluation quality and efficiency of HDR.

In the first aspect, an embodiment of the present invention provides a high dynamic range image evaluation method based on a deep neural network, wherein the evaluation method includes: recognizing and Region of interest division; the high dynamic range image after the scene recognition and region of interest division forms a training data set; by building a data analysis model and a scoring model, sharing the training data set; through efficient convolution of mobile vision applications The neural network is used as the basis to complete the training of the data analysis model; the data analysis model analyzes the scene recognition information and extracts the region of interest according to the input data; the scoring model is output according to the data analysis model The scene recognition information and the region of interest complete the scoring of the high dynamic range image; the collected verification set images are sent to the trained analysis model and scoring model in turn, and the division of the region of interest and the final scoring and scoring are verified. expected compliance.

The beneficial effect of the code scanning device provided by the embodiment of the present invention is that the present invention aims to eliminate the influence of other images such as color and composition on the automatic evaluation of the dynamic range, compared with the manual way of scoring the dynamic range of HDR images , the use of deep learning to complete image data analysis and effect scoring can effectively improve the efficiency of image quality evaluation and reduce the cost of image evaluation. At the same time, using deep learning to build a standard scoring model can effectively avoid subjectivity that is easy to occur in the manual scoring process. Impression and effect bias greatly improve the quality and objectivity of evaluation.

Compared with the traditional deep learning model that uses one model to evaluate the dynamic range of HDR images, it is easily affected by other image features such as color, distortion, composition, etc., causing the dynamic range evaluation to be affected, resulting in the introduction of other dynamic range evaluation results. Noise elements ultimately affect the dynamic range evaluation results. At the same time, the evaluation criteria of the human eye for the dynamic range of different scenes are very different, and the evaluation of different scenes is quite different.

Compared with the above two schemes, the present invention classifies the collection of data features according to the scene, and uses the pre-data processing model to complete the extraction of data features and the acquisition of scene information; the scene information and feature information output by the pre-feature collection module As the input of the scoring model, it effectively avoids the influence of other noise information such as color and composition on the dynamic range evaluation. At the same time, because the scoring model distinguishes scene information, it is not sensitive to the scene and the evaluation is more objective.

In the evaluation method provided in a further embodiment, the high-efficiency convolutional neural network of the mobile vision application is used as a basis to complete the training of the data analysis model, including: introducing the attention module into the data analysis model, and using hard_s i gmo id replaces the ReLU function as an activation function, and the attention module captures the connection between the global information features and local information features of the high dynamic range images in the training data set.

In the evaluation method provided in a further embodiment, the attention module consists of a first convolutional layer, a second convolutional layer, a pooling layer, a normalization layer, a hard_s igmo id activation function, and a third convolutional layer layer composition.

In the evaluation method provided in some embodiments, the scoring model is consistent with the data analysis model.

The evaluation method provided in some other embodiments includes: constructing an initial training set, the initial training set includes the high dynamic range images of different exposure levels collected by different smart mobile terminals; Regions of interest in range images are scored and marked separately, which cannot be reproduced.

The evaluation method provided in some other embodiments includes: constructing a data analysis model by using the mobileNet V3 model, and training the initial training set.

In the evaluation method provided in some optional embodiments, the scoring model is constructed through the mobileNet V3 model, and the initial training set is trained.

In a possible implementation solution, the second aspect of the present invention provides a high dynamic range image evaluation device based on a deep neural network, including:

The training data set includes the high dynamic range images of different exposure levels collected by different smart mobile terminals; the data analysis module is used for scene recognition and interest area division in the high dynamic range images;

A scoring module that completes the scoring of the high dynamic range image for the scene identification information and the region of interest output according to the data analysis model;

The data set is shared by the data analysis model and the scoring model with a comprehensive data verification module, collecting images of the verification set, sending the images of the verification set into the trained analysis model and scoring model in turn, and checking the area of interest. The degree of conformity between division and final scoring and expectations.

In a possible implementation solution, the third aspect of the present invention provides an intelligent evaluation device, which is characterized in that it includes a processor, a memory, and a computer program stored in the memory, and the processor executes the A computer program to realize the evaluation method according to any one of claims 1 to 7.

In a fourth aspect, an embodiment of the present invention also provides a computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any one of claims 1-7 is implemented. The steps of the evaluation method.

Description of drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings that need to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For Those of ordinary skill in the art can also obtain other drawings based on these drawings without making creative efforts.

FIG. 1 is a schematic flow diagram of a dual-model HDR analysis and scoring model provided by an embodiment of the present invention;

Fig. 2 is a schematic diagram of a block diagram of a Squeeze-and-excitati on model provided by an embodiment of the present invention;

Fig. 3 is a schematic diagram of an evaluation device module provided by an embodiment of the present invention;

Fig. 4 is a schematic diagram of dividing a daytime region of interest according to an embodiment of the present invention;

Fig. 5 is a schematic diagram of dividing an area of interest at night according to an embodiment of the present invention;

FIG. 6 is a schematic diagram of an electronic device provided by an embodiment of the present invention;

Fig. 7 is a schematic diagram of a hardware configuration block diagram of an evaluation device provided by an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be described below with reference to the drawings in the embodiments of the present invention. Wherein, in the description of the embodiments of the present invention, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to limit the application. As used in the specification and appended claims of this application, the singular expressions "a", "the", "above", "the" and "this" are intended to also include, for example, "a or more" unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of the present application, "at least one" and "one or more" refer to one or more than two (including two). The term "and/or" is used to describe the association relationship of associated objects, indicating that there may be three types of relationships; for example, A and/or B may indicate: A exists alone, A and B exist simultaneously, and B exists alone, Wherein A and B can be singular or plural. The character "/" generally indicates that the contextual objects are an "or" relationship.

Reference to "one embodiment" or "some embodiments" or the like in this specification means that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the present application. Thus, appearances of the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in other embodiments," etc. in various places in this specification are not necessarily All refer to the same embodiment, but mean "one or more but not all embodiments" unless specifically stated otherwise. The terms "including", "comprising", "having" and variations thereof mean "including but not limited to", unless specifically stated otherwise. The term "connected" includes both direct and indirect connections, unless otherwise stated. "First" and "second" are used for descriptive purposes only, and should not be understood as indicating or implying relative importance or implicitly specifying the quantity of indicated technical features.

In the embodiments of the present invention, words such as "exemplarily" or "for example" are used as examples, illustrations or descriptions. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of the present invention shall not be interpreted as being more preferred or more advantageous than other embodiments or design solutions. Rather, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a concrete manner.

The evaluation method provided in the embodiment of the present invention, as shown in Figure 1, is a high dynamic range image evaluation method based on a deep neural network. Scene recognition and interest region division; the high dynamic range image after the scene recognition and interest region division forms a training data set; by building a data analysis model and a scoring model, share the training data set; through mobile vision applications The high-efficiency convolutional neural network is used as the basis to complete the training of the data analysis model; the data analysis model analyzes the scene recognition information and extracts the region of interest according to the input data; the scoring model is based on the The scene identification information and the region of interest output by the data analysis model complete the scoring of the high dynamic range image.

In the implementation example of the optional evaluation method, since the HDR image focuses on the evaluation of the dynamic range of the image, it is easy to introduce the influence of color, distortion, composition and other factors on the evaluation of the dynamic range for the method of using the whole image for training. The scene information and ROI area generated by the pattern recognition and ROI division model are used as input, which can effectively avoid the influence of irrelevant image elements other than the ROI area on the dynamic range evaluation.

Using artificial methods to subjectively evaluate distorted image quality has the disadvantages of low efficiency and high cost, and it is difficult to evaluate image quality accurately, real-time, and efficiently. Therefore, it is an inevitable trend to study subjective evaluation algorithms for objective images. With the development of artificial intelligence, it is possible to use computers to simulate the perception process of human visual system.

The biggest advantage of the deep neural network is that it integrates the image feature extraction and regression process in an optimized framework, and truly realizes end-to-end learning. Because humans have different evaluation standards for high dynamic images in different scenes, and for high dynamic images, human concerns and areas are also different; therefore, Mobil eNet V3 is used to complete image scene recognition and ROI area division , and iteratively use the Mobil eNet V3 model to train the image scoring model.

The training model of the region of interest is shown in Figure 2; firstly, the 1*1 convolutional layer is used for dimension enhancement processing, and after the convolution is completed, there will be batch normalization (Batch Normalization, BN) and hard_s i gmo i d activation function; the second layer is a 3*3 Depthwise (DW) convolution, and the BN normalization layer and hard_s i gmo i d activation function are still connected after the convolution layer; the last convolution layer is 1* The convolutional layer of 1 plays a role in dimensionality reduction. Finally, the neural network outputs high and low light ROI areas and scene types. The scene types are divided into two categories according to day and night, and different scene information is distinguished according to pattern recognition, such as: sky, buildings, green plants, people, etc.

As shown in Figure 2, 100-training set image; 200-1*1 convolutional layer; 300-3*3Depthwise (DW) convolutional layer; 400-pooling layer; 500-batch normalization (Batch Norma l i ze, BN); 600-hard_s i gmo i d activation function; 700-1*1 convolutional layer for dimensionality reduction; 800-Squeeze-and-exc i tat i on (SE) module.

As shown in Figure 4, the R box is the highlight area identified by the model, and the B box is the dark area identified by the model. After the scene recognition is completed, the scoring of the corresponding scene and the HDR effect of the ROI area is manually completed. From then on, the data preparation for the automatic scoring function of HDR images can be completed.

As shown in Figure 5, the B box is the dark area identified by the above model, and the R box is the highlight area identified by the model one by one. After the scene recognition is completed, the scoring of the corresponding scene and the HDR effect of the ROI area is manually completed. From then on, the data preparation for the automatic scoring function of HDR images can be completed.

Training set image collection method: Use different grades of shooting equipment to collect images with different exposure levels to ensure the diversity of the environment in the data set.

Manual marking stage: select 10 persons who divide and divide the area of interest. In order to obtain a more comprehensive and objective evaluation, the composition of the personnel includes: 3 image review professionals, 3 photographers, 2 image algorithm related personnel, 2 social recruiters. Manual scoring adopts the censored mean method, that is, removes the highest score and the lowest score and calculates the average.

The scoring model structure also uses Mob i l eNet V3, as shown in Figure 2, and can share the training set with the data analysis model;

After the model training is completed, collect the verification set images, send the verification set images to the trained analysis model and scoring model in turn, and check whether the division of the region of interest and the final scoring meet expectations;

In some optional embodiments, a deep convolutional neural network (Deep convolutional neural networks, DCNN) is used for scene recognition and region-of-interest division. Since the human eye perceives the dynamic range of different images differently in different scenes, by using the DCNN network to identify the scene information and the region of interest (ROI), the input of the evaluation model is more specific, and the image on the image is excluded. Other effects such as color, composition, etc. on the automatic evaluation of dynamic range.

In other embodiments, the initial training data set is used in combination; the high light and low light areas on the image are divided and marked by manual labeling to form a data set for dividing the region of interest, and the scoring is completed by manually scoring the HDR image The establishment of the data set; the two can share the image data source. It is worth mentioning that the manual labeling method is unique and cannot be copied.

In some other embodiments, the data analysis model and the scoring model are independent of each other, but they complement each other. The data analysis model completes the analysis of the scene information and the extraction of the ROI area according to the input data; while the scoring model completes the scoring of the image according to the ROI and scene information output by the data analysis model. The two can share the training data set and can be trained at the same time , do not interfere with each other.

In an optional embodiment, use MobileNetV3 as a basis to complete data analysis and scoring model training. Among them, Mob i l eNetV3 introduces the SE (Squeeze-and-exci tat i on) module, as shown in Figure 2, and uses hard_s igmo i d to replace the ReLU function as the activation function. The SE module is similar to an attention module, which can flexibly capture the connection between global information and local information. Its purpose is to let the model obtain the target area that needs to be focused on, and put more weight on this part, highlight the salient and useful features, suppress and ignore irrelevant features, and SE mainly has the following steps:

Global average pooling, which compresses the corresponding spatial information on each channel of mostly data obtained after the first layer of convolution to a constant of the corresponding channel. At this time, a pixel represents a channel and becomes a vector; The vector obtained in the previous step is input into two fully connected layers, and the weight value is obtained through the hard_s i gmo i d activation function; the feature is operated according to the weight table obtained in the previous step to complete the output of the feature emphasis degree;

In an optional embodiment evaluation device, as shown in Figure 3, a high dynamic range image evaluation device 1000 based on a deep neural network is characterized in that it includes:

The training data set 100 includes the high dynamic range images of different exposure levels collected by different smart mobile terminals; the data analysis module 120 is used for scene recognition and region-of-interest division in the high dynamic range images;

The scoring module 130 is configured to score the high dynamic range image for the scene identification information and the region of interest output according to the data analysis model;

The data set is shared by the data analysis model and the scoring model;

The comprehensive data verification module 150 collects images of the verification set, sends the images of the verification set to the trained analysis model and scoring model in turn, and checks whether the division of the region of interest and the final scoring are consistent with expectations.

Fig. 6 is a schematic diagram of an example of an intelligent testing device according to an embodiment of the present application. As shown in FIG. 6 , the electronic device 900 of this embodiment includes: a processor 910 , a memory 920 , and a computer program 930 stored in the memory 920 and operable on the processor 910 . When the processor 910 executes the computer program 930, the steps in the foregoing evaluation method embodiments are realized.

In some embodiments, FIG. 7 shows a hardware configuration block diagram of the intelligent evaluation device 30 . The intelligent code scanning device 30 includes a tuner and demodulator 310, a mobile communication module 320, a wireless communication module 330, a collector 340, an external device interface 350, a controller 360, a display 370, an audio output interface 380, a memory, a power supply, and a user interface at least one of the

In some other embodiments, the tuner and demodulator 310 senses electromagnetic waves through the antenna, converts the sensed electromagnetic waves into electrical signals, and then processes and converts them into electrical signals, and finally converts them into sounds, such as receiving broadcast signals through wireless reception, And demodulate the audio signal from the broadcast signal.

The mobile communication module 320 can provide wireless communication solutions including 2G/3G/4G/5G applied on the smart code scanning device 30 . The mobile communication module 320 may include at least one filter, switch, power amplifier, low noise amplifier (low noise amplifier, LNA) and the like. The mobile communication module 320 can receive electromagnetic waves through the antenna, filter and amplify the received electromagnetic waves, and send them to the tuner-demodulator 310 for demodulation. The mobile communication module 320 can also amplify the signal modulated by the tuner and demodulator 310, and convert it into electromagnetic wave and radiate it through the antenna. In some embodiments, at least part of the functional modules of the mobile communication module 320 may be set in the controller 360 . In some embodiments, at least part of the functional modules of the mobile communication module 320 and at least part of the modules of the controller 360 may be set in the same device.

The wireless communication module 330 can provide wireless local area networks (wireless local area networks, WLAN) (such as wireless fidelity (Wireless fidelity, Wi-Fi) network), Bluetooth (bluetooth, BT), global Wireless communication solutions such as global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared technology (IR). The wireless communication module 330 may be one or more devices integrating at least one communication processing module. The wireless communication module 330 receives electromagnetic waves via the antenna, frequency-modulates and filters the electromagnetic wave signals, and sends the processed signals to the controller 360 . The wireless communication module 330 can also receive the signal to be transmitted from the controller 360, frequency-modulate it, amplify it, and convert it into electromagnetic wave and radiate it through the antenna.

In some other embodiments, the collector 340 is used to collect external environment or signals interacting with the outside. For example, the collector 340 includes a light receiver, which is a sensor for collecting ambient light intensity; or, the collector 340 includes an image collector, such as a camera, which can be used to collect external environmental scenes, user attributes or user interaction gestures, or , the collector 340 includes a sound collector, such as a microphone, for receiving external sound.

In some other embodiments, the external device interface 350 may include but not limited to the following: high-definition multimedia interface (HDMI), analog or data high-definition component input interface (component), composite video input interface (CVBS), USB input interface (USB ), RGB port, etc. any one or more interfaces. It may also be a composite input/output interface formed by the above-mentioned multiple interfaces.

In some other embodiments, the controller 360 and the tuner-demodulator 310 may be located in different split devices, that is, the tuner-demodulator 310 may also be located in an external device of the main device where the controller 360 is located, such as an external Set-top boxes, etc.

In some other embodiments, the controller 360 controls the work of the display device and responds to user operations through various software control programs stored in the memory. The controller 360 controls the overall operation of the smart code scanning device 30 . For example, in response to receiving a user command for selecting a UI object to be displayed on the display 370, the controller 360 may perform an operation related to the object selected by the user command.

In some possible embodiments, the controller 360 includes a central processing unit (central processing unit, CPU), a video processor, an audio processor, a graphics processing unit (graphics processing unit, GPU), RAM, ROM, for input/output At least one of the first interface to the nth interface, a communication bus (Bus) and the like.

The central processing unit is used to execute operating system and application program instructions stored in the memory, and to execute various application programs, data and content according to various interactive instructions received from external inputs, so as to finally display and play various audio and video content . A central processing unit may include multiple processors. For example, including a main processor and one or more sub-processors.

In some embodiments, the graphics processor is configured to generate various graphic objects, such as at least one of icons, operation menus, and user input instruction display graphics. The graphics processor includes an arithmetic unit, which performs calculations by receiving various interactive instructions input by users, and displays various objects according to display attributes; it also includes a renderer, which renders various objects obtained based on the arithmetic unit, and the above-mentioned rendered objects are used to be displayed on the display.

In some embodiments, the video processor is used to receive an external video signal and perform video decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, image synthesis, etc. according to the standard codec protocol of the input signal. At least one of the processing can obtain the signal displayed or played directly on the intelligent code scanning device 30 .

In some embodiments, the video processor includes at least one of a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, a display formatting module, and the like. Wherein, the demultiplexing module is used for demultiplexing the input audio and video data streams. The video decoding module is used to process the demultiplexed video signal, including decoding and scaling. An image compositing module, such as an image compositor, is used to superimpose and mix the graphics generator with the video image after zooming processing, according to the user input or the GUI signal generated by itself, so as to generate an image signal available for display. The frame rate conversion module is used for converting the input video frame rate. The display formatting module is used to convert the received frame rate to the video output signal, and change the signal to conform to the display format signal, such as outputting RGB data signal.

In some embodiments, the audio processor is used to receive an external audio signal, perform decompression and decoding according to the standard codec protocol of the input signal, and perform at least one of processing such as noise reduction, digital-to-analog conversion, and amplification processing , to obtain a sound signal that can be played on a loudspeaker.

In some embodiments, the user can input user commands on the graphical user interface displayed on the display 370, and the user input interface receives user input commands through the graphical user interface. Alternatively, the user may input a user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through a sensor to receive the user input command.

In some embodiments, "user interface" is a medium interface for interaction and information exchange between an application program or an operating system and a user, and it realizes the conversion between the internal form of information and the form acceptable to the user. Graphical user interface refers to the user interface related to computer operation displayed in a graphical way. It can be an icon, window, control and other interface elements displayed on the display screen of an electronic device, where the control can include icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, etc. At least one of the interface elements.

In some embodiments, the display 370 includes a display screen component for presenting images, and a drive component for driving image display, for receiving image signals output from the controller, and displaying video content, image content, and menu manipulation interface Components and user interface etc.

In some other embodiments, the display 370 may be at least one of a liquid crystal display, an organic light emitting diode (OLED) display, and a projection display, and may also be a projection device and a projection screen.

In still other embodiments, the audio output interface 380 includes speakers, external audio output electronics, and the like.

In some embodiments, the user interface is an interface that can be used to receive control input (such as: physical buttons on the display device body, or others).

During specific implementation, the above-mentioned intelligent code scanning device 30 can be a mobile phone, a tablet computer, a handheld computer, a personal computer (personal computer, PC), a cellular phone, a personal digital assistant (personal digital assistant, PDA), a wearable device ( Such as smart watches), smart home equipment (such as televisions), vehicle-mounted computers, game consoles, and augmented reality (augmented reality, AR)\virtual reality (virtual reality, VR) equipment and other electronic products that include cameras. The specific device form of the smart code scanning device 30 is not particularly limited.

Through the description of the above embodiments, those skilled in the art can clearly understand that for the convenience and brevity of the description, only the division of the above-mentioned functional modules is used as an example for illustration. In practical applications, the above-mentioned functions can be allocated according to needs It is completed by different functional modules, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. For the specific working process of the above-described system, device, and unit, reference may be made to the corresponding process in the foregoing method embodiments, and details are not repeated here.

Each functional unit in each embodiment of the embodiment of the present invention may be integrated into one processing unit, or each unit may physically exist separately, or two or more units may be integrated into one unit. The above-mentioned integrated units can be implemented in the form of hardware or in the form of software functional units.

If the integrated unit is realized in the form of a software function unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage The medium includes several instructions to enable a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: flash memory, mobile hard disk, read-only memory, random access memory, magnetic disk or optical disk, and other various media capable of storing program codes.

The above is only a specific implementation of the embodiment of the present invention, but the scope of protection of the embodiment of the present invention is not limited thereto, and any changes or replacements within the technical scope disclosed in the embodiment of the present invention shall be covered by this Within the protection scope of the invention embodiment. Therefore, the protection scope of the embodiments of the present invention should be determined by the protection scope of the claims.

Claims (10)

1. a high dynamic range image evaluation method based on deep neural network, is characterized in that, described evaluation method comprises: Scene recognition and region-of-interest division in the high dynamic range image through a deep volume neural network; The high dynamic range image after the scene recognition and the region of interest is divided into a training data set; Sharing the training data set by building a data analysis model and a scoring model; The data analysis model training is completed by using the efficient convolutional neural network for mobile vision applications as a basis; The data analysis model analyzes the scene recognition information and extracts the region of interest according to the input data; The scoring model completes the scoring of the high dynamic range image according to the scene recognition information output by the data analysis model and the region of interest; Send the collected verification set images into the trained analysis model and scoring model in turn, and verify the division of the region of interest and the degree of conformity of the final scoring with expectations. 2. The evaluation method according to claim 1, wherein, the efficient convolutional neural network of the mobile vision application is used as a basis to complete the training of the data analysis model, comprising: introducing the attention module into the data analysis model , and use hard_sigmoid to replace the ReLU function as an activation function, and the attention module captures the connection between the global information feature and the local information feature of the high dynamic range image in the training data set. 3. The evaluation method according to claim 2, wherein the attention module consists of a first convolutional layer, a second convolutional layer, a pooling layer, a normalization layer, a hard_sigmoid activation function, a third volume Layered composition. 4. The evaluation method according to claim 2, wherein the scoring model is consistent with the data analysis model. 5. The evaluation method according to claim 1, comprising: constructing an initial training set, the initial training set comprising the high dynamic range images of different exposure levels collected by different smart mobile terminals; The training set is marked with non-reproducible scores for the regions of interest of the high dynamic range images. 6. evaluation method as claimed in claim 1, is characterized in that, comprises: by using mobileNet V3 model to construct data analysis model, described initial training set is trained. 7. The evaluation method according to claim 6, wherein the scoring model is constructed by the mobileNet V3 model, and the initial training set is trained. 8. A high dynamic range image evaluation device based on a deep neural network, characterized in that it comprises: The training data set includes the high dynamic range images of different exposure levels collected by different smart mobile terminals; the data analysis module is used for scene recognition and interest area division in the high dynamic range images; A scoring module that completes the scoring of the high dynamic range image for the scene identification information and the region of interest output according to the data analysis model; The data set is shared by the data analysis model and the scoring model; The integrated data verification module collects the images of the verification set, sends the images of the verification set to the trained analysis model and scoring model in turn, and checks the division of the region of interest and the degree of conformity between the final scoring and the expectation. 9. A kind of intelligent evaluation equipment as claimed in claim 7, is characterized in that, comprises processor, memory and the computer program that is stored in described memory, described processor executes described computer program, to realize as claimed in claim The evaluation method described in any one of 1 to 7. 10. A computer-readable storage medium, the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the evaluation method according to any one of claims 1-7 are realized.

Images