HK40081243B - Image processing method and electronic device - Google Patents

Description

Image processing methods and electronic devices

Technical Field

This application relates to the field of image processing, specifically to an image processing method and an electronic device.

Background Technology

Three-dimensional lookup tables (3D LUTs) are widely used in the field of image processing. Typically, by loading 3D LUTs onto electronic devices, color correction or image style adjustment can be performed on images to achieve different image effects.

To implement 3D LUT functionality in electronic devices, it is necessary to load color lookup tables and perform three-dimensional interpolation calculations. A color lookup table is a three-dimensional mapping table between discrete original color data and corrected color data. However, implementing 3D LUT functionality places high demands on the storage size and computing power of electronic devices. When storage size is limited, or when there are performance and power consumption constraints, electronic devices cannot meet the requirements for using 3D LUTs.

Therefore, given the limitations of electronic devices' performance, how to process images to achieve color correction or style adjustment has become an urgent problem to be solved.

Summary of the Invention

This application provides an image processing method and an electronic device, which processes the image to be processed by using a second mapping in a two-dimensional lookup table and a first mapping in gamma processing, thereby effectively reducing the computational load of the electronic device.

Firstly, an image processing method is provided, including:

In the first shooting mode, a first interface is displayed, the first interface including a first control; a first operation on the first control is detected; in response to the first operation, a first mapping in gamma processing and a second mapping in a two-dimensional lookup table are determined, the first mapping corresponds to the first control, the second mapping corresponds to the first control, and the first mapping and the second mapping are associated; an image to be processed is acquired; the image to be processed is processed according to the first mapping and the second mapping to obtain a first image.

It should be understood that the second mapping in a two-dimensional lookup table can refer to a functional relationship, through which the color information of each pixel can be repositioned to obtain new color information. The second mapping in the two-dimensional lookup table can determine an image mode; this image mode can achieve the effects of different filters in a camera.

In embodiments of this application, the electronic device, in response to a detected first operation, can determine a first mapping in gamma processing and a second mapping in a two-dimensional lookup table; based on the first mapping and the second mapping, the image to be processed is processed to obtain a first image; since the embodiments of this application use the second mapping in the two-dimensional lookup table and the first mapping in gamma processing to process the image to be processed, wherein the first mapping is associated with the second mapping; therefore, compared with processing the image to be processed through a three-dimensional lookup table, the image processing method of this application can effectively reduce the computational load of the electronic device.

In conjunction with the first aspect, in some implementations of the first aspect, the first shooting mode refers to a shooting mode in which the frame rate of the image is greater than a preset threshold.

In one possible implementation, the first shooting mode could refer to a shooting mode with a higher frame rate; for example, it could refer to a slow-motion shooting mode in a camera application.

In conjunction with the first aspect, in some implementations of the first aspect, the first control refers to a control used to indicate the second mapping table in the two-dimensional lookup table.

In one possible implementation, the first interface may refer to the shooting preview interface, which includes an image mode preview box. The image mode preview box may include multiple different image modes, which may refer to the filter effect of the image. Detecting the first operation on the first control may refer to detecting the operation of the user selecting the target style in the image mode preview box. An image mode may correspond to a mapping of a two-dimensional lookup table, that is, an image mode may correspond to a target two-dimensional lookup table.

In conjunction with the first aspect, in some implementations of the first aspect, the first control refers to a second mapping used to instruct automatic identification of the two-dimensional lookup table.

In one possible implementation, the first interface can be the settings interface of the camera application, and the first operation can refer to the user enabling the automatic image recognition mode in the settings interface; after the automatic image recognition mode function is enabled, the camera can automatically identify and determine the target style according to the shooting scene, that is, it can automatically determine the second mapping in the two-dimensional lookup table according to the shooting scene.

In conjunction with the first aspect, in some implementations of the first aspect, processing the image to be processed according to the first mapping and the second mapping to obtain a first image includes: processing the image to be processed according to the first mapping to obtain a second image;

A first adjustment amount and a second adjustment amount are determined based on the second image and the second mapping, wherein the first adjustment amount is used to represent the adjustment amount of the first color component in the two images, and the second adjustment amount is used to represent the adjustment amount of the second color component in the second image;

The first image is obtained based on the second image, the first adjustment amount, and the second adjustment amount.

In conjunction with the first aspect, in some implementations of the first aspect, the first mapping and the second mapping are associated to indicate the determination of the second mapping in the two-dimensional lookup table based on the first mapping in the gamma processing.

In embodiments of this application, the first mapping in the gamma processing is associated with the second mapping in the two-dimensional lookup table; that is, the second mapping in the two-dimensional lookup table can be performed based on the first mapping in the gamma processing.

In conjunction with the first aspect, in certain implementations of the first aspect, determining the second mapping in the two-dimensional lookup table based on the first mapping in the gamma processing includes:

Obtain the third mapping in the gamma processing and the fourth mapping in the three-dimensional lookup table, wherein the fourth mapping in the three-dimensional lookup table corresponds to the second mapping in the two-dimensional lookup table;

The first color space is uniformly divided to obtain an image in the initial first color space and an image in the initial second color space;

The image in the initial second color space is processed according to the first mapping of the gamma processing, the third mapping of the gamma processing, and the fourth mapping in the three-dimensional lookup table to obtain a third image, which is an image in the second color space.

Convert the third image into a fourth image in the first color space;

The second mapping in the two-dimensional lookup table is determined based on the pixel difference between the image in the initial first color space and the fourth image in the first color space.

In conjunction with the first aspect, in some implementations of the first aspect, determining the second mapping in the two-dimensional lookup table based on the pixel difference between the image in the initial first color space and the third image in the first color space includes:

Determine the first brightness value;

When the brightness is the first brightness value, the second mapping in the two-dimensional lookup table is determined based on the pixel difference between the image in the initial first color space and the fourth image in the first color space.

It should be understood that excessively bright or dark brightness values will introduce more image noise into the image, so excessively bright or dark brightness values should be avoided when determining the first brightness value.

In conjunction with the first aspect, in some implementations of the first aspect, the first color space refers to the HSL color space or the HSV color space; the second color space refers to the RGB color space.

In a second aspect, an electronic device is provided, comprising: one or more processors, a memory, and a display screen; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, the one or more processors invoking the computer instructions to cause the electronic device to perform: in a first shooting mode, displaying a first interface, the first interface including a first control; detecting a first operation on the first control; in response to the first operation, determining a first mapping in gamma processing and a second mapping in a two-dimensional lookup table, the first mapping corresponding to the first control, the second mapping corresponding to the first control, and the first mapping and the second mapping being associated; acquiring an image to be processed; and processing the image to be processed according to the first mapping and the second mapping to obtain a first image.

In conjunction with the second aspect, in some implementations of the second aspect, the one or more processors invoke the computer instructions to cause the electronic device to execute:

The image to be processed is processed according to the first mapping to obtain the second image;

A first adjustment amount and a second adjustment amount are determined based on the second image and the second mapping, wherein the first adjustment amount is used to represent the adjustment amount of the first color component in the two images, and the second adjustment amount is used to represent the adjustment amount of the second color component in the second image;

The first image is obtained based on the second image, the first adjustment amount, and the second adjustment amount.

In conjunction with the second aspect, in some implementations of the second aspect, the first mapping and the second mapping are associated to indicate the determination of the second mapping in the two-dimensional lookup table based on the first mapping in the gamma processing.

In conjunction with the second aspect, in some implementations of the second aspect, the one or more processors invoke the computer instructions to cause the electronic device to execute:

Obtain the third mapping in the gamma processing and the fourth mapping in the three-dimensional lookup table, wherein the fourth mapping in the three-dimensional lookup table corresponds to the second mapping in the two-dimensional lookup table;

The first color space is uniformly divided to obtain an image in the initial first color space and an image in the initial second color space;

The image in the initial second color space is processed according to the first mapping of the gamma processing, the third mapping of the gamma processing, and the fourth mapping in the three-dimensional lookup table to obtain a third image, which is an image in the second color space.

Convert the third image into a fourth image in the first color space;

The second mapping in the two-dimensional lookup table is determined based on the pixel difference between the image in the initial first color space and the fourth image in the first color space.

In conjunction with the second aspect, in some implementations of the second aspect, the one or more processors invoke the computer instructions to cause the electronic device to execute:

Determine the first brightness value;

When the brightness is the first brightness value, the second mapping in the two-dimensional lookup table is determined based on the pixel difference between the image in the initial first color space and the fourth image in the first color space.

In conjunction with the second aspect, in some implementations of the second aspect, the first color space refers to the HSL color space or the HSV color space; the second color space refers to the RGB color space.

In conjunction with the second aspect, in some implementations of the second aspect, the first shooting mode refers to a shooting mode in which the frame rate of the image is greater than a preset threshold.

In conjunction with the second aspect, in some implementations of the second aspect, the first control refers to a control used to indicate the second mapping table in the two-dimensional lookup table.

In conjunction with the second aspect, in some implementations of the second aspect, the first control refers to a control used to instruct the automatic identification of the second mapping table in the two-dimensional lookup table.

It should be understood that the extensions, limitations, explanations and descriptions of the relevant content in the first aspect above also apply to the same content in the second aspect.

Thirdly, an electronic device is provided, comprising: one or more processors, a memory, and a display screen; the memory is coupled to the one or more processors, the memory being used to store computer program code, the computer program code including computer instructions, wherein the one or more processors invoke the computer instructions to cause the electronic device to perform any of the image processing methods of the first aspect.

Fourthly, a chip system is provided, the chip system being applied to an electronic device, the chip system including one or more processors, the processors being configured to invoke computer instructions to cause the electronic device to perform any of the image processing methods in the first aspect.

Fifthly, a computer-readable storage medium is provided, the computer-readable storage medium storing computer program code, which, when executed by an electronic device, causes the electronic device to perform any of the image processing methods in the first aspect.

In a sixth aspect, a computer program product is provided, the computer program product comprising: computer program code, which, when executed by an electronic device, causes the electronic device to perform any of the image processing methods in the first aspect.

In embodiments of this application, the electronic device, in response to a detected first operation, can determine a first mapping in gamma processing and a second mapping in a two-dimensional lookup table; based on the first mapping and the second mapping, the image to be processed is processed to obtain a first image; since the embodiments of this application use the second mapping in the two-dimensional lookup table and the first mapping in gamma processing to process the image to be processed, wherein the first mapping is associated with the second mapping; therefore, compared with processing the image to be processed through a three-dimensional lookup table, the image processing method of this application can effectively reduce the computational load of the electronic device.

Attached Figure Description

Figure 1 is a schematic diagram of a hardware system applicable to the device of this application;

Figure 2 is a schematic diagram of a software system applicable to the apparatus of this application;

Figure 3 is a schematic diagram of the application scenario provided in the embodiments of this application;

Figure 4 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 5 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 6 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 7 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 8 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 9 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 10 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 11 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 12 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 13 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 14 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 15 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 16 is a schematic diagram of the processing flow of the image processor provided in an embodiment of this application;

Figure 17 is a schematic diagram of the method for determining a two-dimensional lookup table provided in an embodiment of this application;

Figure 18 is a schematic diagram of an image processing method provided in an embodiment of this application;

Figure 19 is a schematic diagram of an image processing method provided in an embodiment of this application;

Figure 20 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 21 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 22 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 23 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 24 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 25 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 26 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 27 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 28 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 29 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 30 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 31 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 32 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 33 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 34 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 35 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 36 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 37 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 38 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 39 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 40 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 41 is a schematic diagram of the display interface of an electronic device provided in an embodiment of this application;

Figure 42 is a schematic diagram of an image processing method provided in an embodiment of this application;

Figure 43 is a schematic diagram of an image processing apparatus provided in an embodiment of this application;

Figure 44 is a schematic diagram of an electronic device provided in an embodiment of this application.

Detailed Implementation

The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

Figure 1 illustrates a hardware system for an electronic device applicable to this application.

Electronic device 100 can be a mobile phone, smart screen, tablet computer, wearable electronic device, in-vehicle electronic device, augmented reality (AR) device, virtual reality (VR) device, laptop computer, ultra-mobile personal computer (UMPC), netbook, personal digital assistant (PDA), projector, etc. This application embodiment does not limit the specific type of electronic device 100.

Electronic device 100 may include processor 110, external memory interface 120, internal memory 121, universal serial bus (USB) interface 130, charging management module 140, power management module 141, battery 142, antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, sensor module 180, button 190, motor 191, indicator 192, camera 193, display screen 194, and subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

It should be noted that the structure shown in FIG1 does not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than those shown in FIG1, or the electronic device 100 may include a combination of some of the components shown in FIG1, or the electronic device 100 may include sub-components of some of the components shown in FIG1. The components shown in FIG1 may be implemented in hardware, software, or a combination of software and hardware.

Processor 110 may include one or more processing units. For example, processor 110 may include at least one of the following processing units: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and neural network processing unit (NPU). The different processing units may be independent devices or integrated devices.

The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

In some embodiments, processor 110 may include one or more interfaces. For example, processor 110 may include at least one of the following interfaces: an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver/transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input/output (GPIO) interface, a SIM interface, and a USB interface.

The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple I2C buses. The processor 110 can couple to the touch sensor 180K, charger, flash, camera 193, etc., through different I2C bus interfaces. For example, the processor 110 can couple to the touch sensor 180K through the I2C interface, enabling the processor 110 and the touch sensor 180K to communicate through the I2C bus interface, thereby realizing the touch function of the electronic device 100.

The I2S interface can be used for audio communication. In some embodiments, the processor 110 may include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to enable communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the I2S interface to enable the function of answering phone calls through a Bluetooth headset.

The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via the PCM interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface, enabling the function of answering phone calls through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 170 can transmit audio signals to the wireless communication module 160 via the UART interface to enable music playback through Bluetooth headphones.

The MIPI interface can be used to connect the processor 110 to peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to enable the electronic device 100 to capture images. The processor 110 and the display screen 194 communicate via the DSI interface to enable the electronic device 100 to display images.

The GPIO interface can be configured via software. It can be configured as a control signal interface or a data signal interface. In some embodiments, the GPIO interface can be used to connect the processor 110 to the camera 193, display screen 194, wireless communication module 160, audio module 170, and sensor module 180. The GPIO interface can also be configured as an I2C interface, I2S interface, UART interface, or MIPI interface.

USB interface 130 is an interface compliant with USB standards, such as a Mini USB interface, a Micro USB interface, or a USB Type-C interface. USB interface 130 can be used to connect a charger to charge electronic device 100, to transfer data between electronic device 100 and peripheral devices, and to connect headphones for audio playback. USB interface 130 can also be used to connect other electronic devices 100, such as AR devices.

The connection relationships between the modules shown in Figure 1 are merely illustrative and do not constitute a limitation on the connection relationships between the modules of the electronic device 100. Optionally, the modules of the electronic device 100 may also adopt a combination of various connection methods described in the above embodiments.

The charging management module 140 receives power from a charger, which can be either a wireless or wired charger. In some wired charging embodiments, the charging management module 140 receives current from the wired charger via the USB interface 130. In some wireless charging embodiments, the charging management module 140 receives electromagnetic waves (current path shown as dashed lines) via the wireless charging coil of the electronic device 100. While charging the battery 142, the charging management module 140 can also supply power to the electronic device 100 via the power management module 141.

The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and/or the charging management module 140, providing power to the processor 110, internal memory 121, display screen 194, camera 193, and wireless communication module 160, etc. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (e.g., leakage current, impedance). Optionally, the power management module 141 can be located within the processor 110, or the power management module 141 and the charging management module 140 can be located in the same device.

The wireless communication function of electronic device 100 can be realized through devices such as antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor and baseband processor.

Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with tuning switches.

The mobile communication module 150 can provide a wireless communication solution applied to the electronic device 100, such as at least one of the following: a ^second -generation (2G) mobile communication solution, a ^third -generation (3G) mobile communication solution, a ^fourth -generation (5G) mobile communication solution, and a fifth ^- generation (5G) mobile communication solution. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low-noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves via the antenna 1, and perform filtering and amplification on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modem processor, and the amplified signal is converted into electromagnetic waves and radiated by the antenna 1. In some embodiments, at least some functional modules of the mobile communication module 150 may be housed in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be housed in the same device.

The modem processor may include a modulator and a demodulator. The modulator modulates a low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates a received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through audio devices (e.g., speaker 170A, receiver 170B) or displays images or videos through the display screen 194. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 110 and may be housed in the same device as the mobile communication module 150 or other functional modules.

Similar to the mobile communication module 150, the wireless communication module 160 can also provide a wireless communication solution for use on the electronic device 100, such as at least one of the following: wireless local area networks (WLAN), Bluetooth (BT), Bluetooth Low Energy (BLE), ultra-wideband (UWB), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technology. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via antenna 2, performs frequency modulation and filtering of the electromagnetic wave signal, and sends the processed signal to processor 110. The wireless communication module 160 can also receive signals to be transmitted from processor 110, perform frequency modulation and amplification on them, and convert the signals into electromagnetic waves for radiation via antenna 2.

In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 of electronic device 100 is coupled to wireless communication module 160, enabling electronic device 100 to communicate with networks and other electronic devices via wireless communication technology. This wireless communication technology may include at least one of the following communication technologies: Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and IR technologies. The GNSS may include at least one of the following positioning technologies: Global Positioning System (GPS), Global Navigation Satellite System (GLONASS), BeiDou Navigation Satellite System (BDS), Quasi-Zenith Satellite System (QZSS), and Satellite Based Augmentation Systems (SBAS).

Electronic device 100 can implement display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

Display screen 194 can be used to display images or videos. Display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a mini light-emitting diode (Mini LED), a micro light-emitting diode (Micro LED), a micro OLED (Micro OLED), or a quantum dot light-emitting diode (QLED). In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display screen 194 and application processor.

The ISP (Image Signal Processor) is used to process data fed back from the camera 193. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can perform algorithmic optimization of image noise, brightness, and color. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

Camera 193 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into a standard red-green-blue (RGB), YUV, or other image signal format. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

Digital signal processors (DSPs) are used to process digital signals. Besides digital image signals, they can also process other digital signals. For example, when electronic device 100 selects a frequency, the DSP can perform Fourier transforms on the frequency energy.

Video codecs are used to compress or decompress digital video. Electronic device 100 may support one or more video codecs. Thus, electronic device 100 can play or record video in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, and MPEG 4.

An NPU (Neural Processing Unit) is a processor that borrows from the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, to rapidly process input information and continuously learn. NPUs can enable intelligent cognitive functions in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

The external storage interface 120 can be used to connect an external memory card, such as a secure digital (SD) card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be stored on the external memory card.

Internal memory 121 can be used to store computer executable program code, which includes instructions. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function (e.g., sound playback and image playback). The data storage area may store data created during the use of electronic device 100 (e.g., audio data and phonebook). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, and universal flash storage (UFS). Processor 110 executes various processing methods of electronic device 100 by running instructions stored in internal memory 121 and/or instructions stored in memory disposed in the processor.

Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

The audio module 170 is used to convert digital audio information into analog audio signals for output, and can also be used to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170, or some functional modules of the audio module 170, can be located in the processor 110.

The speaker 170A, also known as a loudspeaker, is used to convert audio electrical signals into sound signals. The electronic device 100 can listen to music or make hands-free calls through the speaker 170A.

The receiver 170B, also known as the earpiece, is used to convert audio electrical signals into sound signals. When a user uses electronic device 100 to answer a phone call or voice message, they can listen to the voice by bringing the receiver 170B close to their ear.

Microphone 170C, also known as a microphone or transducer, is used to convert sound signals into electrical signals. When a user makes a phone call or sends a voice message, they can input a sound signal into microphone 170C by speaking close to it. Electronic device 100 may have at least one microphone 170C. In some embodiments, electronic device 100 may have two microphones 170C to achieve noise reduction. In other embodiments, electronic device 100 may also have three, four or more microphones 170C to achieve functions such as sound source identification and directional recording. Processor 110 can process the electrical signals output by microphone 170C. For example, audio module 170 and wireless communication module 160 may be coupled through a PCM interface. After microphone 170C converts ambient sound into an electrical signal (such as a PCM signal), it transmits the electrical signal to processor 110 through the PCM interface. Processor 110 performs volume and frequency analysis on the electrical signal to determine the volume and frequency of the ambient sound.

The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

Pressure sensor 180A is used to sense pressure signals and convert them into electrical signals. In some embodiments, pressure sensor 180A can be disposed on display screen 194. Pressure sensor 180A can be of many types, such as resistive pressure sensor, inductive pressure sensor, or capacitive pressure sensor. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to pressure sensor 180A, the capacitance between the electrodes changes, and electronic device 100 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to display screen 194, electronic device 100 detects the touch operation based on pressure sensor 180A. Electronic device 100 can also calculate the touch position based on the detection signal from pressure sensor 180A. In some embodiments, touch operations applied to the same touch position but with different touch operation intensities can correspond to different operation commands. For example, when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS message is executed; when a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS message is executed.

The gyroscope sensor 180B can be used to determine the motion attitude of the electronic device 100. In some embodiments, the gyroscope sensor 180B can determine the angular velocity of the electronic device 100 around three axes (i.e., the x-axis, y-axis, and z-axis). The gyroscope sensor 180B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 180B detects the angle of the shake of the electronic device 100, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the electronic device 100 by moving in the opposite direction, thus achieving image stabilization. The gyroscope sensor 180B can also be used in scenarios such as navigation and motion-sensing games.

The barometric pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates altitude using the air pressure value measured by the barometric pressure sensor 180C to assist in positioning and navigation.

The magnetic sensor 180D includes a Hall effect sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip cover. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover based on the magnetic sensor 180D. The electronic device 100 can set features such as automatic flip unlocking based on the detected opening and closing state of the cover or the flip cover.

The accelerometer 180E can detect the magnitude of acceleration of the electronic device 100 in various directions (typically the x-axis, y-axis, and z-axis). When the electronic device 100 is stationary, it can detect the magnitude and direction of gravity. The accelerometer 180E can also be used to identify the attitude of the electronic device 100, serving as input parameters for applications such as screen orientation switching and pedometers.

The distance sensor 180F is used to measure distance. The electronic device 100 can measure distance using infrared or laser. In some embodiments, such as in a shooting scenario, the electronic device 100 can utilize the distance sensor 180F to measure distance for fast focusing.

The proximity sensor 180G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The LED may be an infrared LED. The electronic device 100 emits infrared light outward through the LED. The electronic device 100 uses the photodiode to detect infrared reflected light from nearby objects. When reflected light is detected, the electronic device 100 can determine that an object is nearby. When no reflected light is detected, the electronic device 100 can determine that no object is nearby. The electronic device 100 can use the proximity sensor 180G to detect whether the user is holding the electronic device 100 close to their ear for a call, so as to automatically turn off the screen to save power. The proximity sensor 180G can also be used for automatic unlocking and automatic screen locking in folding or pocket modes.

The ambient light sensor 180L is used to sense the brightness of ambient light. The electronic device 100 can adaptively adjust the brightness of the display screen 194 based on the sensed ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also work with the proximity sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.

The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to perform functions such as unlocking, accessing application locks, taking photos, and answering calls.

Temperature sensor 180J is used to detect temperature. In some embodiments, electronic device 100 uses the temperature detected by temperature sensor 180J to execute a temperature handling strategy. For example, when the temperature reported by temperature sensor 180J exceeds a threshold, electronic device 100 performs thermal protection by reducing the performance of a processor located near temperature sensor 180J to reduce power consumption. In other embodiments, when the temperature is below another threshold, electronic device 100 heats battery 142 to prevent abnormal shutdown of electronic device 100 due to low temperature. In still other embodiments, when the temperature is below yet another threshold, electronic device 100 boosts the output voltage of battery 142 to prevent abnormal shutdown due to low temperature.

Touch sensor 180K, also known as a touch device, can be disposed on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a touch screen. Touch sensor 180K is used to detect touch operations applied to or near it. Touch sensor 180K can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be disposed on the surface of electronic device 100, and in a different location from display screen 194.

The bone conduction sensor 180M can acquire vibration signals. In some embodiments, the bone conduction sensor 180M can acquire vibration signals from the vibrating bone segments of the human vocal cords. The bone conduction sensor 180M can also contact the human pulse to receive blood pressure signals. In some embodiments, the bone conduction sensor 180M can also be incorporated into headphones to form bone conduction headphones. The audio module 170 can parse the voice signals from the vibrating bone segments of the vocal cords acquired by the bone conduction sensor 180M to realize voice functionality. The application processor can parse heart rate information from the blood pressure signals acquired by the bone conduction sensor 180M to realize heart rate detection functionality.

Button 190 includes a power button and volume buttons. Button 190 can be a mechanical button or a touch button. Electronic device 100 can receive button input signals and implement functions related to the button input signals.

Motor 191 can generate vibrations. Motor 191 can be used for incoming call notifications or for haptic feedback. Motor 191 can produce different vibration feedback effects for touch operations applied to different applications. Motor 191 can also produce different vibration feedback effects for touch operations applied to different areas of the display screen 194. Different application scenarios (e.g., time reminders, receiving messages, alarm clocks, and games) can correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

Indicator 192 can be an indicator light, which can be used to indicate charging status and power changes, or to indicate messages, missed calls and notifications.

The SIM card interface 195 is used to connect a SIM card. The SIM card can be inserted into the SIM card interface 195 to make contact with the electronic device 100, or it can be removed from the SIM card interface 195 to detach from the electronic device 100. The electronic device 100 can support one or N SIM card interfaces, where N is a positive integer greater than 1. Multiple cards can be inserted into the same SIM card interface 195 simultaneously; the multiple cards can be of the same or different types. The SIM card interface 195 is also compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to realize functions such as voice calls and data communication. In some embodiments, the electronic device 100 uses an embedded SIM (eSIM) card, which can be embedded in the electronic device 100 and cannot be separated from it.

The hardware system of electronic device 100 has been described in detail above. The software system of electronic device 100 is described below. The software system can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application embodiment takes a layered architecture as an example to exemplarily describe the software system of electronic device 100.

As shown in Figure 2, a layered architecture software system is divided into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the software system can be divided into four layers, from top to bottom: the application layer, the application framework layer, the Android Runtime and system libraries, and the kernel layer.

The application layer can include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The application framework layer may include some predefined functions.

For example, the application framework layer includes a window manager, content providers, a view system, a phone manager, a resource manager, and a notification manager.

The window manager is used to manage windowed applications. It can obtain the screen size, determine if a status bar is present, lock the screen, and capture the screen.

Content providers store and retrieve data, making that data accessible to applications. This data may include videos, images, audio, made and received phone calls, browsing history and bookmarks, and phone books.

A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views; for example, a display interface including a text notification icon can include views for displaying text and views for displaying images.

The phone manager is used to provide communication functions for electronic devices 100, such as the management of call status (connected or disconnected).

The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, and video files.

The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of download completions and message alerts. The notification manager can also manage notifications that appear as icons or scrollbar text in the system's top status bar, such as notifications from background applications. Furthermore, the notification manager can manage notifications that appear on the screen as dialog boxes, such as text messages in the status bar, sound alerts, electronic device vibrations, and indicator light flashing.

The Android Runtime consists of core libraries and a virtual machine. The Android Runtime is responsible for the scheduling and management of the Android system.

The core library consists of two parts: one part is the functionalities that need to be called by the Java language, and the other part is the Android core library.

The application layer and application framework layer run in a virtual machine. The virtual machine executes the Java files of the application layer and application framework layer as binary files. The virtual machine is used to perform functions such as object lifecycle management, stack management, thread management, security and exception management, and garbage collection.

The system library can include multiple functional modules, such as: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES for embedded systems) and 2D graphics engines (e.g., Skia graphics library (SGL)).

The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

The media library supports playback and recording of various audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, Moving Picture Experts Group Audio Layer III (MP3), Advanced Audio Coding (AAC), Adaptive Multi-rate (AMR), Joint Photographic Experts Group (JPG), and Portable Network Graphics (PNG).

The 3D graphics processing library can be used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

A 2D graphics engine is a graphics engine for 2D drawing.

The kernel layer is the layer between hardware and software. It can include driver modules such as display drivers, camera drivers, audio drivers, and sensor drivers.

The following example, using a photographing scenario, illustrates the workflow of the software and hardware systems of the electronic device 100.

When a user touches the touch sensor 180K, a corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into raw input events, which include information such as touch coordinates and a timestamp of the touch operation. These raw input events are stored in the kernel layer. The application framework layer retrieves these raw input events from the kernel layer, identifies the corresponding control, and notifies the application (APP) that controls the control. For example, if the touch operation is a single click and the APP corresponding to the control is a camera APP, the camera APP, after being activated by the single click, can call the camera driver in the kernel layer via API to control the camera 193 to take a picture.

Currently, 3D LUTs are typically used in image processing ISPs. Implementing 3D LUT functionality in electronic devices requires loading color lookup tables and performing 3D interpolation calculations. A color lookup table is a 3D mapping table between discrete original color data and corrected color data. For example, color data can consist of three components (R, G, and B components), and color correction is achieved by discretely sampling these three components at equal intervals. However, implementing 3D LUT functionality places high demands on the storage capacity and computing power of electronic devices. In situations with storage limitations, or performance and power consumption constraints, such as slow-motion video or high frame rate shooting modes, electronic devices may not be able to meet the requirements for using 3D LUTs.

In embodiments of this application, the electronic device, in response to a detected first operation, can determine a first mapping in gamma processing and a second mapping in a two-dimensional lookup table; based on the first mapping and the second mapping, the image to be processed is processed to obtain a first image; since the embodiments of this application use the second mapping in the two-dimensional lookup table and the first mapping in gamma processing to process the image to be processed, wherein the first mapping is associated with the second mapping; therefore, compared with processing the image to be processed through a three-dimensional lookup table, the image processing method of this application can effectively reduce the computational load of the electronic device.

The image processing method of this application embodiment will be described in detail below with reference to the application scenarios of this application.

For example, as shown in FIG3, the image processing method of this application can be applied to image color style processing; for example, the color style of a video (or image) can be adjusted according to the user's needs to achieve various special effects of the image.

For example, the original image frame can be obtained, and the target image frame can be obtained by image processing of the original image frame; where the original image frame can refer to an image in RGB color space, and the target image frame can be obtained by image processing of the original image frame according to LUT; it can be regarded as a kind of function, through which the color information of each pixel can be repositioned to obtain a new color value.

The image processing method provided in the embodiments of this application will be described in detail below with reference to Figures 4 to 42.

Application Scenario 1: Target image frame obtained by processing image frames according to a two-dimensional lookup table; for example, selecting the image color style corresponding to the 2D LUT in a camera for video recording.

It should be understood that image style can refer to filter effects in the camera interface.

In one example, the image processing method provided in this application embodiment can be applied to video shooting; when shooting video, the slow motion mode of the camera can be selected, and in the slow motion mode, a LUT can be clicked, which can refer to a 2D LUT; in the preview interface of the 2D LUT, the target image style can be selected according to the user's needs, and the camera can shoot the target video in real time according to the target image style.

For example, Figure 4 illustrates a graphical user interface (GUI) of an electronic device, which is the desktop 210 of the electronic device. When the electronic device detects that the user clicks the icon 220 of the camera application (APP) on the desktop 210, it can launch the camera application and display another GUI as shown in Figure 5. The GUI shown in Figure 5 can be the display interface of the camera APP in shooting mode, and the GUI can include a shooting interface 230. The shooting interface 230 can include a viewfinder 231 and shooting controls; for example, the shooting interface 230 can include controls 232 for indicating shooting. In preview mode, the preview image can be displayed in real time within the viewfinder 231; wherein, preview mode can refer to the state before the user opens the camera and does not press the photo/video button, at which time the preview image can be displayed in real time within the viewfinder.

It should be understood that the size of the viewfinder 231 can be the same or different in photo mode and video mode. For example, the viewfinder 231 can be the viewfinder in photo mode; in video mode, the viewfinder 231 can also be the entire display screen.

After the camera application is launched, the shooting mode interface can be displayed; the electronic device can detect the user's operation of clicking the slow motion mode, and in response to the user's operation, the electronic device enters the slow motion mode, as shown in Figure 5; the shooting interface 230 can also include a slow motion option, and after the electronic device detects the user's click on the slow motion option, the interface shown in Figure 8 can be displayed.

In one example, as shown in Figure 6, the shooting interface 230 in shooting mode may also include more options 234; after the electronic device detects the user's operation of clicking more options 234, the settings interface shown in Figure 7 is displayed; the settings interface may include options such as slow motion and high resolution; according to the user's shooting needs, clicking the slow motion option 235 in the settings interface displays the interface shown in Figure 8; that is, the camera enters slow motion mode.

As shown in Figure 8, in the slow-motion mode of the camera, the shooting interface 230 may also include a LUT option 236; the electronic device detects the user's click on the LUT option 236 and displays the interface shown in Figure 9; the shooting interface can display multiple preview frames 237 with different image styles (also known as filter effects), and the preview frames 237 may include style 1, style 2, style 3 and other image styles; the image styles may include, but are not limited to: orange-green style, warm sunlight style, glowing style, cyberpunk style and other image styles.

It should be understood that Figure 9 shows the shooting interface of the electronic device in portrait mode LUT mode; Figure 10 shows the shooting interface of the electronic device in landscape mode LUT mode; the electronic device can determine whether to display in portrait or landscape mode according to the user's state of using the electronic device.

It should also be understood that Figures 9 and 10 provide examples of the image styles displayed in preview box 237; other image styles may also be included in the preview box, and the names of the image styles mentioned above are for illustrative purposes only, and this application does not impose any limitations on them.

In one example, using the shooting interface shown in Figure 11 as an example, the electronic device detects that the user clicks on Style 2 in the preview box 237, and then displays the preview image of Style 2 in the viewfinder 231, as shown in Figure 11. Similarly, the user can also click on other image styles in the preview box 237 to preview. After the user selects a target image style (i.e., the target filter effect), the viewfinder 231 will display the preview image corresponding to the target image style. As shown in Figure 12, after the user confirms the target image style, the electronic device detects that the user clicks on the control indicating recording; in response to the user's operation, the electronic device starts recording, that is, the electronic device enters video shooting.

It should be noted that after the user selects an image style, the preview image displayed in preview box 231 is the image corresponding to the image style.

In one possible implementation, in slow-motion mode (an example of the first shooting mode), the electronic device can display a first interface, the first interface including a first control; detect a first operation on the first control; in response to the first operation, the electronic device can determine a first mapping in gamma processing and a second mapping in a two-dimensional lookup table; wherein, the first interface may refer to the interface shown in FIG9, the first control may refer to the control used to indicate style 2; detecting the first operation on the first control may refer to detecting the operation of the user clicking the control used to indicate mode 2, as shown in FIG11; the electronic device can determine the target two-dimensional lookup table corresponding to mode 2 as shown in FIG11 by detecting the first operation on the first control, that is, it may refer to the second mapping in the two-dimensional lookup table.

It should be noted that the above example uses a click operation as the second operation; the second operation can also refer to the operation of selecting the first control via voice instruction, or it can include other actions that instruct the electronic device to select the first control. The above is for illustrative purposes only and does not limit this application in any way.

In one example, the shooting interface shown in Figure 13 may also include controls for indicating stop or pause; during video shooting, images can also be captured by clicking the capture control, and the style of the captured image is the same as the style selected for the video shooting.

For example, after a user selects a target image style, the preview image displayed in the shooting interface can be switched to the target image style as a whole; the shooting interface shown in Figure 9 can be the preview image in the original image mode; the shooting interface shown in Figure 11 can be the preview image in style 2.

For example, after a user selects a target image style, a portion of the preview image in the shooting interface can be switched to the target image style.

For example, as shown in Figure 14, when the user clicks on Style 3 in the preview box 237, the electronic device can identify the target object and background object in the preview image based on the preview image in the viewfinder 231, and process the background object with Style 3 while keeping the target object in its original image style. For example, the electronic device can determine the target object and background object based on the distance between different objects and the electronic device. As shown in Figure 14, the viewfinder 231 can include the target object 238 and the background object 239. After detecting that the user has selected Style 3, the background object can be processed with Style 3, and the interface shown in Figure 15 can be displayed. After the user confirms Style 3, the electronic device detects that the user has clicked the control indicating recording. In response to the user's operation, the electronic device starts recording, that is, it uses Style 3 to shoot the video.

It should be noted that Figures 11 to 15 above show the determination of a target image style based on the user's selection; in one possible implementation, the electronic device can also automatically determine a target image style based on the identified shooting scene. For example, an Artificial Intelligence (AI) color grading mode can be set for the camera, enabling the electronic device to automatically determine the target image style based on the identified shooting scene when shooting videos or images.

In one possible implementation, in slow-motion mode (an example of a first shooting mode), the electronic device may display a first interface, the first interface including a first control; detect a first operation on the first control; in response to the first operation, the electronic device may determine a first mapping in gamma processing and a second mapping in a two-dimensional lookup table; wherein, the first interface may refer to the settings interface of a camera application; the first control may refer to the control for AI movie color grading; detecting the first operation on the first control may refer to the electronic device detecting an operation to determine the image style based on the identified shooting scene.

It should be understood that the user's actions to instruct recording may include clicking a recording control, the user device instructing the electronic device to record via voice, or other user-instructed actions to record. The above are illustrative examples and do not constitute any limitation on this application.

The above describes the graphical display interface for user operation on the electronic device in conjunction with Figures 4 to 15. The following describes the image processing algorithm running on the electronic device in conjunction with Figures 16 to 19.

Figure 16 is a schematic diagram of the processing flow of the image processor provided in an embodiment of this application.

The image sensor 201 is used to convert the light image on the photosensitive surface into an electrical signal that is proportional to the light image, based on the photoelectric conversion function of the optoelectronic device.

Defect Pixel Correction (DPC) is used to correct defects in the array of points that collect light on a sensor, or errors that occur during the conversion of light signals; it is usually done by averaging the values of surrounding pixels in the brightness domain to eliminate defective pixels.

Black Level Correction (BLC) is used to correct black levels, which refer to the video signal level on a calibrated display device where no line of light is output. The reasons for performing black level correction are twofold: firstly, image sensors have dark current, causing pixels to output voltage even in the absence of light; secondly, image sensors lack precision during analog-to-digital conversion. For example, with 8 bits, the effective range of each pixel is 0 to 255, and the image sensor may not be able to convert information close to 0. Based on users' visual characteristics (sensitivity to details in dark areas), image sensor manufacturers typically add a fixed offset during analog-to-digital conversion to ensure the output pixel value is between 5 (not a fixed value) and 255. This offset is then sent to the ISP for subtraction, adjusting pixel 5 (not a fixed value) to 0, thus ensuring the effective range of each pixel is 0 to 255.

Noise Reduction 204 is used to reduce noise in images; noise in images can affect the user's visual experience, and noise reduction can improve image quality to a certain extent.

Lens Shading Correction (LSC) is used to eliminate inconsistencies in color and brightness around the edges of an image compared to the center, caused by lens optics.

Auto White Balance (AWB) 206 is used to ensure that the camera can reproduce white as white at any color temperature. Due to the influence of color temperature, white paper will appear yellowish at low color temperatures and bluish at high color temperatures. The purpose of white balance is to ensure that white objects appear white at any color temperature with R=G=B.

Color interpolation 207 is used to ensure that each pixel contains all three components (RGB).

Color Correction Matrix (CCM) 208 is used to calibrate the accuracy of colors other than white. Gamma Processing 209 is used to adjust the brightness, contrast, dynamic range, etc. of an image by adjusting the gamma curve. Two-dimensional and three-dimensional lookup tables 210 are used for image color correction; the lookup table can be regarded as a function, and the color information of each pixel can be mapped to obtain a new color value after being mapped by the lookup table; RGB to YUV conversion 211 refers to converting the color values obtained by mapping through the lookup table into luminance and chrominance representations; where Y represents luminance (Luminance or Luma), that is, grayscale value; U and V represent chrominance (Chrominance or Chroma).

It should be noted that the image processing method provided in this application embodiment can be applied to the processing flow of gamma processing 209 and two-dimensional lookup table/three-dimensional lookup table 210.

Figure 17 is a schematic diagram of a method for constructing a two-dimensional lookup table provided in an embodiment of this application.

It should be understood that, as shown in Figure 17, the three-dimensional path represents the image processing of the original image frame using a 3D LUT to obtain the first target image frame; the two-dimensional path represents the image processing of the original image frame using a 2D LUT to obtain the second target image frame. In the embodiments of this application, under scenarios limited by the performance of electronic devices, image processing through a two-dimensional path can make the image color and image style of the second target image frame similar to or the same as those of the first target image frame; wherein, the first target image frame and the second target image frame can refer to image frames in the RGB color space.

For example, as shown in Figure 17, in order to make the image color and image style of the second target image frame similar or the same as those of the first target image frame, the function of the first gamma processing (an example of the first mapping of gamma processing) and the two-dimensional lookup table can be determined based on the function of the second gamma processing (an example of the third mapping of gamma processing) and the three-dimensional lookup table.

For example, the process of constructing a two-dimensional lookup table includes the following steps:

Step 1: Acquire data; for example, evenly divide the Hue, Saturation, and Lightness (HSL) color space into H*S*L grid points (e.g., number of grid points in the Hue direction = 25, number of grid points in the Saturation direction = 17, number of grid points in the Lightness direction = 257), and generate the corresponding initial HSL image (an example of an initial first color space image) and initial RGB image (an example of an initial second color space image);

Step 2: The initial RGB image can be the RGB image at node B1 in the 3D path; perform color correction processing on the RGB image at node B1 according to a 3D lookup table (an example of the fourth mapping in the 3D lookup table) to obtain the RGB image at node C1, which is the first target RGB image;

Step 3: Perform inverse second gamma processing on the RGB image at node B1 to obtain the RGB image corresponding to node A1;

Step 4: For the two-dimensional path, acquire the RGB image at node A1, extract the gray pixels (i.e., pixels with hue = 0 and saturation = 0) of the RGB image at node A1 as the gray pixels of the RGB image at node A2. Use the gray pixels at node A2 as input data, and use the pixels at the corresponding positions of the RGB image at node C1 as output data. Here, the corresponding position refers to the position of the gray pixel before mapping according to the three-dimensional lookup table. Fit the points respectively, and the three curves obtained by fitting are the gamma curves corresponding to the r, g, and b channels, which are the parameters corresponding to the first gamma processing.

Step 5: Using the initial RGB image generated in Step 1 as the RGB image of node B2, perform inverse first gamma processing on this image to obtain the RGB image at node A2; using the RGB image at node A2 as the RGB image at node A1, and then performing second gamma processing on the RGB image at node A1 to obtain the RGB image at node B1; performing three-dimensional lookup table processing on the RGB image at node B1 to obtain the RGB image at node C1; converting the RGB image at node C1 (an example of the third image) to the HSL color space (an example of the second color space) to obtain the HSL image at node C1 (an example of the fourth image); using the initial HSL image generated in Step 1 as the HSL image of node B2, using the HSL image at node B2 as input data, and using the HSL image at node C1 as output data, calculate the difference between them to obtain a two-dimensional lookup table corresponding to hue and saturation.

In one example, the difference between and can be calculated to obtain the hue adjustment, saturation adjustment, and brightness adjustment corresponding to the initial HSL image, denoted as . Based on and fixing the brightness value, a two-dimensional lookup table corresponding to hue and saturation can be obtained.

In one example, with a fixed brightness value, the difference between hue and saturation can be calculated to obtain a two-dimensional lookup table for hue and saturation.

For example, in the HSL color space, with a fixed brightness value, the hue can correspond to a two-dimensional lookup table; the horizontal axis of the two-dimensional lookup table corresponding to this hue can represent the hue, and the vertical axis can represent the saturation; each value in the two-dimensional table can represent the adjustment amount of the hue and the hue at that saturation, as shown in Table 1.

Table 1

Saturation/Hue 0 degrees (hue) 72 144 216 288 360 0 (saturation) 15 (ΔH/degrees) 20 10 -5 -10 15 0.25 10 -10 -15 10 5 10 0.5 5 5 10 15 20 5 0.75 -15 10 5 20 15 -15 1.0 -20 15 20 20 20 -20

For example, in the HSL color space, with a fixed brightness value, saturation can correspond to a two-dimensional lookup table; the horizontal axis of the two-dimensional lookup table corresponding to this saturation can represent hue, and the vertical axis can represent saturation. Each value in the two-dimensional lookup table can represent the adjustment amount of saturation for that hue and that saturation level, as shown in Table 2.

Table 2

Saturation/Hue 0 degrees (hue) 72 144 216 288 360 0 (saturation) -0.1(ΔS) -0.02 -0.08 -0.1 0.02 -0.1 0.25 0.05 0.05 -0.1 0.1 0.04 0.05 0.5 0.03 -0.08 0.05 -0.04 -0.06 0.03 0.75 0.04 0.04 0.04 0.05 0.07 0.04 1.0 -0.05 -0.05 0.02 0.08 0.05 -0.05

In one example, assume i = 0~24, j = 0~16, k = 0~256; take k = 128 (i.e., brightness = 0.5, data range 0.0~1.0); that is, with the brightness value fixed, the corresponding hue adjustment and saturation adjustment are taken as the coordinate positions of the two-dimensional lookup table.

It should be understood that other brightness values can be used when a fixed brightness value is selected; however, excessively bright or dark brightness values should be avoided as much as possible, because 3D LUTs may make other adjustments to overly dark or overly bright pixels due to noise or overexposure.

It should also be understood that the process of constructing the two-dimensional lookup table described above is carried out in the HSL color space; similarly, it can also be carried out in the Hue, Saturation, and Value (HSV) color space, where H represents hue, S represents saturation, and V represents value.

Figure 18 is a schematic flowchart of an image processing method provided in an embodiment of this application. The method 300 includes steps S310 to S340, which are described in detail below.

It should be understood that the first gamma processing and the two-dimensional lookup table processing shown in Figure 18 are the same as those shown in Figure 17; that is, the parameters and the two-dimensional lookup table corresponding to the first gamma processing can be determined by the method shown in Figure 17.

Step S310: Obtain the image frame to be processed (an example of the image to be processed).

For example, the image frame to be processed can be an image captured in real time by an electronic device through a camera; the real-time captured image can refer to the image output by the CCM 208.

It should be understood that the image frame to be processed can refer to an image that requires color correction.

Step S320: Perform first gamma processing on the image frame to be processed (an example of the first mapping of gamma processing).

For example, the three curves obtained by fitting in step four of Figure 17 are the gamma curves corresponding to the three channels r, g, and b, and the first gamma processing is performed on the RGB image to be processed.

Furthermore, the image after the first gamma processing can be converted from the RGB color space to the HSL color space.

Step S330, Two-dimensional lookup table processing (an example of the second mapping of a two-dimensional lookup table).

For example, a two-dimensional lookup table corresponding to different image styles can be generated according to the method shown in Figure 17 above; here, the image to be processed can be obtained by processing the image according to the two-dimensional lookup table corresponding to the target image style; wherein, the target image style can be an image style determined by the user, or the target image style can be an image style automatically recognized by the electronic device.

For example, the adjustment amount can be obtained by looking up a two-dimensional table, thereby obtaining a color-corrected image.

In one example, the user can select the image style for color correction of the RGB image to be processed according to their needs, so that the electronic device determines the two-dimensional lookup table corresponding to the image style; and performs color correction on the RGB image to be processed according to the two-dimensional lookup table.

In one example, by setting an AI color grading mode on an electronic device, the device can automatically determine the style of the target image based on the identified shooting scene; and then perform color correction on the RGB image to be processed according to the two-dimensional lookup table corresponding to the target image style.

In one example, for the HSL color space, the L value can be fixed, and the corrected H value and corrected S value can be obtained respectively according to the two-dimensional lookup table corresponding to H and the two-dimensional lookup table corresponding to S. Here, the corrected H value can refer to the original H value with an added hue adjustment amount ΔH; similarly, the corrected S value refers to the original S value with an added saturation adjustment amount ΔS. The original H value and original S value refer to the original H value and original S value obtained after converting the original RGB pixels to the HSL color space, and ΔH and ΔS refer to the values obtained through the two-dimensional lookup table.

Step S340: Obtain the target image frame (an example of the first image).

For example, the HSL image processed by the two-dimensional lookup table in step S330 is converted into the RGB color space to obtain the target RGB image, which can refer to the target image frame.

In one example, the target image frame can refer to an RGB image that has undergone color correction processing on the image to be processed according to a two-dimensional lookup table selected by the user, resulting in an image with the image effect required by the user.

In one example, the target image frame can refer to the RGB image obtained after color correction processing of the image to be processed by determining a two-dimensional lookup table based on scene detection by an electronic device.

It should also be understood that the above description is based on images; similarly, the image obtained above can also be a video to be processed, and the above processing can be performed on each frame of the video to be processed to obtain the target video.

In the embodiments of this application, a three-dimensional lookup table can be compressed to obtain a two-dimensional lookup table with an effect similar to or the same as that of a three-dimensional lookup table. Color correction of the image using this two-dimensional lookup table can achieve an image effect similar to that of a three-dimensional lookup table. In scenarios where the performance of electronic devices is limited, the image processing method provided in the embodiments of this application can meet the needs of users and achieve different image effects.

Application Scenario 2: Obtaining target video based on 3D and 2D lookup tables; for example, in different shooting scenarios, the camera can switch between 3D LUT and 2D LUT for video shooting.

In one example, the image processing method provided in this application embodiment can be applied to video shooting. When shooting video, the camera's movie mode can be selected first, and a 3D LUT can be clicked in movie mode. In the preview interface of the 3D LUT, the target image style can be selected according to the user's needs, and the camera can shoot video in real time according to the target image style. During the video shooting process, for scenarios where the computing power of electronic devices is weak (e.g., scenarios where the image frame rate requirement is high), the camera mode can be switched from movie mode to slow motion mode. After switching to slow motion mode, the 3D LUT is triggered to switch to the corresponding 2D LUT, thereby continuing to shoot video to obtain the target video.

Figure 19 is a schematic flowchart of an image processing method provided in an embodiment of this application. The method 400 includes steps S410 to S460, which are described in detail below.

It should be understood that the image processing method shown in Figure 19 can be designed in parallel with three-dimensional and two-dimensional pathways; that is, color correction of the image to be processed can be performed through the three-dimensional pathway; or, color correction of the image to be processed can be performed through the two-dimensional pathway; or, color correction processing of the image to be processed can be performed by switching between the three-dimensional and two-dimensional pathways according to specific needs.

Step S410: Obtain the RGB image to be processed.

For example, the RGB image to be processed can be an image captured in real time by an electronic device through a camera.

Step S420, Second Gamma Processing.

For example, the second gamma processing may refer to performing gamma processing on the image to be processed using the gamma parameters of a 3D LUT in the prior art.

Step S430: 3D lookup table processing.

For example, three-dimensional lookup table processing can refer to processing the RGB image to be processed using a three-dimensional lookup table in the prior art.

Step S440, First Gamma Processing.

For example, the parameters corresponding to the first gamma processing can be determined by the method shown in Figure 17; the R, G, and B pixel values of the RGB image to be processed can be adjusted by the first gamma processing in the two-dimensional path.

Furthermore, based on the type of the two-dimensional lookup table, it is determined whether the RGB image after the first gamma processing will be converted from the RGB color space to the HSL color space or the HSV color space.

For example, if the brightness value in the two-dimensional lookup table is fixed, the RGB image after the first gamma processing is converted from the RGB color space to the HSL color space; if the luminance value in the two-dimensional lookup table is fixed, the RGB image after the first gamma processing is converted from the RGB color space to the HSV color space.

Step S450: Two-dimensional lookup table processing.

It should be understood that a two-dimensional lookup table reduces one dimension compared to a three-dimensional lookup table; this is because the computational load is reduced to some extent compared to a three-dimensional lookup table.

In one example, for the HSL color space, the brightness value can be fixed, and the saturation and hue can be adjusted using a two-dimensional lookup table.

In one example, for the HSL color space, the L value can be fixed, and the corrected H value and corrected S value can be obtained respectively according to the two-dimensional lookup table corresponding to H and the two-dimensional lookup table corresponding to S. Here, the corrected H value can refer to the original H value with an added hue adjustment amount ΔH; similarly, the corrected S value refers to the original S value with an added saturation adjustment amount ΔS. The original H value and original S value refer to the original H value and original S value obtained after converting the original RGB pixels to the HSL color space, and ΔH and ΔS refer to the values obtained through the two-dimensional lookup table.

In one example, for the HSV color space, the brightness value can be fixed, and the saturation and hue can be adjusted using a two-dimensional lookup table.

In one example, for the HSL color space, the luminance value can be fixed, and the corrected H value and corrected S value can be obtained respectively from the two-dimensional lookup table corresponding to H and the two-dimensional lookup table corresponding to S. Here, the corrected H value can refer to the original H value with an added hue adjustment amount ΔH; similarly, the corrected S value refers to the original S value with an added saturation adjustment amount ΔS. The original H value and original S value refer to the original H value and original S value obtained after converting the original RGB pixels to the HSL color space, and ΔH and ΔS refer to the values obtained through the two-dimensional lookup table.

Furthermore, the corrected HSL image is converted from the HSL color space to the RGB color space; or, the corrected HSV image is converted from the HSV color space to the RGB color space.

Step S460: Target RGB image.

In one example, when the electronic device is recording video, the user indicates movie mode. In movie mode, the user selects a target image style in the 3D LUT table to perform color correction on the images in the video. At some point, the electronic device receives an operation from the user to switch to slow motion mode. In response to the user's operation, the electronic device can switch the target image style in the 3D LUT table to the 2D LUT table corresponding to the target image style. The images in the video are then color corrected according to the 2D LUT.

For example, Figure 20 shows a schematic diagram of the display desktop 510 of an electronic device. When the electronic device detects that the user clicks the camera application icon 520 on the desktop 510, it can launch the camera application and display another GUI as shown in Figure 21. The GUI shown in Figure 21 can be the display interface of the camera app in movie mode, and the GUI can include a shooting interface 530. The shooting interface 530 can include a viewfinder 531 and shooting controls. For example, the shooting interface 530 can include controls 532 for indicating shooting. In preview mode, the preview image can be displayed in real time within the viewfinder 531. The preview mode refers to the state before the user opens the camera and presses the photo/video button, at which time the preview image can be displayed in real time within the viewfinder.

As shown in Figure 21, in the shooting mode interface, the electronic device can detect the user's click on the movie mode operation and enter the movie mode in response to the user's operation. As shown in Figure 22, in movie mode, the shooting interface 530 also includes a slow motion mode option 533 and a LUT option 534. The slow motion mode option 533 is used to indicate whether the slow motion mode is turned on or off. Figure 22 shows that the slow motion mode is currently off. When the slow motion mode is off, the LUT option 534 is used to indicate a three-dimensional lookup table. When the slow motion mode is on, as shown in Figure 28, the LUT option is used to indicate a two-dimensional lookup table. After the electronic device detects that the user clicks on the LUT option 534, the shooting interface can display multiple preview boxes 535 with different image styles, as shown in Figure 23. The preview box 535 can include style 1, style 2, style 3, and other image styles. The image styles can include, but are not limited to, the following: orange-green style, warm sunlight style, glowing style, cyberpunk style, etc.

It should be understood that Figure 23 above can be the shooting interface of the electronic device in portrait mode movie mode; the electronic device can determine whether to display in portrait or landscape mode according to the user's state of using the electronic device.

It should also be understood that Figure 23 illustrates the image style displayed in preview frame 535; other image styles may also be included in the preview frame, and this application does not limit this. In one example, using the shooting interface shown in Figure 24 as an example, if the electronic device detects that the user clicks on style 2 in preview frame 535, then a preview image of style 2 will be displayed in viewfinder 531; similarly, the user can also click on the image styles of other image modes in preview frame 535 for preview; after the user selects a target image style, the preview image corresponding to the target image style will be displayed in viewfinder 531. As shown in Figure 25, after the user confirms the target image style, the electronic device detects that the user clicks on the control indicating recording; in response to the user's operation, the electronic device begins recording, i.e., the electronic device enters video shooting mode.

As shown in Figure 26, during video recording, assuming the slow-motion mode is off at the 8-second mark, the electronic device detects the user clicking the slow-motion mode option. When the user clicks the slow-motion mode option, the camera's slow-motion mode is activated, and a notification box appears on the display interface saying "Slow-motion mode on." The notification box disappears automatically after a few seconds. At this time, the camera switches from movie mode to slow-motion mode, displaying the interface shown in Figure 28. Alternatively, as shown in Figure 27, a slow-motion option can be included in the preview box 535. The electronic device detects the user clicking the slow-motion option. In response to the user's operation, the electronic device switches the camera from movie mode to slow-motion mode, displaying the slow-motion display interface, as shown in Figure 28. On the slow-motion display interface, if the electronic device detects the user's operation of the recording control, the electronic device continues recording the video.

It should be noted that when the electronic device detects that the user clicks the slow motion option, the electronic device can switch the three-dimensional lookup table in movie mode to a two-dimensional lookup table; in other words, after the user clicks the slow motion option, the electronic device performs color correction processing on the image in the video by using the lookup table in the two-dimensional lookup table that corresponds to style 2 in the three-dimensional lookup table.

It should be understood that the user's actions to instruct recording may include clicking a recording control, the user device instructing the electronic device to record via voice, or other user-instructed actions to record. The above are illustrative examples and do not constitute any limitation on this application.

Application Scenario 3: Acquire the captured video to be processed, perform 2D LUT processing on the video to be processed, and obtain the processed target video.

In one example, the method for determining a two-dimensional lookup table provided in this application embodiment can be applied to image processing; after acquiring and capturing a video, the captured video can be color-corrected according to the 2D LUT to obtain a target video with an image style that meets the user's requirements.

For example, Figure 29 shows a schematic diagram of the display desktop 610 of the electronic device. When the electronic device detects that the user clicks the album application icon 620 on the desktop 610, it can launch the album application and display another GUI as shown in Figure 30; Figure 30 shows the electronic device's album, which includes videos and images; the electronic device detects that the user clicks on video 6 in the album, and in response to the user's operation, displays video 6 on the electronic device's display interface, as shown in Figure 31; the display interface 630 shown in Figure 31 includes a viewfinder 631, which displays the content of video 6; the shooting interface 630 also includes more options 632, and the electronic device detects that the user clicks more. The operation of option 632, in response to the user's operation, the electronic device displays a settings interface with more options, as shown in Figure 32. The settings interface with more options includes LUT option 633, watermark option, and other options. After the electronic device detects the user's operation of clicking the LUT option, it displays the display interface shown in Figure 33. The display interface 630 displays multiple preview boxes 634 with different image styles. The preview boxes 634 can include style 1, style 2, style 3, and other image styles. The image styles can include, but are not limited to: orange-green style, warm sunlight style, glowing style, cyberpunk style, and other image styles.

In one example, as shown in Figure 34, the electronic device detects that the user clicks on style 3 in the preview box 634, and then the electronic device performs color correction processing on video 6 according to style 3.

It should be understood that the LUT shown in Figure 33 above is a two-dimensional lookup table; the different style images displayed in the preview box 634 refer to the different image styles (filter effects) corresponding to the two-dimensional lookup table.

Application Scenario 4: Perform 2D LUT processing on the captured video in an image processing application to obtain the processed target video.

In one example, the method for determining a two-dimensional lookup table provided in this application embodiment can be applied to image processing; after acquiring a video that has been captured, the video can be color-corrected according to the 2D LUT in the image processing APP to obtain a target video with an image style that meets the user's requirements.

For example, Figure 35 shows a schematic diagram of the display desktop 710 of an electronic device; when the electronic device detects that the user clicks the icon 720 of the color correction application on the desktop 710, it can launch the color correction application and display another GUI as shown in Figure 36; the GUI shown in Figure 36 can be the display interface of the color correction application, and the GUI can include a display interface 730; the display interface 730 can include a viewfinder 731 and controls; for example, the display interface 730 can include a photo album 732 and more options 733; when the electronic device detects that the user clicks the photo album 732, in response to the user's operation, the electronic device displays the photo album and displays the display interface as shown in Figure 37; the photo album includes videos and images; when the electronic device detects that the user clicks the video 7 in the photo album, in response to the user's operation, it displays the photo album and displays the display interface as shown in Figure 37; the photo album includes videos and images; when the electronic device detects that the user clicks the video 7 in the photo album, in response to the user's operation, it displays the video album 732, and ... The user's operation displays video 7 on the display interface of the electronic device, as shown in Figure 38. The electronic device detects the user's click on the more options 733 and, in response, displays the settings interface for the more options, as shown in Figure 39. The settings interface for the more options includes LUT option 734, watermark option, and other options. After the electronic device detects the user's click on LUT option 734, it displays the display interface shown in Figure 40. As shown in Figure 40, multiple preview boxes 735 with different image styles are displayed in the display interface 730. The preview boxes 735 may include style 1, style 2, style 3, and other image styles. The image styles may include, but are not limited to, the following: orange-green style, warm sunlight style, glowing style, cyberpunk style, etc.

In one example, as shown in Figure 41, the electronic device detects that the user clicks on Style 2 in the preview box 735, and then the electronic device performs color correction processing on the video 7 according to Style 2.

It should be understood that the LUT shown in Figure 40 above is a two-dimensional lookup table; the different style images displayed in the preview box 735 refer to the different image styles (filter effects) corresponding to the two-dimensional lookup table.

The above description, in conjunction with Figures 29 to 41, describes the graphical display interface through which the user operates on the electronic device. The following description, in conjunction with Figure 42, describes the image processing algorithm running on the electronic device.

Figure 42 is a schematic flowchart of an image processing method provided in an embodiment of this application. The method 800 includes steps S810 to S860, which are described in detail below.

It should be understood that Figures 18 and 19 above are used for color correction of images when capturing images or videos in real time; the method shown in Figure 42 is applied to acquiring images or videos that have already been captured and performing color correction processing on the completed images or videos.

Step S810: Obtain the video to be processed.

For example, the video to be processed can refer to a video that requires color correction. This video can be obtained from within an electronic device (e.g., a video stored in the device's photo album, or an RGB video retrieved from the cloud). When the obtained video to be processed is in another format, it needs to be converted to RGB video.

Step S820: Anti-gamma processing.

It should be noted that since the video to be processed is already a video that has been shot, if color correction is to be performed on the video to be processed, it is necessary to obtain the original RGB image corresponding to each frame of the video to be processed, that is, the original RGB pixels output by the color correction 208 as shown in Figure 16.

Step S830: Obtain the original RGB pixels.

It should be understood that raw RGB pixels can refer to RGB pixels that have not undergone gamma processing.

Step S840: Perform the first gamma processing on the original RGB pixels.

For example, the three curves obtained by fitting in step four of Figure 17 are the gamma curves corresponding to the three channels r, g, and b, and the first gamma processing is performed on the RGB image to be processed.

Furthermore, the image after the first gamma processing can be converted from the RGB color space to the HSL color space.

Step S850: Two-dimensional lookup table processing.

For example, the adjustment amount can be obtained by looking up a two-dimensional table, thereby obtaining a color-corrected image.

It should be noted that different two-dimensional lookup tables can correspond to different image styles. Users can select the image style for color correction in the video to be processed according to their needs, and the electronic device will determine the two-dimensional lookup table corresponding to that image style; then, the color correction will be performed on the image in the video to be processed based on the two-dimensional lookup table.

In one example, for the HSL color space, the L value can be fixed, and the corrected H value and corrected S value can be obtained respectively according to the two-dimensional lookup table corresponding to H and the two-dimensional lookup table corresponding to S. Here, the corrected H value can refer to the original H value with an added hue adjustment amount ΔH; similarly, the corrected S value refers to the original S value with an added saturation adjustment amount ΔS. The original H value and original S value refer to the original H value and original S value obtained after converting the original RGB pixels to the HSL color space, and ΔH and ΔS refer to the values obtained through the two-dimensional lookup table.

Step S860: Obtain the target video.

For example, the HSL image processed by the two-dimensional lookup table in step S850 is converted into the RGB color space to obtain the target video.

It should be understood that the target video can refer to a video that has the image effects required by the user, based on a two-dimensional lookup table selected by the user.

It should also be understood that the above description is based on a video example; similarly, the image obtained above can also be an image to be processed, and the above processing can be performed on the image to be processed to obtain the target image.

In the embodiments of this application, a two-dimensional lookup table is constructed using a three-dimensional lookup table; the target image frame is obtained by image processing of the original image frame according to the two-dimensional lookup table, which enables the image quality of the target image frame to be the same as or close to that of the image obtained by the three-dimensional lookup table; since the embodiments of this application use a two-dimensional lookup table to process the original image frame, the computational load of the electronic device can be reduced while ensuring image quality.

It should be understood that the above examples are provided to help those skilled in the art understand the embodiments of this application, and are not intended to limit the embodiments of this application to the specific values or scenarios illustrated. Those skilled in the art can obviously make various equivalent modifications or changes based on the above examples, and such modifications or changes also fall within the scope of the embodiments of this application.

The image processing method of the present application embodiment has been described in detail above with reference to Figures 1 to 42. The apparatus embodiment of the present application will be described in detail below with reference to Figures 43 and 44. It should be understood that the image processing apparatus in the embodiments of the present application can execute the various image processing methods of the foregoing embodiments of the present application. That is, the specific working processes of the various products described below can be referred to the corresponding processes in the foregoing method embodiments.

Figure 43 is a schematic diagram of the structure of an image processing apparatus provided in an embodiment of this application. The image processing apparatus 900 includes a display unit 910 and a processing unit 920.

The display unit 910 is configured to display a first interface in a first shooting mode, the first interface including a first control; the processing unit 920 is configured to detect a first operation on the first control; in response to the first operation, determine a first mapping in gamma processing and a second mapping in a two-dimensional lookup table, the first mapping corresponding to the first control, the second mapping corresponding to the first control, and the first mapping and the second mapping being associated; acquire an image to be processed; and process the image to be processed according to the first mapping and the second mapping to obtain a first image.

In conjunction with the second aspect, in some implementations of the second aspect, the processing unit 920 is specifically used for:

The image to be processed is processed according to the first mapping to obtain the second image;

A first adjustment amount and a second adjustment amount are determined based on the second image and the second mapping, wherein the first adjustment amount is used to represent the adjustment amount of the first color component in the two images, and the second adjustment amount is used to represent the adjustment amount of the second color component in the second image;

The first image is obtained based on the second image, the first adjustment amount, and the second adjustment amount.

Optionally, as an embodiment, the first mapping and the second mapping are associated to indicate the determination of the second mapping in the two-dimensional lookup table based on the first mapping in the gamma processing.

Optionally, as an embodiment, the processing unit 920 is specifically used for:

Obtain the third mapping in the gamma processing and the fourth mapping in the three-dimensional lookup table, wherein the fourth mapping in the three-dimensional lookup table corresponds to the second mapping in the two-dimensional lookup table;

The first color space is uniformly divided to obtain an image in the initial first color space and an image in the initial second color space;

The image in the initial second color space is processed according to the first mapping of the gamma processing, the third mapping of the gamma processing, and the fourth mapping in the three-dimensional lookup table to obtain a third image, which is an image in the second color space.

Convert the third image into a fourth image in the first color space;

The second mapping in the two-dimensional lookup table is determined based on the pixel difference between the image in the initial first color space and the fourth image in the first color space.

Optionally, as an embodiment, the processing unit 920 is specifically used for:

Determine the first brightness value;

When the brightness is the first brightness value, the second mapping in the two-dimensional lookup table is determined based on the pixel difference between the image in the initial first color space and the fourth image in the first color space.

Optionally, as an embodiment, the first color space refers to the HSL color space or the HSV color space; the second color space refers to the RGB color space.

Optionally, as an embodiment, the first shooting mode refers to a shooting mode in which the frame rate of the image is greater than a preset threshold.

Optionally, as an embodiment, the first control refers to a control used to indicate the second mapping table in the two-dimensional lookup table.

Alternatively, as an embodiment, the first control refers to a control used to instruct the automatic identification of a second mapping table in the two-dimensional lookup table.

It should be noted that the image processing device 900 described above is embodied in the form of a functional unit. The term "unit" here can be implemented in software and/or hardware, and there is no specific limitation on this.

For example, a "unit" can be a software program, a hardware circuit, or a combination of both that implements the above functions. The hardware circuit may include an application-specific integrated circuit (ASIC), electronic circuitry, a processor (e.g., a shared processor, a proprietary processor, or a group processor) and memory for executing one or more software or firmware programs, integrated logic circuitry, and/or other suitable components that support the described functions.

Therefore, the units of the various examples described in the embodiments of this application can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

Figure 44 shows a schematic diagram of the structure of an electronic device provided in this application. The dashed lines in Figure 44 indicate that the unit or module is optional. The electronic device 1000 can be used to implement the image processing method described in the above method embodiments.

Electronic device 1000 includes one or more processors 1001, which support the implementation of the methods in the method embodiments. Processor 1001 can be a general-purpose processor or a special-purpose processor. For example, processor 1001 can be a central processing unit (CPU), a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, such as discrete gates, transistor logic devices, or discrete hardware components.

The processor 1001 can be used to control the electronic device 1000, execute software programs, and process data from the software programs. The electronic device 1000 may also include a communication unit 1005 for inputting (receiving) and outputting (transmitting) signals.

For example, electronic device 1000 may be a chip, communication unit 1005 may be the input and/or output circuit of the chip, or communication unit 1005 may be the communication interface of the chip, and the chip may be a component of terminal device or other electronic device.

For example, electronic device 1000 can be a terminal device, communication unit 1005 can be the transceiver of the terminal device, or communication unit 1005 can be the transceiver circuit of the terminal device.

The electronic device 1000 may include one or more memories 1002, which store a program 1004. The program 1004 can be executed by the processor 1001 to generate instructions 1003, causing the processor 1001 to execute the image processing method described in the above method embodiments according to the instructions 1003.

Optionally, the memory 1002 may also store data. Optionally, the processor 1001 may also read data stored in the memory 1002, which may be stored at the same memory address as the program 1004, or the data may be stored at a different memory address than the program 1004.

The processor 1001 and memory 1002 can be configured separately or integrated together; for example, integrated on the system-on-chip (SOC) of the terminal device.

For example, the memory 1002 can be used to store the related program 1004 of the image processing method provided in the embodiments of this application, and the processor 1001 can be used to call the related program 1004 of the image processing method stored in the memory 1002 during image processing to execute the image processing method of the embodiments of this application; for example, in a first shooting mode, a first interface is displayed, the first interface includes a first control; a first operation on the first control is detected; in response to the first operation, a first mapping in gamma processing and a second mapping in a two-dimensional lookup table are determined, the first mapping corresponds to the first control, the second mapping corresponds to the first control, and the first mapping and the second mapping are associated; an image to be processed is obtained; the image to be processed is processed according to the first mapping and the second mapping to obtain a first image.

This application also provides a computer program product that, when executed by processor 1001, implements the image processing method described in any of the method embodiments of this application.

The computer program product can be stored in memory 1002, for example, program 1004. Program 1004 is finally converted into an executable object file that can be executed by processor 1001 after processing such as preprocessing, compilation, assembly and linking.

This application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a computer, implements the image method described in any of the method embodiments of this application. The computer program may be a high-level language program or an executable object program.

Optionally, the computer-readable storage medium is, for example, memory 1002. Memory 1002 can be volatile memory or non-volatile memory, or memory 1002 can include both volatile and non-volatile memory. The non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. The volatile memory can be random access memory (RAM), which serves as an external cache. By way of example, but not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous linked dynamic random access memory (SLDRAM), and direct rambus RAM (DR RAM).

Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process and technical effects of the above-described apparatus and equipment can be referred to the corresponding processes and technical effects in the foregoing method embodiments, and will not be repeated here.

In the several embodiments provided in this application, the systems, apparatuses, and methods disclosed can be implemented in other ways. For example, some features of the method embodiments described above can be ignored or not performed. The apparatus embodiments described above are merely illustrative; the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Multiple units or components can be combined or integrated into another system. Furthermore, the coupling between units or components can be direct coupling or indirect coupling, including electrical, mechanical, or other forms of connection.

It should be understood that in the various embodiments of this application, the sequence number of each process does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

Furthermore, the terms "system" and "network" are often used interchangeably in this paper. The term "and/or" in this paper merely describes the relationship between related objects, indicating that three relationships can exist. For example, A and/or B can represent: A alone, A and B simultaneously, or B alone. Additionally, the character "/" in this paper generally indicates that the preceding and following related objects have an "or" relationship.

In summary, the above description is merely a preferred embodiment of the technical solution of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims (11)

1. An image processing method, characterized in that it is applied to an electronic device, the image processing method comprising: In the first shooting mode, a first interface is displayed. The first interface includes a first control. The first shooting mode includes a slow-motion shooting mode. The first control is used to indicate a two-dimensional lookup table. A first operation on the first control was detected; In response to the first operation, a first mapping in the gamma processing and a second mapping in the two-dimensional lookup table are determined, the first mapping corresponding to the first control and the second mapping corresponding to the first control, and the first mapping and the second mapping being associated; wherein, the association of the first mapping and the second mapping is used to indicate: the second mapping in the two-dimensional lookup table is determined according to the first mapping in the first gamma processing; The step of determining the second mapping in the two-dimensional lookup table based on the first mapping in the first gamma processing includes: obtaining the third mapping in the second gamma processing and the fourth mapping in the three-dimensional lookup table, wherein the fourth mapping in the three-dimensional lookup table corresponds to the second mapping in the two-dimensional lookup table; uniformly dividing the first color space to obtain an image in the initial first color space and an image in the initial second color space; processing the image in the initial second color space according to the first mapping in the first gamma processing, the third mapping in the second gamma processing, and the fourth mapping in the three-dimensional lookup table to obtain a third image, wherein the third image is an image in the second color space; converting the third image into a fourth image in the first color space; and determining the second mapping in the two-dimensional lookup table based on the pixel difference between the image in the initial first color space and the fourth image in the first color space. Based on the first mapping and the second mapping, the image to be processed is processed to obtain the first image; When switching from the first shooting mode to the second shooting mode, a second interface is displayed; the second shooting mode is different from the first shooting mode; the second interface includes a second control, which is used to indicate a three-dimensional lookup table. Upon detecting a second operation on the second control, in response to the second operation, a second gamma processing and a three-dimensional lookup table processing are performed on the image to be processed to obtain an image corresponding to the second shooting mode. 2. The image processing method as described in claim 1, characterized in that, processing the image to be processed according to the first mapping and the second mapping to obtain a first image includes: The image to be processed is processed according to the first mapping to obtain the second image; A first adjustment amount and a second adjustment amount are determined based on the second image and the second mapping, wherein the first adjustment amount is used to represent the adjustment amount of the first color component in the two images, and the second adjustment amount is used to represent the adjustment amount of the second color component in the second image; The first image is obtained based on the second image, the first adjustment amount, and the second adjustment amount. 3. The image processing method as described in claim 1, characterized in that, determining the second mapping in the two-dimensional lookup table based on the pixel difference between the image in the initial first color space and the fourth image in the first color space includes: Determine the first brightness value; When the brightness is the first brightness value, the second mapping in the two-dimensional lookup table is determined based on the pixel difference between the image in the initial first color space and the fourth image in the first color space. 4. The image processing method as described in claim 1, wherein the first color space refers to the HSL color space or the HSV color space; and the second color space refers to the RGB color space. 5. The image processing method according to any one of claims 1 to 4, wherein the first shooting mode refers to a shooting mode in which the frame rate of the image is greater than a preset threshold. 6. The image processing method according to any one of claims 1 to 4, wherein the second shooting mode refers to a shooting mode in which the frame rate of the image is not greater than a preset threshold. 7. The image processing method according to any one of claims 1 to 4, wherein the first control refers to a control used to indicate the second mapping table in the two-dimensional lookup table. 8. The image processing method according to any one of claims 1 to 4, wherein the first control refers to a control for instructing automatic recognition of the second mapping table in the two-dimensional lookup table. 9. An electronic device, characterized in that the electronic device comprises: One or more processors, memory, and a display screen; The memory is coupled to the one or more processors, the memory being used to store a computer-executable program, the one or more processors calling the computer-executable program to cause the electronic device to perform the image processing method as described in any one of claims 1 to 8. 10. A chip system, characterized in that the chip system is applied to an electronic device, the chip system comprising one or more processors, the processors being configured to invoke computer instructions to cause the electronic device to perform the image processing method as described in any one of claims 1 to 8. 11. A computer-readable storage medium, characterized in that the computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the image processing method according to any one of claims 1 to 8.