CN1511303A - Scalable and extensible system and method for optimizing image quality stochastic algorithm system - Google Patents

Abstract

An optimizing video processing method and system selects algorithms for best obtainable video quality for the available computation resources. A video processing module, which processes an input of a video stream, architectural parameters for identifying an order of cascaded video functions and determining a bit precision between data of any consecutive cascaded functions according to an associated complexity level which correlates with a value of available computational resources. An optimizer module optimizes processing of the video stream and includes a plurality of optimization engines each having an associated complexity level. The optimizer module selects an optimization engine according a complexity level which correlates with the value of available computational resources. An Object Image Quality (OIQ) evaluator module evaluates an image quality of an output of the video stream from the video processing module. The OIQ evaluator module includes a plurality of objective image quality metrics having an associated complexity level. The OIQ evaluator module selects a metric according to a correlation factor and a complexity level for said value of available computation resources.

Description

Scalable and extensible system and method for optimizing image quality stochastic algorithm system

technical field

The present invention relates to methods and systems for optimizing video quality. In particular, the invention relates to a scalable approach to video algorithms with improved image quality.

technical background

In a video processing system, multiple video functions are used to process video signals (such as enhancing sharpness, reducing noise, correcting color, etc.). These functions all require multiple (small or large) control parameters.

However, some of these parameters can have a large impact on image quality, while others have little impact. Furthermore, the order in which the individual functions are applied can be a parameter (before constituting the hardware implementing the software-emulated video processing functions, or if we have a highly flexible reconfigurable hardware), or this video system is static and cannot be changed .

In addition to the video processing functions themselves, there are two other modules whose complexity determines the final quality of the video system.

The complexity of an objective image quality (OIQ) evaluation unit can range from simple measurements of simple signals (like the rise time of a luminance signal) to very complex systems that mimic the psychophysical processes of the human visual system (HVS). The complexity of the optimization process can range from a greedy exhaustive search engine (which requires a lot of computing resources, which is almost impossible to use in many practical situations) to a less computationally intensive, intelligent heuristic search method.

Contents of the invention

In accordance with the present invention, a system for optimizing video quality includes a scalable optimization paradigm for utilizing available computing resources to provide the best objective image quality.

An optimized video processing system comprising:

A video processing module for processing an incoming video stream, the video processing module including structural parameters for determining the order of concatenated video functions, an arbitrary consecutive concatenation of bit precision between functions;

an optimization module for optimizing processing of the video stream, the optimization module in communication with the video processing module, the optimization module including a plurality of optimization engines each having an associated complexity level, the optimization module including means for selecting the optimization engine according to a level of complexity related to a value of available computing resources; and

An objective image quality (OIQ) evaluation module for evaluating the image quality of the video stream output of the video processing module, the OIQ evaluator includes a plurality of objective image quality metrics related to complexity, the OIQ evaluation module includes means for according to A correlation coefficient ri and a complexity of values of available computing resources select a metric.

The means by which the OIQ evaluation module selects a metric may include determining a correlation coefficient R according to the following formula:

$\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; R</span>}}{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; \{</span>{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \}</span><span\; class="notranslate">\; ,</span>$

where F is a final metric (of video quality evaluated by the system), and F is determined by finding a set of weights w _i that are correlated with each metric f _i of multiple objective metrics (which has a value between 1 and n When multiplied together, the correlation coefficient R will reach the maximum with the predetermined subjective impression.

The system can also have a computing resource analyzer for selecting a level of complexity for at least one of the video processing module, the optimization module, and the OIQ evaluation module.

This optimization module can include deterministic and non-deterministic optimization engines.

The optimization module may include a heuristic search engine including at least one of genetic algorithm (GA), simulated annealing (SA) heuristic search engine, forbidden search (TS), simulated evolution (SE) and stochastic evolution (SE).

At least one of the video processing module, the optimization module and the OIQ evaluation module is scalable.

The computing resource analysis module may select a level of complexity for at least one of the video processing module, the optimization module, and the OIQ evaluation module by detecting available computing resources.

Methods for optimizing video algorithms for available computing resources include:

(a) the video processing module determines the order of the cascaded video functions that process the video streams input to the video processing module according to the relative complexity associated with a value of available computing resources;

(b) selecting an optimization method for optimizing the processing of the video system, the optimization method being selected from a plurality of optimization methods according to the degree of complexity associated with that value of available computing resources;

(c) assessing the objective image quality of the video stream after the video stream is output from the video processing module;

Wherein the evaluation of the objective image quality of the video stream is a metric selected from a plurality of metrics according to a correlation coefficient and the degree of complexity associated with that value of computing resources.

The assessment of objective image quality in step (c) includes determining a correlation coefficient R according to the following formula:

$\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; R</span>}}{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; \{</span>{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \}</span><span\; class="notranslate">\; ,</span>$

where F is a final metric (of video quality evaluated by the system), and F is determined by finding a set of weights w _i that are correlated with each metric f _i of multiple objective metrics (which has a value between 1 and n When multiplied together, the correlation coefficient R will reach the maximum with the predetermined subjective impression.

The method can also include:

(d) Computing the relative complexity of at least one of the resource analyzer selection steps (a), (b) and (c).

The plurality of optimization methods selected in step (b) may include deterministic and non-deterministic optimization methods.

The plurality of optimization methods includes a heuristic search engine with at least one of genetic algorithm (GA), simulated annealing (SA), forbidden search (TS), simulated evolution (SE) and stochastic evolution (SE).

The relevant level of complexity selected in step (d) may include the step of detecting available computing resources for at least one of steps (a), (b) and (c).

The video processing module mentioned in step (a) is scalable.

Step (b) may include a scalable optimizer for selecting an optimization method.

Step (c) can provide a scalable objective image quality evaluator for evaluating objective image quality.

The system can also include a video processing module, an optimization module, a scalable objective image quality (OIQ) assessment module, and a computing resource analyzer.

The video processing module includes multiple video processing functions F ₁ , F ₂ , . . . , F _n . Each function has a set of parameters P _i , 1≤i≤n, arranged in ascending order according to their influence on the quality of the resulting image. This video processing module has its own set of structural parameters that describe the order of cascaded video processing functions and the bit precision of the data bus between any two consecutive functions.

This optimization module is a scalable optimizer with multiple possible optimization mechanisms. This optimization module can include multiple optimized search engines of varying complexity and required resources. These search engines can be exhaustive or heuristic.

The scalable OIQ assessment module includes multiple OIQ measures of varying complexity. The complexity table is maintained by the OIQ assessment module and includes all the constituent measures and the assumed complexity for each measure.

The Computing Resource Analysis module is an arbiter that decides which complexity level all other modules should adopt based on the available computing resources.

Description of drawings

Fig. 1 is a general schematic diagram of the scalable optimization system in the present invention.

FIG. 2 is a detailed block diagram of the optimization module shown in FIG. 1 .

FIG. 3A is a detailed block diagram of the objective image quality assessor shown in FIG. 1. FIG.

Figure 3B is a scalable dynamic objective metric flow.

Fig. 4 is a flow chart of a method of the present invention.

FIG. 5 is a continuation of the flowchart shown in FIG. 4 .

Detailed ways

Figure 1 shows a general diagram of the scalable optimization system of the present invention. As shown in FIG. 1 , there is a video processing module 100 , a system optimization module 200 , an objective image quality assessment module 300 and an optional computing resource analysis module 400 .

This video processing module 100 includes some structural parameters for determining the order of the concatenated video functions, and for determining the one-bit precision between the data of any consecutive concatenated functions.

As shown in FIG. 1 , there are multiple video processing functions 102 (in the range of F ₁ ~F _n ), and each function has a set of structural parameters P _i 105 in the range of P ₁ ~P _n . The set of parameters P _i (where 1≤i≤n) are arranged in descending order according to their influence on the obtained image quality.

FIG. 2 shows a detailed example of the optimization module 110 shown in FIG. 1 . This module includes a plurality of optimization engines (m search engines), which can be called optimization methods 220, and differ in their complexity, formulation and required computing resources. Optimization method 220 may include a simple exhaustive search method (which perturbs all predefined parameters within a range of values), and multiple heuristic engines.

The optimization module also maintains in table 230 a record of each method's predetermined level of complexity. This optimization module is extensible, since any engine found can be added behind it, as long as its relative complexity is defined relative to the complexity of other methods. The parameter and control signal scheduler 235 in the optimization module invokes the appropriate optimization engine based on the appropriate level of complexity that the available resources can bear. The scheduler gets control signals and calls the appropriate method (ie engine) and structure parameters. In this embodiment, the computing resource analyzer 130 (shown in FIG. 1 ) selects and/or provides a suggested level of complexity, but the computing resource analyzer is an optional feature. The level of complexity of the proposal can be selected by, for example, an optimization module.

As an illustration and not a limitation, some of the methods in the optimization module can be heuristic, they can go from a greedy approach where good results are constructed by cascading, to a more localized heuristic search method, Examples include genetic algorithm (GA), simulated annealing (SA), forbidden search (TS), simulated evolution (SE) and stochastic evolution (SE) and any number of mixtures of these methods.

With regard to this optimization module, several examples of the aforementioned heuristic search methods are given here in more detail. However, those methods are known to those skilled in the art.

When used with a heuristic approach, the video processing algorithm may employ, for example, a genetic algorithm. Genetic Algorithms are developed towards a system architecture that allows for the best image quality.

A genetic algorithm is an iterative procedure that maintains a set of potential "candidate" solutions, evaluates these "candidate" solutions, and assigns them a suitability value. Genetic algorithm is a well-known algorithm for solving complex problems. In the book "Progress in Parallel Algorithms" published in 1990 by Kronsjo and Shumshesuddin, "Genetic Algorithms in Optimization and Adaptation", pages 227-276, are introduced here as background material.

A genetic algorithm is an iterative procedure that maintains a large number of candidate solutions encoded in the form of chromosome strings. Each chromosome gives a specific way of connecting different video processing modules, as well as the processing order of the sequences. Each chromosome then includes a number of genes, which are video processing functions and their order for the video optimization process.

Simulated evolution is a method, rather than a fixed algorithm, in which a global minimum is computed for the solution at the level of complexity to be employed by the optimization block.

TABU search is an adaptive procedure for solving combinatorial optimization problems that is able to make heuristics, continuing to explore descending paths without returning to places it has traveled before.

Simulation evolution is a method that uses a series of equations to determine the appropriate level of complexity.

Stochastic evolution utilizes genetic random variables that usually depend on a parameter that accounts for the timing of the genetic program.

Figure 3A details the OIQ assessment module. The OIQ assessment module 300 includes multiple objective image quality metrics (K metrics) of varying complexity. The OIQ module maintains in table 330 a record of the methods it constitutes a measure of, and the level of complexity assumed by each method. The OIQ module is extensible because any proposed metric can be added, given its complexity in advance. The video stream scheduler 310 in the OIQ module initiates appropriate OIQ metrics based on an appropriate level of complexity that the available resources can bear.

Regarding metrics, each objective metric 320 has an evaluation, called a quality factor, according to the required performance and allowed complexity. In other words, the figure of merit indicates the quality of the video signal based on this metric. The correlation coefficients of human perception of video quality allow for scalable models where new objective measures can be added to or removed from the system, given its correlation with human perception.

FIG. 3B illustrates the scalable objective metric 320 shown in FIG. 3A, which illustrates the complexity table 330 in more detail. Each measure has a correlation coefficient (R, 1≤i≤n), with 1 corresponding to the first measure f _i n corresponding to the nth measure f _n . On the basis of each correlation coefficient, the evaluator gives a weight w _i to each quality factor, trying to maximize the total correlation coefficient R of the final composite measure F expressed by the following formula for predetermined subjective results:

$\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; R</span>}}{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; \{</span>{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \}</span>$

For fast systems (real time), the complex measurement metric can be turned off and the judgment is made without its quality factor. For simulation and video chain optimization, which can take longer in this case, more complex metrics are turned on, their results taken into account in the final objective measurement.

The optional computing resource analysis module 400 can detect available computing resources and determine an appropriate analyzer complexity level and an appropriate OIQ module complexity level.

In this way, since certain complex metrics are turned off to take into account real-time requirements, it is possible to provide computing resource availability to the OIQ evaluation module, removing certain metrics from a series of choices, because the required resources may exceed the available capacity. This value is also received by the system optimization module 200, and the resource requirements of the selected optimization method 220 must be compatible with the given available resources.

The end result is that the selected algorithm is optimal in terms of achieving the best objective image quality with the available resources that can be used. This objective image quality then correlates to the subjective image quality of the human visual system. Based on resource availability at any point at any time, different algorithms and/or different metrics may be chosen for a given image. This flexible approach results in the highest image quality, since for static systems a conservative threshold is required in terms of choosing algorithms or metrics such that resource availability constraints are not exceeded. If resources are overloaded by algorithmic or metric requirements, a system outage will occur when an alternative algorithm is selected to accommodate the resources at a given moment, at least a time outage perceivable by human vision.

4 and 5 are flow charts of the method of the present invention.

In step (a), the order of the concatenated video functions needs to be determined.

In step (b), an optimization method is selected to optimize the processing of the video stream.

After the video stream is output from the video processing module in step (c), the objective image quality of this video stream is evaluated by selecting a metric for the value of computing resources according to the correlation coefficient and the associated complexity.

Fig. 5 illustrates how in step (c) an objective image quality is evaluated by determining a correlation coefficient according to the previous formula.

As mentioned earlier, the Computational Resource module can be bypassed if required to achieve a certain level of complexity in the Optimization module and/or OIQ module.

Various modifications to the above systems and methods may be made by those skilled in the art without departing from the spirit and scope of the invention.

Claims (14)

1. An optimized video processing system, comprising: A video processing module (100) for processing an incoming video stream, said video processing module comprising structural parameters (105) for determining a concatenated video function ( 102), determining the bit precision between the data of any successive concatenated functions; an optimization module (200) for optimizing the processing of said video stream, said optimization module being in communication with said video processing module, said optimization module comprising a plurality of optimization engines (220), each having an associated degree of complexity ( 230), the optimization module comprising means for selecting an optimization engine (235) according to a degree of complexity related to said value of available computing resources; An objective image quality (OIQ) evaluation module (300) for evaluating the image quality of the video stream (310) output by the video processing module, the OIQ evaluator includes a plurality of objective image quality metrics ( 320), this OIQ evaluation module comprising means for selecting a measure from a plurality of objective measures according to the correlation coefficient R and complexity of said value of available computing resources (310). 2. The system of claim 1, wherein said means for selecting a metric by said OIQ evaluation module comprises determining said correlation coefficient R according to the following formula: $\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; R</span>}}{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; \{</span>{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \}</span><span\; class="notranslate">\; ,</span>$ where F is a final measure of the video quality evaluated by the system, and F is determined by finding a set of weights w _i , which are related to each measure f _i of multiple objective measures (the value of f _i is between 1 and n ) will make the correlation coefficient R reach the maximum with the predetermined subjective impression. 3. The optimized video processing system of claim 1, further comprising a computing resource analyzer (400) for selecting a relative complexity level of at least one of said video processing module, said optimization module and said OIQ evaluation module. 4. The system of claim 1, wherein the optimization module includes deterministic and non-deterministic optimization engines. 5. The system of claim 4, wherein the optimization module includes a heuristic search engine comprising at least one of genetic algorithm (GA), simulated annealing (SA), forbidden search (TS), simulated evolution (SE) and stochastic evolution. 6. The system of claim 2, wherein the optimization module includes a heuristic search engine comprising at least one of genetic algorithm (GA), simulated annealing (SA), forbidden search (TS), simulated evolution (SE) and stochastic evolution. 7. The video processing system of claim 1, wherein the optimization module is scalable. 8. The video processing system of claim 2, wherein the optimization module is scalable. 9. The video processing system of claim 1, wherein the OIQ evaluation module is scalable. 10. The video processing system of claim 2, wherein the OIQ evaluation module is scalable. 11. The video processing system of claim 7, wherein the OIQ evaluation module is scalable. 12. The video processing system of claim 2 , wherein the computing resource analysis module provides for the video processing module, At least one of the optimization module and the OIQ evaluation module selects a level of complexity. 13. A method of optimizing a video algorithm for available computing resources, the method comprising: (a) the video processing module determines the order of the concatenated video functions for processing the video stream input to the video processing module according to the relative complexity associated with a value of available computing resources; (b) selecting an optimization method for optimizing the processing of the video stream, which optimization method is selected from a plurality of optimization methods according to the degree of complexity in relation to said value of available computing resources; (c) assessing the objective image quality of the video stream after the video stream is output from the video processing module; The evaluation of the objective image quality of the video stream is determined by selecting a metric from a plurality of metrics according to a correlation coefficient R and a degree of complexity related to a value of computing resources. 14. The system of claim 13, wherein the evaluation of the objective image quality in step (c) comprises determining the correlation coefficient R according to the following formula: $\mathrm{<span\; class="notranslate">\; f</span>}<span\; class="notranslate">\; =</span>\underset{\mathrm{<span\; class="notranslate">\; R</span>}}{\mathrm{<span\; class="notranslate">\; max</span>}}<span\; class="notranslate">\; \{</span>{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}^{\mathrm{<span\; class="notranslate">\; no</span>}}{\mathrm{<span\; class="notranslate">\; f</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; \}</span><span\; class="notranslate">\; ,</span>$ where F is a final measure of the video quality evaluated by the system, and F is determined by finding a set of weights w _i , which are related to each measure f _i of multiple objective measures (the value of f _i is between 1 and n ) will make the correlation coefficient R reach the maximum with the predetermined subjective impression.

Images