CN108876811B - Image processing method, device and computer readable storage medium - Google Patents
Image processing method, device and computer readable storage medium Download PDFInfo
- Publication number
- CN108876811B CN108876811B CN201710325571.6A CN201710325571A CN108876811B CN 108876811 B CN108876811 B CN 108876811B CN 201710325571 A CN201710325571 A CN 201710325571A CN 108876811 B CN108876811 B CN 108876811B
- Authority
- CN
- China
- Prior art keywords
- analysis
- subgraphs
- rectangular image
- subgraph
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/246—Analysis of motion using feature-based methods, e.g. the tracking of corners or segments
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/20—Analysis of motion
- G06T7/269—Analysis of motion using gradient-based methods
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10016—Video; Image sequence
Landscapes
- Engineering & Computer Science (AREA)
- Multimedia (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Image Analysis (AREA)
Abstract
The invention provides an image processing method, an image processing device and a computer readable storage medium, wherein the method comprises the following steps: setting each of frame images in video the movable object is arranged as a rectangular image block; covering the rectangular image block by setting an analysis subgraph with a set size in the set frame image, and determining the number range of the analysis subgraphs; under the limitation of preset constraint conditions and the number range of the analysis subgraphs, setting an adaptability function of the analysis subgraphs based on a genetic algorithm according to the positions and the number of the rectangular image blocks, and carrying out iterative computation on the adaptability function to obtain configuration parameters of the analysis subgraphs; and detecting a rectangular image block in the analysis subgraph corresponding to the configuration parameter to acquire the movable object in the rectangular image block. Compared with the existing detection algorithm, the method has higher precision, and the detection speed is effectively improved compared with the method for detecting the blocking of any frame of image in the video.
Description
Technical Field
The present invention relates to the field of image processing technologies, and in particular, to an image processing method, an image processing device, and a computer readable storage medium.
Background
Deep learning-based target detection is a research hotspot in recent years, and by improving convolutional neural networks, the speed of target detection is greatly improved, and even real-time is achieved. However, in many existing deep learning methods, the input picture is often downsampled, and a small-resolution image is used, so that the operand is reduced, and the detection speed is further improved. However, for high definition large graphs containing many detail features, loss of detail features will reduce detection accuracy. If the high-definition picture is directly input into the depth network for training without downsampling, huge calculation amount causes great speed loss.
In the field of monitoring, the quality level of video is uneven due to shooting angle, environmental factors and complexity of the monitoring scene itself. When the size difference of the target objects in the monitoring video is larger, or the targets are denser, or the local targets are in an environment with poorer light, the detection effect becomes very poor when the picture is directly reduced by using the traditional detection method. If the picture is not reduced, the original picture is sliced and then is segmented into a depth network for detection, so that the detection accuracy can be improved to a certain extent, but the efficiency is very low, and a large number of image blocks are likely to contain few targets, even background and no targets.
Disclosure of Invention
The invention aims to solve the technical problem of providing an image processing method, image processing equipment and a computer readable storage medium, and overcomes the defect of low target detection efficiency under the condition that an original image is not compressed in the prior art.
The technical scheme adopted by the invention is that the image processing method comprises the following steps:
setting each movable object in the set frame image in the video as a rectangular image block;
covering the rectangular image block by setting an analysis subgraph with a set size in the set frame image, and determining the number range of the analysis subgraphs;
under the limitation of preset constraint conditions and the number range of the analysis subgraphs, setting an adaptability function of the analysis subgraphs based on a genetic algorithm according to the positions and the number of the rectangular image blocks, and carrying out iterative computation on the adaptability function to obtain configuration parameters of the analysis subgraphs;
and detecting a rectangular image block in the analysis subgraph corresponding to the configuration parameter to acquire the movable object in the rectangular image block.
Further, the rectangular image block is an image within a smallest circumscribed rectangle of the movable object.
Further, the configuration parameters of the analysis subgraph at least comprise one of the following: the number of the analysis subgraphs, the positions of the analysis subgraphs in the set frame image, and the positions of the rectangular image blocks in the analysis subgraphs.
Further, before the setting each movable object in the set frame image in the video as the rectangular image block, the method further includes:
calculating a plurality of continuous frame images of the video based on a preset movable object analysis model to obtain an immovable object in any frame image of the video;
and subtracting all the objects in the set frame image of the video from the immovable object to obtain the movable object in the set frame image.
Further, after the obtaining the movable object in the set frame image of the video, the method further includes:
and filtering noise on the movable object in the set frame image of the video based on a preset filtering model.
Further, the determining the number range of the analysis subgraphs by setting the analysis subgraphs with set sizes in the set frame image to cover the rectangular image block includes:
determining a lower limit of the number range of the analysis subgraphs based on the areas of the analysis subgraphs and the sum of the areas of all the rectangular image blocks;
and determining the upper limit of the number range of the analysis subgraphs by setting the analysis subgraphs to cover all the rectangular image blocks in the image based on a greedy algorithm.
Further, the determining the lower limit of the number range of the analysis subgraphs based on the sum of the areas of the analysis subgraphs with the set size and the areas of all the rectangular image blocks includes:
and rounding up the ratio of the sum of the areas of all the rectangular image blocks to the area of the analysis subgraph, and setting the ratio as the lower limit of the number range of the analysis subgraph.
Further, the determining, by setting the analysis subgraph to cover at least one rectangular image block at a preset position in the image and by setting the analysis subgraph to cover all the rectangular image blocks in the image based on a greedy algorithm, an upper limit of a number range of the analysis subgraph includes:
based on a greedy algorithm, the first quantity value of the analysis subgraph is obtained by setting the analysis subgraph to cover all the rectangular image blocks in the image;
the second quantity value of the analysis subgraph is obtained by setting the analysis subgraph to cover at least one rectangular image block at a preset position in the image;
the smaller of the first and second quantity values is set as an upper limit of a quantity range of the analysis sub-graph.
Further, the preset constraint condition includes:
and moving one or more rectangular image blocks to a blank area which does not cover any rectangular image block in the analysis subgraph, so that the analysis subgraph covers all the rectangular image blocks.
Further, the fitness function is to respectively carry out weighted summation on the objective function and the penalty function;
the objective function includes: respectively carrying out weighted summation on functions representing the number of the analysis subgraphs, functions representing the number of rectangular image blocks which are not covered by any analysis subgraph, functions representing the number of rectangular image blocks which are moved to the blank area and functions representing the distance between any two analysis subgraphs;
the penalty function includes: when the area of the blank area which does not cover the rectangular image blocks in all the analysis subgraphs is smaller than or equal to the set multiple of the sum of the areas of the rectangular image blocks which are not covered by any analysis subgraph, the punishment function is a first preset value;
and when the areas of the blank areas which are not covered by the rectangular image blocks in all the analysis subgraphs are larger than the set times of the sum of the areas of the rectangular image blocks which are not covered by any analysis subgraph, the penalty function is a second preset value.
Further, before the detecting the rectangular image block in the analysis subgraph corresponding to the configuration parameter, the method further includes:
and under the condition that the analysis subgraph corresponding to the configuration parameters of the analysis subgraph does not cover all the rectangular image blocks, carrying out iterative computation on the fitness function again based on a genetic algorithm by adjusting the set multiple in the penalty function and/or the number range of the analysis subgraphs to obtain the configuration parameters of the analysis subgraphs.
The present invention also provides an image processing apparatus including a processor and a memory;
the processor is configured to execute an image processing program stored in the memory to implement the steps of the above image processing method.
The present invention also provides a computer-readable storage medium storing one or more programs executable by one or more processors to implement the steps of the above-described image processing method.
By adopting the technical scheme, the invention has at least the following advantages:
the image processing method, the device and the computer readable storage medium can preliminarily determine the approximate position of the movable object in the video by extracting the movable object when the movable object in the video is monitored, and replace any frame of image in the video by a certain number of analysis subgraphs containing all the movable objects of any frame of image in the video according to the prior characteristic of the distribution of the movable object, thereby retaining the detail characteristic of the image, having higher precision than the prior detection algorithm and effectively improving the detection speed compared with the method for detecting any frame of image in the video in a blocking way.
Drawings
FIG. 1 is a flowchart of an image processing method according to a first embodiment of the present invention;
FIG. 2 is a flowchart of an image processing method according to a second embodiment of the present invention;
FIG. 3 is a schematic view of a third embodiment of the present invention in which each movable object is arranged as a rectangular image block;
FIG. 4 is a flowchart of a genetic algorithm method according to a third embodiment of the present invention.
Detailed Description
In order to further describe the technical means and effects adopted by the present invention for achieving the intended purpose, the following detailed description of the present invention is given with reference to the accompanying drawings and preferred embodiments.
In a first embodiment of the present invention, as shown in fig. 1, an image processing method includes the following specific steps:
step S101, each movable object in the set frame image in the video is set as a rectangular image block.
Wherein the method comprises the steps of the process comprises, a rectangular image block is an image within the smallest bounding rectangle of the movable object.
Step S102, the number range of the analysis subgraphs is determined by setting the analysis subgraphs with set sizes in the set frame image to cover the rectangular image blocks.
Optionally, step S102 includes:
determining the lower limit of the number range of the analysis subgraph based on the area of the analysis subgraph and the sum of the areas of all the rectangular image blocks;
the upper limit of the number range of the analysis subgraphs is determined by setting the way that the analysis subgraphs cover at least one rectangular image block at a preset position in the image and by setting the way that the analysis subgraphs cover all rectangular image blocks in the image based on a greedy algorithm.
Step S103, under the limitation of preset constraint conditions and the number range of the analysis subgraphs, setting the fitness function of the analysis subgraphs according to the positions and the number of the rectangular image blocks based on a genetic algorithm, and carrying out iterative computation on the fitness function to obtain configuration parameters of the analysis subgraphs.
Wherein the configuration parameters of the analysis subgraph comprise at least one of the following: analyzing the number of sub-images, analyzing the position of the sub-images in the set frame image, and analyzing the position of the rectangular image block in the sub-images.
Optionally, step S103 includes:
under the preset constraint condition and the limitation of the number range of the analysis subgraphs, based on genetic algorithm, according to the position and the number of the rectangular image blocks, setting an adaptability function of the analysis subgraph, and carrying out iterative calculation on the adaptability function to obtain configuration parameters of the analysis subgraph.
The fitness function is to respectively carry out weighted summation on the objective function and the penalty function;
the objective function includes: respectively carrying out weighted summation on functions representing the number of analysis subgraphs, functions representing the number of rectangular image blocks which are not covered by any analysis subgraph, functions representing the number of rectangular image blocks which are moved to a blank area and functions representing the distance between any two analysis subgraphs;
the penalty function includes: when the area of the blank area which is not covered by the rectangular image block in all the analysis subgraphs is smaller than or equal to the set multiple of the sum of the areas of the rectangular image blocks which are not covered by any analysis subgraph, the penalty function is a first preset value;
and when the areas of the blank areas which are not covered by the rectangular image blocks in all the analysis subgraphs are larger than the set times of the sum of the areas of the rectangular image blocks which are not covered by any analysis subgraph, the penalty function is a second preset value.
The configuration parameters of the analysis subgraph include at least one of: analyzing the number of sub-images, analyzing the position of the sub-images in the set frame image, and analyzing the position of the rectangular image block in the sub-images.
And step S104, detecting the rectangular image block in the analysis subgraph corresponding to the configuration parameter to acquire the movable object in the rectangular image block.
According to the image processing method disclosed by the first embodiment of the invention, when the movable object in the video is monitored, the approximate position of the movable object in the video can be preliminarily determined by the method of extracting the movable object, and according to the prior characteristic of the distribution of the movable object, a certain number of analysis subgraphs containing all the movable objects of any frame of images in the video are used for replacing any frame of images in the video, so that the detail characteristic of the picture is reserved, the accuracy is higher than that of the conventional detection algorithm, and the detection speed is effectively improved compared with that of the method of carrying out block detection on any frame of images in the video.
In a second embodiment of the present invention, as shown in fig. 2, an image processing method includes the following specific steps:
step S201, calculating a plurality of continuous frame images of the video based on a preset movable object analysis model to obtain an immovable object in any frame image of the video;
and subtracting all the objects in the set frame image of the video from the immovable object to obtain the movable object in the set frame image.
Wherein the preset movable object analysis model includes but is not limited to: a mixed Gaussian model (Gaussian Mixture Model), a VIBE (Background Generation And Foreground Detection) model, a three-frame differential algorithm model and the like.
For example: step S201 includes: based on the Gaussian mixture model, calculating a plurality of continuous frame images of the video to obtain an immovable object B in any frame image of the video (x,y) ；
All objects I in the set frame image of the video t And immovable object B (x,y) Subtracting to obtain the set frame imageMovable object in (a)Wherein (x, y) is the pixel position; t is a frame sequence number of the video, and t is a positive integer; i is the number of the movable object in the set frame image, i is [1, M]M is the total number of movable objects in the set frame image.
Step S202, filtering noise on a movable object in a set frame image of the video based on a preset filtering model.
Wherein the preset filtering model includes but is not limited to: a median filtering model, a morphological filtering model, and the like.
For example: step S202, including: based on a preset median filtering model and a morphological filtering model, a movable object in a set frame image of a video is subjected toFiltering the noise;
step S203 sets each movable object in the set frame image in the video as a rectangular image block.
Wherein the rectangular image block is an image within the smallest circumscribed rectangle of the movable object.
The manner of setting each movable object as a rectangular image block includes:
extracting the outermost contour of each movable object;
setting the minimum circumscribed rectangle for the outermost contour of each movable object;
the image within each minimum bounding rectangle is set to a rectangular image block.
For example: step S203 includes:
each movable object is movedSet as rectangular image block E i The method comprises the steps of carrying out a first treatment on the surface of the Wherein, rectangular image block A i Is a movable objectA minimum circumscribed rectangular inner image of (2);
the manner of setting each movable object as a rectangular image block includes:
extracting each movable objectIs the outermost contour of (a);
each movable object is movedSetting a minimum circumscribed rectangle on the outermost contour of the frame;
setting the image in each minimum bounding rectangle as a rectangular image block E i 。
Wherein, rectangular image block E i Length of (h) i Rectangular image block E i Is w of width of i 。
Step S204, the number range of the analysis subgraphs is determined by setting the analysis subgraphs with set sizes in the set frame image to cover the rectangular image blocks.
Optionally, step S204 includes:
determining the lower limit of the number range of the analysis subgraph based on the area of the analysis subgraph and the sum of the areas of all the rectangular image blocks;
determining the upper limit of the number range of the analysis subgraphs by setting a mode of covering at least one rectangular image block by the analysis subgraphs at preset positions in the image and by setting a mode of covering all rectangular image blocks by the analysis subgraphs in the image based on a greedy algorithm;
wherein determining a lower limit of the number range of the analysis subgraph based on the area of the analysis subgraph and the sum of the areas of all the rectangular image blocks comprises:
the ratio of the sum of the areas of all the rectangular image blocks to the area of the analysis sub-graph is rounded up and set as the lower limit of the number range of the analysis sub-graph.
Determining an upper limit of a number range of the analysis subgraph by setting a manner that the analysis subgraph covers at least one rectangular image block at a preset position in the image and by setting a manner that the analysis subgraph covers all rectangular image blocks in the image based on a greedy algorithm, comprising:
based on a greedy algorithm, setting an analysis subgraph in an image to cover all rectangular image blocks, and obtaining a first quantity value of the analysis subgraph;
the second quantity value of the analysis subgraph is obtained by setting the analysis subgraph to cover at least one rectangular image block at a preset position in the image;
the smaller of the first and second quantity values is set as the upper limit of the quantity range of the analysis sub-graph.
For example, step S204 includes:
the ratio of the sum of the areas of all the rectangular image blocks to the area of the analysis subgraph is rounded upwards and is set as the lower limit min of the number range of the analysis subgraph;
where H is the height of the analysis subgraph and W is the width of the analysis subgraph.
Based on a greedy algorithm, setting an analysis subgraph in an image to cover all rectangular image blocks, and obtaining a first quantity value max1 of the analysis subgraph;
the second quantity value max2 of the analysis subgraph is obtained by setting the analysis subgraph to cover at least one rectangular image block at a preset position in the image;
setting the smaller value of the first quantity value max1 and the second quantity value max2 as the upper limit max of the quantity range of the analysis subgraph;
the number range of analysis subgraphs is determined [ min, max ].
Step S205, under the limitation of preset constraint conditions and the number range of the analysis subgraphs, setting the fitness function of the analysis subgraphs according to the positions and the number of the rectangular image blocks based on a genetic algorithm, and carrying out iterative computation on the fitness function to obtain configuration parameters of the analysis subgraphs.
Wherein the configuration parameters of the analysis subgraph comprise at least one of the following: analyzing the number of sub-images, analyzing the position of the sub-images in the set frame image, and analyzing the position of the rectangular image block in the sub-images.
The preset constraint conditions comprise:
the analysis subgraph is made to cover all the rectangular image blocks by moving one or more rectangular image blocks to a blank area in the analysis subgraph that does not cover any rectangular image block.
Optionally, step S205 includes:
under the limitation of preset constraint conditions and the number range of the analysis subgraphs, setting the fitness function of the analysis subgraphs based on a genetic algorithm according to the positions and the number of the rectangular image blocks, and carrying out iterative calculation on the fitness function to obtain the configuration parameters of the analysis subgraphs.
The fitness function is to respectively carry out weighted summation on the objective function and the penalty function;
the objective function includes: respectively carrying out weighted summation on functions representing the number of analysis subgraphs, functions representing the number of rectangular image blocks which are not covered by any analysis subgraph, functions representing the number of rectangular image blocks which are moved to a blank area and functions representing the distance between any two analysis subgraphs;
the penalty function includes: when the area of the blank area which is not covered by the rectangular image block in all the analysis subgraphs is smaller than or equal to the set multiple of the sum of the areas of the rectangular image blocks which are not covered by any analysis subgraph, the penalty function is a first preset value;
and when the areas of the blank areas which are not covered by the rectangular image blocks in all the analysis subgraphs are larger than the set times of the sum of the areas of the rectangular image blocks which are not covered by any analysis subgraph, the penalty function is a second preset value.
The configuration parameters of the analysis subgraph include at least one of: analyzing the number of sub-images, analyzing the position of the sub-images in the set frame image, and analyzing the position of the rectangular image block in the sub-images.
Step S206, under the condition that the analysis subgraph corresponding to the configuration parameters of the analysis subgraph does not cover all the rectangular image blocks, the configuration parameters of the analysis subgraph are obtained by adjusting the set times in the penalty function and/or the number range of the analysis subgraphs and performing iterative computation on the fitness function again based on a genetic algorithm.
Step S207, detecting a rectangular image block in the analysis subgraph corresponding to the configuration parameters of the analysis subgraph to obtain a movable object in the rectangular image block.
According to the image processing method disclosed by the second embodiment of the invention, when the movable object in the video is monitored, the approximate position of the movable object in the video can be preliminarily determined by the method of extracting the movable object, and according to the prior characteristic of the distribution of the movable object, a certain number of analysis subgraphs containing all the movable objects of any frame of images in the video are used for replacing any frame of images in the video, so that the detail characteristic of the picture is reserved, the accuracy is higher than that of the conventional detection algorithm, and the detection speed is effectively improved compared with that of the method of carrying out block detection on any frame of images in the video.
In a third embodiment of the present invention, as shown in fig. 3 to 4, an image processing method includes the following specific steps:
step S701, calculating from continuous K frames of images in video by using a Gaussian mixture model (Gaussian Mixture Model) to obtain an immovable object set B in any frame of images (x,y) The method comprises the steps of carrying out a first treatment on the surface of the Wherein K is a positive integer greater than 1;
calculating t-th frame image I t With immovable object set B (x,y) Obtaining a movable object set in the t-th frame image
For movable object setPerforming binarization processing to obtain binarized movable object set +.>
Binarizing a set of movable objects by median filtering and morphological filteringFiltering the noise in the filter;
wherein (x, y) is the pixel position in the image, t is the image frame sequence number in the video, t is [1, A ]]A is the total number of image frames in the video. Binarized movable object setThe non-zero pixel points are movable object pixel points.
Step S702, based on binarized movable object setObtaining the profile C of each movable object i ；
As shown in fig. 3, each movable object is set to a rectangular image block D according to the contour of each movable object i 。
i is E [1, M ], M is the total number of movable objects in the image of the t frame;
rectangular image block D i Center pixel position (x) i ,y i ) Is the position of the ith movable object; rectangular image block D i Length of (h) i The method comprises the steps of carrying out a first treatment on the surface of the Rectangular image block D i Is w of width of i 。
Rectangular image block D i To be able to contain the contour C of the movable object i Rectangular image block D of all pixels in the image block i The abscissa of the top left corner vertex is the contour C of the movable object i Minimum value of abscissa in all pixel point sets in the image block D i The ordinate of the top left corner vertex is the contour C of the movable object i Minimum value of ordinate in all pixel point sets; rectangular image block D i Is the contour C of the movable object i Maximum value of difference value of two-point longitudinal coordinates in pixel point setRectangular image block D i Is the outline C of the movable object i And the maximum value of the difference value of the two-point horizontal coordinates in the pixel point set.
Step S703, setting r horizontal lines and c vertical lines at equal distances in the t-th frame image; r is not less than 1, a step of; c is more than or equal to 1;
an analysis subgraph is arranged at the intersection point position of the transverse line and the longitudinal line, and a rectangular image block D is covered by the analysis subgraph i Obtaining an overlay rectangular image block D i The number N of the analysis subgraphs;
N＝Ψ-n；
where n is the number of tiles D that do not cover any rectangle i Is a sub-graph number;
Ψ＝ceil((h1/H)×(w1/W))；
ceil ((H1/H) × (W1/W)) is the smallest integer greater than or equal to ((H1/H) × (W1/W));
h1 is the height of any frame of image of the video; w1 is the width of any frame of image of the video;
h is the high of the analytical subgraph; w is the width of the analysis subgraph;
min=ceil (sum of all rectangular image block areas/(h×w));
ceil (sum of all rectangular image block areas/(h×w)) is a minimum integer greater than or equal to (sum of all rectangular image block areas/h×w);
max1=ceil ((p0×sum of rectangular image block areas not covered by any analysis sub-graph+sum of rectangular image block areas covered by analysis sub-graph)/(h×w));
ceil ((p0×sum of rectangular image block areas not covered by any analysis sub-graph + sum of rectangular image block areas covered by analysis sub-graph)/(hxw)) is a minimum value of greater than or equal to (p0×sum of rectangular image block areas not covered by any analysis sub-graph + sum of rectangular image block areas covered by analysis sub-graph)/(hxw); p0 > 1;
based on a greedy algorithm, setting Max2 analysis subgraphs in the t-th frame image, and covering all rectangular image blocks through the Max2 analysis subgraphs;
setting Max3 to be the minimum value of Max1 and Max2 by comparing Max1 and Max2;
obtaining the number range [ Min, max3] of the analysis subgraphs;
setting total analysis subgraphs in a t-th frame image, wherein at least one rectangular image block is covered by any analysis subgraph, and different analysis subgraphs cover different rectangular image blocks;
total＝p1×(Max3+Min)×(Max3-Min+1)/2；
setting parameters of an elite retention policy according to total:
mem=ceil (total/p 2); ceil (total/p 2) is the smallest integer greater than or equal to (total/p 2);
s1=ceil (mem/p 3); ceil (mem/p 3) is the smallest integer greater than or equal to (mem/p 3);
S2＝mem-S1；
p1, p2 and p3 are preset values.
Step S704, setting an objective function f in the case that one or more rectangular image blocks can be moved to the blank of the analysis sub-graph;
f＝α 1 f 1 +α 2 f 2 +α 3 f 3 +α 4 f 4 ；
wherein alpha is 1 ＞0,α 2 ＞0,α 3 ＞0,α 4 ＞0；
f 1 ＝(Ψ-N)×H×W；
g' (i, j) is the movable target C j Distance analysis subgraph P i The sum of the larger value of the vertical boundary distance and the larger value of the horizontal boundary, J i Analyzed subgraph P i Covered movable object C j Is a set of numbers of (c).
dist (i) returns an N matrix D, where D (i, j) represents P i And P j The distance between the center points, MIN { D } represents the minimum value by column and returns an N-dimensional vector.
The constraint terms are:
movable object C i ，/>Analysis subgraph P j Covering the movable object Ci, i.e. [1, M]，j∈[1,N]。
The penalty term is:
wherein J is a movable target C covered by at least one analysis sub-graph j I is the movable target C not covered by any analysis subgraph i Is a preset value, and beta is more than 1.
As shown in fig. 4, based on the fitness function formed by the objective function and the penalty term, performing iterative calculation on the number of analysis subgraphs, the positions of the analysis subgraphs and the positions of the movable targets in the analysis subgraphs, and specifically, the steps are as follows:
step S7041, calculating the fitness fit of the solution by using the fitness function f=f+δf' i ；
Step S7042, with probability ρ 1 Carrying out local gradient search on the solution and moving along the ascending direction; the maximum moving step length is n units; one unit is the minimum distance between the candidate locations of the analysis subgraph;
step S7043, the individual with the largest fitness value is extracted from the candidate solutions and recorded. Judging whether the optimal fitness value is reduced or not through four continuous iterative computations; if the optimal fitness value is judged to be reduced, an optimal solution converged by the iterative algorithm is obtained, and the step is transferred to the step S7047; otherwise, step S7044 is executed;
step S7044, the solution concentration c is obtained i Combining fitness and concentration to solve propagation rate e i ；
Wherein the concentration c i Is the ratio of the number of solutions that are identical to the i-th solution to the total number of candidate solutions;
μ 1 sum mu 2 Is a preset value;
according to the propagation rate e i Sorting from big to small, and selecting by using a roulette method;
selecting S from candidate solutions 1 Individual with greatest fitness value and S 2 Individuals with the greatest reproductive rate;
step S7045, with probability ρ 2 Performing single-point cross operation on the offspring individuals;
with probability ρ 3 Single-point mutation operation is carried out on offspring individuals;
step S7046, directly placing the mem individuals obtained in step S7045 into the next generation population, and turning to step S7041;
in step S7047, the iterative calculation ends.
Step S705, verifying rationality of the number of analysis subgraphs, the positions of the analysis subgraphs and the positions of the movable targets in the analysis subgraphs obtained by iterative calculation in step S704;
specifically, step S705 includes:
iterative calculation is carried out to obtain a solution omega; the solution omega comprises the following steps: analyzing the number N' of the subgraphs, analyzing the positions of the subgraphs and analyzing the positions of the movable targets in the subgraphs;
step S7051, finding out at most R blank rectangles in each analysis subgraph, wherein three blank rectangles do not existIn inclusion relationship, denoted as R k ＝{R k1 ,R k2 ,...,R kM },k∈[1,N']If the number of such blank rectangles is smaller than M, the insufficient portion is replaced with a dot. Movable object C in the image not covered by any analysis sub-graph i I e I, where I is the numbered set of movable objects not covered by any analytical subgraph; movable object C in the image not covered by any analysis sub-graph j J e J, where J is the set of numbers of the movable object covered by the analysis subgraph.
Step S7052, from C i Is selected to have the largest area of movable object C max The method comprises the steps of carrying out a first treatment on the surface of the Judging whether or not there is one R ki ,i∈[1,M]May contain C max The method comprises the steps of carrying out a first treatment on the surface of the If so, go to step S7053; otherwise, the process goes to step S7054, where Ω does not satisfy the constraint.
Step S7053, C max Removed from I and added to J. If I is empty, the test is ended, Ω satisfies the constraint condition, otherwise, the maximum three blank rectangles in the kth analysis sub-graph are recalculated, and the process goes to step S7052.
Step S7054, judging whether the solution omega meets constraint items after the test is finished;
if the solution omega is determined to meet the constraint term, the solution omega is an approximate optimal solution;
if the solution Ω satisfies the constraint condition, the step=p is used 4 Increasing a penalty term coefficient beta, and adding 1 to the minimum number Min of the analysis subgraphs; if min=max3, max3 is incremented by 1, and the process proceeds to step S703.
According to the image processing method, when the movable object in the video is monitored, the approximate position of the movable object in the video can be initially determined by the method of extracting the movable object, and according to the prior characteristic of the distribution of the movable object, a certain number of analysis subgraphs containing all the movable objects of any frame of images in the video are used for replacing any frame of images in the video, so that the detail characteristic of the picture is reserved, the accuracy is higher than that of the conventional detection algorithm, and the detection speed is effectively improved compared with the method of detecting any frame of images in the video in a blocking mode.
A fourth embodiment of the present invention is an image processing apparatus including the following components:
a processor 110 and a memory 109. In some embodiments of the invention, these components may be connected by a bus or other means.
The processor 110 may be a general-purpose processor, such as a central processing unit (Central Processing Unit, CPU), a digital signal processor (Digital Signal Processor, DSP), an application specific integrated circuit (English: application Specific Integrated Circuit, ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. Wherein the memory is used for storing executable instructions of the processor;
a memory 109 for storing program code and for transmitting the program code to the processor 110. Memory 109 may include Volatile Memory (Volatile Memory), such as random access Memory (Random Access Memory, RAM); the Memory 109 may also include a nonvolatile Memory (Non-Volatile Memory), such as a Read-Only Memory (ROM), a Flash Memory (Flash Memory), a Hard Disk (HDD), or a Solid State Drive (SSD); memory 109 may also include a combination of the above types of memory. The memory 109 is connected to the processor 110 via a bus.
Wherein the processor 110 is configured to invoke the program code management code stored in the memory 109 to perform part or all of the steps of any one of the first to third embodiments of the present invention.
According to the image processing device disclosed by the fourth embodiment of the invention, when the movable object in the video is monitored, the approximate position of the movable object in the video can be preliminarily determined by a method for extracting the movable object, and according to the prior characteristic of the distribution of the movable object, a certain number of analysis subgraphs containing all the movable objects of any frame of images in the video are used for replacing any frame of images in the video, so that the detail characteristic of the picture is reserved, the accuracy is higher than that of the conventional detection algorithm, and the detection speed is effectively improved compared with that of a method for detecting any frame of images in the video in a blocking way.
In a fifth embodiment of the present invention, a computer-readable storage medium.
The computer storage medium may be RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.
The computer-readable storage medium stores one or more programs executable by one or more processors to implement some or all of the steps of any one of the first to third embodiments of the present invention.
A computer readable storage medium according to a fifth embodiment of the present invention stores one or more programs executable by one or more processors, which can preliminarily determine an approximate position of a movable object in a video by extracting the movable object when monitoring the movable object in the video, and replace any one frame image in the video with a certain number of analysis subgraphs including all the movable object in any one frame image in the video according to a priori feature of the movable object distribution, thereby preserving detailed features of the picture, having higher accuracy than existing detection algorithms, and effectively improving a detection speed than a method of detecting any one frame image in the video in blocks.
While the invention has been described in connection with specific embodiments thereof, it is to be understood that these drawings are included in the spirit and scope of the invention, it is not to be limited thereto.
Claims (11)
1. An image processing method, comprising:
setting each movable object in the set frame image in the video as a rectangular image block;
determining a number range of the analysis subgraphs by setting analysis subgraphs of a set size in the set frame image to cover the rectangular image blocks, wherein a lower limit of the number range of the analysis subgraphs is determined based on the area of the analysis subgraphs and the sum of the areas of all the rectangular image blocks, a mode of setting the analysis subgraphs to cover at least one rectangular image block in a preset position in the image, and an upper limit of the number range of the analysis subgraphs is determined based on a greedy algorithm by setting the analysis subgraphs to cover all the rectangular image blocks in the image;
under the limitation of preset constraint conditions and the number range of the analysis subgraphs, setting an adaptability function of the analysis subgraphs based on a genetic algorithm according to the positions and the number of the rectangular image blocks, and carrying out iterative computation on the adaptability function to obtain configuration parameters of the analysis subgraphs, wherein the configuration parameters of the analysis subgraphs at least comprise one of the following: the number of the analysis subgraphs, the positions of the analysis subgraphs in the set frame image and the positions of the rectangular image blocks in the analysis subgraphs;
and detecting a rectangular image block in the analysis subgraph corresponding to the configuration parameter to acquire the movable object in the rectangular image block.
2. The method of claim 1, wherein the rectangular image block is an image within a smallest bounding rectangle of the movable object.
3. The method of claim 1, wherein prior to said setting each movable object in the set frame image in the video as a rectangular image block, the method further comprises:
calculating a plurality of continuous frame images of the video based on a preset movable object analysis model to obtain an immovable object in any frame image of the video;
and subtracting all the objects in the set frame image of the video from the immovable object to obtain the movable object in the set frame image.
4. A method according to claim 3, wherein after said obtaining a movable object in a set frame image of said video, said method further comprises:
and filtering noise on the movable object in the set frame image of the video based on a preset filtering model.
5. The method of claim 1, wherein determining the lower limit of the number range of the analysis subgraphs based on the area of the sized analysis subgraph and the sum of the areas of all the rectangular image blocks comprises:
and rounding up the ratio of the sum of the areas of all the rectangular image blocks to the area of the analysis subgraph, and setting the ratio as the lower limit of the number range of the analysis subgraph.
6. The method according to claim 1, wherein said determining the upper limit of the number range of said analysis subgraphs by setting said analysis subgraphs to cover at least one of said rectangular image blocks at preset positions in said image and by setting said analysis subgraphs to cover all of said rectangular image blocks in said image based on a greedy algorithm comprises:
based on a greedy algorithm, the first quantity value of the analysis subgraph is obtained by setting the analysis subgraph to cover all the rectangular image blocks in the image;
the second quantity value of the analysis subgraph is obtained by setting the analysis subgraph to cover at least one rectangular image block at a preset position in the image;
the smaller of the first and second quantity values is set as an upper limit of a quantity range of the analysis sub-graph.
7. The method of claim 1, wherein the preset constraints comprise:
and moving one or more rectangular image blocks to a blank area which does not cover any rectangular image block in the analysis subgraph, so that the analysis subgraph covers all the rectangular image blocks.
8. The method of claim 1, wherein the fitness function is a weighted sum of an objective function and a penalty function, respectively;
the objective function includes: respectively carrying out weighted summation on functions representing the number of the analysis subgraphs, functions representing the number of rectangular image blocks which are not covered by any analysis subgraph, functions representing the number of rectangular image blocks which are moved to a blank area of the analysis subgraph and functions representing the distance between any two analysis subgraphs;
the penalty function includes: when the area of the blank area which does not cover the rectangular image blocks in all the analysis subgraphs is smaller than or equal to the set multiple of the sum of the areas of the rectangular image blocks which are not covered by any analysis subgraph, the punishment function is a first preset value;
and when the areas of the blank areas which are not covered by the rectangular image blocks in all the analysis subgraphs are larger than the set times of the sum of the areas of the rectangular image blocks which are not covered by any analysis subgraph, the penalty function is a second preset value.
9. The method of claim 8, wherein prior to detecting a rectangular image block in the analysis sub-graph corresponding to the configuration parameter, the method further comprises:
and under the condition that the analysis subgraph corresponding to the configuration parameters of the analysis subgraph does not cover all the rectangular image blocks, carrying out iterative computation on the fitness function again based on a genetic algorithm by adjusting the set multiple in the penalty function and/or the number range of the analysis subgraphs to obtain the configuration parameters of the analysis subgraphs.
10. An image processing apparatus, characterized in that the image processing apparatus comprises a processor and a memory;
the processor is configured to execute an image processing program stored in a memory to realize the steps of the image processing method according to any one of claims 1 to 9.
11. A computer-readable storage medium, characterized in that the computer-readable storage medium stores one or more programs executable by one or more processors to implement the steps of the image processing method according to any one of claims 1 to 9.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710325571.6A CN108876811B (en) | 2017-05-10 | 2017-05-10 | Image processing method, device and computer readable storage medium |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710325571.6A CN108876811B (en) | 2017-05-10 | 2017-05-10 | Image processing method, device and computer readable storage medium |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108876811A CN108876811A (en) | 2018-11-23 |
| CN108876811B true CN108876811B (en) | 2024-02-02 |
Family
ID=64287716
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710325571.6A Active CN108876811B (en) | 2017-05-10 | 2017-05-10 | Image processing method, device and computer readable storage medium |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108876811B (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5721595A (en) * | 1996-06-19 | 1998-02-24 | United Microelectronics Corporation | Motion estimation block matching process and apparatus for video image processing |
| EP1443460A1 (en) * | 2003-02-03 | 2004-08-04 | C.F.D. Elettronica S.P.A. | A process for digital image processing, in particular in the video monitoring field |
| CN102043953A (en) * | 2011-01-27 | 2011-05-04 | 北京邮电大学 | Real-time-robust pedestrian detection method aiming at specific scene |
| JP2013003860A (en) * | 2011-06-16 | 2013-01-07 | Iwate Univ | Object detection device and object detection program |
| CN103262121A (en) * | 2010-12-20 | 2013-08-21 | 国际商业机器公司 | Detection and tracking of moving objects |
| CN103679271A (en) * | 2013-12-03 | 2014-03-26 | 大连大学 | Collision detection method based on Bloch spherical coordinates and quantum computing |
-
2017
- 2017-05-10 CN CN201710325571.6A patent/CN108876811B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US5721595A (en) * | 1996-06-19 | 1998-02-24 | United Microelectronics Corporation | Motion estimation block matching process and apparatus for video image processing |
| EP1443460A1 (en) * | 2003-02-03 | 2004-08-04 | C.F.D. Elettronica S.P.A. | A process for digital image processing, in particular in the video monitoring field |
| CN103262121A (en) * | 2010-12-20 | 2013-08-21 | 国际商业机器公司 | Detection and tracking of moving objects |
| CN102043953A (en) * | 2011-01-27 | 2011-05-04 | 北京邮电大学 | Real-time-robust pedestrian detection method aiming at specific scene |
| JP2013003860A (en) * | 2011-06-16 | 2013-01-07 | Iwate Univ | Object detection device and object detection program |
| CN103679271A (en) * | 2013-12-03 | 2014-03-26 | 大连大学 | Collision detection method based on Bloch spherical coordinates and quantum computing |
Non-Patent Citations (2)
| Title |
|---|
| 刘丽 ; 徐浩 ; 陈瑞生 ; .改进的遗传算法在PCNN参数标定中的应用.机械制造.2013,(第05期),全文. * |
| 巨永锋,蔺广逢,蔡占华.基于遗传算法的图像识别方法.长安大学学报(自然科学版).2004,(第06期),全文. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108876811A (en) | 2018-11-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN113109368B (en) | Glass crack detection method, device, equipment and medium | |
| WO2022134996A1 (en) | Lane line detection method based on deep learning, and apparatus | |
| CN106934795B (en) | A kind of automatic testing method and prediction technique of glue into concrete beam cracks | |
| CN110913243B (en) | Video auditing method, device and equipment | |
| CN109886218B (en) | SAR image ship target detection method based on superpixel statistic dissimilarity | |
| CN116152231B (en) | Method for detecting impurities in lubricating oil based on image processing | |
| CN103955949B (en) | Moving target detecting method based on Mean-shift algorithm | |
| CN111695373B (en) | Zebra stripes positioning method, system, medium and equipment | |
| CN111968081B (en) | Automatic fish shoal counting method and device, electronic equipment and storage medium | |
| CN109376740A (en) | A kind of water gauge reading detection method based on video | |
| CN109961065B (en) | A method for target detection of surface ships | |
| CN109785356B (en) | Background modeling method for video image | |
| CN106683073A (en) | License plate detection method, camera and server | |
| CN101114337A (en) | A method for identifying and locating buildings on the ground | |
| CN112669301B (en) | High-speed rail bottom plate paint removal fault detection method | |
| CN115620279A (en) | Pod phenotype analysis method, system and device based on computer vision | |
| CN108229473A (en) | Vehicle annual inspection label detection method and device | |
| CN120047440A (en) | Method, device, equipment and medium for detecting surface cracks of aviation structural component | |
| CN104182731B (en) | A grid-shaped radar detection method based on cross detection | |
| CN113947723A (en) | High-resolution remote sensing scene target detection method based on size balance FCOS | |
| CN108876811B (en) | Image processing method, device and computer readable storage medium | |
| CN115170507B (en) | Grouting pipe surface defect detection method and system based on image data | |
| Muzammil et al. | Application of image processing techniques for the extraction of vehicle number plates over ARM target board | |
| KR20080094503A (en) | Extracting and Determining Images of Features Based on Region Expansion | |
| CN106340038B (en) | A kind of depth order inference method based on object in monocular image |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |