CN107403409B - Image processing method and device before image splicing and image splicing method and device - Google Patents
Image processing method and device before image splicing and image splicing method and device Download PDFInfo
- Publication number
- CN107403409B CN107403409B CN201710602597.0A CN201710602597A CN107403409B CN 107403409 B CN107403409 B CN 107403409B CN 201710602597 A CN201710602597 A CN 201710602597A CN 107403409 B CN107403409 B CN 107403409B
- Authority
- CN
- China
- Prior art keywords
- image
- processed
- coordinates
- stretching
- processing
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
- G06T3/4038—Image mosaicing, e.g. composing plane images from plane sub-images
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T3/00—Geometric image transformations in the plane of the image
- G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
Landscapes
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Image Processing (AREA)
Abstract
The invention provides an image processing method and device before image splicing and an image splicing method and device, wherein the image processing method before image splicing comprises the following steps: acquiring an image to be processed; determining a target area; performing stretching processing in a first direction on an image to be processed by calling a two-dimensional linear stretching mechanism, and taking an image output after the processing of the two-dimensional linear stretching mechanism as an image stretched in the first direction; and calling a two-dimensional linear stretching mechanism to perform stretching processing in a second direction on the image stretched in the first direction, and taking the image output after the two-dimensional linear stretching mechanism processing as an image for image splicing. The invention carries out secondary expansion processing on the image to be processed by setting a two-dimensional linear expansion mechanism, thereby realizing the correction of the image, reducing the code amount of correction processing, simplifying a programming algorithm and an execution process, simultaneously reducing the transmission delay of the image and improving the operation efficiency of the system.
Description
Technical Field
The invention relates to the technical field of video image preprocessing, in particular to an image processing method and device before image splicing and an image splicing method and device.
Background
At present, in the aspect of image display, the user's demands for a large screen, a wide viewing angle, and a low delay time are gradually increased. In order to meet the above requirements, a plurality of, for example, 3 cameras are often fixedly placed at a certain angle in image acquisition, so as to increase the viewing angle range, and a plurality of, for example, 3 screens are used for tiled display, so as to meet the above requirements of users. This requires that the input images from multiple cameras be seamlessly stitched together and displayed on multiple screens. The existing image splicing method mainly comprises the following three types:
the first type: and (4) image fusion splicing method. The method realizes the splicing of two images by utilizing the steps of feature point extraction, coordinate transformation and image fusion. This approach requires that there be a sufficient number of matchable feature points between the images being stitched. This requires that the overlapping area of the images is typically above 30%. This greatly reduces the actual footprint of the camera in the system, thereby reducing the efficiency of use of the camera and increasing the cost of the system. Meanwhile, the calculation process involves a large number of equations and repeated image processing, the process is complex, and resources consumed by calculation directly result in cost increase and large-area popularization is not possible. Meanwhile, the complexity of calculation also makes the image splicing process unable to meet the current requirements of high definition and real time.
The second type: the optical center sharing method. The method requires that the placement positions of a plurality of camera lenses of an input image optically meet the requirement of a common optical center, and then the input images are directly spliced. This method requires high accuracy of the physical position of the camera. However, due to the volume limitation of the cameras, the physical position of the optical center of each camera is inside the camera, so that the superposition of the physical positions of the cameras cannot be realized on the preset arrangement of the cameras, and the requirement of the common optical center cannot be met in practice. The images directly spliced out can generate visual difference due to inconsistent perspective relation, and the spliced object is deformed.
In the third category: and (5) simply cutting and stretching. This method requires a physical location of the camera to be defined, and although a complete co-optical center cannot be achieved, an approximate co-optical center can be achieved. Some relatively simple processing is then done on the image: such as cropping, stretching, etc., and finally, image stitching is achieved. Although it is still difficult to avoid distortion due to inconsistent perspective from the visual effect, the distortion can be accepted on a broadside display screen if the position of the camera is reasonable.
By adopting the third type of image stitching method, the image problems needing to be processed generally comprise small-scale rotation, trapezoid and cutting, displacement and stretching of overlapped visual fields. Because it is difficult to accurately position a plurality of cameras in actual work, it is not possible to ensure that the positions of the cameras are completely horizontal, and especially in the presence of a pitch angle, the same horizontal line in two frames rotates by a certain angle, and the rotation direction is not fixed, as shown in fig. 1, the same horizontal line is 101 and 102 in the two frames, and the rotation by a certain angle is generated, and at this time, it is necessary to correct the image by using a small-angle rotation mechanism as shown in fig. 2. Meanwhile, due to the existence of the pitching angle of the camera, the shot picture has a ratio maladjustment in the up-down or left-right direction, as shown in fig. 3, the object at a shorter distance is larger in the shot picture, and the object at a longer distance is smaller, which causes the shot picture of a standard rectangle to become a trapezoid, and the correction needs to be performed by adopting a stretching mechanism as shown in fig. 4. And because the cameras cannot be arranged in a common optical center in practical situations, as shown in fig. 5, if the spliced pictures are continuous, the overlapped fields of view which are always existed between the spliced pictures are corrected by adopting a cutting and stretching mechanism as shown in fig. 6, namely, the overlapped parts are cut, then are moved to be aligned and then are spliced, and the cut images are reduced in size, so that the cut images need to be stretched before being spliced to reach the image size before being cut again.
The third method is a practical and low-cost splicing method. Although the method is relatively simple to implement, the problem still exists at present that different correction processing mechanisms need to be adopted according to different images, for example, for an image inclined in the horizontal or vertical direction, a small-angle rotation mechanism needs to be adopted to correct the image; for an image shot by a camera with a pitch (left-right inclination) angle (the shot image is in a vertical or horizontal maladjustment proportion), an angle adjusting mechanism is required,
for the region where the image fields overlap, it is necessary to perform processing by using a mechanism such as cropping or displacement stretching. The implementation process of each mechanism needs to adopt different processing mechanisms, which causes the program code amount for realizing image correction and splicing to be very large and the processing flow to be more, thus causing the image processing speed to be slow.
Disclosure of Invention
Therefore, the technical problem to be solved by the invention is as follows: in the prior art, the image correction and splicing program has large code amount and more processing flows, so that the image correction and splicing processing speed is low, and the image processing method and device before image splicing and the image splicing method and device are provided.
According to a first aspect, an embodiment of the present invention provides an image processing method before image stitching, including: acquiring an image to be processed, and determining the vertex coordinates of the image to be processed as the coordinates of the telescopic characteristic points; determining a target area, and acquiring a vertex coordinate of the target area as a target point coordinate; the target area is an image area of an image to be processed, and the coordinates of the telescopic characteristic points correspond to the coordinates of the target point one by one; performing stretching processing in a first direction on an image to be processed by calling a two-dimensional linear stretching mechanism, and taking an image output after the processing of the two-dimensional linear stretching mechanism as an image stretched in the first direction; a two-dimensional linear stretching mechanism is called to stretch and stretch the image stretched and stretched in the first direction in a second direction, wherein the second direction and the first direction are two directions which are perpendicular to each other, and the image output after being processed by the two-dimensional linear stretching mechanism is used as an image for image splicing; wherein, the two-dimensional linear scaling mechanism comprises: step S1, acquiring the coordinates of the telescopic characteristic points and the coordinates of the target points; step S2, stretching the image by stretching the coordinates of the stretching feature points to the coordinate positions of the quasi-target points according to the stretching direction by taking the image as a whole, wherein the coordinates of the quasi-target points coincide with the projection of the coordinates of the target points corresponding to the coordinates of the stretching feature points in the stretching direction; in step S3, the zoomed image is output.
Optionally, the acquiring the image to be processed includes: acquiring an image shot by a camera; determining a reference object from the shot image, wherein at least one surface or line direction of the reference object in the actual scene is parallel to the first direction or the second direction; and intercepting the image to be processed according to the reference object, wherein at least two edges of the intercepted image to be processed are parallel to the first direction or the second direction in the actual scene.
Optionally, intercepting the image to be processed according to the reference object includes: and taking a plane or a line of the reference object parallel to the first direction or the second direction in the actual scene as a reference, and intercepting the to-be-processed image by using a dividing line which is parallel to the reference and contains a vertex on the shot image.
Optionally, the image to be processed is a quadrangle with at least one pair of sides parallel, and the target area is a rectangle.
According to a second aspect, an embodiment of the present invention provides an image stitching method, including: processing a first image to be stitched and a second image to be stitched respectively according to the image processing method provided by the first aspect to obtain a processed first stitched image and a processed second stitched image; and splicing the processed first spliced image and the second spliced image to obtain a spliced image.
According to a third aspect, an embodiment of the present invention provides an image processing apparatus before image stitching, including: the image acquisition module is used for acquiring an image to be processed, determining the vertex coordinates of the image to be processed as the coordinates of the telescopic characteristic points; the target area determining module is used for determining a target area, and acquiring vertex coordinates of the target area as target point coordinates; the target area is an image area of an image to be processed, and the coordinates of the telescopic characteristic points correspond to the coordinates of the target point one by one; the first processing module is used for calling a two-dimensional linear stretching mechanism to perform stretching processing in a first direction on an image to be processed, and taking the image output after the processing of the two-dimensional linear stretching mechanism as an image stretched in the first direction; the second processing module is used for calling the two-dimensional linear stretching mechanism to perform stretching processing in a second direction on the image stretched in the first direction, wherein the second direction and the first direction are mutually vertical, and the image output after the two-dimensional linear stretching mechanism processing is used as an image for image splicing; wherein, the two-dimensional linear scaling mechanism comprises: step S1, acquiring the coordinates of the telescopic characteristic points and the coordinates of the target points; step S2, stretching the image by stretching the coordinates of the stretching feature points to the coordinate positions of the quasi-target points according to the stretching direction by taking the image as a whole, wherein the coordinates of the quasi-target points coincide with the projection of the coordinates of the target points corresponding to the coordinates of the stretching feature points in the stretching direction; in step S3, the zoomed image is output.
Optionally, the image acquisition module comprises: an acquisition unit configured to acquire an image captured by a camera; the determining unit is used for determining a reference object from the shot image, wherein at least one surface or line direction of the reference object in the actual scene is parallel to the first direction or the second direction; and the intercepting unit is used for intercepting the image to be processed according to the reference object, and at least two edges of the intercepted image to be processed are parallel to the first direction or the second direction in the actual scene.
Optionally, the clipping unit is further configured to take a plane or a line of the reference object in the actual scene, which is parallel to the first direction or the second direction, as a reference, and clip the to-be-processed image with a dividing line, which is parallel to the reference and includes a vertex on the captured image.
Optionally, the image to be processed is a quadrangle with at least one pair of sides parallel, and the target area is a rectangle.
According to a fourth aspect, an embodiment of the present invention further provides an image stitching apparatus, including: the image processing module is configured to process the first image to be stitched and the second image to be stitched respectively by using the image processing method provided by the first aspect of the embodiment of the present invention, so as to obtain a processed first stitched image and a processed second stitched image; and the splicing module is used for splicing the processed first spliced image and the second spliced image to obtain a spliced image.
According to a fifth aspect, an embodiment of the present invention provides an electronic device, including: at least one processor; and a memory communicatively coupled to the at least one processor; the memory stores instructions executable by a processor, and the instructions are used by at least one processor to execute any one of the image processing method before image stitching or the image stitching method.
According to a sixth aspect, the embodiment of the present invention further provides a computer-readable storage medium, on which computer instructions are stored, and the instructions, when executed by a processor, implement any one of the image processing method before image stitching or the image stitching method.
According to the embodiment of the invention, the two-dimensional linear scaling mechanism is arranged to perform secondary scaling processing on the image to be processed, so that the image is corrected, wherein the execution logic of the two-dimensional linear scaling mechanism adopted in each processing is the same, and only the input image and the output image are different, so that the code amount of correction processing is reduced, the programming algorithm and the execution process are simplified, the image transmission delay is reduced, and the system operation efficiency is improved.
Drawings
In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the embodiments or the prior art will be briefly described below, and it is obvious that the drawings in the following description are some embodiments of the present invention, and other drawings can be obtained by those skilled in the art without creative efforts.
FIG. 1 is a schematic view of a horizontal line rotated at a certain angle;
FIG. 2 is a diagram of an example of a small-scale rotation correction processing image;
FIG. 3 is an exemplary illustration of an image being out of scale up, down, or left and right;
FIG. 4 is a diagram of an example of a keystone correction mechanism processing an image;
FIG. 5 is an exemplary diagram of overlapping field-of-view images;
FIG. 6 is an exemplary diagram of a cropping, stretching, and translating linear stretching mechanism processing an image;
FIG. 7 is a flowchart of an image processing method before image stitching according to an embodiment of the present invention;
FIG. 8 is a schematic diagram of a first exemplary process of an image processing method before image stitching according to this embodiment;
FIG. 9 is a schematic process diagram of a two-dimensional linear scaling mechanism of the image processing method before image stitching according to this embodiment;
FIG. 10 is a schematic diagram of a second exemplary process of the image processing method before image stitching according to the embodiment;
FIG. 11 is a schematic diagram of a third exemplary process of the image processing method before image stitching according to the embodiment;
FIG. 12 is a schematic diagram of images obtained after image stitching according to the embodiment;
FIG. 13 is a block diagram of an image processing apparatus before image stitching according to the present embodiment;
FIG. 14 is a block diagram of an image capturing module according to the present embodiment;
FIG. 15 is a block diagram of the image stitching apparatus according to the present embodiment;
fig. 16 is a schematic diagram of a hardware structure of the electronic device according to the embodiment.
Detailed Description
The technical solutions of the present invention will be described clearly and completely with reference to the accompanying drawings, and it should be understood that the described embodiments are some, but not all embodiments of the present invention. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present invention.
In the description of the present invention, it should be noted that the terms "vertical", "horizontal", and the like indicate orientations or positional relationships based on those shown in the drawings, which are merely for convenience of describing the present invention and simplifying the description, and do not indicate or imply that the referred device or element must have a specific orientation, be constructed in a specific orientation, and be operated, and thus, should not be construed as limiting the present invention. Furthermore, the terms "first," "second," and "third" are used for descriptive purposes only and are not to be construed as indicating or implying relative importance.
In order to simplify the algorithm for processing the large-screen wide-viewing-angle video conference image and reduce the time delay, this embodiment provides an image processing method before image stitching, please refer to fig. 7, which is a flowchart of the image processing method before image stitching, and the image processing method before image stitching includes:
step S110, an image to be processed is acquired. In this embodiment, the vertex coordinates of the image to be processed are determined as the coordinates of the telescopic feature points. Referring to state a of fig. 8, the vertex coordinates of the to-be-processed image are e, f, g, and h, which are used as the coordinates of the telescopic feature points. In a specific embodiment, the to-be-processed image may be obtained by receiving the to-be-processed image transmitted by the external device, reading a locally stored to-be-processed image, or acquiring a video image by an image acquisition device such as a video camera. In an embodiment, the image capturing device, such as one or more cameras, may be provided, for example, three cameras are fixed at a certain angle, so as to increase the range of viewing angle for image capturing. Generally, a camera acquires a video image at a certain pitch angle, and due to the existence of the pitch angle, the acquired video image usually has a certain deformation.
Step S120, a target area is determined. In the embodiment, the vertex coordinates of the target area are obtained as the coordinates of the target point; the target area is an image area of the processed image to be processed, and the coordinates of the telescopic characteristic points correspond to the coordinates of the target points one to one. In this embodiment, the coordinates of the telescopic feature points correspond to the closest target point. In a specific embodiment, referring to state a, state b, and state c of fig. 8, the target area is determined to be ABCD, and the vertex coordinates A, B, C and D of the target area are used as the coordinates of the target point; e. and the image area of the processed image area formed by the telescopic characteristic points of the image to be processed, which are formed by f, g and h, corresponds to the target point coordinates A, B, C and D of the target area one by one.
Step S130, a two-dimensional linear scaling mechanism is called to perform scaling processing in a first direction on the image to be processed. In this embodiment, an image output after being processed by the two-dimensional linear scaling mechanism is used as an image scaled in the first direction. Referring to state a and state b of fig. 8, in an embodiment, the state a is an image to be processed, and a two-dimensional linear scaling mechanism is invoked to perform scaling processing in the horizontal direction on the state a. The two-dimensional linear scaling mechanism described herein is the method logic for performing two-dimensional linear scaling processing on an image.
Please refer to fig. 9, which is a process diagram of a two-dimensional linear scaling mechanism, the two-dimensional linear scaling mechanism includes:
and step S1, acquiring the coordinates of the telescopic characteristic point and the coordinates of the target point. Referring to state a of fig. 8, in the present embodiment, the feature point coordinates are points e, f, g, and h, and the target point coordinates are points A, B, C and D.
And step S2, performing stretching processing on the image by stretching the stretching feature point coordinates to the quasi-target point coordinate position according to the stretching direction by taking the image as a whole. In this embodiment, the quasi-target point coordinates coincide with the projection of the target point coordinates corresponding to the telescopic feature point coordinates in the telescopic direction. Referring to state a and state b of fig. 8, in the embodiment, the state a is expanded and contracted in the horizontal direction, the feature points e and f are expanded and contracted to the quasi target points, and the target point coordinates corresponding to the coordinates of the expanded and contracted feature points are overlapped with the projection AB edge in the expansion and contraction direction as shown by e and f in the state b. The feature points g and h are projected and overlapped with the CD edge after being stretched, so that the e point is overlapped with the projection of the point A in the horizontal direction, the g point is overlapped with the projection of the point C in the horizontal direction, and the f point is overlapped with the point B.
In step S3, the zoomed image is output. In this embodiment, the corrected image may be output to an image display device, where the image display device may be one or more screens, and the corrected image is presented in a visual manner, specifically, the screen may be an LED screen, or may be another display screen.
Step S140, a two-dimensional linear scaling mechanism is invoked to perform scaling processing in a second direction on the image scaled in the first direction. In this embodiment, the second direction and the first direction are two directions perpendicular to each other, and an image output after being processed by the two-dimensional linear scaling mechanism is used as an image for image stitching. In step S140, the image processed in step S130 is processed by calling the two-dimensional linear scaling mechanism again. For example, in the state b of fig. 8, a two-dimensional linear scaling mechanism is invoked to perform scaling processing in the vertical direction on the graph, the point e is extended to the point a, and the point g is extended to the point C.
According to the embodiment of the invention, the two-dimensional linear scaling mechanism is arranged to perform secondary scaling processing on the image to be processed, so that the image is corrected, wherein the execution logic of the two-dimensional linear scaling mechanism adopted in each processing is the same, and only the input image and the output image are different, so that the code amount of correction processing is greatly reduced, the programming algorithm and the execution process are simplified, the image transmission delay is reduced compared with the prior art, and the system operation efficiency is improved.
In the step S110, the image to be processed is obtained, in an optional embodiment, the image captured by the capturing camera determines a reference object from the captured image, where at least one surface or line direction of the reference object in the actual scene is parallel to the first direction or the second direction, the image to be processed is captured according to the reference object, and at least two sides of the captured image to be processed are parallel to the first direction or the second direction in the actual scene. The reference object in this embodiment may be an object horizontally placed in an actual scene or a vertically placed object, such as a lateral road sign, a vertical utility pole, and the like. The reference object is deformed, for example, from a rectangular shape to a trapezoidal shape, in the captured image due to the problem of the capturing angle. After the reference object is determined, the to-be-processed image to be corrected is cut out by taking the reference object as a reference, and the cut-out to-be-processed image may be a rectangle, a rhombus or a quadrilateral with at least one pair of sides parallel.
Regarding the step of capturing the image to be processed based on the reference object, in an alternative embodiment, the image to be processed is captured based on a plane or a line of the reference object in the actual scene, the plane or the line being parallel to the first direction or the second direction, and a segmentation line being parallel to the reference and including a vertex on the captured image. Referring to fig. 10, the reference object CD is parallel to the horizontal direction in the actual scene, and is captured by taking the CD as a reference and taking vertices gh and ef which are parallel to the CD and included in the captured image as dividing lines, so as to obtain the image to be processed efgh.
In an alternative embodiment, the image to be processed is a quadrilateral with at least one pair of sides parallel, and the target area is a rectangle. Referring to fig. 11, in the present embodiment, the image to be processed is efgh, ef is parallel to gh, and the target area is ABCD.
In an alternative embodiment, the physical position of the image capture device is first coarsely adjusted, and then the captured image is finely adjusted. Specifically, when the image is processed, a calibrated calibration object can be placed in the range of the shot picture as a vertical baseline or a horizontal baseline, and the image is adjusted by adopting the two-dimensional linear telescopic mechanism provided in the embodiment, so that the calibrated object in the picture is vertical or horizontal; the calibration objects with equal length are respectively placed at the upper and lower boundaries or the left and right boundaries of the shot picture, and the two-dimensional linear telescopic mechanism provided in the embodiment is adopted to adjust the image, so that the calibration objects at the upper and lower boundaries or the left and right boundaries in the image have equal length. In a specific embodiment, the coordinates of four feature points of the image are adjusted, that is, "+/-" operation is performed on the coordinates of the corner points in the horizontal and/or vertical directions, so that the positions of the markers in the picture are equal to the positions of the boundary markers in the horizontal or vertical direction.
The embodiment also provides an image stitching method, which comprises the following steps:
respectively processing the first image to be spliced and the second image to be spliced by adopting the image processing method provided by the embodiment to obtain a processed first spliced image and a processed second spliced image; and splicing the first spliced image and the second spliced image to obtain a spliced image. Referring to fig. 10, 11 and 12, in this embodiment, fig. 10 and 11 are images captured by two adjacent cameras, and two processed images e (a), (b), (g), (d) h (c) and f (a), (b), (d) h (c) are spliced to coincide the points f (b) and f (a), and the points h (c) and e (d) to obtain a spliced continuous image e (a), (g), (b), g (d) h (c).
In a specific embodiment, if a gap exists between or coincides with the first stitched image and the second stitched image, the object displayed in the two images is perfectly stitched by performing a "+/-" left-right movement operation on the coordinates of the corner points of the two images. If the two images have vertical deviation after the left-right displacement operation, the coordinates of the angular points of the images are subjected to +/-adjustment in the vertical direction until the heights are equivalent, and the increment values of the adjustment of the upper and lower angular points are required to be consistent; if the two images have the problem of inconsistent scaling of the object after the operation of moving left and right, the coordinates of the angular points of one or two images can be subjected to +/-operation, and at the moment, the incremental values of the adjustment of the upper and lower angular points have a difference value, the difference value is amplified when being larger than 0 and is reduced when being smaller than 0 until the scales of the objects in the two images are equivalent.
In an alternative embodiment, a plurality of images processed by the two-dimensional scaling mechanism are stitched. Specifically, a first image to be stitched and a second image to be stitched are selected for stitching, for convenience of operation, during actual adjustment, adjustment is performed for image feature points, that is, angular points of the images, and in an algorithm module, it is required to obtain coordinates of inner points corresponding to angular points of a certain boundary, which need to be stretched. The adjustment of the image corner points and the coordinates of the inner points in the algorithm have a mapping relation, taking the corner point at the upper left corner of the image as an example:
the original corner coordinates are (x, y) ═ 0, where x is the abscissa and y is the ordinate. Performing horizontal telescopic operation on the inner point: modifying the value of the abscissa x, for example, the coordinates of the inner point before modification are (a, 0), and when a is greater than 0, performing a right shift operation in the horizontal direction on the inner point, wherein the shift distance is a; when a is smaller than 0, the inner point is horizontally moved to the left by a distance of-a. And finally, adjusting the coordinates of the corner point at the upper left corner to (0, 0). The value of a may be 1 or other values. The operation in the vertical direction is similar to that in the horizontal direction.
The operation of the remaining three corner points is similar to the top left corner (0, 0) point described above.
The two images can be spliced by the displacement operation of the four corner points of the images.
Referring to fig. 13, an embodiment of the present invention further discloses an image processing apparatus before image stitching, including: an image acquisition module 131, a target area determination module 132, a first processing module 133, and a second processing module 134, wherein:
the image acquisition module 131 is configured to acquire an image to be processed, determine vertex coordinates of the image to be processed, and use the vertex coordinates as coordinates of the telescopic feature points;
a target area determining module 132, configured to determine a target area, and obtain vertex coordinates of the target area as target point coordinates; the target area is an image area of an image to be processed, and the coordinates of the telescopic characteristic points correspond to the coordinates of the target point one by one;
the first processing module 133 is configured to perform stretching processing in a first direction on an image to be processed, and use an image output after being processed by a two-dimensional linear stretching mechanism as an image stretched in the first direction;
the second processing module 134 is configured to perform stretching processing in a second direction on the image subjected to stretching in the first direction, where the second direction and the first direction are two directions perpendicular to each other, and take the image output after the processing by the two-dimensional linear stretching mechanism as an image for image stitching;
wherein, the two-dimensional linear scaling mechanism comprises:
step S1, acquiring the coordinates of the telescopic characteristic points and the coordinates of the target points;
step S2, stretching the image by stretching the coordinates of the stretching feature points to the coordinate positions of the quasi-target points according to the stretching direction by taking the image as a whole, wherein the coordinates of the quasi-target points coincide with the projection of the coordinates of the target points corresponding to the coordinates of the stretching feature points in the stretching direction;
in step S3, the zoomed image is output.
Referring to fig. 14, in an alternative embodiment, the image obtaining module 131 includes: an obtaining unit 1311, a determining unit 1312, and a truncating unit 1313, wherein:
an acquisition unit 1311 configured to acquire an image captured by a camera;
a determining unit 1312 configured to determine a reference object from the captured image, wherein at least one plane or line direction of the reference object in the actual scene is parallel to the first direction or the second direction;
a capturing unit 1313, configured to capture the to-be-processed image according to the reference object, where at least two sides of the captured to-be-processed image are parallel to the first direction or the second direction in the actual scene.
In an alternative embodiment, the clipping unit 1313 is further configured to, with a plane or a line of the reference object in the actual scene, which is parallel to the first direction or the second direction as a reference, clip the to-be-processed image with a dividing line that is parallel to the reference and includes a vertex on the captured image.
With reference to the third aspect of the present invention, the image to be processed is a quadrilateral with at least one pair of parallel sides, and the target area is a rectangle.
The embodiment of the present invention further discloses an image stitching device, as shown in fig. 15, the image stitching device includes: the image processing module 151 is configured to process the first image to be stitched and the second image to be stitched respectively by using the image processing method disclosed in the embodiment of the present invention, so as to obtain the processed first stitched image and the processed second stitched image, and the stitching module 152 is configured to stitch the first stitched image and the second stitched image, so as to obtain the stitched image.
The image processing apparatus and the image stitching apparatus before image stitching in this embodiment are presented in the form of functional units, where the units refer to ASIC circuits, processors and memories executing one or more software or fixed programs, and/or other devices that can provide the above-mentioned functions.
Further functional descriptions of the modules are the same as those of the corresponding embodiments, and are not repeated herein.
Referring to fig. 16, fig. 16 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present invention, and as shown in fig. 16, the terminal may include: at least one processor 161, such as a CPU (Central Processing Unit), at least one communication interface 163, memory 164, and at least one communication bus 162. Wherein a communication bus 162 is used to enable the connection communication between these components. The communication interface 163 may include a Display (Display) and a Keyboard (Keyboard), and the optional communication interface 163 may also include a standard wired interface and a standard wireless interface. The Memory 164 may be a Random Access Memory (RAM) or a Non-volatile Memory (Non-volatile Memory), such as at least one disk Memory. The memory 164 may optionally be at least one memory device located remotely from the processor 161. Wherein the processor 161 may be combined with the apparatus described in fig. 13 or fig. 15, the memory 164 stores a set of program codes, and the processor 161 calls the program codes stored in the memory 164 to execute any one of the image processing method before image stitching or the image stitching method.
The communication bus 162 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus 162 may be divided into an address bus, a data bus, a control bus, and the like. For ease of illustration, only one thick line is shown in FIG. 16, but this is not intended to represent only one bus or type of bus.
The Memory 164 may include a Volatile Memory (english: Volatile Memory), such as a random access Memory; the Memory may also include a Non-volatile Memory (english: Flash Memory), such as a Flash Memory, a Hard Disk (Hard Disk Drive, HDD) or a Solid-state Drive (SSD); the memory 1004 may also comprise a combination of memories of the kind described above.
The Processor 161 may be a Central Processing Unit (CPU), a Network Processor (NP), or a combination of CPU and NP.
The processor 161 may further include a hardware chip. The hardware chip may be an Application-specific Integrated Circuit (ASIC), a Programmable Logic Device (PLD), or a combination thereof. The PLD may be a Complex Programmable Logic Device (CPLD), a Field-Programmable Gate Array (FPGA), General Array Logic (GAL), or any combination thereof.
Optionally, memory 164 is also used to store program instructions. Processor 161 may invoke program instructions to implement the methods as shown in the above-described embodiments of the present application.
The embodiment of the invention also provides a non-transitory computer storage medium, wherein the computer storage medium stores computer executable instructions, and the computer executable instructions can execute the image processing method or the image splicing method before image splicing in any method embodiment. The storage medium may be a magnetic Disk, an optical Disk, a Read-Only Memory (ROM), a random access Memory (ram), a Flash Memory, a Hard Disk (Hard Disk Drive, abbreviated as HDD), a Solid-State Drive (SSD), or the like; the storage medium may also comprise a combination of memories of the kind described above.
The image processing method before image splicing disclosed in the above embodiment obtains an image to be processed; determining a target area; performing stretching processing in a first direction on an image to be processed by calling a two-dimensional linear stretching mechanism, and taking an image output after the processing of the two-dimensional linear stretching mechanism as an image stretched in the first direction; and calling a two-dimensional linear stretching mechanism to perform stretching processing in a second direction on the image stretched in the first direction, and taking the image output after the two-dimensional linear stretching mechanism processing as an image for image splicing. Because various image correction processes are integrated into an image stretching process, a programming algorithm and an execution process are simplified, time delay is reduced compared with the prior art, and the system operation efficiency is improved.
It will be understood by those skilled in the art that all or part of the processes of the methods of the embodiments described above can be implemented by a computer program, which can be stored in a computer-readable storage medium, and when executed, can include the processes of the embodiments of the methods described above. The storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM), a Random Access Memory (RAM), or the like. The computer processor is used to execute a computer program stored in a storage medium to implement the following method: acquiring an image to be processed, and determining the vertex coordinates of the image to be processed as the coordinates of the telescopic characteristic points; determining a target area, and acquiring a vertex coordinate of the target area as a target point coordinate; the target area is an image area of an image to be processed, and the coordinates of the telescopic characteristic points correspond to the coordinates of the target point one by one; performing stretching processing in a first direction on an image to be processed by calling a two-dimensional linear stretching mechanism, and taking an image output after the processing of the two-dimensional linear stretching mechanism as an image stretched in the first direction; and calling a two-dimensional linear stretching mechanism to perform stretching processing in a second direction on the image stretched in the first direction, wherein the second direction and the first direction are two directions perpendicular to each other, and taking the image output after the two-dimensional linear stretching mechanism processing as an image for image splicing.
It should be understood that the above examples are only for clarity of illustration and are not intended to limit the embodiments. Other variations and modifications will be apparent to persons skilled in the art in light of the above description. And are neither required nor exhaustive of all embodiments. And obvious variations or modifications therefrom are within the scope of the invention.
Claims (8)
1. An image processing method before image stitching is characterized by comprising the following steps:
acquiring an image to be processed, and determining the vertex coordinates of the image to be processed as the coordinates of the telescopic characteristic points;
determining a target area, and acquiring a vertex coordinate of the target area as a target point coordinate; the target area is an image area of the image to be processed, and the coordinates of the telescopic characteristic points correspond to the coordinates of the target points one to one;
a two-dimensional linear stretching mechanism is called to perform stretching processing in a first direction on the image to be processed, and the image output after the processing of the two-dimensional linear stretching mechanism is used as the image stretched in the first direction;
calling the two-dimensional linear stretching mechanism to perform stretching processing in a second direction on the image stretched in the first direction, wherein the second direction and the first direction are two directions perpendicular to each other, and taking the image output after the two-dimensional linear stretching mechanism processing as an image for image splicing;
wherein the two-dimensional linear scaling mechanism comprises:
step S1, acquiring the coordinates of the telescopic characteristic points and the coordinates of the target points;
step S2, stretching the image by stretching the coordinates of the stretching feature points to the coordinate positions of the quasi-target points according to the stretching direction by taking the image as a whole, wherein the coordinates of the quasi-target points coincide with the projection of the coordinates of the target points corresponding to the coordinates of the stretching feature points in the stretching direction;
step S3, outputting the image after expansion and contraction,
wherein, obtaining the image to be processed comprises:
acquiring an image shot by a camera;
determining a reference object from the shot image, wherein at least one surface or line direction of the reference object in the actual scene is parallel to the first direction or the second direction; and intercepting the image to be processed according to the reference object, wherein at least two edges of the intercepted image to be processed are parallel to the first direction or the second direction in the actual scene.
2. The image processing method according to claim 1, wherein intercepting the image to be processed according to the reference object comprises:
and taking a plane or a line of the reference object parallel to the first direction or the second direction in an actual scene as a reference, and cutting out the to-be-processed image by using a dividing line which is parallel to the reference and contains a vertex on the shot image.
3. The image processing method according to claim 2, wherein the image to be processed is a quadrangle with at least one pair of sides parallel, and the target area is a rectangle.
4. An image stitching method, comprising:
respectively processing a first image to be spliced and a second image to be spliced by adopting the image processing method of any one of claims 1 to 3 to obtain a processed first spliced image and a processed second spliced image;
and splicing the processed first spliced image and the second spliced image to obtain a spliced image.
5. An image processing apparatus before image stitching, comprising:
the image acquisition module is used for acquiring an image to be processed, determining the vertex coordinates of the image to be processed as the coordinates of the telescopic characteristic points;
the target area determining module is used for determining a target area, acquiring the vertex coordinates of the target area as target point coordinates; the target area is an image area of the image to be processed, and the coordinates of the telescopic characteristic points correspond to the coordinates of the target points one to one;
the first processing module is used for calling a two-dimensional linear stretching mechanism to perform stretching processing in a first direction on the image to be processed, and taking the image output after the processing of the two-dimensional linear stretching mechanism as the image stretched in the first direction;
the second processing module is used for calling the two-dimensional linear stretching mechanism to perform stretching processing in a second direction on the image stretched in the first direction, wherein the second direction and the first direction are two directions perpendicular to each other, and the image output after the two-dimensional linear stretching mechanism processing is used as an image for image splicing;
wherein the two-dimensional linear scaling mechanism comprises:
step S1, acquiring the coordinates of the telescopic characteristic points and the coordinates of the target points;
step S2, stretching the image by stretching the coordinates of the stretching feature points to the coordinate positions of the quasi-target points according to the stretching direction by taking the image as a whole, wherein the coordinates of the quasi-target points coincide with the projection of the coordinates of the target points corresponding to the coordinates of the stretching feature points in the stretching direction;
step S3, outputting the image after expansion and contraction,
wherein the image acquisition module comprises:
an acquisition unit configured to acquire an image captured by a camera;
the determining unit is used for determining a reference object from the shot image, wherein at least one surface or line direction of the reference object in the actual scene is parallel to the first direction or the second direction;
and the intercepting unit is used for intercepting the image to be processed according to the reference object, and at least two edges of the intercepted image to be processed are parallel to the first direction or the second direction in the actual scene.
6. The image processing apparatus according to claim 5, wherein the clipping unit is further configured to clip the image to be processed with a dividing line that is parallel to the reference and includes a vertex on the captured image, with a plane or a line of the reference parallel to the first direction or the second direction in the actual scene as a reference.
7. An image stitching device, comprising:
an image processing module, configured to process the first image to be stitched and the second image to be stitched respectively by using the image processing method according to any one of claims 1 to 3, so as to obtain a processed first stitched image and a processed second stitched image;
and the splicing module is used for splicing the processed first spliced image and the second spliced image to obtain a spliced image.
8. An electronic device, comprising: at least one processor; and a memory communicatively coupled to the at least one processor; wherein the memory stores instructions executable by the one processor to cause the at least one processor to perform the method of any one of claims 1-4.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710602597.0A CN107403409B (en) | 2017-07-21 | 2017-07-21 | Image processing method and device before image splicing and image splicing method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201710602597.0A CN107403409B (en) | 2017-07-21 | 2017-07-21 | Image processing method and device before image splicing and image splicing method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN107403409A CN107403409A (en) | 2017-11-28 |
| CN107403409B true CN107403409B (en) | 2021-02-02 |
Family
ID=60402346
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201710602597.0A Active CN107403409B (en) | 2017-07-21 | 2017-07-21 | Image processing method and device before image splicing and image splicing method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107403409B (en) |
Families Citing this family (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110290332A (en) * | 2018-03-19 | 2019-09-27 | 北京蛙视软件技术有限公司 | A kind of method and apparatus of superimposed image |
| US10776972B2 (en) * | 2018-04-25 | 2020-09-15 | Cognex Corporation | Systems and methods for stitching sequential images of an object |
| CN110120037B (en) * | 2019-04-29 | 2021-07-13 | 国网江苏省电力有限公司电力科学研究院 | Image processing and defect detection method, device, equipment and computer storage medium |
| CN112449100B (en) * | 2019-09-03 | 2023-11-17 | 中国科学院长春光学精密机械与物理研究所 | Stitching method, device, terminal and storage medium for aerial camera oblique images |
| CN110675358B (en) * | 2019-09-30 | 2023-05-16 | 上海扩博智能技术有限公司 | Image stitching method, system, equipment and storage medium for long object |
| CN111002287B (en) * | 2019-12-26 | 2021-11-30 | 北京海益同展信息科技有限公司 | Robot, image acquisition method, electronic device, and computer-readable storage medium |
| CN112801874B (en) * | 2021-02-03 | 2024-04-05 | 广州六环信息科技有限公司 | Vehicle image stitching method and device and vehicle |
Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102857704A (en) * | 2012-09-12 | 2013-01-02 | 天津大学 | Multisource video stitching method with time domain synchronization calibration technology |
| CN102890824A (en) * | 2011-07-19 | 2013-01-23 | 株式会社东芝 | Moving object contour tracking method and device, myocardial motion analysis method and device |
| CN102982516A (en) * | 2012-10-25 | 2013-03-20 | 西安理工大学 | Panoramic picture method based on hemisphere annular panoramic camera |
| CN103500337A (en) * | 2013-09-30 | 2014-01-08 | 上海合合信息科技发展有限公司 | Quadrangular side frame identifying method and device applied to intelligent wireless communication terminal |
| CN103533279A (en) * | 2013-10-24 | 2014-01-22 | 深圳市晶日盛科技有限公司 | Calibration method for multi-projection splicing |
| CN104794701A (en) * | 2014-01-21 | 2015-07-22 | 富士通株式会社 | Image splicing device, method and image processing equipment |
| CN106162125A (en) * | 2016-08-23 | 2016-11-23 | Tcl集团股份有限公司 | A kind of gentle curve projection correction method and system thereof |
| CN106447602A (en) * | 2016-08-31 | 2017-02-22 | 浙江大华技术股份有限公司 | Image mosaic method and device |
Family Cites Families (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140270449A1 (en) * | 2013-03-15 | 2014-09-18 | John Andrew HIPP | Interactive method to assess joint space narrowing |
-
2017
- 2017-07-21 CN CN201710602597.0A patent/CN107403409B/en active Active
Patent Citations (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102890824A (en) * | 2011-07-19 | 2013-01-23 | 株式会社东芝 | Moving object contour tracking method and device, myocardial motion analysis method and device |
| CN102857704A (en) * | 2012-09-12 | 2013-01-02 | 天津大学 | Multisource video stitching method with time domain synchronization calibration technology |
| CN102982516A (en) * | 2012-10-25 | 2013-03-20 | 西安理工大学 | Panoramic picture method based on hemisphere annular panoramic camera |
| CN103500337A (en) * | 2013-09-30 | 2014-01-08 | 上海合合信息科技发展有限公司 | Quadrangular side frame identifying method and device applied to intelligent wireless communication terminal |
| CN103533279A (en) * | 2013-10-24 | 2014-01-22 | 深圳市晶日盛科技有限公司 | Calibration method for multi-projection splicing |
| CN104794701A (en) * | 2014-01-21 | 2015-07-22 | 富士通株式会社 | Image splicing device, method and image processing equipment |
| CN106162125A (en) * | 2016-08-23 | 2016-11-23 | Tcl集团股份有限公司 | A kind of gentle curve projection correction method and system thereof |
| CN106447602A (en) * | 2016-08-31 | 2017-02-22 | 浙江大华技术股份有限公司 | Image mosaic method and device |
Non-Patent Citations (1)
| Title |
|---|
| 一种基于鱼眼图像的全景展开算法;王志刚等;《通信技术》;20121231;第65-71页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107403409A (en) | 2017-11-28 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN107403409B (en) | Image processing method and device before image splicing and image splicing method and device | |
| JP7372199B2 (en) | Projection system, projection device, and calibration method for its displayed image | |
| CN109104596B (en) | Projection system and correction method of display image | |
| CN113365041B (en) | Projection correction method, projection correction device, storage medium and electronic equipment | |
| KR100796849B1 (en) | How to take panoramic mosaic photos for mobile devices | |
| CN114727081B (en) | Projector projection correction method and device and projector | |
| CN108574825B (en) | A method and device for adjusting a PTZ camera | |
| CN111694528B (en) | Typesetting identification method of display wall and electronic device using the method | |
| CN114332246A (en) | Virtual simulation method and device for camera distortion | |
| CN106576140B (en) | Image processing device and camera device | |
| TWI443604B (en) | Image correction method and image correction device | |
| US9654749B2 (en) | Projection methods and projection devices | |
| CN115086631B (en) | Image generation method and information processing device | |
| CN112770095A (en) | Panoramic projection method and device and electronic equipment | |
| CN108924521A (en) | Trapezoidal correction method and equipment for projection system | |
| WO2018152710A1 (en) | Image correction method and device | |
| CN102469249A (en) | Image correction method and image correction device | |
| JP2014033342A (en) | Image correction device | |
| CN115797210A (en) | Calibration method, device and storage medium for taking pictures underwater | |
| CN114339179A (en) | Projection correction method, projection correction device, storage medium and projection equipment | |
| JP6716019B2 (en) | IMAGE CORRECTION DEVICE, IMAGE CORRECTION METHOD, AND PROGRAM | |
| CN115994949A (en) | An automatic calibration method, device, system and computer-readable storage medium | |
| JP6285657B2 (en) | Image processing apparatus, image processing method, and program | |
| CN104616246B (en) | A kind of method, apparatus and terminal device for handling image | |
| CN110310374B (en) | Automatic tripod eliminating method and device in panoramic photo |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |