WO2019010704A1 - Panoramic image and video recognition method, classifier establishment method and electronic apparatus - Google Patents

Abstract

The present disclosure discloses a method for recognizing a 360-degree panoramic image, comprising: acquiring an image file to be identified; performing feature extraction on the image file to obtain a three-dimensional feature value; and transforming the three-dimensional feature value into a two-dimensional feature value; Comparing the two-dimensional feature value with a first region and a second region defined by a linear classifier, determining an area where the two-dimensional feature value is distributed, and determining the image file according to the region where the two-dimensional feature value is distributed Whether it is a 360 degree panoramic image. The present application also discloses a method for identifying a 360-degree panoramic video, a method for establishing a linear classifier, and an electronic device. The electronic device and the 360-degree panoramic video and image recognition method of the present application can more accurately recognize a 360-degree panoramic video or a 360-degree panoramic image in a faster manner.

Description

Panoramic image, video recognition method, classifier establishment method and electronic device Technical field

The invention relates to a video image recognition method, in particular to a 360-degree panoramic image recognition method and a 360-degree panoramic video recognition method, which have been used in the recognition process of the 360-degree panoramic image and the 360-degree panoramic video. A method of establishing a linear classifier, and an electronic device applying the identification method and the method of establishing the method.

Background technique

At present, 360-degree panoramic camera technology has been widely used, and 360-degree panoramic image or 360-degree panoramic video can be obtained through 360-degree panoramic photography technology. In some situations, when playing an image or video file, it is usually necessary to identify whether it is a 360-degree panoramic video to perform corresponding playback settings, reflecting the effect of 360-degree panoramic. The existing recognition method is usually based on the aspect ratio of the image frame in the image file or the video file and/or the RGB (red, green and blue primary colors) values for judgment and recognition, which is often not accurate enough.

Summary of the invention

The embodiment of the invention discloses a 360-degree panoramic image and a 360-degree panoramic video recognition method, a method for establishing a linear classifier for identifying a 360-degree panoramic image and a 360-degree panoramic video, and an electronic device, which can accurately recognize 360 degrees. Panoramic image or 360 degree panoramic video.

The method for recognizing a 360-degree panoramic image disclosed by the embodiment of the present invention includes: acquiring an image file to be identified; performing feature extraction on the image file to obtain a three-dimensional image including the first feature value, the second feature value, and the third feature value The eigenvalue is transformed into a two-dimensional eigenvalue; the two-dimensional eigenvalue is compared with the first region and the second region defined by a linear classifier to determine a region in which the two-dimensional eigenvalue is distributed; When the area of the value distribution is the first area, it is confirmed that the image file to be recognized is a 360-degree panoramic image; and when the area of the two-dimensional feature value distribution is the second area, it is determined that the image file to be recognized is not a 360-degree panoramic image.

The method for identifying a 360-degree panoramic video disclosed by the embodiment of the present invention includes: obtaining Identifying the video file; extracting at least one frame representative image frame from the video file to be identified; performing feature extraction on each representative image frame to obtain a first feature value, a second feature value, and a third feature of each representative image frame a three-dimensional feature value of the value; transforming the three-dimensional feature value of each representative image frame into a two-dimensional feature value; comparing the two-dimensional feature value of each representative image frame with the first region and the second region defined by a linear classifier Determining a region in which the two-dimensional feature value is distributed; when a region representing a two-dimensional feature value distribution of the image frame is the first region, confirming that the representative image frame is a 360-degree panoramic image; when a certain representative image frame is two-dimensional When the region of the feature value distribution is the second region, it is determined that the representative image frame is not a 360-degree panoramic image; and after the at least one frame representative image frame is recognized, the proportion of the 360-degree panoramic image in the image frame is represented according to at least one frame. Determine if the video file to be identified is a 360-degree panoramic video.

The method for establishing a linear classifier disclosed in the embodiment of the present invention includes the steps of: storing a first preset number of 360-degree panoramic images to a first folder and a second preset number of non-360-degree panoramic images to a second a folder; extracting features of the first feature value, the second feature value, and the third feature value for each 360-degree panoramic image to form a three-dimensional feature value of each 360-degree panoramic image and storing the same to the first folder; a non-360 degree panoramic image is subjected to feature extraction of the first feature value, the second feature value, and the third feature value to form a three-dimensional feature value of each non-360 degree panoramic image and stored in the second folder; according to the first preset The three-dimensional feature value of the 360-degree panoramic image and the three-dimensional feature value of the second preset number of non-360-degree panoramic images generate a three-dimensional scattergram, wherein the three-dimensional feature value of each 360-degree panoramic image or a non-360-degree panoramic image Corresponding to a feature point in a three-dimensional scatterplot; converting a three-dimensional scatterplot into a two-dimensional scatterplot; and eigenvalues of a 360-degree panoramic image on a two-dimensional scatterplot and non-360-degree panoramic images The distribution of the eigenvalue determines a boundary value of the eigenvalue of the 360-degree panoramic image and the eigenvalue of the non-360-degree panoramic image to obtain a linear classifier, wherein the eigenvalue of the 360-degree panoramic image on the side of the boundary line is the first region The area where the feature value of the non-360 degree panoramic image on the other side of the boundary line is located is the second area.

The electronic device disclosed in the embodiment of the present invention includes a processor and a memory. The memory stores an image file to be identified, a video file to be identified, a plurality of recognized 360-degree panoramic images, and a non-360-degree panoramic image, and the processor is configured to execute The foregoing method for recognizing a 360-degree panoramic image to identify whether the image file to be recognized is a 360-degree panoramic image, and/or a recognition method for performing the aforementioned 360-degree panoramic video to identify whether the video file to be recognized is a 360-degree panoramic view Image, and/or for The method of establishing the aforementioned linear classifier is performed to establish a linear classifier.

The 360-degree panoramic image and the 360-degree panoramic video recognition method, the linear classifier establishing method and the electronic device of the present invention obtain the two-dimensional feature value of the image frame in the image to be recognized and/or the image to be recognized, and then linearly The first region and the second region defined by the classifier are compared, and the region where the two-dimensional feature value is located is determined whether the image to be identified is a 360-degree panoramic image, or whether the video to be identified is a 360-degree video, and the recognition is improved. Speed and can effectively improve recognition accuracy.

DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings to be used in the embodiments will be briefly described below. It is obvious that the drawings in the following description are only some embodiments of the present invention. Those skilled in the art can also obtain other drawings based on these drawings without paying any creative work.

FIG. 1 is a flowchart of a method for recognizing a 360-degree panoramic image according to an embodiment of the present invention.

2 is a sub-flow diagram of step S12 of FIG. 1 in an embodiment of the present invention.

3 is a schematic diagram of an image file in an embodiment of the present invention.

FIG. 4 is a flowchart of a method for recognizing a 360-degree panoramic video according to an embodiment of the present invention.

FIG. 5 is a flowchart of a method for establishing a linear classifier according to an embodiment of the present invention.

FIG. 6 is a schematic diagram of a three-dimensional scattergram in an embodiment of the present invention.

FIG. 7 is a schematic diagram of a two-dimensional scattergram according to an embodiment of the present invention.

FIG. 8 is a schematic diagram of a two-dimensional scattergram in another embodiment of the present invention.

FIG. 9 is a structural block diagram of an electronic device according to an embodiment of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention are clearly and completely described in the following with reference to the accompanying drawings in the embodiments of the present invention. It is obvious that the described embodiments are only a part of the embodiments of the present invention, and not all of the embodiments. Those of ordinary skill in the art are not creative based on the embodiments of the present invention. All other embodiments obtained under the premise of labor are within the scope of the invention.

Please refer to FIG. 1 , which is a flowchart of a method for recognizing a 360-degree panoramic image according to an embodiment of the present invention. The method of recognizing the 360-degree panoramic image is not limited to the following execution order. The method for recognizing a 360-degree panoramic image includes the steps of:

S11: Acquire an image file to be identified.

S12: Perform feature extraction on the image file to obtain a three-dimensional feature value including the first feature value, the second feature value, and the third feature value.

S13: Transform the three-dimensional feature value into a two-dimensional feature value. In some embodiments, step S13 specifically includes: transforming the three-dimensional feature values into two-dimensional feature values through a preset three-dimensional coordinate transformation matrix. Further, by multiplying the three-dimensional feature value and the three-dimensional coordinate transformation matrix, a corresponding two-dimensional feature value including two dimensional features is obtained.

S14: Comparing the two-dimensional feature value with the first region and the second region defined by a linear classifier to determine a region in which the two-dimensional feature value is distributed. In some embodiments, the two-dimensional feature value is located in the same coordinate system as the first region and the second region defined by the linear classifier, the first region defined by the linear classifier corresponds to a first coordinate set, and the second region corresponds to a first a set of two coordinates, the two-dimensional feature value corresponding to a coordinate, by comparing the coordinates of the two-dimensional feature value with the first coordinate set and the second coordinate set, determining whether the coordinate of the two-dimensional feature value is located in the first coordinate set or the second coordinate set When the coordinates corresponding to the two-dimensional feature value are located in the first coordinate set, determining that the region where the two-dimensional feature value is located is the first region, and when the coordinate corresponding to the two-dimensional feature value is located in the second coordinate set, determining two The area in which the dimension feature value is located is the second area.

S15: When the area of the two-dimensional feature value distribution is the first area, confirm that the image file to be recognized is a 360-degree panoramic image.

S16: When the area of the two-dimensional feature value distribution is the second area, it is determined that the image file to be identified is not a 360-degree panoramic image.

The first region is a region of a two-dimensional feature value distribution of a 360-degree panoramic image defined in the linear classifier, and the second region is a region of a two-dimensional feature value distribution of the non-360-degree panoramic image defined in the linear classifier.

Therefore, in the present application, by comparing the two-dimensional feature value of the image file with a preset linear classifier, it is possible to quickly determine whether the image file is a 360-degree panoramic image, and the recognition process is simpler. Single and more accurate.

Referring to FIG. 2 and FIG. 3 together, FIG. 2 is a sub-flow chart of step S12 in an embodiment, and FIG. 3 is a schematic diagram of an image file M1. As shown in FIG. 3, the image file M1 includes m*n pixels. P1, that is, pixel point P1 of m rows and n columns, where m and n are positive integers greater than 1, and m and n may be equal or unequal. Step S12 includes:

S121: Acquire gray values of pixel points of the first row C1, the last row Cm, the first column L1, and the last column Ln of the image file M1. In some embodiments, the grayscale values of all the pixel points of the first row C1, the last row Cm, the first column L1, and the last column Ln of the image file M1 are acquired in step S122. The first row C1 may be the first row of the image file M1, the last row Cm may be the last row of the image file M1, the first column L1 may be the first column of the image file M1, and the last column Ln may be the image file M1. last row.

S123: Extract the gray values of the plurality of row sampling points from the first row C1 and the last row Cm, and extract the gray values of the plurality of column sampling points from the first column L1 and the last column Ln, respectively. In some embodiments, the gray values of the first preset line sampling points are respectively extracted from the first row C1 and the last row Cm at even intervals; and the first column L1 and the last column Ln are separately extracted at even intervals. The gray value of the two preset column sampling points. In some embodiments, if the pixel point of the image file M1 is too small, a portion of the pixel of the corresponding row or column of the copyable image file M1 is supplemented into a number of row or column sample points. In some embodiments, the number of the plurality of row sampling points and the column sampling points may be equal or unequal, for example, a plurality of row sampling points are 300, and a plurality of column sampling points are 150.

S125: Calculate a mean value of variances of gray values of a plurality of row sampling points of the first row C1 and the last row Cm to obtain a first feature value.

Specifically, the variance of the gray value of the plurality of row sampling points of the first row C1 and the variance of the gray values of the plurality of row sampling points of the last row C1 are respectively calculated, and then the average of the two variances is averaged to obtain the first eigenvalue. . Let the variance of the gray value of several row sampling points of the first row C1 be F1, and the variance of the gray value of several row sampling points of the last row Cm be F2, then the first characteristic value=(F1+F2)/ 2. The variance of the gray values of the plurality of row sampling points of the first row C1 represents the difference between the gray values of the plurality of row sampling points of the first row C1, and the variance of the gray values of the plurality of row sampling points of the last row Cm. It represents the difference between the gray values of several line sampling points of the last line Cm. The mean of the variance of the gray value of the first row C1 gray value and the gray value of several row sample points of the last row Cm, that is, the first feature value represents the first row C1 and The difference between the gray values of all the sample points of the last line Cm. Wherein, if the image file M1 is a 360-degree panoramic image, generally, the variance of the gray value of the sampling points of the plurality of rows of the first row C1 and the variance of the gray values of the plurality of row sampling points of the last row of Cm are small, close to each other. At zero.

S127: Calculate a mean value of variances of gray values of the plurality of column sampling points of the first column L1 and the last column Ln to obtain a second feature value. Specifically, after calculating the variance of the gray value of the plurality of column sampling points of the first column L1 and the variance of the gray values of the plurality of column sampling points of the last column L1, the average of the two variances is obtained. Eigenvalues. Let the variance of the gray values of several column sampling points of the first column L1 be F3, and the variance of the gray values of several column sampling points of the last column Ln is F4, and the second characteristic value=(F3+F4)/2 .

S129: respectively calculating the square of the difference between the gray value of the plurality of column sampling points of the first column L1 and the gray value of the column sampling point of the position corresponding to the last column Ln to obtain a square of the difference, and then calculating the difference The average of the squares of the values yields a third eigenvalue.

Specifically, the positions of the plurality of column sampling points in the first column L1 are in one-to-one correspondence with the positions of the plurality of column sampling points of the last column L1, and the positions of the column sampling points of the first column L1 and the last column Ln are respectively calculated. The difference between the sampling points of the column, and then squaring the difference, respectively obtaining the square of the difference between the gray values of the column sampling points at the positions corresponding to the respective column sampling points of the first column L1 and the last column Ln, thereby Get the square of several differences. For example, calculating the difference between the gray value between the first column sample point of the first column L1 and the first column sample point at the position corresponding to the last column Ln, and then squaring, to obtain the square of the first difference, Calculating the difference between the gray value between the second column sample point of L1 of the first column and the second column sample point at the position corresponding to the last column Ln, and then squared to obtain the square of the second difference value, etc. , thus obtaining the square of several differences. Wherein, if the image file M1 is a 360-degree panoramic image, the squares of the plurality of differences are both small and close to zero.

Please refer to FIG. 4 , which is a flowchart of a method for recognizing a 360-degree panoramic video according to an embodiment of the present invention. As shown in FIG. 4, the method for recognizing a 360-degree panoramic video includes the following steps:

S31: Acquire a video file to be identified.

S32: Extract at least one frame representative image frame from the video file to be identified.

In some embodiments, the representative image frame is selected based on at least one of the following selection principles: 1. The key frame in the video file is selected as the representative image frame, and the general key frame includes a complete frame picture, compared to other types of frames. With higher image quality, keyframes often include a camera lens In the beginning period, there is a big difference between adjacent key frames, which is beneficial to improve the diversity of the recognition samples; 2. Select a frame image with rich picture changes as a representative image frame; 3. Select multiple video files in the video file. The time frame covers a plurality of image frames whose time period exceeds a preset value as a plurality of representative image frames. For example, if the duration of the video file is 2 hours, the interval is selected as much as possible from the beginning to the last time. The image frames are represented as representative image frames, and the time span of the plurality of image frames may be 1 hour and 50 minutes, and the like.

S33: Perform feature extraction on each representative image frame to obtain a first feature value, a second feature value, and a third feature value of the third feature value of each representative image frame.

S34: Convert the three-dimensional feature value of each representative image frame into a two-dimensional feature value.

S35: Compare the two-dimensional feature value of each representative image frame with the first region and the second region defined by a linear classifier, and determine an area where the two-dimensional feature value is distributed. In some embodiments, the two-dimensional feature value is located in the same coordinate system as the first region and the second region defined by the linear classifier, the first region defined by the linear classifier corresponds to a first coordinate set, and the second region corresponds to a first A set of two coordinates. The two-dimensional feature value of each representative image frame corresponds to a coordinate, and the coordinates of the two-dimensional feature value of each representative image frame are compared with the first coordinate set and the second coordinate set to determine that the coordinates of the two-dimensional feature value are located at the A coordinate set is also a second coordinate set. When the coordinate corresponding to the two-dimensional feature value is located in the first coordinate set, determining that the corresponding two-dimensional feature value of the representative image frame is in the first region, when the two-dimensional feature value When the corresponding coordinate is located in the second coordinate set, it is determined that the region where the corresponding two-dimensional feature value of the representative image frame is located is the second region.

S36: When the area of the two-dimensional feature value distribution of the representative image frame is the first area, it is confirmed that the representative image frame is a 360-degree panoramic image.

S37: When the area of the two-dimensional feature value distribution of the representative image frame is the second area, it is determined that the representative image frame is not a 360-degree panoramic image.

S38: After the at least one frame representative image frame is identified, determining whether the video file to be identified is a 360-degree panoramic video according to a proportion of the 360-degree panoramic image in the at least one frame representative image frame. Specifically, determining whether at least one frame represents a proportion of a 360-degree panoramic image in the image frame is greater than or equal to a preset ratio; if yes, determining that the video file is a 360-degree panoramic video, and if not, determining that the video file is not 360 Degree panoramic video. Among them, the preset ratio can be 80%, 100%, and the like.

Wherein, steps S33-S37 correspond to steps S12-S16 in FIG. 1, respectively. Steps S33-S37 For specific steps, please refer to the foregoing detailed description of steps S12-S16. That is, any one of the representative image frames corresponds to one image file, and the method of recognizing whether any one of the representative image frames is a 360-degree panoramic image in FIG. 4 is the same as the method of identifying whether the image file is a 360-degree panoramic image in FIG. A detailed description of each step in FIG. 1 is included, including, for example, the sub-steps described in FIG. 2 for step S12.

For example, step S33 may specifically include: extracting pixel values of pixel points of each of the first row, the last row, the first column, and the last column of each representative image frame; respectively extracting from the first row and the last row of each representative image frame a plurality of rows of sampling points, and extracting a plurality of series of sampling points from the first column and the last column of each representative image frame; calculating a plurality of rows of sampling points of each of the first row and the last row of each representative image frame The mean value of the variance obtains a first feature value of each representative image frame; calculating a mean value of variances of the plurality of series of sample points of the first column and the last column of each representative image frame to obtain a second feature value of each representative image frame; And separately calculating a square of a difference between a plurality of series of sampling points of the first column of each representative image frame and a sampling point of the most one of the corresponding positions to obtain a square of a plurality of differences of each representative image frame, and then calculating the obtained squares The average of the squares of the differences yields a third eigenvalue for each representative image frame.

Similarly, in the present application, by obtaining two-dimensional feature values of each representative image frame in the video file and comparing with a preset linear classifier, it is possible to quickly determine whether each representative image frame is a 360-degree panoramic image. Then, according to the recognition result of each representative image frame, it can be quickly determined whether the video file is a 360-degree video file, and the recognition process is simpler and more accurate.

Please refer to FIG. 5 to FIG. 7. FIG. 5 is a schematic flowchart diagram of a method for establishing a linear classifier according to an embodiment of the present invention. Wherein, the steps shown in FIG. 5 can be performed before the aforementioned 360-degree video recognition method or 360-degree image recognition method for pre-establishing a linear classifier for 360-degree video recognition and 360-degree image recognition. The method for establishing a linear classifier includes:

S61: Store the first preset number of 360-degree panoramic images to the first folder and the second preset number of non-360-degree panoramic images to the second folder in advance.

S62: Extracting the feature of the first feature value, the second feature value, and the third feature value for each 360-degree panoramic image to form a three-dimensional feature value of each 360-degree panoramic image and storing the same to the first folder.

S63: extracting features of the first feature value, the second feature value, and the third feature value for each non-360 degree panoramic image to form a three-dimensional feature value of each non-360 degree panoramic image and storing the same to the second file. folder.

S64: Generate a three-dimensional scatterplot according to the three-dimensional feature value of the first preset number of 360-degree panoramic images and the three-dimensional feature values of the second preset number of non-360-degree panoramic images, wherein each 360-degree panoramic image or one non-360 The three-dimensional feature value of the panoramic image corresponds to one feature point in the three-dimensional scattergram.

As shown in FIG. 6, the three-dimensional coordinates of the three-dimensional scattergram T1 are the coordinates of the first feature value, the second feature value, and the third feature value, respectively, and the three-dimensional feature values of the first preset number of 360-degree panoramic images. And each of the three-dimensional feature values of the second predetermined number of non-360 degree panoramic images constitutes a point in the three-dimensional scattergram T1. For example, as shown in FIG. 6, the black dots are points corresponding to the three-dimensional feature values of the non-360 degree panoramic image, and the white dots are points corresponding to the three-dimensional feature values of the 360-degree panoramic image.

S65: Converting a three-dimensional scattergram into a two-dimensional scatterplot.

In some embodiments, a two-dimensional scattergram formed by all the two-dimensional feature values is obtained by converting all three-dimensional feature values in the three-dimensional scattergram into two-dimensional feature values through a preset three-dimensional coordinate transformation matrix. Among them, the three-dimensional coordinate transformation matrix is the same as the three-dimensional coordinate transformation matrix in the previous recognition process, thereby ensuring that the conversion criteria in the recognition process are exactly the same. As shown in FIG. 7, the two-dimensional scattergram T2 can be regarded as a three-dimensional scattergram rotated at a preset angle and projected onto a two-dimensional plane in which the feature points are not overlapped as much as possible. As shown in the figure, a black point representing a three-dimensional feature value of a non-360 degree panoramic image and a white point representing a three-dimensional feature value of a 360-degree panoramic image are two-dimensionally distributed on the two-dimensional scattergram, which is more intuitive.

S66: determining a boundary value J1 of the feature value of the 360-degree panoramic image and the feature value of the non-360-degree panoramic image according to the feature value of the 360-degree panoramic image on the two-dimensional scattergram and the distribution of the feature values of the non-360-degree panoramic image, to Get a linear classifier. The area where the feature value of the 360-degree panoramic image on one side of the boundary line is located is the first area, that is, the area A1 of the white point distribution as shown in FIG. 7, and the characteristic value of the non-360 degree panoramic image on the other side of the boundary line. The area in which it is located is the second area, that is, the area A2 in which black dots are distributed as shown in FIG. In some embodiments, the linear classifier is a two-dimensional map that is divided into a first region and a second region by a boundary line J1. In other embodiments, the linear classifier can be viewed as a set of two coordinates divided by a set of coordinates defined by the boundary line J1 in a two-dimensional coordinate system.

As shown in FIG. 7, the boundary line J1 is a curve between the black point and the white point, that is, the first area and the second area are separated by a curve. As shown in FIG. 8, in another embodiment, the boundary line J1 may also be a multi-segment line between a black point and a white point, that is, a plurality of straight lines between the first area and the second area Lines are separated.

The feature of the first feature value, the second feature value, and the third feature value is extracted for each 360-degree panoramic image in step S62 to form a three-dimensional feature value of each 360-degree panoramic image and each of the steps S63 The non-360 degree panoramic image performs feature extraction of the first feature value, the second feature value, and the third feature value to form a three-dimensional feature value of each non-360 degree panoramic image, which is the same as step S12 in FIG. For example, step S62 or S63 includes: extracting gray values of pixels of the first row, the last row, the first column, and the last column of the 360-degree panoramic image or the non-360-degree panoramic image; respectively, extracting from the first row and the last row respectively The gray value of a plurality of row sampling points, and the gray values of the plurality of column sampling points are respectively extracted from the first column and the last column; and the variance of the gray value of the plurality of row sampling points of the first row and the last row is calculated. Mean value obtains a first eigenvalue; calculating a mean value of variances of gradation values of a plurality of column sampling points of the first column and the last column to obtain a second eigenvalue; and calculating gray values of the plurality of column sampling points of the first column respectively The square of the difference between the gray values of the sampling points corresponding to the last column obtains the square of the difference, and then the average of the squares of the calculated differences yields the third characteristic value. For more specific steps, reference may be made to the flowchart shown in FIG. 2 and its related description.

The step S14 in the foregoing method for recognizing the 360-degree panoramic image may further be: determining, by a preset vector cross-multiplication algorithm, which side of the boundary between the first region and the second region the two-dimensional feature value is, thereby determining Whether the area where the two-dimensional feature value is located is the first area or the second area.

The step S35 in the foregoing method for recognizing the 360-degree panoramic video may further be: calculating, by a preset vector cross-multiplication algorithm, which of the boundary points of the first region and the second region of the two-dimensional feature value of each representative image frame Side, thereby determining whether the area in which the two-dimensional feature value of each representative image frame is located is the first area or the second area.

Specifically, the two-dimensional feature value may be vector-multiplied with the point located in the first region or the second region of the linear classifier, and the cross-product is used to determine whether the two points are in the same by the principle of "same direction method" The same side of the boundary line J1, thereby judging whether the two-dimensional feature value is located in the first region or in the second region.

Please refer to FIG. 9 , which is a structural block diagram of an electronic device 100 according to an embodiment of the invention. As shown in FIG. 1 , the electronic device 100 includes a processor 10 and a memory 20 .

The memory 20 stores image files and/or video files to be identified. The image files and/or video files stored in the memory 20 are pre-stored in the memory 20, and may also be temporarily downloaded from the server or received from other electronic devices 100. Wherein, the aforementioned linear classifier is also stored in In the memory 20, the aforementioned first folder and second folder are all folders in the memory 20.

In some embodiments, the electronic device 100 further includes a communication unit 30, and the processor 10 can establish a communication connection with a server or other electronic device through the communication unit 30 in advance, and receive an image file to be recognized from a server or other electronic device and/or The video file is stored in the memory 20. The communication unit 30 can be a wired or wireless communication module, and can be, for example, a wired network interface unit, a WIFI module, a Bluetooth module, or the like.

The processor 10 is configured to analyze the image file and/or the video file to be recognized stored in the memory 20 to identify whether it is a 360-degree panoramic image and/or a 360-degree panoramic video.

The processor 10 is configured to perform at least any of the methods shown in FIG. 1-2 and FIG. 5 to identify whether the image file to be recognized is a 360-degree panoramic image.

The processor 10 is further configured to perform at least the method steps shown in FIG. 4-5 and the related method steps shown in FIG. 2 to identify whether the video file to be identified is a 360-degree panoramic video.

In some embodiments, a plurality of program instructions are stored in the memory 20, and after the processor 10 calls to execute a plurality of program instructions, performing any of the methods shown in FIGS. 1-2 and 5 to identify whether the image file to be identified is 360. The panoramic image, and/or the method steps shown in Figures 4-5 and the associated method steps shown in Figure 2 are used to identify whether the video file to be identified is a 360-degree panoramic video.

In some embodiments, the memory 20 also stores a number of already recognized 360 degree panoramic images and non-360 degree panoramic images. The processor 10 is further configured to execute or invoke a method of establishing a linear classifier as shown in FIG. 5 after executing a plurality of program instructions.

For example, the processor 10 performs the following method to identify whether the image file to be identified is a 360-degree panoramic image: acquiring an image file to be identified; performing feature extraction on the image file to obtain a first feature value, a second feature value, and a third feature value. a three-dimensional eigenvalue; transforming the three-dimensional eigenvalue into a two-dimensional eigenvalue; comparing the two-dimensional eigenvalue with a first region and a second region defined by a linear classifier to determine a region in which the two-dimensional eigenvalue is distributed; When the area of the dimension feature value distribution is the first area, it is confirmed that the image file to be recognized is a 360-degree panoramic image; and when the area of the two-dimensional feature value distribution is the second area, it is determined that the image file to be recognized is not a 360-degree panoramic image. image.

For another example, the processor 10 performs the following method: determining whether the video file to be identified is a 360-degree panoramic video: acquiring a video file to be identified; extracting at least one frame representative image frame from the video file to be identified; for each representative image frame Performing feature extraction to obtain a first feature value of each representative image frame, a three-dimensional feature value of the second feature value and the third feature value; transforming the three-dimensional feature value of each representative image frame into a two-dimensional feature value; and defining a two-dimensional feature value of each representative image frame with a linear classifier A region and a second region are compared to determine a region in which the two-dimensional feature value is distributed; when a region of the two-dimensional feature value distribution of the representative image frame is the first region, the representative image frame is confirmed to be a 360-degree panoramic image; When a region representing a two-dimensional feature value distribution of the image frame is the second region, determining that the representative image frame is not a 360-degree panoramic image; and determining a to-be-recognized according to a proportion of the 360-degree panoramic image in the at least one frame representative image frame Whether the video file is a 360-degree panoramic video.

For another example, the processor 10 performs a method of establishing a linear classifier: storing a first preset number of 360-degree panoramic images to the first folder and a second preset number of non-360-degree panoramic images to the second folder in advance. And extracting, by each of the 360-degree panoramic images, features of the first feature value, the second feature value, and the third feature value to form a three-dimensional feature value of each 360-degree panoramic image and storing the same to the first folder; The 360-degree panoramic image performs feature extraction of the first feature value, the second feature value, and the third feature value to form a three-dimensional feature value of each non-360 degree panoramic image and stores the same to the second folder; according to the first preset number The three-dimensional feature value of the 360-degree panoramic image and the three-dimensional feature value of the second preset number of non-360-degree panoramic images generate a three-dimensional scattergram, wherein the three-dimensional feature value of each 360-degree panoramic image or a non-360-degree panoramic image corresponds to a feature point in a 3D scatter plot; converting a 3D scatter plot into a 2D scatter plot; according to the eigenvalues of the 360 degree panoramic image on the 2D scatter plot and the eigenvalues of the non-360 degree panoramic image Determining a boundary value of the feature value of the 360-degree panoramic image and the feature value of the non-360-degree panoramic image to obtain a linear classifier, wherein the region where the feature value of the 360-degree panoramic image on one side of the boundary line is the first region, the boundary line The area where the feature value of the non-360 degree panoramic image on the other side is located is the second area.

The processor 40 can be a microcontroller, a microprocessor, a single chip, a digital signal processor, or the like.

The memory 20 can be any storage device that can store information such as a memory card, a solid state memory, a micro hard disk, an optical disk, or the like.

In some embodiments, the present invention further provides a computer readable storage medium having a plurality of program instructions stored therein for execution by the processor 10 to perform the operations of FIG. 1 and FIG. 3-7. Any method step to identify whether the image file is a 360 degree panoramic image and/or whether the video file is a 360 degree panoramic video. In some embodiments, the computer storage medium is the memory 20, which may be any storage information such as a memory card, a solid state memory, a micro hard disk, an optical disk, or the like. Storage device.

The electronic device 100 can be a mobile phone, a tablet computer, a notebook computer, a desktop computer, etc., and can also be a head-mounted device such as a smart helmet or smart glasses.

Thus, by the 360 degree panoramic image and the 360 degree panoramic video recognition method and the electronic device of the present invention, by determining the target image frame, and then analyzing the gray between the pixel of the first column of the target image frame and the pixel of the last column Determining whether the target image frame is a 360-degree panoramic image frame, and then determining whether the target image frame is a 360-degree panoramic image frame by analyzing the grayscale difference between the pixel points of the first row of the target image frame and the grayscale difference between the pixels of the last row. Whether the corresponding image or video is a 360-degree panoramic image or a 360-degree panoramic video can effectively improve the recognition accuracy.

The above is a preferred embodiment of the present invention, and it should be noted that those skilled in the art can also make several improvements and retouchings without departing from the principles of the present invention. These improvements and retouchings are also considered as The scope of protection of the invention.

Claims (19)

A method for recognizing a 360-degree panoramic image, characterized in that the identification method comprises: Obtaining an image file to be identified; Performing feature extraction on the image file to obtain a three-dimensional feature value including a first feature value, a second feature value, and a third feature value; Transforming the three-dimensional feature value into a two-dimensional feature value; Comparing the two-dimensional feature value with a first region and a second region defined by a linear classifier to determine an area in which the two-dimensional feature value is distributed; When the area of the two-dimensional feature value distribution is the first area, confirming that the image file to be recognized is a 360-degree panoramic image; When the area of the two-dimensional feature value distribution is the second area, it is determined that the image file to be recognized is not a 360-degree panoramic image. The method for recognizing a 360-degree panoramic image according to claim 1, wherein the step of "extracting the image file to obtain a three-dimensional shape including a first feature value, a second feature value, and a third feature value. Characteristic values" include: Extracting gray values of pixels of the first row, the last row, the first column, and the last column of the image file; Extracting gray values of a plurality of row sampling points from the first row and the last row, and extracting gray values of the plurality of column sampling points from the first column and the last column respectively; Calculating an average of the variances of the gray values of the plurality of row sampling points of the first row and the last row to obtain the first feature value; Calculating an average of the variances of the gray values of the plurality of column sampling points of the first column and the last column to obtain the second characteristic value; Calculating the square of the difference between the gray value of the plurality of column sampling points of the first column and the gray value of the sampling point corresponding to the last column to obtain a square of the difference, and then calculating the obtained difference The average of the squares gives the third characteristic value. The method for recognizing a 360-degree panoramic image according to claim 2, wherein said step "extracting gray values of a plurality of row sampling points from the first row and the last row, respectively, and from the first column And the last column separately extracts the gray values of several column sampling points" including: Extracting gray values of the first preset line sampling points at even intervals from the first line and the last line; The gray values of the second predetermined column sample points are extracted from the first column and the last column at even intervals. The method for identifying a 360-degree panoramic image according to any one of claims 1 to 3, wherein the method further comprises: Pre-storing the first preset number of 360-degree panoramic images to the first folder and the second preset number of non-360-degree panoramic images to the second folder; And extracting, by each of the 360-degree panoramic images, features of the first feature value, the second feature value, and the third feature value to form a three-dimensional feature value of each 360-degree panoramic image and storing the same to the first folder; Performing feature extraction of the first feature value, the second feature value, and the third feature value for each non-360 degree panoramic image to form a three-dimensional feature value of each non-360 degree panoramic image and storing the same to the second folder; Generating a three-dimensional scattergram according to the three-dimensional feature value of the first preset number of 360-degree panoramic images and the three-dimensional feature values of the second preset number of non-360-degree panoramic images, wherein each 360-degree panoramic image or one The three-dimensional feature value of the non-360 degree panoramic image corresponds to one feature point in the three-dimensional scatterplot; Converting the three-dimensional scattergram into a two-dimensional scattergram; Determining the feature value of the 360-degree panoramic image and the feature value of the non-360-degree panoramic image according to the feature value of the 360-degree panoramic image on the two-dimensional scattergram and the distribution of the feature values of the non-360-degree panoramic image a boundary line to obtain the linear classifier, wherein an area where the feature value of the 360-degree panoramic image on one side of the boundary line is located is the first area, and the non-the other side of the boundary line The area where the feature value of the 360-degree panoramic image is located is the second area. The method for identifying a 360-degree panoramic image according to claim 4, wherein the step of "transforming the three-dimensional feature value into a two-dimensional feature value" comprises: Converting the three-dimensional feature value into a two-dimensional feature value by a preset three-dimensional coordinate transformation matrix; The step of "converting the three-dimensional scattergram into a two-dimensional scatterplot" includes: Converting all the three-dimensional feature values in the three-dimensional scattergram into two-dimensional feature values by using the preset three-dimensional coordinate transformation matrix to obtain a two-dimensional scattergram formed by all the two-dimensional feature values. The method for identifying a 360-degree panoramic image according to claim 4, wherein said step "comparing said two-dimensional feature value with a first region and a second region defined by a linear classifier to determine said The area where the two-dimensional eigenvalues are distributed" includes: Determining, by a preset vector cross-multiplication algorithm, which side of the boundary between the first region and the second region is the two-dimensional feature value, thereby determining whether the region in which the two-dimensional feature value is located is the first region or Second area. The method for recognizing a 360-degree panoramic image according to any one of claims 1 to 5, wherein the step of "performing the two-dimensional feature value with a first region and a second region defined by a linear classifier Comparing, determining the area in which the two-dimensional feature values are distributed" includes: Comparing the coordinate value corresponding to the two-dimensional feature value with the first coordinate set corresponding to the first region and the second coordinate set corresponding to the second region; When the coordinate value corresponding to the two-dimensional feature value is located in the first coordinate set, determining that the area where the two-dimensional feature value is located is the first area, and when the coordinate corresponding to the two-dimensional feature value When the value is in the second set of coordinates, the area in which the two-dimensional feature value is determined is determined to be the second area. A method for recognizing a 360-degree panoramic video, characterized in that the identification method comprises: Obtain a video file to be identified; Extracting at least one frame representative image frame from the video file to be identified; Perform 360-degree image recognition processing on each representative image frame, including: Performing feature extraction on each representative image frame to obtain a first feature value, a second feature value, and a third feature value of the third feature value of each representative image frame; Converting a three-dimensional feature value of each representative image frame into a two-dimensional feature value; Comparing the two-dimensional feature value of each representative image frame with the first region and the second region defined by a linear classifier to determine an area in which the two-dimensional feature value is distributed; When a region representing a two-dimensional feature value distribution of the image frame is the first region, confirming that the representative image frame is a 360-degree panoramic image; Determining that the representative image frame is not a 360-degree panoramic image when a region representing a two-dimensional feature value distribution of the image frame is the second region; After the at least one frame representative image frame is identified, determining whether the to-be-recognized video file is a 360-degree panoramic view according to a ratio of the 360-degree panoramic image in the at least one frame representative image frame frequency. The method for identifying a 360-degree panoramic video according to claim 8, wherein the step of performing feature extraction on each representative image frame to obtain a first feature value, a second feature value, and a third feature value 3D eigenvalues include: Extracting gray values of pixel points of each of the first row, the last row, the first column, and the last column of each representative image frame; Extracting gray values of a plurality of row sampling points from the first row and the last row of each representative image frame, and extracting gray values of the plurality of column sampling points from the first column and the last column of each representative image frame respectively ; Calculating an average of the variances of the gray values of the plurality of row sample points of the first row and the last row of each representative image frame to obtain the first feature value of each representative image frame; Calculating an average of the variances of the gray values of the plurality of column sample points of the first column and the last column of each representative image frame to obtain the second feature value of each representative image frame; Calculating, respectively, a difference between a gray value of a plurality of column sampling points of a first column of each representative image frame and a gray value of a sampling point of a corresponding position of the last column to obtain a plurality of differences of each representative image frame The square of the square, and then the average of the squares of the obtained differences is calculated to obtain the third characteristic value of each representative image frame. The method for identifying a 360-degree panoramic video according to claim 9, wherein said step "extracting gray values of a plurality of row sampling points from the first row and the last row of each representative image frame, and Extracting the gray values of several column sampling points in each of the first column and the last column of each representative image frame" includes: Extracting gray values of the first preset line sample points of each representative image frame from the first line and the last line of each representative image frame at even intervals; The gray values of the second predetermined column sample points of each representative image frame are extracted from the first column and the last column of each representative image frame at even intervals. The method for identifying a 360-degree panoramic video according to any one of claims 8 to 10, wherein the method further comprises: Pre-storing the first preset number of 360-degree panoramic images to the first folder and the second preset number of non-360-degree panoramic images to the second folder; And extracting, by each of the 360-degree panoramic images, features of the first feature value, the second feature value, and the third feature value to form a three-dimensional feature value of each 360-degree panoramic image and storing the same to the first folder; Performing feature extraction of the first feature value, the second feature value, and the third feature value for each non-360 degree panoramic image to form a three-dimensional feature value of each non-360 degree panoramic image and storing the same to the second folder; Generating a three-dimensional scattergram according to the three-dimensional feature value of the first preset number of 360-degree panoramic images and the three-dimensional feature values of the second preset number of non-360-degree panoramic images, wherein each 360-degree panoramic image or one The three-dimensional feature value of the non-360 degree panoramic image corresponds to one feature point in the three-dimensional scatterplot; Converting the three-dimensional scattergram into a two-dimensional scattergram; Determining the feature value of the 360-degree panoramic image and the feature value of the non-360-degree panoramic image according to the feature value of the 360-degree panoramic image on the two-dimensional scattergram and the distribution of the feature values of the non-360-degree panoramic image a boundary line to obtain the linear classifier, wherein an area where the feature value of the 360-degree panoramic image on one side of the boundary line is located is the first area, and the non-the other side of the boundary line The area where the feature value of the 360-degree panoramic image is located is the second area. The method for identifying a 360-degree panoramic video according to claim 11, wherein the step of converting the three-dimensional feature value of each representative image frame into a two-dimensional feature value comprises: Converting the three-dimensional feature value of each representative image frame into a two-dimensional feature value by using a preset three-dimensional coordinate transformation matrix; The step of "converting the three-dimensional scattergram into a two-dimensional scatterplot" includes: Converting all the three-dimensional feature values in the three-dimensional scattergram into two-dimensional feature values by using the preset three-dimensional coordinate transformation matrix to obtain a two-dimensional scattergram formed by all the two-dimensional feature values. The method for identifying a 360-degree panoramic video according to claim 11, wherein said step of "sending said two-dimensional feature value of each representative image frame with a first region and a second region defined by a linear classifier Comparing and determining the area in which the two-dimensional feature values are distributed includes: Calculating which side of the boundary line between the first region and the second region of the two-dimensional feature value of each representative image frame is determined by a preset vector cross-multiplication algorithm, thereby determining the two of each representative image frame Whether the region in which the dimension feature value is located is the first region or the second region. The method for identifying a 360-degree panoramic video according to any one of claims 8 to 12, wherein the step of "defining the two-dimensional feature value of each representative image frame with a linear classifier The first region and the second region are compared, and the region in which the two-dimensional feature value is distributed is determined to include: Comparing coordinate values corresponding to the two-dimensional feature values of each representative image frame with a first coordinate set corresponding to the first region and a second coordinate set corresponding to the second region; When the coordinate value corresponding to the two-dimensional feature value is located in the first coordinate set, determining that the area where the two-dimensional feature value is located is the first area, and when the coordinate corresponding to the two-dimensional feature value When the value is in the second set of coordinates, the area in which the two-dimensional feature value is determined is determined to be the second area. The method for recognizing a 360-degree panoramic video according to claim 8, wherein the step of determining "determining whether the video file to be recognized is based on a ratio of a 360-degree panoramic image in the image frame of the at least one frame 360 degree panoramic video" includes: Determining whether the ratio of the at least one frame representing the 360-degree panoramic image in the image frame is greater than or equal to a preset ratio; If so, it is determined that the video file is a 360-degree panoramic video, and if not, it is determined that the video file is not a 360-degree panoramic video. A method for establishing a linear classifier includes the steps of: Pre-storing the first preset number of 360-degree panoramic images to the first folder and the second preset number of non-360-degree panoramic images to the second folder; And extracting, by each of the 360-degree panoramic images, features of the first feature value, the second feature value, and the third feature value to form a three-dimensional feature value of each 360-degree panoramic image and storing the same to the first folder; Performing feature extraction of the first feature value, the second feature value, and the third feature value for each non-360 degree panoramic image to form a three-dimensional feature value of each non-360 degree panoramic image and storing the same to the second folder; Generating a three-dimensional scattergram according to the three-dimensional feature value of the first preset number of 360-degree panoramic images and the three-dimensional feature values of the second preset number of non-360-degree panoramic images, wherein each 360-degree panoramic image or one The three-dimensional feature value of the non-360 degree panoramic image corresponds to one feature point in the three-dimensional scatterplot; Converting the three-dimensional scattergram into a two-dimensional scatter plot; Determining the feature value of the 360-degree panoramic image and the feature value of the non-360-degree panoramic image according to the feature value of the 360-degree panoramic image on the two-dimensional scattergram and the distribution of the feature values of the non-360-degree panoramic image a boundary line to obtain the linear classifier, wherein an area where a feature value of the 360-degree panoramic image on one side of the boundary line is located is the first area, and the other side of the boundary line The area where the feature value of the non-360 degree panoramic image is located is the second area. The method for establishing a linear classifier according to claim 16, wherein the step of "performing each 360-degree panoramic image with features of the first feature value, the second feature value, and the third feature value to form each a three-dimensional feature value of a 360-degree panoramic image or the step of extracting features of the first feature value, the second feature value, and the third feature value for each non-360 degree panoramic image to form each non-360 degree panoramic image The 3D feature values include: Extracting gray values of pixels of the first row, the last row, the first column, and the last column of the 360-degree panoramic image or the non-360-degree panoramic image; Extracting gray values of a plurality of row sampling points from the first row and the last row, and extracting gray values of the plurality of column sampling points from the first column and the last column respectively; Calculating an average of the variances of the gray values of the plurality of row sampling points of the first row and the last row to obtain the first feature value; Calculating an average of the variances of the gray values of the plurality of column sampling points of the first column and the last column to obtain the second characteristic value; Calculating the square of the difference between the gray value of the plurality of column sampling points of the first column and the gray value of the sampling point corresponding to the last column to obtain a square of the difference, and then calculating the obtained difference The average of the squares gives the third characteristic value. The method for establishing a linear classifier according to claim 16, wherein the step of converting the three-dimensional scattergram into a two-dimensional scattergram comprises: Converting all the three-dimensional feature values in the three-dimensional scattergram into two-dimensional feature values by using the preset three-dimensional coordinate transformation matrix to obtain a two-dimensional scattergram formed by all the two-dimensional feature values. An electronic device includes a processor and a memory, wherein the memory stores an image file to be identified, a video file to be recognized, a plurality of recognized 360-degree panoramic images, and a non-360-degree panoramic image, wherein the A processor for performing the method of any of claims 1-7 to identify whether the image file to be identified is a 360-degree panoramic image, and/or for performing the method of any of claims 8-15 A method for identifying whether the video file to be identified is a 360-degree panoramic image, and/or for performing the method of any of claims 16-18 to establish a linear classifier.

Images