TWI628626B - Multiple image source processing methods - Google Patents

Abstract

The present invention is a method for processing multiple image sources, first receiving a plurality of first images of a first device, arranging a plurality of first images according to a time sequence of each first image capturing, and receiving at least one of the at least one second device And the second image is arranged according to the first shooting positioning information of the first image and the second shooting positioning information of the second image, and the second image is aligned with the first image closest to the geographical space, and then the first geographic space is selected. After an image and a second image are classified into the same range group, the same still life background class is used as the same background group, and the image of the same still background is compared to the image of each background group. The large range of backgrounds are arranged and arranged into a video to be played. The method of the present invention can provide a user with a smooth and unobtrusive feeling when watching a movie.

Description

Multiple image source processing methods

The invention relates to a method for processing image data, in particular to a method for processing multiple images source which can integrate images in a plurality of devices at the same time and assemble them into a film, so that the film is smooth and has no ambiguity.

With the rapid development of technology, camera and photography functions have become more and more popular in various electronic products. At present, almost all communication mobile devices have built-in functions of photography and photography and have positioning service functions. Therefore, each user is When you travel, you can instantly capture images of your travels through the devices at your fingertips.

These retained images can be obtained and edited by the user. The user can use the application in the computer to arrange the photos taken during the trip and then integrate the images into a single film as a travel documentary. Of course, in addition to editing the image in a single photographic device, it is also possible to add images from multiple photographic devices to the same device, thereby combining the images captured by multiple users to make the content richer and more diverse. . However, most of the current applications can only use the order of the shooting time to arrange the photos, but only use the time to sort the photos, which is easy to cause the background of the front and back photos to be different, resulting in inconsistent images, which makes the user easy to produce when viewing photos. Uncoordinated aura.

In addition to the use of time to sequence the images, there are other applications that use location services such as the Global Positioning System (GPS), the Beidou satellite navigation and positioning system of the People's Republic of China, and GLONASS of Russia. The system or the European Union has launched positioning information such as the Galileo program to sort the images. The images contain photos and videos. However, the photos contain Exchangeable image file format (EXIF). For location services, EXIF information includes location information, but EXIF information is not covered in the video. At this time, the video cannot be sorted by location information. In this case, you need to set the location of the representative photo of one of the multiple photos in the movie. The information and direct use of location location information when the device is photographed can be used to sort the film and photos. However, when using the positioning information to sort the images, if the multiple images of the single camera device are sorted, there should be no ambiguity, but when combining the images of multiple cameras, the positioning information of each camera is not the same. There may be errors in the positioning measurement between the cameras, or depending on the type of lens, such as a standard lens or a zoom lens (zoom lens), the same positioning information may have different photo views. In addition, most of the professional monocular cameras produced in the past have no positioning function. It is necessary to query the positioning information of the map such as GPS information afterwards to supplement the positioning information of the registered photos, but even if the photos are sorted according to the positioning information, The various positioning measurement errors between the shooting devices make the same back photos not continuous in the playback order, which leads to the film being inconsistent after the image is integrated, and will improve the residual image caused by the background change when playing the video, so that after integration, The film has become uncoordinated.

Therefore, in order to make the sorting of images imageable, most users will choose to manually sort the image sequence, but manual sorting is quite time consuming. For example, when processing the image data of photos of all members of a tour group, it is even bigger. engineering.

It can be seen from the above that how to integrate the images of users in outbound tourism, become a travel documentary shared by the tour group and enhance the quality of travel documentary production and increase community interaction is a very important part of the current social network age.

In view of the above, the present invention proposes a multiple image source processing method to effectively overcome the above problems in view of the above-mentioned shortcomings of the prior art.

The main object of the present invention is to provide a multi-image source processing method, which can combine images in multiple devices, and simultaneously sort images in multiple devices according to the position of each image and the background of still images, through the still life background. Sorting can reduce various positioning measurement errors between the cameras, enabling the final integrated film to improve the consistency of the playback content, and can be changed during playback according to the angle of the same still life of the image and the next image. Step by step to reduce the swaying feeling when the screen is switched, and adjust the playing time according to the ratio of the same still life of the image and the next image, so as to improve the playing time diversity of each image. In addition, an image playback can be added. The path axis is played in the movie so that the user can select the image to be viewed according to the tourist location.

For the above purposes, the present invention provides a method for processing multiple image sources, comprising the steps of: receiving a first image of a first device and shooting time information of each first image and first shooting positioning information; Receiving a plurality of first images in a shooting sequence of a shooting time information; then receiving at least one second image of the at least one second device and the second shooting positioning information; and second shooting positioning information according to the second image and each first The first shooting positioning information of the image is arranged, and the second image is aligned with the first image closest to the geographical space; and the first shooting positioning information of the first image and the second shooting positioning information of the adjacent second image or the first A shooting position information is subtracted to generate distance information. When the distance information of the adjacent first image or the second image is less than a predetermined distance, the information is classified into the same range group; then each range group is compared. The first image and the second image in the still image background, the first image or the second image having the same still background is classified into the same background group; Group within a complex background of the first image and the second image to the background of a wide range of small-scale background, or the background of small to large range background arrangement.

The purpose, technical content, features and effects achieved by the present invention will be more readily understood by the detailed description of the embodiments.

10‧‧‧ first device

12‧‧‧second device

14‧‧‧ Terminal

20, 20’, 20” ‧ ‧ play video

22, 22’ ‧ ‧ intersection range

24‧‧‧Image playback path axis

26‧‧‧Playing a video

A‧‧‧ center point

The first figure is a block diagram of the system used in the present invention.

The second figure is a flow chart of the sequence of steps of the image of the present invention.

The third figure is a flow chart of the playing steps of the playing movie of the present invention.

4A to 4D are schematic diagrams showing steps of playing a video of the present invention.

The fifth figure is a schematic diagram of converting the playback image of the present invention into the next playback image.

The sixth figure is a schematic diagram of the video playback path axis playing interface of the present invention.

Referring to the first figure, the system architecture used in the present invention is first described, which includes a first device 10, a second device 12, and a terminal device 14. The first device 10 is an electronic device capable of connecting to a network to upload or download images and play a movie, such as a smart mobile communication device, a smart phone or a computer, etc., and the embodiment is provided with photography, photographing and positioning. The smart phone of the service function is used as an embodiment of the first device 10. The first device 10 can generate a first image by using a shooting image. The first image in this embodiment can include a movie and a photo. The first device 10 can be used when shooting. At the same time, the shooting time information and the first shooting position positioning information of the shooting location are captured, wherein the first shooting positioning information can be a Global Positioning System (GPS) signal, the Beidou satellite navigation positioning system of the People's Republic of China, Russia The GLONASS system or the European Union launches the Galileo program, but if the first device 10 does not have the location service enabled, so that the first location information cannot be obtained, the user can manually enter the first shot. Positioning information to define the location of each of the first images. The second device 12 can be a plurality of devices. However, the second device 12 can be connected to the network as an electronic product for uploading or downloading images and playing a movie. The second device 12 can also be a smart mobile device with a function of photographing, photographing and positioning services, such as a smart phone or a computer, which can be used to generate a second image, and can be used to capture a picture. At the same time, the second location information of the shooting location is captured. The second information can also be the Global Positioning System (GPS) signal, the Beidou satellite navigation and positioning system of the People's Republic of China, and GLONASS of Russia. The system or the European Union launches the "Galileo" program, but if the second device 12 is unable to obtain the second location information, the user can manually enter the second location information to define The position of each second image. The first image and the second image include the photo and the movie, wherein the photos include an Exchangeable image file format (EXIF). If the positioning service is provided when the photo is taken, the EXIF information includes the positioning. Information, but the EXIF information is not included in the video. In this case, the video and the photo cannot be sorted. You need to set the representative photo of one of the multiple photos in the movie to set the positioning information or directly use the position of the device when shooting. Information substitution.

The terminal device 14 is a computer that can be used to process information. In this embodiment, the terminal device 14 is used as a computer host installed in the cloud, and the terminal device 14 is connected to the first device 10 and the second device 12 to receive the first through the network. The information in the device 10 and the second device 12 enables the terminal device 14 to view the first image of the first device 10 and the second device 12, the first shooting positioning information, and the second shooting position positioning information. And the second image arrangement sequence is arranged, and the first image and the second image after the arrangement are made into a play movie, so that the first device 10, the second device 12 or the remaining devices (not shown) can be connected to the network. It can be connected to the terminal 14 through the network to browse and play the video, in addition to editing the individual images through the first device 10 or the second device 12, such as adding text or selecting a favorite soundtrack to match Integrated videos to increase community interaction.

After the system architecture of the present invention is described, please refer to the first figure and the second figure for details on the method for arranging the images of the present invention. First, please refer to step S10. First, the terminal device 14 can receive a first a plurality of first images of the device 10, and shooting time information of each of the first images and the first shooting position positioning information, and the terminal device 14 can simultaneously determine the first image when receiving the first image, when the screen has In an unclear situation, such as a blurred picture, poor focus, or a situation in which a person in a first image has a grimace, such as a closed eye, a white eye, or a slanted face, the terminal 14 can take the first image having the above states from The terminal 14 is deleted to prevent the unclear photos from being retained in the terminal 14. Then, referring to step S12, the terminal device 14 arranges the plurality of first images according to the time of the shooting sequence of each shooting time information, and the first images generated by the first shooting are arranged at the forefront, and are sequentially arranged according to the chronological order. An image, starting from the location where the first shooting position is located at the beginning of the time The location of the head, the first shooting position positioning information of the last time is the end point.

After the terminal device 14 arranges the first image according to the time, please refer to step S14, and the terminal device 14 receives at least one second image in the second device 12. In this embodiment, the second device 12 transmits the second image to the second device. In the terminal device 14, the second device 12 transmits the second shooting positioning information of each second image to the terminal device 14. At the same time, the terminal device 14 can also determine the picture of the second image. When the picture is unclear, such as when the picture is blurred, the focus is poor, or the character in the second image has a ghost face, such as a closed eye, a white eye or a slanted face. The terminal device 14 can delete the second image having the above states from the terminal device 14 to prevent the unclear photos from being retained in the terminal device 14; in addition, the terminal device 14 can receive the second image. After the step, a time shooting path formed by arranging the first images in the shooting sequence time is used as a reference, and when the second shooting positioning information of each second image is compared with the shooting path distance by more than a distance setting value, The second image terminal device 14 is deleted to prevent the outlier image from being retained in the terminal device 14, which can reduce the sudden sensation caused by the user's apparently different location when viewing the image.

Next, as shown in step S15, the terminal device 14 approximates the second image to the space according to the second shooting positioning information of the second image and the first shooting positioning information of each first image, for example, by calculating the minimum moving distance. The first image arrangement of the plurality of images, that is, the second image selects the first image closest to the distance to be sorted as a line. Then, proceeding to step S16, the terminal device 14 captures the plurality of first shooting positioning information of the plurality of first images, and the plurality of second shooting positioning information of the plurality of second images, and returns the first image and the second image having the same geographic space. The class is the same range group, wherein the step of classifying the first image and the second image into the same range group is to compare the first shooting positioning information of the adjacent first image or the second shooting positioning information of the second image. Subtracting the distance information, when the distance information generated by the first shooting positioning information of the subtracted first image or the second shooting positioning information of the second image is less than a preset distance, indicating the two subtracted first images The same range group as the second image can be accessed to the following step S17. The setting of the above-mentioned preset distance is generally due to the scene of the open space. The president is consistent, so generally the preset distance set by the open grassland area will generally be equal to or greater than the city street. For example, the pre-set distance of the open area, such as the Selim Lake area in Xinjiang, China, can be set to 1,000 meters, while the urban street, such as the pre-set distance of Urumqi, Xinjiang, China, is set to 500 meters. Just fine.

After classifying the first image and the second image into different range groups, proceeding to step S17 to compare the first image with the same still life background with the still image of the first image and the second image in each range group Or the second image is classified into the same background group, wherein the still life background may be an item, a building, a texture, or a scene such as a road, a lake, a mountain, a utility pole, an obvious landmark, or the like. Finally, proceeding to step S18, the first image and the second image in each background group are arranged from a wide range background to a small range background, or a small range background to a large range background, and the embodiment is exemplified by a wide range to a small range. Sort. The large-scale background and the small-scale background are judged according to the number of still-life backgrounds in the image of the image, and the image with more still-life background can be judged as a large-scale background, and the less-remaining background can be judged as a small-scale background.

Finally, proceeding to step S19, according to capturing the first shooting positioning information or the second shooting positioning information with the highest altitude in each background group, to become a representative height information representing the background group, and according to each background group The representative height information is arranged by the background group with the highest altitude to the lowest background group, but if the altitude of the two background groups representing the height information is the same, then the early shooting time information in the background group is taken as per A representative of the background group, and then sequence the shooting of the two background groups according to the shooting time information. In terms of shooting images, the higher the height, the more the still life background will be. For example, the more the still life background will be taken in the aerial photo, so the altitude sorting can more effectively integrate the image sorting. Therefore, the above-mentioned continuous image content can be harmonious and have a sense of continuity by the above method.

After the first image and the second image are arranged according to the above method, the terminal 14 can edit the first image and the second image into a plurality of playback images to become one of the playable videos. Please refer to the first, third, fourth, fourth, fourth, and fifth figures to illustrate the hair. In the playback step, the process first proceeds to step S20, and the terminal device 14 first provides a panoramic picture of the playback image 20 for display on the display screen of the first device 10 or the second device 12. Next, proceeding to step S21 and referring to FIG. 4A, the terminal device 14 captures the intersection range 22 of the next playback image on the playback image 20, wherein the intersection range 22 is a range having the same still life background. Then, proceeding to step S22, the terminal device 14 determines whether the ratio of the intersection range 22 to the playback image 20 is greater than a proportional value, and if not, proceeds to step S24, and directly plays the next playback image 20 after playing the image 20; If yes, the process proceeds to step S26 and the image frames (Zoom in) are gradually enlarged to the intersection range 22 of the playback image 20 with reference to the fourth A and fifth figures. At this time, the terminal 14 can adjust the intersection range 22 of the playback images 20. And the playing area of the intersection area 22' of the next playback image 20', in order to conform to the aspect ratio of a playback screen, and then to the center point A of the intersection range 22 on the playback image 20, that is, As shown in FIG. 4B, the terminal unit 14 adjusts the intersection range 22 until the intersection range 22 occupies the screen for playing the screen. The smaller the ratio of the intersection range 22 of the playback image 20 to the playback image is, the longer the time of moving to the center point of the intersection range 22 is. Furthermore, when the comparison intersection range 22 accounts for the ratio of the playback image 20, the ratio of the Zoom in intersection range 22 can be determined as the terminal unit 14 can avoid the image placement in the intersection range being too large. Image distortion caused by poor resolution. Please continue to refer to step S26 and refer to the fourth B and fifth figures to zoom in on the intersection of the playback images to display only the intersection of the playback image 20, such as playing the video on the first device 10. When the intersection range 22 is just in accordance with the aspect ratio of the display screen of the first device 10, the screen of the intersection range fills the display screen of the first device 10; of course, while the intersection range 22 is enlarged, the rotation is simultaneously coordinated. The angle of the intersection range 22 is such that the angle of the intersection range 22 is the same as the angle 22' of the intersection of the next play image 20'.

Finally, proceeding to step S28 and referring to the fourth C and fifth figures, the screen jumps to display the intersection range 22' of the next play image 20', and moves to the next one with reference to the fourth D map and the fifth map. The center point of the panoramic image of the playback image 20' will be zoomed out at the same time (Zoom out) the intersection of the next playback image 20' 22' until the entire screen is played. Like the panoramic view of 20', then play the next playback image 20", the playback mode is the same as above, so it is not repeated. The intersection of the next playback image 20' is 22'. The smaller the ratio of 20', the longer the time to convert to a panoramic picture. For example, when the ratio of the intersection range 22' is 80% of the panoramic image of the playback image 20, the time to convert to the panoramic picture is about 1.25 seconds, if the intersection range When the proportion of 22' accounts for 90% of the 20' panoramic image of the playback image, the time to convert to the panoramic image is 1.11 seconds, and the time is relatively small.

In addition, referring to the sixth figure, this embodiment can further generate an image playback path axis 24 to the play movie 26, which can be displayed in the screen of the play movie 26, or as shown in the sixth figure. The time axis of the playback device movie is the same and is displayed in the playback movie 26. When the user clicks on the location in the video playback path axis 24, the playback image of the corresponding location can be played, so that the user can know at which location the current playback image is taken, which is beneficial to the community friends on the network. Know where to shoot. The image playback path axis 24 is a moving path generated by the second shooting positioning information of the playback image and the first shooting positioning information, and may also utilize a regression analysis method such as a least square method, a moving average method or most Like the method, the first shooting positioning information and the second shooting positioning information in the same range group are smoothed, and the curve of the moving path in the smoothing group group is smoothed, so that the path is smoothed, the jagged shape is reduced, and the viewer clicks the operation.

In summary, the present invention can combine images in a plurality of devices, and simultaneously sorts images in multiple devices according to positional positioning information of each image and still life background, and can reduce various types of photography devices by sorting still images. The positioning measurement error enables the final integrated film to improve the consistency of the playback content, and reduces the residual image caused by the background change when playing the video, and can be adjusted according to the angle and scale range of the image and the next image. Image playback angle and playback time to reduce the playback time of the two images due to the tilt angle of the two images and the playback time of different images. In addition, an image playback path axis can be added to In the video, the user can select the image to be viewed according to the tourist location.

The above is only the preferred embodiment of the present invention and is not intended to limit the scope of the present invention. Therefore, any changes or modifications of the features and spirits of the present invention should be included in the scope of the present invention.

Claims (10)

A method for processing multiple image sources, comprising the steps of: receiving a first image of a first device and shooting time information of each of the first images and first shooting positioning information; according to a shooting sequence of each shooting time information Time-aligning the first images; receiving at least one second image of the at least one second device and second shooting positioning information of the second image; and the second shooting positioning information according to the second image and each of the first images Aligning the second image with the first image that is closest to the geographic space; and the second image capturing location information of the first image and the second image of the adjacent second image The shooting positioning information or the first shooting positioning information is subtracted to generate the distance information, and when the distance information of the adjacent first image or the second image is less than a preset distance, it is classified into the same range group; Comparing the first image and the second image still life background in each of the range groups, classifying the first image or the second image having the same still background as the same background Group; and each of those within the first group of background image and the second image to the background by a wide range of small-scale background, or the background of small to large range background arrangement. The method for processing multiple image sources according to claim 1, wherein after the step of receiving the second image, time-lapse path formed by arranging the first images in the shooting sequence is used as a reference to compare When the second shooting positioning information of the second image and the shooting path distance exceed a distance setting value, the second image may be deleted. The multiple image source processing method of claim 1, further comprising extracting each of the background groups The first shooting positioning information or the second shooting positioning information with the highest altitude in the group to become a representative height information representing the background group, and the highest altitude information is highest according to the representative height information of each of the background groups The background group is arranged to the lowest background group, but if at least two of the background groups of the representative height information have the same altitude, the background groups are arranged in the order of the shooting time. The method of processing multiple image sources as claimed in claim 1, wherein the still life background is an item, a building, a texture, or a scene. The processing method of the multiple image sources described in claim 1 further includes sequentially arranging the first image and the second image to form a plurality of playback images, and playing one of the playable videos to play the playback movie. The step of playing the played video includes: displaying a panoramic image of the played image; capturing, on the play image, an intersection range of the same still image as the next play image; determining that the intersection range occupies the play image Whether the ratio is greater than a proportional value: if not, playing the next playback image directly; and if so, proceeding to the next step; gradually zooming in the intersection range of the playback image until only the intersection of the playback image is displayed And displaying the intersection range of the next played image and moving to the center point of the panoramic picture of the next played image, gradually reducing the picture of the next played image until only the next one is displayed The panoramic view of the playback image. The processing method of the multiple image source according to claim 5, wherein the intersection of the playback image and the next playback image are further adjusted before the step of moving to the center point of the intersection range on the playback image The playing area of the intersection range to match one Play the aspect ratio of the screen. In the processing method of the multiple image sources described in claim 5, the smaller the ratio of the intersection of the playback image to the playback image is, the longer the time is moved to the intersection range. For the processing method of the multiple image sources described in claim 5, the smaller the ratio of the intersection of the next playback image to the next playback image, the longer the conversion to the panoramic image. The processing method of the multiple image sources described in claim 1 further includes sequentially arranging the first image and the second image to form a plurality of playback images, and playing one of the playable videos to play the playback movie. After that, an image playback path axis is generated into the playing movie. When the location in the video playback path axis is clicked, the played image corresponding to the location may be played, and the video playback path axis is according to each of the played images. The first shooting positioning information and the second shooting positioning information sequentially generate a moving path. The processing method of the multiple image source according to claim 9, wherein the image playback path axis transmits a movement path generated by the second shooting positioning information of the playback image and the first shooting positioning information, and utilizes A regression analysis method smoothes the first shooting positioning information and the second shooting positioning information in the same group of the range, so that the curve of the moving path is smooth.