CN114390219B - Photography methods, devices, electronic equipment and storage media - Google Patents

Abstract

This application discloses a shooting method, device, electronic equipment and storage medium. The shooting method is applied to electronic equipment. The electronic equipment includes a camera that can move in a preset direction. The shooting method includes: obtaining the data collected by the camera. A first framing image and a second framing image; based on the first framing image and the second framing image, control the camera to move along the preset direction, and capture images during the movement to obtain multiple images to be synthesized ; Perform image reconstruction on each of the plurality of images to be synthesized, and generate a panoramic image based on the plurality of images to be synthesized after reconstruction. This method can quickly realize the shooting of panoramic images, and can improve the quality of panoramic images.

Description

Photography methods, devices, electronic equipment and storage media

Technical field

The present application relates to the technical field of electronic equipment, and more specifically, to a photographing method, device, electronic equipment and storage medium.

Background technique

Electronic devices, such as mobile phones, tablets, etc., have become one of the most commonly used consumer electronic products in people's daily lives. With the development of science and technology, electronic devices are usually equipped with cameras to realize the shooting function. Moreover, most current electronic devices support the shooting of panoramic images, thereby allowing users to obtain images with a larger field of view. However, the current convenience of shooting panoramic images is insufficient.

Contents of the invention

This application proposes a shooting method, device, electronic equipment and storage medium, which can quickly realize the shooting of panoramic images.

In a first aspect, embodiments of the present application provide a shooting method applied to an electronic device. The electronic device includes a camera that can move in a preset direction. The method includes: acquiring a first view image collected by the camera; a second framing image; based on the first framing image and the second framing image, control the camera to move along the preset direction, and capture images during the movement to obtain multiple images to be synthesized; Perform image reconstruction on each of the images to be synthesized, and generate a panoramic image based on the multiple images to be synthesized after reconstruction.

In the second aspect, embodiments of the present application provide a shooting device, which is applied to electronic equipment. The electronic equipment includes a camera that can move in a preset direction. The device includes: an image acquisition module, an image shooting module, and an image reconstruction module. and an image generation module, wherein the image acquisition module is used to acquire the first view image and the second view image collected by the camera; the image capture module is used to obtain the first view image and the second view image based on the first view image and the second view image, Control the camera to move along the preset direction and capture images during the movement to obtain multiple images to be synthesized; the image reconstruction module is used to reconstruct each image to be synthesized in the multiple images to be synthesized, Carry out image reconstruction; the image generation module is used to generate a panoramic image based on the multiple images to be synthesized after the image reconstruction.

In a third aspect, embodiments of the present application provide an electronic device, including: one or more processors; a memory; and one or more application programs, wherein the one or more application programs are stored in the memory and Configured to be executed by the one or more processors, the one or more application programs are configured to execute the photographing method provided by the above-mentioned first aspect.

In a fourth aspect, embodiments of the present application provide a computer-readable storage medium. The computer-readable storage medium stores program code. The program code can be called by a processor to perform the shooting provided in the first aspect. method.

In the solution provided by this application, the electronic device is equipped with a camera that can move in a preset direction. When the electronic device is shooting, by acquiring the first view image and the second view image collected by the camera, based on the first view image and the second view image, image, control the camera to move along the preset direction, and capture images during the movement to obtain multiple images to be synthesized, and then perform image reconstruction on each of the multiple images to be synthesized, and then reconstruct the image based on the reconstructed image. Multiple images to be synthesized to generate a panoramic image. As a result, the user can use the electronic device to capture the first view image and the second view image, control the camera to move and capture panoramic images, and perform image reconstruction on the multiple captured images to generate a panoramic image, which can improve Image quality of the obtained panoramic image.

Description of the drawings

In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed to be used in the description of the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained based on these drawings without exerting creative efforts.

Figure 1 shows a schematic diagram of the application environment provided by the embodiment of the present application.

Figure 2 shows a flow chart of a shooting method according to an embodiment of the present application.

Figure 3 shows a flow chart of a shooting method according to another embodiment of the present application.

Figure 4 shows a flow chart of a shooting method according to yet another embodiment of the present application.

Figure 5 shows a schematic diagram of a panoramic image provided by an embodiment of the present application.

Figure 6 shows a flow chart of a shooting method according to yet another embodiment of the present application.

Figure 7 shows a block diagram of a photographing device according to an embodiment of the present application.

FIG. 8 is a block diagram of an electronic device used to perform a photographing method according to an embodiment of the present application.

Figure 9 is a storage unit used to save or carry the program code for implementing the shooting method according to the embodiment of the present application.

Detailed ways

In order to enable those in the technical field to better understand the solution of the present application, the technical solution in the embodiment of the present application will be clearly and completely described below in conjunction with the drawings in the embodiment of the present application.

Currently, the popularity of electronic devices (such as smartphones, tablets, etc.) in daily life has reached nearly universal coverage. Among them, the camera module has become the main function point of electronic devices. Users can take photos through the camera function of electronic devices. With video, the resulting images are instantly captured, which is convenient and fast. Not only that, users also often upload the captured images to the Internet to share them with others.

When shooting through electronic devices, expanded field of view (Field of View, FOV) imaging has an important application in users' photography. Currently, the imaging method to achieve a larger FOV can use an ultra-wide-angle imaging module. Ultra-wide-angle imaging refers to capturing a lens that is larger and wider than the FOV of the main camera. The commonly used ultra-wide-angle equivalent focal length is 12-15mm, and the FOV is approximately 120 degrees. , due to the short focal length, wide viewing angle, and long depth of field of the ultra-wide-angle lens, it is usually used to take landscape photos, and it can also be used to take close-up shots to show a broad perspective effect. Of course, there are also panoramic image shootings through the main camera to expand the field of view of imaging. In this method, the user uses a panoramic camera based on the main camera to expand the FOV imaging. Panoramic imaging requires the user to frame the view along a specific axis or border. Then the phone performs image synthesis internally to obtain an image with an extended FOV.

In related technologies, in the method of using the main camera to capture panoramic images, the user needs to keep holding the electronic device to move, so there is an inconvenience problem; moreover, the shaking of the user's hand during shooting will prevent the block areas from being fixed in the same position. The horizontal plane first makes it difficult to capture clear photos. Secondly, the captured images are difficult to align due to jitter and have edge distortion or blur, which greatly affects the shooting experience. In addition, this method also has shortcomings such as out-of-focus at close range and low resolution. The resulting image quality is poor, affecting the user's shooting experience.

The scenarios involved in the embodiments of this application are first introduced below.

As shown in Figure 1, the scene shown in Figure 1 includes an electronic device 100. The electronic device 100 includes a terminal body 130, a mobile component 140 and a camera 150. The camera 150 is installed on the mobile component 140, thereby realizing the mobile component 140. The camera 150 is driven to move.

In some embodiments, the camera 150 is driven by the moving component 140 and can move in a preset direction. The preset direction may be a horizontal direction, a vertical direction, or may include both a horizontal direction and a vertical direction.

In the embodiment of the present application, as shown in Figure 1, the moving component 140 can be a movable guide rail, and the camera 150 can move along the guide rail. The driving method of the guide rail can be piezoelectric, electromagnetic, etc. In this method, the camera 150 can The direction of movement is determined by the arrangement direction of the guide rails.

In addition, other functional devices of the electronic device 100 may also be provided on the mobile component 140 , where other functional devices may include proximity light sensors, earpieces, distance sensors, ambient light sensors, temperature sensors, pressure sensors and other devices.

The photographing method provided by the embodiment of the present application will be introduced in detail below with reference to the accompanying drawings.

Please refer to Figure 2. Figure 2 shows a schematic flowchart of a shooting method provided by an embodiment of the present application. In a specific embodiment, the photographing method is applied to the photographing device 400 shown in Fig. 8 and the electronic device 100 (Fig. 9) configured with the photographing device 400. The following will take an electronic device as an example to illustrate the specific process of this embodiment. Of course, it can be understood that the electronic device applied in this embodiment can be a smart phone, a tablet computer, an e-book, etc., and is not limited here. The process shown in Figure 2 will be described in detail below. The shooting method may specifically include the following steps:

Step S110: Obtain the first view image and the second view image collected by the camera.

In this embodiment of the present application, the electronic device can acquire the first view image and the second view image collected by the camera, so as to capture the panoramic image based on the first view image and the second view image. Wherein, the first framing image and the second framing image may be images taken by the user using an electronic device to determine the framing range of the panoramic image, and the first framing image and the second framing image are when the electronic device is located in the first framing image and the second framing image. When framing the image's framing range, the image can be collected when the camera moves in the preset direction. It is understandable that when a user uses an electronic device to shoot a panoramic image, he usually uses the electronic device to start shooting the image at a certain position, and then moves the electronic device to another position to complete the shooting of the panoramic image. Based on the shooting principle of the panoramic image , the electronic device can acquire images at the boundary of the panoramic image (ie, the first framing image and the second framing image) for controlling the shooting of the panoramic image.

In some implementations, the electronic device may output prompt information to prompt the user to use the electronic device to collect the first view image and the second view image. The output prompt information can be used to prompt the user to collect two images used to determine the viewing range of the panoramic image, and the collected images should be images that can be collected during the movement of the camera in the preset direction. As a result, the above-mentioned first view image and second view image can be obtained.

Step S120: Based on the first framing image and the second framing image, control the camera to move along the preset direction, and capture images during the movement to obtain multiple images to be synthesized.

In this embodiment of the present application, after the electronic device acquires the first framing image and the second framing image, it can use the first framing image and the second framing image to capture the panoramic image. Wherein, the electronic device can control the camera to move in a preset direction based on the first view image and the second view image, and capture images during the movement, and obtain multiple images to be synthesized into a panoramic image based on the captured images. Composite image. That is to say, the above-mentioned preset direction is the viewing direction when shooting panoramic images. The multiple images captured by the electronic device while the camera moves in the preset direction may include the image content of the first viewing image, the second viewing image The image content, and the image content between the image content of the first framing image and the image content of the second framing image, therefore, the image to be synthesized for synthesizing the panoramic image can be obtained based on this. The multiple images to be synthesized may include image content within the framing range determined based on the first framing image and the second framing image. That is to say, the image content of the multiple images to be synthesized includes the image content of the first framing image, the image content of the second framing image, and the image content of the first framing image. The image content of the two framing images, and the image content between the image content of the first framing image and the image content of the second framing image.

In some embodiments, the image content that the electronic device needs to obtain includes the image content of the first framing image, the image content of the second framing image, and the image content between the image content of the first framing image and the image content of the second framing image. Therefore, in the process of controlling the movement of the camera in a preset direction, the image content of the captured image can be detected. If the captured image contains the image content of the first framing image and the image content of the second framing image, the movement of the camera can be stopped, and the image containing the first framing image can be determined from the multiple images taken during the movement. The image content of the image, the image containing the image content of the second view image, and the image containing the image content in between are used as the image to be synthesized.

In some embodiments, the electronic device may output prompt information to prompt the user to hold the electronic device so that the camera of the electronic device is within the viewing range of the first and second viewing images, and the electronic device keeps the posture of the electronic device in The target posture is to ensure that the image content of the previously collected first view image and the second view image can be collected when the camera moves in the preset direction. This ensures that when the electronic device controls the camera to move in the preset direction, the image content required by the user can be collected to generate a panoramic image.

In some embodiments, when the electronic device controls the movement of the camera and collects images through the camera, it can collect images according to the moving distance of the camera to ensure that the image content in the collected images is continuous, so that it can be generated later. panoramic image.

Step S130: Perform image reconstruction on each of the multiple images to be synthesized.

In the embodiment of the present application, after obtaining multiple images to be synthesized, the electronic device can perform image reconstruction on the multiple images to be synthesized, so as to improve the image quality of the images to be synthesized. Among them, image reconstruction can be image reconstruction based on stacked imaging, image denoising, contrast improvement, saturation improvement, clarity improvement, resolution improvement and other reconstruction processes. By performing image reconstruction on the image to be synthesized, the subsequently generated panorama can be The image quality of the image is better.

Step S140: Generate a panoramic image based on the multiple images to be synthesized after the image reconstruction.

In this embodiment of the present application, after the electronic device performs image reconstruction on multiple images to be synthesized, the panoramic image can be synthesized based on the multiple images to be synthesized after image reconstruction, so that a panoramic image with better image quality can be obtained.

In some embodiments, the electronic device can splice the images to be synthesized according to the matching areas between the images to be synthesized, thereby generating a panoramic image. Among them, for two images to be synthesized that are adjacent in shooting time sequence, the matching area between the two images to be synthesized can be used as the overlapping area, and the two images to be synthesized are overlapped together according to the overlapping area; repeat this to complete All the images to be synthesized are stitched together to obtain the generated panoramic image. Since the image content in the multiple images to be synthesized includes the image content of the first framing image, the image content of the second framing image, and the image content between the image content of the first framing image and the image content of the second framing image, therefore , the panoramic image obtained by splicing multiple images to be synthesized can contain the image content required by the user. Of course, the way in which the electronic device synthesizes multiple images to be synthesized to obtain a panoramic image can also be other synthesis methods used to generate a panoramic image.

The shooting method provided by the embodiment of the present application allows the user to use an electronic device to shoot the first view image and the second view image, and then control the camera to move and capture panoramic images without holding the electronic device and maintaining a corresponding posture to move the electronic device. Shooting with an electronic device can facilitate users to obtain panoramic images through shooting with electronic devices; and reconstructing multiple captured images to generate panoramic images can improve the image quality of the obtained panoramic images.

Please refer to FIG. 3 , which shows a schematic flowchart of a shooting method provided by another embodiment of the present application. This shooting method is applied to the above-mentioned electronic equipment. The process shown in Figure 3 will be described in detail below. The shooting method may specifically include the following steps:

Step S210: Obtain the first view image and the second view image collected by the camera.

Step S220: Based on the first framing image and the second framing image, control the camera to move along the preset direction, and capture images during the movement to obtain multiple images to be synthesized.

In the embodiment of the present application, the content of steps S210 and S220 can be referred to other embodiments, and will not be described again here.

Step S230: Perform image reconstruction based on stacked imaging for each of the multiple images to be synthesized to obtain a panoramic image.

In the embodiment of the present application, after obtaining multiple images to be synthesized, the electronic device can perform image reconstruction based on stacked imaging on the multiple images to be synthesized. Since the image reconstruction based on stacked imaging is used to reconstruct the image, It reconstructs images collected at different locations into higher-resolution images, so it can not only complete image synthesis, but also improve image quality. Among them, image reconstruction based on stack imaging can be Fourier stack imaging. In Fourier stack imaging, the principle is: since different low-resolution images correspond to specific sub-regions in the sample spectrum, optimization algorithms can be used to reconstruct the image in the frequency domain. The information of low-resolution images is fused to obtain high-resolution amplitude and phase images that exceed the limit resolution of the system objective lens and retain its original field of view. Since image reconstruction based on stacked imaging itself processes multiple images at different locations, this reconstruction processing method can be used to reconstruct and synthesize multiple images to be synthesized, so that the synthesized panoramic image can be directly obtained. .

In some embodiments, image reconstruction based on stacked imaging is performed on each of the multiple images to be synthesized to obtain multiple images to be synthesized after the image reconstruction, including: Aperture function, obtains the correspondence between each image to be synthesized and the reconstructed spectrum; performs Fourier transform on each image to be synthesized, and obtains the frequency domain information corresponding to each image to be synthesized; based on the correspondence relationship and the frequency domain information corresponding to each image to be synthesized, replace the amplitude at the position corresponding to each image to be synthesized in the reconstructed spectrum; repeat the aperture function based on the camera, The step of obtaining the corresponding relationship between each image to be synthesized and the reconstructed spectrum is to, based on the corresponding relationship and the frequency domain information corresponding to each image to be synthesized, to compare the reconstructed spectrum with the The step of replacing the amplitude at the corresponding position of each image to be synthesized until the preset iteration conditions are met; transforming the reconstructed spectrum into the time domain to obtain the panoramic image.

In this method, the electronic device can determine the corresponding area of the frequency domain area corresponding to the image to be synthesized in the reconstructed spectrum based on the aperture function; then, the image to be synthesized is convolved with the point spread function corresponding to the camera, and then the Fourier transform is performed , obtain the frequency domain information; then based on the corresponding area of the frequency domain area corresponding to the image to be synthesized in the reconstructed spectrum in the correspondence relationship, and the frequency domain information corresponding to the image to be synthesized, the position corresponding to each image to be synthesized in the reconstructed spectrum is The amplitude at the corresponding region is replaced, and the phase of the corresponding region is kept unchanged, thereby completing the update of the reconstructed spectrum. This is repeated until the preset iteration conditions are met, and the reconstructed spectrum is subjected to inverse Fourier transform to transform it into the time domain to obtain the final panoramic image. The preset iteration condition may be that the number of iterations reaches the target number, or that the difference between the reconstructed spectrum after the update and the reconstructed spectrum before the update is smaller than the target difference. The specific iteration conditions may not be limited.

As a possible implementation, before performing image reconstruction based on stacked imaging, the electronic device can obtain the point spread function corresponding to the camera, and then determine the aperture function of the camera based on the point spread function. Among them, the electronic device can determine the pre-designed point spread function based on the pre-designed optical parameters of the camera; and then determine the aperture function corresponding to the camera based on the designed point spread function and the measured point spread function when shooting the image. Alternatively, the electronic device can obtain the point spread function information of multiple fields of view and wavelengths at the camera's focus position based on the lens design optical file of the camera, and then obtain the R, G, and B channels of the image sensor through a weighted algorithm based on the sensor spectral response curve. For point spread function information, the parameter interval of the point spread function is downsampled to the pixel size of the image sensor through a downsampling algorithm, thereby obtaining a pre-designed point spread function. Among them, the lens design optical files of the camera, such as Zemax lens design optical files, etc., are not limited here.

The shooting method provided by the embodiment of the present application, due to the image reconstruction based on stacked imaging, can not only synthesize multiple images collected at different positions, but also improve the image resolution and perform aberration correction. Therefore, the multiple captured images can be When the images to be synthesized are reconstructed and a panoramic image is generated, using image reconstruction based on stacked imaging to process multiple images to be synthesized can improve the image quality of the generated panoramic image.

Please refer to FIG. 4 , which shows a schematic flowchart of a shooting method provided by yet another embodiment of the present application. This shooting method is applied to the above-mentioned electronic equipment. The process shown in Figure 4 will be described in detail below. The shooting method may specifically include the following steps:

Step S310: Obtain the first view image and the second view image collected by the camera.

In the embodiment of the present application, reference may be made to the content of the previous embodiment for step S310, which will not be described again here.

Step S320: Control the camera to move along the preset direction, and capture images during the movement.

In this embodiment of the present application, after obtaining the first framing image and the second framing image, the electronic device can control the camera to move in a preset direction and capture images during the movement. Optionally, the electronic device can output prompt information to prompt the user to place the electronic device at a position where the image content of the first framing image and the second framing image can be collected by moving the camera in a preset direction; and then receive the user's request based on When prompted to perform a shooting operation, the camera is controlled to move in a preset direction, and images are captured during the movement.

Step S330: When the image captured by the camera includes first image content and second image content, stop the movement of the camera and image capture, and the first image content matches the first view image, and the The second image content matches the second view image.

In the embodiment of the present application, when the electronic device controls the movement of the camera, the collected image can be matched with the first framing image and the second framing image. If the image captured by the camera already includes the first image content and the second framing image, Image content means that the camera has collected the image content of the first framing image, the image content of the second framing image, and the image content between the image content of the first framing image and the image content of the second framing image. At this time, there is no need Continue to control the camera movement, so you can stop the camera movement and image capture.

In some embodiments, the preset direction includes a direction moving from the first side of the electronic device to the second side of the electronic device, and a direction moving from the second side of the electronic device to the first side of the electronic device. If the electronic device After controlling the camera to move in one of the preset directions to the maximum distance, if the image content of the first view image or the image content of the second view image has not been captured, you can control the camera to move in another direction to collect at the same time. to the image content of the first framing image and the image content of the second framing image. It can be understood that since the shooting direction of the position where the camera is when starting image collection may be between the image content of the first view image and the image content of the second view image, or may be located outside the two, the camera moves along a By moving in one direction, the image content of the first framing image and the image content of the second framing image may not be captured at the same time. Therefore, after controlling the camera to move in one direction to the maximum distance, if the image of the first framing image is not captured at the same time The content of the first framing image or the image content of the second framing image can be moved in the reverse direction, so that the image content of the first framing image and the image content of the second framing image are simultaneously captured.

Step S340: Based on the first view image and the second view image, select a plurality of images to be synthesized from the images captured by the camera.

In this embodiment of the present application, after controlling the camera to stop moving and capturing images, the electronic device can filter out multiple images to be synthesized from the images captured by the camera based on the first framing image and the second framing image. It can be understood that since the shooting direction of the position where the camera is when starting image collection may be between the image content of the first view image and the image content of the second view image, or may be located outside the two, the camera captures The images may include images corresponding to image content outside the viewing range composed of the first viewing image and the second viewing image. In this case, in order to meet the user's needs, the panoramic view of the viewing range composed of the first viewing image and the second viewing image can be obtained. image, based on the first framing image and the second framing image, multiple images to be synthesized can be selected from the images captured by the camera.

In some embodiments, based on the first view image and the second view image, filtering out multiple images to be synthesized from the images captured by the camera includes: sorting the images captured by the camera in the order of shooting time, to obtain Image sequence; from the image sequence, obtain at least one first target image, at least one second target image and the image between the first target image and the second target image, and obtain a plurality of images to be synthesized, Wherein, the first target image includes the first image content, and the second target image includes the second image content. It is understandable that since the electronic device may not be able to capture exactly the same image as the first framing image and the second framing image during movement, that is, the image content of the first framing image may be distributed in the two captured images. The image content of the second framing image may also be distributed in the two captured images, but in order to obtain a panoramic image of the framing range composed of the first framing image and the second framing image, it is necessary to make the panoramic image include the first framing image and The image content of the second view image, therefore, the first target image and the second target image may be a single image, or may not be a single image.

Optionally, if the preset direction includes a direction moving from the first side of the electronic device to the second side of the electronic device, and a direction moving from the second side of the electronic device to the first side of the electronic device, the camera may appear. By moving back and forth, the image captured by the camera includes the first image content and the second image content. At this time, it is possible that when the electronic device moves in one direction, the image content of the target framing image (the first framing image or the second framing image) has been captured, and when it moves in another direction, the target framing image may be captured again. Therefore, when acquiring the image between the first target image and the second target image, the image between the first target image and the second target image that are closest in shooting time may be acquired. For example, the image sequence includes: image 1, image 2, image 3..., image 10. Image 2 includes the image content of the first framing image, image 5 includes the image content of the first framing image, and image 10 includes the image content of the second framing image. image content, then the above-mentioned first target image includes image 2 and image 5, and the second target image is image 10. When acquiring the image between the first target image and the second target image, the shooting time difference between image 5 and image 10 is , is less than the shooting time difference between image 2 and image 10, that is, image 5 and image 10 are closer in shooting time, so the image between image 5 and image 10 is obtained as the first target image and the second target images between images.

In some embodiments, if the preset direction in which the camera can move includes multiple directions, the camera can move from the first side to the second side in the horizontal direction, and then move in the vertical direction, and then move in the horizontal direction from the second side to the second side. Move the second side to the first side, and repeat until the camera moves from one side to the other vertically. In this way, the image content of the first viewfinding image and the image content of the second viewfinder image may not be image content that can be collected by moving the camera in the horizontal direction or the vertical direction. The electronic device can control the camera to move in multiple directions. Move so that the electronic device can capture the image content of the first framing image and the image content of the second framing image. Moreover, after collecting the image content of the first view image and the image content of the second view image, when selecting multiple images to be synthesized for generating a panoramic image from the images captured by the electronic device, the moving direction of the camera can be determined. The image to be synthesized. Specifically, the electronic device can arrange all the captured images according to the movement direction, and then determine from the arrangement results the image of the row and column of the third target image as the image of the first row and the image of the first column. image, as well as the image of the row and column where the fourth target image is located, as the image of the second row and the image of the second column; then from the arrangement results, determine the image of the first row and the image of the first column, and the image of the second The image in the row and the image in the second column are included in the range as the image to be synthesized.

Step S350: Perform image reconstruction on each of the multiple images to be synthesized to obtain multiple images to be synthesized after the image reconstruction.

Step S360: Generate a panoramic image based on the multiple images to be synthesized after the image reconstruction.

In the embodiment of the present application, the content of step S350 and step S360 can be referred to the previous embodiment, and will not be described again here.

For example, please refer to Figure 5. While keeping the posture unchanged, the electronic device controls the camera to move in a preset direction, can collect multiple captured images, and can filter out images 1 to 1 from the multiple captured images. Image n, where n is a positive integer, image 1 contains the image content of the first framing image, and image n contains the image content of the second framing image; then image 1 to image n are reconstructed and then synthesized to obtain The final panoramic image A1.

The shooting method provided by the embodiment of the present application enables the user to use an electronic device to shoot a first view image and a second view image, and then control the camera to move in a preset direction according to the first view image and the second view image, and during the movement process When the image captured by the camera includes the first image content and the second image content, the movement of the camera and the image capture are stopped, whereby multiple images used to synthesize the panoramic image can be determined from the captured images. The image is to be synthesized. In this way, a panoramic image with better image quality can be obtained without the user having to hold the electronic device and keep the posture unchanged while moving the electronic device for shooting, which facilitates the user to obtain the desired panoramic image.

Please refer to FIG. 6 , which shows a schematic flowchart of a shooting method provided by yet another embodiment of the present application. This shooting method is applied to the above-mentioned electronic equipment. The process shown in Figure 6 will be described in detail below. The shooting method may specifically include the following steps:

Step S410: Obtain the first viewfinding image and the second viewfinder image collected by the camera. The first viewfinder image corresponds to the current position of the camera. The second viewfinder image and the camera are located along the edge of the current position from the current position. Corresponds to the target position during movement in the preset direction.

In the embodiment of the present application, the first view image may correspond to the current position of the camera, and the second view image may correspond to the target position of the camera when it moves from the current position in the preset direction. That is to say, the first view image is When the camera is at the current position, it can capture the image when the electronic device is at the corresponding position. The second view image is the image that can be captured when the camera moves from the current position in the preset direction while the electronic device maintains its posture. , if the subsequent camera can collect an image that matches the first view image, it controls the camera to move in the preset direction from the current position, so that the image content of the second view image can be collected.

In some embodiments, the electronic device can output second prompt information, which is used to prompt the user to collect the first view image through the electronic device, and to collect the second view image through the electronic device after moving the electronic device in a preset direction. images, so that the user can collect the above-mentioned first view image and second view image through the electronic device according to the prompt information.

Step S420: When the preview image collected by the camera matches the first view image, start image capturing.

Step S430: Control the camera to move from the current position along the preset direction, and capture images during the movement.

In the embodiment of the present application, the electronic device can determine the timing to start collecting images and control the movement of the camera based on the preview image collected by the camera. The electronic device can match the collected preview image with the first viewfinder image. When the electronic device When an image matching the first view image is collected, the camera can be controlled to capture the image, and the camera can be controlled to move from the current position in a preset direction, and capture the image during the movement, so that the captured image is used to generate the image that matches the first view image. A panoramic image that matches the framing range formed by the framing image and the second framing image.

Step S440: When the image captured by the camera matches the second view image, stop the movement of the camera and image capturing.

In the embodiment of the present application, in the process of controlling the movement of the camera, the electronic device can match the captured image with the second framing image. If the image captured by the camera matches the second framing image, the electronic device will stop the camera. Mobile and image capture. Wherein, the photographed image matches the second framing image, which may be that the image content of the photographed image matches the image content of the second framing image.

In some implementations, there may be a situation where the user needs to frame a larger range, and the electronic device keeps its posture unchanged and controls the camera to move in a preset direction, but cannot obtain the image content of the second framing image. Therefore, if the camera does not capture an image that matches the second view image when moving the maximum distance in the preset direction, the first prompt information can be output and the image capture can be paused. The first prompt information is used to prompt the user to move in the preset direction. Mobile electronic device; then, control the camera to move to the initial position; when the preview image collected by the camera matches the image collected when the camera moves to the maximum distance, control the camera to move in the preset direction, and re-capture the image until shooting When the image matches the second view image, stop camera movement and image capture. Therefore, the electronic device can be moved briefly through the electronic device, and then the camera can be controlled to move, so that an image matching the second view image can be captured.

Step S450: Determine multiple images to be synthesized based on the images captured by the camera during movement.

In some embodiments, if the camera moves in the same direction, when the electronic device captures images, the image when it starts collecting matches the first view image, and the image when it stops collecting matches the second view image. Therefore, it can Directly use the images captured by the camera during movement as multiple images to be synthesized.

In some embodiments, if the camera moves in multiple directions, the camera can move from the first side to the second side in the horizontal direction, and then move in the vertical direction, and then move in the horizontal direction from the second side to the second side. First side, repeat until the camera moves from one side to the other vertically. In this method, a variety of images to be synthesized can also be screened out from the images taken during the movement according to the method of screening multiple images to be synthesized in the previous embodiment.

Step S460: Perform image reconstruction on each of the multiple images to be synthesized to obtain multiple images to be synthesized after the image reconstruction.

Step S470: Generate a panoramic image based on the multiple images to be synthesized after the image reconstruction.

In the embodiment of the present application, the content of step S460 and step S470 can be referred to the previous embodiment, and will not be described again here.

The shooting method provided by the embodiment of the present application allows the user to use an electronic device to shoot a first view image and a second view image, and then start image shooting when the preview image collected by the camera matches the first view image, and control the camera to move along the preview image. Set the direction to move, and capture images during the movement. When the image captured by the camera matches the second image, stop the movement of the camera and capture the image, so that the panoramic image used to synthesize the panoramic image can be determined from the captured image. Multiple images to be synthesized. In this way, a panoramic image with better image quality can be obtained without the user having to hold the electronic device and keep the posture unchanged while moving the electronic device for shooting, which facilitates the user to obtain the desired panoramic image.

Please refer to FIG. 7 , which shows a structural block diagram of a photographing device 400 provided by an embodiment of the present application. The shooting device 400 applies the above-mentioned electronic equipment, and the electronic device includes a camera that can move in a preset direction. The shooting device 400 includes: an image acquisition module 410, an image shooting module 420, an image reconstruction module 430, and an image generation module 440. The image acquisition module 410 is used to acquire the first view image and the second view image collected by the camera; the image capture module 420 is used to control the first view image and the second view image based on the first view image and the second view image. The camera moves along the preset direction and captures images during the movement to obtain multiple images to be synthesized; the image reconstruction module 430 is used to perform image reconstruction on each image to be synthesized in the multiple images to be synthesized. Reconstruction is performed to obtain multiple images to be synthesized after the image reconstruction; the image generation module 440 is configured to generate a panoramic image based on the multiple images to be synthesized after the image reconstruction.

In some embodiments, the image reconstruction module 430 may be used to: perform image reconstruction based on stacked imaging for each of the multiple images to be synthesized, to obtain multiple images to be synthesized after the image reconstruction. .

As a possible implementation, the image reconstruction module 430 may be specifically configured to: obtain the spectrum region corresponding to the image to be synthesized based on the aperture function of the camera; perform Fourier transform on each of the images to be synthesized to obtain the The frequency domain information corresponding to each image to be synthesized is based on the frequency domain information corresponding to each image to be synthesized, and the amplitude of the spectrum region corresponding to each image to be synthesized is replaced to obtain all replaced values. Describe the spectrum area corresponding to each image to be synthesized; convert the replaced spectrum area corresponding to each image to be synthesized into the time domain; repeat the above-mentioned method based on the aperture function to obtain the spectrum area corresponding to the image to be synthesized to the step of converting the replaced spectral region corresponding to each of the images to be synthesized into a time domain, until the preset iteration conditions are met, and multiple images to be synthesized after the image reconstruction are obtained.

Optionally, the image reconstruction module 430 may also be used to: obtain a point spread function corresponding to the camera; and determine an aperture function based on the point spread function.

In some embodiments, the image capturing module 420 may be configured to: control the camera to move along the preset direction and capture images during the movement; and include the first image content and the third image content in the image captured by the camera. When there are two image contents, stop the movement of the camera and image shooting, the first image content matches the first view image, and the second image content matches the second view image; based on the first The framing image and the second framing image are to select a plurality of images to be synthesized from the images captured by the camera.

As a possible implementation, the image capture module 420 may be specifically configured to: sort the images captured by the camera according to the order of capture time to obtain an image sequence; and obtain at least one first image from the image sequence. A target image, at least one second target image, and an image between the first target image and the second target image, to obtain multiple images to be synthesized, wherein the first target image includes the first image content , the second target image includes the second image content.

In some embodiments, the first view image corresponds to the current position of the camera, and the second view image corresponds to the target position of the camera when it moves from the current position along the preset direction. The image capturing module 420 may be used to: start image capturing when the preview image collected by the camera matches the first view image; control the camera to move from the current position along the preset direction, and in Image shooting is performed during the movement; when the image captured by the camera matches the second viewfinder image, the movement of the camera and image capture are stopped; and multiple images are determined based on the images captured by the camera during the movement. The image to be synthesized.

As a possible implementation, the image capturing module 420 may also be configured to: when the image captured by the camera matches the second view image, stop the movement of the camera and before capturing the image, if the camera When moving the maximum distance along the preset direction, if no image matching the second viewfinder image is captured, the first prompt information is output and the image capture is paused. The first prompt information is used to prompt the user to move along the preset direction. Move the electronic device in a preset direction; control the camera to move to an initial position; when the preview image collected by the camera matches the image collected when the camera moves to the maximum distance, control the camera to move along the preset direction, and capture the image again.

In some implementations, the shooting device 400 may further include: a prompt module. The prompt module is configured to output second prompt information before obtaining the first view image and the second view image collected by the camera. The second prompt information is used to prompt the user to collect the first view image through the electronic device. , and collecting the second view image through the electronic device after moving the electronic device along the preset direction.

Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the above-described devices and modules can be referred to the corresponding processes in the foregoing method embodiments, and will not be described again here.

In several embodiments provided in this application, the coupling between modules may be electrical, mechanical or other forms of coupling.

In addition, each functional module in each embodiment of the present application can be integrated into one processing module, or each module can exist physically alone, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software function modules.

To sum up, in the solution provided by this application, the electronic device is equipped with a camera that can move in a preset direction. When the electronic device takes pictures, it obtains the first view image and the second view image collected by the camera, based on the first view image. image and the second view image, control the camera to move along the preset direction, and capture images during the movement to obtain multiple images to be synthesized, and then perform image reconstruction on each of the multiple images to be synthesized to obtain The multiple images to be synthesized after image reconstruction are used to generate a panoramic image based on the multiple images to be synthesized after image reconstruction. As a result, the user can use the electronic device to capture the first view image and the second view image, control the camera to move and capture panoramic images, and perform image reconstruction on the multiple captured images to generate a panoramic image, which can improve Image quality of the obtained panoramic image.

Please refer to FIG. 8 , which shows a structural block diagram of an electronic device provided by an embodiment of the present application. The electronic device 100 may be a smart phone, a tablet computer, an e-book, or other electronic device capable of running applications. The electronic device 100 in the present application may include one or more of the following components: a processor 110, a memory 120, and one or more application programs, wherein one or more application programs may be stored in the memory 120 and configured by One or more processors 110 execute, and one or more application programs are configured to execute the method as described in the foregoing method embodiments.

Processor 110 may include one or more processing cores. The processor 110 uses various interfaces and lines to connect various parts of the entire electronic device 100, and executes by running or executing instructions, programs, code sets or instruction sets stored in the memory 120, and calling data stored in the memory 120. Various functions and processing data of the electronic device 100 . Optionally, the processor 110 may adopt at least one of digital signal processing (Digital Signal Processing, DSP), field-programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). implemented in hardware form. The processor 110 may integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processor (Graphics Processing Unit, GPU), a modem, and the like. Among them, the CPU mainly handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the display content; and the modem is used to handle wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 110 and may be implemented solely through a communication chip.

The memory 120 may include random access memory (RAM) or read-only memory (Read-Only Memory). Memory 120 may be used to store instructions, programs, codes, sets of codes, or sets of instructions. The memory 120 may include a program storage area and a data storage area, where the program storage area may store instructions for implementing an operating system and instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.) , instructions for implementing each of the following method embodiments, etc. The storage data area can also store data created during use of the electronic device 100 (such as phone book, audio and video data, chat record data), etc.

Please refer to FIG. 9 , which shows a structural block diagram of a computer-readable storage medium provided by an embodiment of the present application. Program code is stored in the computer-readable medium 800, and the program code can be called by the processor to execute the method described in the above method embodiment.

Computer-readable storage medium 800 may be electronic memory such as flash memory, EEPROM (electrically erasable programmable read-only memory), EPROM, hard disk, or ROM. Optionally, the computer-readable storage medium 800 includes non-transitory computer-readable storage medium. The computer-readable storage medium 800 has storage space for program code 810 that performs any method steps in the above-described methods. These program codes can be read from or written into one or more computer program products. Program code 810 may, for example, be compressed in a suitable form.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application, but not to limit it; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art will understand that: it can still Modifications are made to the technical solutions described in the foregoing embodiments, or equivalent substitutions are made to some of the technical features; however, these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions in the embodiments of the present application.

Claims (10)

1. A photographing method, characterized in that it is applied to electronic equipment, and the electronic equipment includes a camera that can move in a preset direction, and the method includes: Obtain the first view image and the second view image collected by the camera; Based on the first framing image and the second framing image, control the camera to move along the preset direction, and capture images during the movement to obtain multiple images to be synthesized; Based on the aperture function of the camera, obtain the correspondence between each image to be synthesized and the reconstructed spectrum; Perform Fourier transform on each of the images to be synthesized to obtain the frequency domain information corresponding to each of the images to be synthesized; Based on the corresponding relationship and the frequency domain information corresponding to each image to be synthesized, replace the amplitude at the position corresponding to each image to be synthesized in the reconstructed spectrum; Repeat the step of obtaining the corresponding relationship between each image to be synthesized and the reconstructed spectrum based on the aperture function of the camera, until the frequency domain corresponding to the corresponding relationship and each image to be synthesized is obtained. Information, the step of replacing the amplitude value at the position corresponding to each of the images to be synthesized in the reconstructed spectrum until the preset iteration conditions are met; The reconstructed spectrum is transformed into the time domain to obtain a panoramic image. 2. The method according to claim 1, characterized in that, before obtaining the correspondence between each image to be synthesized and the reconstructed spectrum based on the aperture function of the camera, the method further includes: Obtain the point spread function corresponding to the camera; Based on the point spread function, an aperture function is determined. 3. The method according to claim 1, characterized in that, based on the first view image and the second view image, the camera is controlled to move along the preset direction, and image capture is performed during the movement. , and obtain multiple images to be synthesized, including: Control the camera to move along the preset direction and capture images during the movement; When the image captured by the camera includes first image content and second image content, stop the movement of the camera and image capture, the first image content matches the first view image, and the second image The content matches the second framing image; Based on the first view image and the second view image, a plurality of images to be synthesized are selected from the images captured by the camera. 4. The method according to claim 3, characterized in that, based on the first framing image and the second framing image, filtering out a plurality of images to be synthesized from the images captured by the camera, including: Sort the images captured by the camera according to the order of shooting time to obtain an image sequence; From the image sequence, at least one first target image, at least one second target image, and images between the first target image and the second target image are obtained, and a plurality of images to be synthesized are obtained, wherein The first target image includes the first image content, and the second target image includes the second image content. 5. The method according to claim 1, characterized in that the first viewfinding image corresponds to the current position of the camera, and the second viewfinder image and the camera move from the current position along the preset Corresponding to the target position during the direction movement, based on the first view image and the second view image, the camera is controlled to move along the preset direction, and images are captured during the movement to obtain multiple images to be synthesized Images, including: When the preview image collected by the camera matches the first view image, start image capturing; Control the camera to move from the current position along the preset direction, and capture images during the movement; When the image captured by the camera matches the second view image, stop the movement of the camera and image capture; Based on the images captured by the camera during movement, multiple images to be synthesized are determined. 6. The method according to claim 5, wherein when the image captured by the camera matches the second view image and before stopping the movement of the camera and image capture, the method further includes: If the camera moves the maximum distance along the preset direction and does not capture an image that matches the second view image, the first prompt information is output and the image capture is suspended. The first prompt information is used to prompt The user moves the electronic device along the preset direction; Control the camera to move to the initial position; When the preview image collected by the camera matches the image collected when the camera moves to the maximum distance, the camera is controlled to move along the preset direction and the image is captured again. 7. The method according to any one of claims 1 to 6, characterized in that, before obtaining the first view image and the second view image collected by the camera, the method further includes: Output second prompt information, the second prompt information is used to prompt the user to collect the first view image through the electronic device, and to collect the second view image through the electronic device after moving the electronic device along the preset direction. image. 8. A photographing device, characterized in that it is applied to electronic equipment. The electronic equipment includes a camera that can move in a preset direction. The device includes: an image acquisition module, an image shooting module, an image reconstruction module and an image generation module. ,in, The image acquisition module is used to acquire the first view image and the second view image collected by the camera; The image capture module is configured to control the camera to move along the preset direction based on the first view image and the second view image, and capture images during the movement to obtain multiple images to be synthesized; The image reconstruction module is used to obtain the correspondence between each image to be synthesized and the reconstructed spectrum based on the aperture function of the camera; perform Fourier transform on each of the images to be synthesized to obtain each image to be synthesized. Corresponding frequency domain information; based on the corresponding relationship and the frequency domain information corresponding to each of the images to be synthesized, replace the amplitude at the position corresponding to each of the images to be synthesized in the reconstructed spectrum; repeat The step of obtaining the corresponding relationship between each image to be synthesized and the reconstructed spectrum based on the aperture function of the camera, to the frequency domain information corresponding to the corresponding relationship and each image to be synthesized. , the step of replacing the amplitude at the position corresponding to each of the images to be synthesized in the reconstructed spectrum until the preset iteration conditions are met; transforming the reconstructed spectrum into the time domain to obtain a panoramic image. 9. An electronic device, characterized in that it includes: one or more processors; memory; one or more programs, wherein said one or more programs are stored in said memory and configured to be executed by said one or more processors, said one or more programs are configured to perform as claimed The method described in any one of 1-7. 10. A computer-readable storage medium, characterized in that program code is stored in the computer-readable storage medium, and the program code can be called and executed by a processor as described in any one of claims 1-7 Methods.