JP2017098753A - Encoding apparatus, imaging apparatus, encoding apparatus control method, program, and storage medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To divide an image, and reduce a noise on a division boundary generated when performing a compression encoding independently on each division image.SOLUTION: An encoding device comprises: division means of dividing a plurality of frames which is which temporally continued into a plurality of regions including a first region and a second region spatially adjacent each other, and dividing each frame into a plurality of division images so that a final line of a plurality of lines of the division images constructing the first region and a head line of the plurality of lines of the division images constructing the second region construct a division boundary; parallel encoding means of including a plurality of encoding means for independently encoding the plurality of division images; and switching means of controlling an input to the plurality of encoding means of the plurality of division images, and controlling so that the division images constructing the second region of a sequential frame of a frame corresponding to the division image after the input of the division images constructing the first region.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an encoding apparatus, an imaging apparatus, an encoding apparatus control method, a program, and a storage medium.

  A moving image is composed of a plurality of continuous frames. H.264, JPEG2000, and other techniques are used for compression encoding and recording. Compression encoding and decoding techniques are extremely important for the multimedia industry because they can reduce the storage capacity required for storing video signals and the bandwidth required for transmission. In recent years, higher resolution and higher frame rate of moving images to be encoded such as 3840 pixels or more and 60 frames or more per second are required, and very high calculation performance is required for compression encoding. Yes.

  In order to perform compression encoding of such a moving image in real time, a method of spatially dividing one frame and parallelizing compression encoding processing is known. However, when an image is divided and each of them is compression-encoded, the quantization step size differs between the divided images, so that distortion occurs on the division boundary and the compression efficiency is lowered.

  For example, a case where a quantization step size is set for each line is given. If the image is not divided, the quantization step size can be set by referring to the quantization step size of the upper line so as not to cause a large difference in the quantization step size between the lines. However, if the screen is divided and each divided image is encoded in parallel, the quantization step size cannot be referred across the division boundary, so there is a difference in the quantization step size between the lines across the division boundary. Will occur. As a result, image quality deteriorates due to distortion occurring on the dividing boundary line.

  On the other hand, a parallelization method for suppressing a decrease in compression efficiency due to division processing has been proposed (see Patent Document 1).

  Patent Document 1 discloses a method of dividing an image and compressing and coding with an encoder for each divided image. In this method, each encoder acquires encoding information such as a quantization step size near the division boundary from the encoder that processes the divided image adjacent to the divided image to be processed, and the vicinity of the division boundary. The encoding condition is made closer. This is intended to reduce distortion on the division boundary.

JP 9-294262 A

  However, in the method of Patent Document 1, in order to acquire coding information in the vicinity of the division boundary, it is necessary to wait until the encoder that processes the adjacent divided images finishes the processing in the region near the division boundary. For example, when information near the division boundary is exchanged between the encoders for the vertically divided image, in order to start the processing of the divided image positioned below, the compression encoding processing of the divided image positioned above is performed. Must wait for termination. During the standby period, the compression encoding process cannot be executed. Therefore, in the method of Patent Literature 1, a waiting time until information near the division boundary is acquired is generated, and processing performance per real time is deteriorated.

  Therefore, in the present invention, when each of the plurality of encoders independently compresses and encodes the corresponding divided image, a new method for bringing the encoding condition near the division boundary closer is proposed. It is possible to reduce distortion between divided images without degrading the processing performance.

The present invention for solving the above problems is an encoding device,
Dividing means for dividing a plurality of temporally continuous frames into a plurality of regions including a first region and a second region that are spatially adjacent to each other, and a plurality of divided images constituting the first region A dividing unit that divides each frame into a plurality of divided images so that a final line of the lines and a leading line of a plurality of lines of the divided images that form the second region form a dividing boundary;
Parallel encoding means including a plurality of encoding means for independently encoding the plurality of divided images;
A switching unit that controls input of the plurality of divided images to the plurality of encoding units, and after the input of the divided images constituting the first region, Switching means for controlling so that the divided images constituting the second region are input,
Each of the plurality of encoding means is
Quantization means for setting the quantization step size according to the encoding result of the previous line for the data corresponding to the input divided image, and performing quantization for each line using the set quantization step size; ,
Encoding means for encoding the quantized data,
When the quantization means quantizes the data of the first line of the divided image constituting the second area, the quantization means includes the divided image of the first area of the frame preceding the frame corresponding to the divided image. The quantization step size is set according to the encoding result of the final line.

  According to the present invention, in an encoding method in which each of a plurality of encoders independently compress-encodes a corresponding divided image, a new method for bringing the encoding condition near the division boundary closer to the real time. It becomes possible to reduce the distortion between the divided images without degrading the processing performance.

The block diagram which shows an example of a structure of the encoding apparatus 100 which concerns on Embodiment 1 of invention. (A) A diagram illustrating an example of a Bayer array, (b) a color separation of RAW data in the Bayer array, an image forming method for each color component, and (c) an image dividing method according to the first embodiment of the invention. Illustration to explain. The figure for demonstrating an example of the switching method of the transmission destination of the divided image in the encoding process of the encoding apparatus 100 which concerns on Embodiment 1 of invention. (A) The block diagram which shows the structural example of the encoding part of the encoding apparatus which concerns on embodiment of an invention, (b) According to embodiment of invention, the quantization parameter of the division | segmentation image of the last frame based on embodiment of an invention WHEREIN: The figure for demonstrating the method handed over to a quantization parameter. The flowchart which shows an example of the encoding process in the encoding apparatus which concerns on embodiment of invention. The flowchart which shows an example of the setting process of the quantization step size which concerns on embodiment of invention. The block diagram which shows an example of a structure of the encoding apparatus 700 which concerns on Embodiment 2 of invention. (A) The figure explaining the image division | segmentation method which concerns on Embodiment 2 of invention, (b) An example of the switching method of the transmission destination of the division | segmentation image in the encoding process of the encoding apparatus 700 which concerns on Embodiment 2 of invention is demonstrated. Figure to do.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

[Embodiment 1]
The configuration and operation of the encoding apparatus corresponding to the first embodiment of the invention will be described below. FIG. 1 is a block diagram showing an example of a configuration of an encoding apparatus 100 according to Embodiment 1 of the invention. In the present embodiment, a case where a digital camera is assumed as the encoding apparatus 100 will be described. The digital camera compresses the R, G1, G2, and B Bayer array image data shown in FIG. However, the embodiment of the invention is not limited to this, and for example, an RGB image, a YUV image, or a grayscale image may be a target for compression encoding.

  In addition to the digital video camera, the encoding apparatus 100 is an arbitrary information processing apparatus or imaging apparatus having a moving image shooting function, such as a personal computer, a mobile phone, a smartphone, a PDA, a tablet terminal, or a digital still camera. Can do. Note that FIG. 1 shows a configuration including the imaging unit 103 in consideration of the case of functioning as a digital video camera or the like. However, as an embodiment of the invention, an image processing apparatus for processing a captured moving image may be realized with a configuration that does not include the imaging unit 103. In the encoding apparatus 100 of FIG. 1, each block may be configured by hardware using a dedicated logic circuit or a memory, except for a physical device such as an image sensor. Alternatively, the processing program stored in the memory may be configured by software by a computer such as a CPU executing the processing program.

  The encoding apparatus 100 according to the present embodiment includes an operation unit 101 for receiving an instruction to shoot a subject from a user. The operation unit 101 may be configured as a push button such as a shutter switch, or may be configured as a touch panel that displays a predetermined operation interface. The control unit 102 is connected to each component via the internal bus 114, and controls the overall operation of the encoding apparatus 100 according to a signal sent from the operation unit 101 to instruct the start of shooting. For example, the control unit 102 may be configured as a memory that stores a CPU and a control program executed by the CPU. In that case, the CPU reads the control program from the memory and controls the encoding device 100 in accordance with a signal from the operation unit 101. Further, the control unit 102 may be configured as a dedicated logic circuit programmed in accordance with processing corresponding to the present embodiment.

  After receiving the imaging start instruction, the control unit 102 controls the operation of the imaging unit 103 to acquire a captured image of the subject. The imaging unit 103 is configured to include, for example, an imaging lens and an image sensor having a Bayer array color filter illustrated in FIG. 2A, and an optical image of a subject is formed on the image sensor by the imaging lens. The formed optical image is converted into RAW image data. The image sensor is composed of CMOS or CCD.

  The RAW image data generated by the imaging unit 103 is sent to the signal processing unit 104, and a restoration process is performed on the RAW image data. The restoration process includes a process of interpolating a pixel to be restored using a peripheral pixel value with respect to a missing pixel value or a low-reliability pixel value in the image sensor of the imaging unit 103, or subtracting a predetermined offset value. included. The signal processing unit 104 outputs the repaired RAW image data to the scene change detection unit 105 and the color separation unit 106.

  The scene change detection unit 105 detects a scene change based on the RAW image data received from the signal processing unit 104 and supplies scene change information to a parallel encoding unit 109 described later. The scene change detection method in the present embodiment is not particularly limited, and any known method can be applied. For example, a method of obtaining an integrated value or a variance value for each frame and using a case where a change or difference in the integrated value or variance value between the immediately preceding frame and the current frame is larger than a predetermined value as a scene change is used as an example. be able to.

  The color separation unit 106 converts the RAW image data supplied from the signal processing unit 104 into independent plane image data (hereinafter simply referred to as “plane image” for each of R, G1, G2, and B pixels as shown in FIG. 2B. ”). In the following description, only one of the plurality of plane images is taken up and described. However, the encoding apparatus 100 corresponding to the embodiment of the invention is configured to perform the same processing on each plane image. It is configured. A plane image of each color separated and generated by the color separation unit 106 is divided into two upper and lower regions by an image division unit 107 as shown in FIG. In the present embodiment, among the divided images divided in the upper and lower directions, the divided image in the upper region is referred to as a divided image 0, and the divided image in the lower region is referred to as a divided image 1.

  After the image dividing unit 107 divides the image, the divided image is supplied to the parallel encoding unit 109 via the input switching unit 108. The input switching unit 108 switches the supply destination of the divided image 0 and the divided image 1 alternately for each frame between the encoding unit 0 and the encoding unit 1 constituting the parallel encoding unit 109. Here, with reference to FIG. 3, switching of the transmission destination of the divided image by the input switching unit 108 will be described. When the frames 0, 1 and 2 are indexed on the shooting frame, the input switching unit 108 encodes the divided images 0 and 1 depending on whether the index is an even number or an odd number, respectively. Switching between sending to the unit 0 or the coding unit 1 is performed. For example, when encoding an even index frame, the divided image 0 is sent to the encoding unit 0, and the divided image 1 is sent to the encoding unit 1. When an odd index frame is encoded, the divided image 0 is encoded. The input switching unit 108 can operate to send the divided image 1 to the encoding unit 0 to the encoding unit 1. Focusing on one encoding unit, the spatial position in each frame of the input divided image is switched cyclically, such as divided image 0, divided image 1, divided image 0, and divided image 1.

  FIG. 4A is a diagram illustrating an example of the configuration of each encoding unit included in the parallel encoding unit 109. The encoding unit 0 and the encoding unit 1 have the configuration shown in FIG. 4A, and compress and encode the divided images sent to each. Here, the encoding unit 0 and the encoding unit 1 execute compression encoding independently of each other. The encoding unit 0 and the encoding unit 1 perform frequency separation by performing discrete wavelet transform on the divided image supplied from the input switching unit 108 in the discrete wavelet transform unit 401, and each of the divided images includes a plurality of wavelet coefficients. A subband image is generated.

  FIG. 4B shows an example of an array of wavelet coefficients T1HH to T3LL generated by the discrete wavelet transform unit 401. Here, a case is shown in which subband images from decomposition level 1 to decomposition level 3 are generated by performing band division three times. The decomposition level 1 to 3 wavelet coefficients T1HH to T3LL converted by the discrete wavelet transform unit 401 are input to the quantization unit 402 as subband images. The following description of each block constituting the encoding unit 0 and the encoding unit 1 in the parallel encoding unit 109 will be described by taking one subband image out of a plurality of subband images. The parallel encoding unit 109 corresponding to 1 performs the same processing on each subband image. In the present embodiment, a case is described in which discrete wavelet transformation is performed as transformation processing for generating a coefficient to be quantized, but other predetermined transformation processing may be executed instead of the transformation processing. Also in this case, the transform coefficient obtained as a processing result is supplied to the quantization unit 402 and the quantization process is executed.

  The quantization unit 402 performs a quantization process on the subband image supplied from the discrete wavelet transform unit 401 using the quantization parameter set by the quantization control unit 403, that is, the quantization step size. In the present embodiment, the quantization step size is set for each coefficient line constituting the subband image. The quantization control unit 403 sets the quantization step size for each coefficient line. Further, the quantization control unit 403 stores the quantization step size used in the last line of the divided image 0 of the frame to be processed and uses it as the quantization step size of the first line of the divided image 1 of the next frame. Also, the quantization control unit 403 stores the quantization step size used in the previous line of the same divided image of the processing target frame and the generated code amount of the previous line, and the quantization step size of the processing target line Is determined based on the generated code amount of the previous line and the target code amount. This process will be described later with reference to FIG.

  Furthermore, scene change information indicating a scene change determination result in the scene change detection unit 105 is input to the quantization control unit 403. When the scene change information indicates scene change detection, the quantization step size of the first line of the divided image 1 is determined in advance by a statistical value or the like instead of the quantization step size used in the last line of the divided image 0 of the immediately preceding frame. Use the quantization step size. Similarly, when the divided image 1 of the first frame among a plurality of temporally continuous frames to be encoded is input, the quantization control unit 403 uses a predetermined quantization step size for the first line. Use. A predetermined quantization step size is set for all head lines of the divided images included in the head frame. The coefficient lines quantized by the quantizing unit 402 are sent to the entropy coding unit 404 for compression coding. The compression encoding method used by the entropy encoding unit 404 includes, for example, DPCM and Huffman code. The generated code amount of the encoded data generated by the entropy encoding unit 404 is notified to the quantization control unit 403.

  Next, the encoded stream generation unit 110 combines the codes of all the plane images and all the subband images generated by the parallel encoding unit 109, adds a header indicating encoding conditions, and generates a decodable stream. . Also, the encoded stream generation unit 110 saves the code transmitted from the parallel encoding unit 109 in the temporary storage unit 111 until the codes for all subbands and all planes are prepared. The temporary storage unit 111 can be composed of, for example, a DRAM or an SRAM. The encoded stream generation unit 110 reads the code saved in the temporary storage unit 111 when the codes for all subbands and all planes are prepared, and generates a stream. The stream generated by the encoded stream generation unit 110 is written to the storage medium 113 via the memory I / O unit 112. The storage medium 113 can be composed of a magnetic disk, a semiconductor memory, or the like.

  Next, details of compression encoding processing in the encoding unit 0 and the encoding unit 1 constituting the parallel encoding unit 109 will be described based on a flowchart shown in FIG. The processing corresponding to the flowchart can be realized, for example, by executing a program corresponding to one or more processors functioning as the parallel encoding unit 109.

    In step S501, the discrete wavelet transform unit 401 performs discrete wavelet transform on the divided image to perform frequency separation to generate a plurality of subband images. In S502, according to the content of the scene change information from the scene change detection unit 105, the quantization control unit 403 determines whether or not a scene change has been detected. When a scene change is detected (“YES” in S502), the process proceeds to S503, and the predetermined quantization step size is set to the quantization step size of the first line of the divided image to be processed. On the other hand, when a scene change is not detected (“NO” in S502), the process proceeds to S504, and the quantization control unit 403 determines whether the processing target image is the divided image 0 or not. When the processing target image is the divided image 0 (“YES” in S504), the process proceeds to S503, and the quantization control unit 403 sets a predetermined quantization step size to the quantization step size of the first line of the divided image 0. To do. In the divided image 0, since the leading line does not form a dividing boundary, a predetermined quantization step size is set for the leading line.

  On the other hand, when the processing target image is the divided image 1 (“NO” in S504), the process proceeds to S505, and the quantization control unit 403 determines whether the processing target divided image belongs to the first frame. If it is the divided image 1 of the first frame (“YES” in S505), the process proceeds to S503, and the quantization control unit 403 sets the predetermined quantization step size to the quantization step size of the first line of the divided image 1. Set to. On the other hand, if it is the divided image 1 that does not belong to the first frame (“NO” in S505), the process proceeds to S506. In S506, the quantization control unit 403 sets the quantization step size of the last line of the divided image 0 of the frame processed immediately before in the same encoding unit to the quantization step size of the first line.

  This is shown in FIG. The divided image 0 belonging to the immediately preceding frame N-1 is processed in the same encoding unit as the encoding unit that processes the divided image 1 of the current frame N. The quantization control unit 403 of the encoding unit holds the quantization step size of the final line of the divided image 0. When setting the quantization step size of the first line of the divided image 1 belonging to the current frame N, the quantization control unit 403 sets the quantization step size of the final line of the divided image 0 processed immediately before being held. To do.

  When the quantization step size of the first line is set in S503 or S506, the process proceeds to S507, and the quantization unit 402 quantizes one line of the divided image to be processed using the set quantization step size. To do. In subsequent S508, the entropy encoding unit 404 compresses and encodes the quantized line. In subsequent S509, the entropy encoding unit 404 outputs the encoding result to the encoded stream generation unit 110. Also, the generated code amount information is notified to the quantization control unit 403.

  After that, the quantization control unit 403 determines whether or not the last line of the divided image to be processed has been processed in S510. If the processing has not been completed up to the final line (“NO” in S510), the process proceeds to S511. To do. In S511, the quantization control unit 403 sets the quantization step size of the next line, and returns to S507 to repeat the processing. An example of the processing in S511 will be described later with reference to FIG. When the last line has been processed (“YES” in S510), the quantization control unit 403 determines whether or not the processing target image is the divided image 1. If it is the divided image 1 (“YES” in S512), the encoding process is terminated as it is. On the other hand, when the processing target image is not the divided image 1 but the divided image 0 (“NO” in S512), the process proceeds to S513, and the quantization control unit 403 stores the quantization step size of the final line and performs encoding. The process ends.

  FIG. 6 is a flowchart illustrating an example of the quantization step size setting process of the quantization control unit 403 in S511 of FIG. In step S601, the quantization control unit 403 adds the code amount generated by the encoding for one line in step S508 to the accumulated generated code amount (S). The accumulated generated code amount (S) is initialized in units of divided images. In step S602, the quantization control unit 403 adds the target code amount for each line to the integrated target code amount (T). The integration target code amount (T) is also initialized in units of divided images. Here, the target code amount for each line can be calculated as, for example, a value obtained by dividing the target code amount of the divided image by the number of lines of the divided image. The target code amount of the divided image can be calculated as, for example, the product of the resolution of the divided image, the accuracy of the coefficient bits of the divided image, and the compression ratio.

  In subsequent S603, the accumulated generated code amount (S) calculated in the above step is compared with the accumulated target code amount (T). If S = T (“YES” in S603), the process proceeds to S604. At this time, the quantization control unit 403 sets the same value as the previous quantization step size set for the immediately preceding line. On the other hand, if S ≠ T (“NO” in S603), the process proceeds to S605. If S> T is determined in S605 ("YES" in S605), a larger quantization step size than the previous time is set in S606 in order to reduce the generated code amount. On the other hand, if it is determined that S <T in S605 (“NO” in S605), a smaller quantization step size than the previous time is set in S607 in order to increase the generated code amount. Thereafter, the process returns to S507 in FIG. When the quantization step size is changed, for example, the quantization step size can be changed by increasing or decreasing it by one step in a plurality of predetermined quantization step sizes.

  The quantization step size setting method described above is described as an example, and the quantization step size of the encoding target line may be set based on the quantization step size used in the previous line. For example, it is not limited to this. For example, not only the above method based on the magnitude relationship between S and T, but also the magnitude of the difference between S and T for each line, the degree of change of the difference between adjacent lines, the direction of change of the difference (increase Quantization step size may be set on the basis of (decrease).

  As described above, the encoding apparatus 100 according to the present embodiment cyclically switches the encoding unit serving as the transmission destination of each of the divided images divided into the upper and lower parts for each frame, and the previous frame in the same encoding unit. Based on the divided image 0, the quantization step size of the first line of the divided image 1 of the current frame to be processed is set. In general, since temporally adjacent frames have high similarity to each other, the quantization step size of the immediately preceding divided image 0 is substantially the same as the quantization step size of the divided image 0 of the current frame existing at the same position. Can think. Therefore, using the quantization setting of the last line of the divided image 0 of the immediately preceding frame for the quantization setting of the first line of the divided image 1 of the current frame means that the quantization setting of the last line of the divided image 0 of the current frame is set to the current frame It can be considered that it is substantially equivalent to using it for the quantization setting of the head line of the divided image 1.

  In this way, in this embodiment, the fact that the similarity between temporally adjacent frames is very high is used, and the quantization information of the divided image 0 of the immediately preceding frame is used for the quantization control of the divided image 1 of the current frame. Can do. Accordingly, it is possible to reduce the distortion between the divided images by giving the continuity to the quantization control between the divided images, and to reduce the noise on the divided boundary.

[Embodiment 2]
Next, a second embodiment of the invention will be described. A configuration example of an encoding apparatus 700 corresponding to this embodiment is shown in FIG. 7 is different from the encoding apparatus 100 of the first embodiment in the configuration of an image dividing unit 701, an input switching unit 702, and a parallel encoding unit 703. Hereinafter, the configuration will be mainly described, and the description of the configuration common to the encoding device 100 will be omitted.

  First, the image dividing unit 701 divides the plane image of each color separated and generated by the color separation unit 106 into three regions, upper, lower, middle, and lower, as shown in FIG. 8A. In the present embodiment, among the divided images obtained by dividing the upper and lower parts into three, the divided images in the upper region are referred to as divided image 0, divided image 1, and divided image 2. In the following description, only one of the plurality of plane images is taken up and described. However, the encoding apparatus 700 corresponding to the embodiment of the invention performs the same processing on each plane image. It is configured.

  After the image is divided into three by the image dividing unit 701, the divided image is supplied to the parallel encoding unit 703 via the input switching unit 702. The input switching unit 702 supplies the divided image 0, the divided image 1 and the divided image 2 to each frame between the encoding unit 0, the encoding unit 1 and the encoding unit 2 constituting the parallel encoding unit 703. Switch cyclically. Here, switching of the transmission destination of the divided image for each frame by the input switching unit 702 will be described with reference to FIG. When the frames 0, 1 and 2 are indexed with respect to the shooting frame, the input switching unit 702 assigns the divided image 0, the divided image 1 and the divided image 2 according to the remainder when the index is divided by 3, respectively. The transmission is switched to one of the encoding unit 0, the encoding unit 1, and the encoding unit 2.

  For example, when a frame with a remainder of 0 is encoded, the divided image 0 is sent to the encoding unit 0, the divided image 1 is sent to the encoding unit 1, and the divided image 2 is sent to the encoding unit 2. When encoding a frame with a remainder of 1, the divided image 0 is sent to the encoding unit 1, the divided image 1 is sent to the encoding unit 2, and the divided image 2 is sent to the encoding unit 0. When a frame with a remainder of 2 is encoded, the divided image 0 is sent to the encoding unit 2, the divided image 1 is sent to the encoding unit 0, and the divided image 2 is sent to the encoding unit 1. Focusing on one encoding unit, the spatial position in each frame of the input divided image is cyclic, such as divided image 0, divided image 1, divided image 2, divided image 0, divided image 1, and divided image 2. Switch.

  Each configuration of the encoding unit 0, the encoding unit 1, and the encoding unit 2 included in the parallel encoding unit 703 is the same as that shown in FIG. 4A, and the division sent to each encoding unit The image is compression encoded. Also in the present embodiment, the encoding unit 0, the encoding unit 1, and the encoding unit 2 execute compression encoding independently of each other.

  Next, details of compression encoding processing in the encoding unit 0, the encoding unit 1, and the encoding unit 2 constituting the parallel encoding unit 703 will be described. The compression encoding process can be executed by correcting the process in the step of the flowchart shown in FIG. Specifically, the process in S512 executed between S510 and S513 is replaced with a process for determining whether or not the divided image to be processed corresponds to the divided image 2. As a result of this determination, if the divided image to be processed is the divided image 2 (“YES” in S512), the encoding process is terminated as it is. This is because the divided image 2 does not need to store the quantization step size set for the final line because the final line does not form a division boundary. On the other hand, when the processing target image is not the divided image 2 but the divided image 0 or the divided image 1 (“NO” in S512), the process proceeds to S513, and the quantization control unit 403 stores the quantization step size of the final line. Then, the encoding process ends.

  As described above, the encoding device corresponding to the present embodiment switches the encoding unit serving as the transmission destination of each of the divided images divided into the upper and lower parts for each frame, and the divided image of the immediately preceding frame in the same encoding unit and Quantization control is performed by regarding the divided images of the current frame as spatially continuous images. Generally, temporally adjacent frames have high similarity to each other, so the quantization step size of the previous divided image and the quantization step size of the divided image of the current frame existing at the same position should be almost the same. Can do. Therefore, using the quantization setting of the last line of the divided image 0 or 1 of the previous frame as the quantization setting of the first line of the divided image 1 or 2 of the current frame means that the final line of the divided image 0 or 1 of the current frame is used. Can be regarded as being substantially equivalent to using the quantization setting for the first line of the divided image 1 or 2 in the current frame. In this way, this embodiment also uses the fact that the temporally adjacent frames are very similar, and uses the quantization information of the divided image of the immediately preceding frame for the quantization control of the divided image of the current frame. Can do. As a result, the quantization control can be made continuous between the divided images, distortion between the divided images can be reduced, and noise on the division boundary can be reduced.

  In the above-described second embodiment, the example of the upper and lower three divisions has been described. In that case, the image dividing unit 701 is configured to divide the image into upper and lower N divisions, and the encoding unit 703 is configured with N encoding units. Also, S512 in FIG. 5 replaces the process of determining whether or not the last line of the divided image to be processed constitutes a division boundary. At this time, the division number N may be maximized, and the division number may be varied according to the decomposition level of the wavelet coefficient.

(Other examples)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  401: discrete wavelet unit, 402: quantization unit, 403: quantization control unit, 404: entropy coding unit

Claims (14)

Dividing means for dividing a plurality of temporally continuous frames into a plurality of regions including a first region and a second region that are spatially adjacent to each other, and a plurality of divided images constituting the first region A dividing unit that divides each frame into a plurality of divided images so that a final line of the lines and a leading line of a plurality of lines of the divided images that form the second region form a dividing boundary;
Parallel encoding means including a plurality of encoding means for independently encoding the plurality of divided images;
A switching unit that controls input of the plurality of divided images to the plurality of encoding units, and after the input of the divided images constituting the first region, Switching means for controlling the divided images constituting the second area to be input;
With
Each of the plurality of encoding means is
Quantization means for setting the quantization step size according to the encoding result of the previous line for the data corresponding to the input divided image, and performing quantization for each line using the set quantization step size; ,
Encoding means for encoding the quantized data,
When the quantization means quantizes the data of the first line of the divided image constituting the second area, the quantization means includes the divided image of the first area of the frame preceding the frame corresponding to the divided image. An encoding apparatus, wherein a quantization step size is set according to an encoding result of a final line. The quantization means stores the quantization step size when the quantization step size is set for the final line of the divided image of the first region,
The said setting of the said quantization step size at the time of quantizing the data of the head line of the division | segmentation image of a said 2nd area | region is performed based on this memorize | stored quantization step size. Encoding device. When the quantization means sets the quantization step size for the final line of the divided image of the second region,
Storing the set quantization step size when the final line of the divided image of the second region constitutes a dividing boundary;
The encoding apparatus according to claim 2, wherein the set quantization step size is not stored when the final line of the divided image of the second region does not form a division boundary.   The switching unit controls input to the plurality of encoding units so that a spatial position in each frame of a divided image supplied to each of the plurality of encoding units is cyclically switched. The encoding device according to any one of claims 1 to 3.   2. The quantization means sets a predetermined quantization step size for the first line of a divided image in which the first line does not form a division boundary among the plurality of divided images. 5. The encoding device according to any one of items 1 to 4.   The quantization means sets a predetermined quantization step size for all head lines of a plurality of divided images included in a head frame among the plurality of temporally continuous frames. The encoding device according to any one of claims 1 to 5. Detection means for detecting a change between a first frame and a second frame that are temporally adjacent;
When the detection unit detects a predetermined change, the quantization unit sets a predetermined quantization step size for all head lines of the plurality of divided images of the second frame. The encoding device according to any one of claims 1 to 6.   The quantization means, the quantization step size set for the line immediately before the processing target line, the quantization step size of the processing target line excluding the head line among the lines constituting the divided image, The encoding apparatus according to any one of claims 1 to 7, wherein the encoding device is set by correcting based on a processing result by the encoding unit and a target code amount of the immediately preceding line. Further comprising conversion means for performing a conversion process for converting the input divided image into a coefficient to be quantized.
The encoding apparatus according to claim 1, wherein the quantization unit quantizes the coefficient.   The encoding apparatus according to claim 9, wherein the conversion process is a wavelet transform. Imaging means for imaging a subject and generating a plurality of temporally continuous frames;
An imaging apparatus comprising: the encoding apparatus according to claim 1. A method for controlling an encoding device, comprising:
A dividing step of dividing a plurality of temporally continuous frames into a plurality of regions including a first region and a second region that are spatially adjacent to each other, and forming the first region; Each frame is divided into a plurality of divided images such that the last line of the plurality of lines of the image and the first line of the plurality of lines of the divided image constituting the second region form a division boundary. Dividing process;
A plurality of encoding means, wherein the plurality of divided images are independently encoded, and a parallel encoding step;
A switching means for controlling the input of the plurality of divided images to the plurality of encoding means, and after the input of the divided images constituting the first region, the frame corresponding to the divided images; A switching step for controlling the divided images constituting the second region of the next frame to be input;
Have
Each of the plurality of encoding means is
For the data corresponding to the input divided image, a quantization step size is set according to the encoding result of the previous line, and a quantization step is performed for each line using the set quantization step size. ,
Performing an encoding step of encoding the quantized data;
In the quantization step, when the data of the first line of the divided image constituting the second area is quantized, the divided image of the first area of the frame preceding the frame corresponding to the divided image is displayed. A method for controlling an encoding apparatus, characterized in that a quantization step size is set according to an encoding result of a final line.   The program for functioning a computer as an encoding apparatus of any one of Claims 1 thru | or 10.   A computer-readable storage medium storing the program according to claim 13.

Images