JP4649309B2 - Image processing apparatus and image processing method - Google Patents

Description

The present invention relates to an image processing apparatus and an image processing method for digitizing a moving image signal and performing digital signal processing. For example, the present invention relates to a technique suitable for use in performing gradation correction processing of an image.

  Conventionally, an image with a small number of pixels is created with respect to the original image, and a spatially filtered image is created by interpolating and enlarging the image until it becomes equal to the number of pixels of the original image. An image processing method for performing nonlinear processing based on this has been proposed. In these cases, creation of an image with a small number of pixels is realized in a closed form inside the image processing apparatus (see, for example, Patent Document 1).

  On the other hand, in recent years, an apparatus such as a network camera that allows a plurality of users to use video captured by a camera via a network is increasing. In such an apparatus, according to the resolution requested by the user, the resolution of the captured image is converted on the imaging apparatus side and transmitted (for example, see Non-Patent Document 1).

  Also, a technique for sharing an image processing module such as resolution conversion processing for a plurality of outputs has been proposed (see, for example, Patent Document 2). Furthermore, as a technique that associates gradation correction with resolution, a technique of switching the resolution of a correction coefficient for gradation correction according to the pixel value of image data has been proposed (see, for example, Patent Document 3).

JP-A-10-63837 JP 2002-318673 A Japanese Patent No. 3424576 "Network Camera VB-C50i / VB-C50iR", Video Information Industrial, Industrial Development Organization, October 1, 2004, Vol. 36, No. 10, p.68-69

  Applying the image processing to a system for converting the resolution of a moving image input according to the resolution requested by the user, such as the network camera described above, and converting the image subjected to the gradation correction processing to a resolution-converted image You may have thought about. In this case, both the gradation correction processing and the resolution conversion processing such as image enlargement / reduction are performed for each video period which is an update interval of data for one screen constituting a moving image. Needs to be done. However, there has been no apparatus that considers simultaneous execution of gradation correction processing and resolution conversion processing.

  In Patent Document 3, the term “resolution” is used in the gradation correction processing, but this has a strong meaning of frequency characteristics and is different from the resolution conversion processing for generating images of different image sizes.

  In addition, if the gradation correction processing unit and the resolution processing unit are simply present in one system, the hardware resources required for both processes are added as they are, which increases the amount of hardware. There's a problem.

  Further, when the resolution conversion process is performed after the gradation correction process is performed, the delay element is increased by the increase in the processing contents. If the result of processing in units of frame images is simply passed between the processing units, the gradation correction processing unit delays at least one frame in order to perform gradation correction processing after creating a reduced image of the entire screen. In addition, a delay associated with the resolution conversion process is added. From the viewpoint of sending out a photographed image without delay as much as possible, an increase in video processing delay is a problem.

  Further, the gradation correction process is performed for each frame image unit. Therefore, the input frame image can be processed and handled as a plurality of different images within one video period that is the update interval of data for one screen, and gradation correction processing can be performed on each of them. There wasn't.

An object of the present invention is to provide an image processing apparatus and an image processing method in consideration of simultaneous execution of gradation correction processing and resolution conversion processing.

The image processing apparatus of the present invention, filtering against more and resolution converting means for converting the resolution of the input image, the reduced image input image more input to the image input means is reduced to the image input unit create a spatial filtered image by interpolation for the filter image obtained by, based on the spatial filter processing image the created, and gradation correcting means for performing gradation correction for pairs in the input image, the image input unit memory means for holding gradation corrected image by more inputted input image, the resolution conversion unit image resolution is converted by, and said tone correction processing means, which is more input to the image input unit performs a process of converting the resolution of the input image picture, and a process of generating the reduced image to be used for creating a pre-Symbol spatial filtering image time division And a control Gosuru control means cormorants, wherein, in one picture period is the update interval of data for one screen constituting a moving picture, conversion of the resolution for the one screen of data and creation process of the reduced image and is characterized by controlling so complete.
According to another aspect of the image processing apparatus of the present invention, a resolution conversion unit that converts the resolution of an input image input by the image input unit, and a reduced image obtained by reducing the input image input by the image input unit A gradation correction processing means for creating a spatial filter processed image by interpolation processing on the filter image obtained by the filter processing on the image, and performing gradation correction on the input image based on the created spatial filter processed image; and the image input Memory means for holding an input image inputted by the means, an image whose resolution has been converted by the resolution conversion means, and an image whose gradation has been corrected by the gradation correction processing means, and the resolution of the gradation-corrected image And a process of creating the reduced image used for creating the spatial filter processed image in a time-sharing manner. Control means for controlling the resolution, and the control means converts the resolution for the data for one screen within one video period which is an update interval of the data for one screen constituting a moving image. And the process for creating the reduced image is completed.

  According to the present invention, the control means for controlling the resolution conversion and the reduced image creation so as to be completed for each video period which is an update interval of data for one screen constituting the moving image. I had it. As a result, it is possible to perform resolution conversion of the image on which the gradation correction processing has been performed and deliver it.

  Further, the resolution conversion means is used for both the process of converting the resolution of the input image or the image subjected to the gradation correction process and the process of creating a reduced image used for the gradation correction process. Means for use in time division multiplexing were provided. As a result, it becomes possible to create a reduced image using the resolution processing unit for the gradation correction processing, so that it is not necessary to have a processing unit for creating a reduced image by the gradation correction processing unit alone. Therefore, it is possible to reduce hardware resources compared to a case where the system is configured by simply arranging the both.

  In addition, a reduced image used for gradation correction processing is created using image data of one field for time division multiplexing. This makes it possible to reduce the delay of the output image by at least 0.5 frames compared to the case of creating a reduced image using image data for one frame. Further, by controlling the two processing units of the gradation correction processing and the resolution conversion processing with one sequence control unit, it is possible to suppress the entire processing delay within one frame.

  Further, in the resolution conversion means, a plurality of processing means for converting the resolution are operated in parallel. As a result, a plurality of reduced images to be used for gradation correction processing are created corresponding to different angles of view such as a cut-out image, and a plurality of reduced images are used within one video period that is an update interval of data for one screen constituting a moving image. It is possible to perform gradation correction processing on images having different angles of view.

(First embodiment)
Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings.
FIG. 1 is a block diagram illustrating a configuration of an image processing apparatus according to the present embodiment. Reference numeral 1 denotes an image processing unit as a main subject of the proposal, and a video signal including a synchronization signal is input as moving image data from the imaging unit 3 using a sensor such as a CCD. As an example of the video signal including the synchronization signal, there is one that conforms to the format of ITU-R BT.656. The moving image processed by the image processing unit is sent to the image output processing unit 2, where processing such as compression is added and sent out to the network 4.

  The image processing unit 1 includes an image input interface 10 for receiving moving image input, a memory (1) 40 for temporarily storing input images and image data processed by each processing unit. Further, there are a gradation correction processing unit 20 and a resolution conversion unit 30 for performing gradation correction processing.

  The image input interface 10 extracts the synchronization signal from the video signal including the synchronization signal input from the imaging unit 3, extracts the synchronization information such as the start of the frame, switching of the odd field and the even field, and the sequence control unit Communicate to 80. At the same time, the transfer of the input image data to the memory (1) 40 is controlled according to the state of the synchronizing signal, and the sequence controller 80 is also notified of the transfer status. Based on the information of the synchronization signal and the transfer signal to the memory, the sequence control unit 80 recognizes one video period that is an update interval of data for one screen, and completes various processes for each period. Take control.

  The image data transferred to the memory (1) by the image input interface 10 is obtained by extracting only the image signal portion from the video signal including the synchronization signal. The memory (1) 40 can also be accessed from the later-described image output processing unit 2 side for transferring image data.

  The image output processing unit 2 includes a compression processing unit 50 that reads out moving image data from the memory (1) 40 and performs compression based on a predetermined method. Further, a memory (2) 70 for temporarily storing the compressed encoded data, and a communication processing unit 60 that performs processing such as packetization for extracting the encoded data from the memory (2) 70 and sending it to the network 4 Exists. Then, a sequence control unit 80 for controlling the data transfer between the processing modules of the image processing unit 1 and the image output processing unit 2 spans the image processing unit 1 and the image output processing unit 2. These are connected to the respective processing modules. The sequence control unit 80 includes a CPU 81 and appropriately controls each module in response to a request from a user using the network 4 transmitted via the communication control unit 60.

  Here, the memory (1) 40 and the memory (2) 70 are described as separate modules. However, the memory may be configured as separate physical memories, or may be configured as a single physical memory. It is also possible to do.

  In the present embodiment, the image output processing unit 2 performs compression processing and sends encoded data to the connected network. However, the configuration of the image output processing unit 2 is not limited to this. . For example, it is possible to connect to a plurality of monitors having different resolutions and supply image data to the monitors in the form of uncompressed data. The operation of the image processing unit 1, which is the main focus of the present proposal, is not affected by the form of the image output processing unit 2.

FIG. 2 is a diagram showing details of the gradation correction processing unit 20 in the present embodiment.
A data supply unit 21 and a data output unit 25 manage data exchange with the memory (1) 40. The image data read from the memory (1) 40 is distributed to two paths by the data supply unit 21 depending on the type of data.

  When the reduced image of the ODD field Y component, which is the original data for creating the spatial filter processed image, has been read, the reduced image data is first supplied to the preprocessing filter 22 and subjected to a predetermined low-pass filter. Thereafter, the image is transmitted to the spatial filter processed image creating unit 23, where it is interpolated and enlarged to a necessary size to obtain a spatial filter processed image for gradation processing, and is sent to the gradation correction calculating unit 24.

  On the other hand, the image data subjected to gradation correction processing is sent directly from the data supply unit 21 to the gradation correction calculation unit 24. The tone correction calculation unit 24 uses a look-up table for each pixel of the image corresponding to the spatial filter processed image created by the spatial filter processed image creation unit 23 to set a gamma correction value preset for luminance correction. Ask. After that, multiplication is performed between each pixel value of the image data and the gamma correction value, and then an operation such as adding a correction amount is performed as necessary to create a tone corrected image. The created tone corrected image is output to the data output unit 25, and the output tone corrected image is written to the memory (1) 40 by the data output unit 25.

  Note that the sequence control unit 80 manages these series of operations. Then, the filter coefficient of the preprocessing filter 22, the enlargement ratio at the time of image enlargement by the spatial filter processing image creation unit 23, the lookup table and correction amount used at the time of gradation correction calculation, and the address at the time of reading / writing to the memory Etc. to each processing unit. In the conventional example, a reduced image creation unit that creates a reduced image of the ODD field Y component that is the original data for creating the spatial filter processing image exists between the data supply unit 21 and the preprocessing filter 22. However, in the present embodiment, since this portion is shared with the resolution conversion unit 30 shown in FIG. 3, it is omitted from the drawing.

  FIG. 3 is a diagram showing details of the resolution conversion unit 30 in the present embodiment. 31 is a data supply unit, and 34 is a data output unit, which manages the exchange of data with the memory (1) 40. The image data read from the memory (1) 40 is separated into a Y signal and a C signal by the data supply unit 31. Then, the signals are input to the Y data resolution conversion unit 32 and the C data resolution conversion unit 33, respectively, and resolution conversion processing is performed. As the resolution conversion processing, the vertical / horizontal size of the image is enlarged or reduced by a predetermined ratio by the resolution converter 35, and then the data is temporarily stored in the line buffer 36, and then the edge enhancement processing is performed by the post-processing filter 37. Apply. The processing contents are basically the same for the Y system and the C system.

  The image whose resolution has been converted by the Y data resolution conversion unit 32 and the C data resolution conversion unit 33 as described above is converted again into one data string by the data output unit 34, and is stored in the memory (1) 40. Written. The series of operations are managed by the sequence control unit 80, and each processing unit is instructed about the size of the image to be converted, the necessity of processing in the post-processing filter, the address when reading from and writing to the memory, and the like. To do.

  Note that the resolution conversion processing performed by the resolution converter 35 is a combination of filtering processing according to the resolution to be converted and interpolation processing such as bilinear and bicubic. Since it is dedicated to resolution conversion, it is possible to invest hardware resources to provide complicated filtering processing, and the simple gradation correction processing unit that has been retained for creating a reduced image alone can be used. Compared to the reduction processing unit, its characteristics are much better.

  In the image processing system having such a configuration, how the resolution conversion unit 30 is shared by time-division multiplexing to achieve both gradation correction processing and resolution conversion processing is shown in FIGS. This will be described with reference to the timing chart and the flowcharts of FIGS. FIG. 4 is a diagram showing in what order the sequence control unit 80 controls the gradation correction processing unit 20, the resolution conversion unit 30, and the compression processing unit 50. 5 and 6 are flowcharts showing the flow of processing performed by the CPU 81 included in the sequence control unit 80 in performing the control. FIG. 5 shows a flowchart relating to tone correction processing, and FIG. 6 shows a flowchart relating to resolution conversion processing.

  In FIG. 4, it is assumed that input of a new image starts to the image input interface 10 at time t = 1. Here, the input image data is a progressive image. Of the lines constituting the image data, an odd field (ODD field, hereinafter referred to as ODD) in which odd lines are collected and an even field (EVEN field, hereinafter referred to as EVEN) in which even lines are collected are divided into two fields. Then, it is assumed that they are input for each field using a format such as ITU-R BT.656.

  When the image input interface 10 extracts the synchronization signal and recognizes the start of the ODD field, a new sequence is started. At this time, it is assumed that the input of a new image started at time t = 1 is the N-th frame ODD.

  When the image input interface 10 recognizes the start of the ODD field, it immediately starts to transfer the ODD data to the memory (1) 40. As a result, the ODD data is accumulated in the memory (1) 40 from the time t = 2. Then, the resolution conversion unit 30 reads out the ODD data of the Nth frame from the memory (1) 40. Then, a reduction process for creating a reduced image of the ODD field Y component that is the original data of the spatial filter processed image necessary for the gradation correction process using the Y data resolution conversion unit 32 is started at time t = 3.

  This reduction processing ends at time t = 4 after a certain processing delay after all of the ODD data of the Nth frame is input to the system. At that time, the generated reduced image of the ODD field Y component of the Nth frame is output on the memory (1) 40.

  When the input of the ODD data of the Nth frame is finished, the input of the EVEN data of the Nth frame is started from time t = 5 after a predetermined time has elapsed. When the image input interface 10 recognizes the start of the EVEN field by extracting the synchronization signal, it immediately starts to transfer the EVEN data to the memory (1) 40. As a result, the EVEN data is accumulated in the memory (1) 40 from the time t = 6. As a result, the progressive image data of the Nth frame is sequentially constructed on the memory (1) 40.

  The image data on the memory (1) 40 including the input image data is double-buffered, and the image data being used in the subsequent process is not overwritten by the output of the previous stage. And

  At time t = 7, the Nth frame of progressive image data can be read. In this state, the gradation correction processing unit 20 performs gradation correction processing on the progressive image of the Nth frame which is created by alternately reading out the ODD data and the EVEN data of the Nth frame from the memory (1) 40 line by line. As an object, gradation correction processing is started. At that time, the reduced image of the ODD field Y component of the Nth frame previously created at time t = 3 to 4 is also read out from the memory (1) 40 and used. Then, the result of the gradation correction process is started to be written in the memory (1) 40 from time t = 8.

  When the image data of the Nth frame after gradation correction processing starts to be stored in the memory (1) 40, the resolution conversion unit 30 reads the image data and performs resolution conversion from time t = 9. Here, it is assumed that the resolution conversion process is a reduction process. In substantially the same manner, the compression processing unit starts compression processing at time t = 10 for the Nth frame image data that has been subjected to the gradation correction processing using the same data. The tone-corrected Nth frame image data that has been compressed into encoded data is stored in the memory (2) 70 from time t = 11. This data is read out by the communication processing unit 60, subjected to processing such as packetization, and then sent to the user at the end of the network 4.

  On the other hand, the resolution conversion unit 30 that is performing the reduction process of the Nth frame image data that has been subjected to the gradation correction process ends the reduction process at time t = 12. The reduced N-th frame image data that has been subjected to the gradation correction processing is stored in the memory (1) 40. In response to the end of the previous compression process and the end of the resolution conversion process (t = 12), the compression processing unit 50 applies to the Nth frame image data that has been subjected to the gradation correction process reduced from time t = 13. Start the compression process.

  The image data of the Nth frame that has been subjected to the reduction / gradation correction processing that has been compressed into encoded data is stored in the memory (2) 70 from time t = 14. This data is also read out by the communication processing unit 60, subjected to processing such as packetization, and then transmitted to the user at the end of the network 4.

  It should be noted that the time required to compress the reduced N-th frame image data that has been subjected to the gradation correction processing is shorter than that of an image without reduction, but this is because the image to be processed itself is reduced. This is because the amount of processing data is reduced. Since the time required for the compression process is shortened, the time required for writing the encoded data on the memory (2) 70 is also shortened.

FIG. 5 is a flowchart showing the flow of processing of the CPU 81 in the sequence control unit 80 regarding the tone correction processing in the present embodiment.
When the process related to the gradation correction process is started (step S501), the sequence control unit first checks whether a gradation correction process request is generated (step S502). This tone correction processing request is a request from the communication processing unit 60 to the sequence control unit 80 in response to a request from a user ahead of the network 4.

  When a tone correction processing request is generated (step S502—YES), the process proceeds to step S503. In step S <b> 503, the sequence control unit 80 sets a parameter for creating a reduced image of the ODD field Y component that is the original data of the spatial filter processing image used for the gradation correction processing in the resolution conversion unit 30.

  The parameters set here include the total of the data transfer source address and transfer size for the data supply unit 31, and the setting of the path to the data supply destination (Y data resolution conversion unit 32). Also included are the image size (image reduction ratio) after conversion for the Y data resolution converter 32, and settings for supplying the data whose resolution has been converted to the data output unit 34 without being processed by the post-processing filter. It is. Furthermore, the address of the reduced image transfer destination for the data output unit 34, the total transfer size, and the like are included.

  After that, it waits until image data of an amount that can start creation of a reduced image is transferred to the memory (1) (step S504). When it is recognized by a notification from the image input interface 10 that a sufficient amount of image data capable of starting to create a reduced image has been transferred to the memory (1) (step S504—YES), the process proceeds to the next. Then, the sequence controller 80 instructs the resolution converter 30 to create a reduced image of the ODD field Y component (step S505), and the resolution converter 30 starts the reduction process (t = 3 in FIG. 4).

  When the end of the generation of the reduced image of the ODD field Y component is recognized by a notification from the data output unit data output unit 34 (step S506-YES, t = 4 in FIG. 4), the process proceeds to the next. Then, the sequence control unit 80 sets parameters necessary for executing the tone correction processing of the current frame to the tone correction processing unit (step S507).

  The parameters set here include the total of the transfer source address and transfer size of the reduced image of the ODD field Y component and the gradation-corrected image data for the data supply unit 21, and the filter coefficient of the preprocessing filter 22. . Also included are an enlargement ratio at the time of image enlargement by the spatial filter processing image creation unit 23, a lookup table and a correction amount used at the time of gradation correction calculation by the gradation correction calculation unit 24. Furthermore, the address of the transfer destination of the gradation corrected data for the data output unit 25, the total transfer size, and the like are included.

  After that, it waits until image data of an amount capable of starting the gradation correction processing is transferred to the memory (1) (step S508). This is because the ODD image data already exists on the memory (1) 40, so that the EVEN image data is transferred to the ODD image data and waits until a progressive image is formed on the memory (1) 40. means. When the image input interface 10 recognizes that the amount of image data that can start the gradation correction processing has been transferred to the memory (1) (YES in step S508), the process proceeds to the next. Then, the sequence control unit 80 instructs the tone correction processing unit 20 to perform tone correction processing for the current frame (step S509), and the tone correction processing unit 20 starts the tone correction processing (t in FIG. 4). = 7).

  When the end of the gradation correction processing is recognized by a notification from the data output unit 25 (step S510), the sequence control unit 80 checks whether the gradation correction processing request is continued for the next frame. To do. If it continues (step S511-YES), it is checked whether there is a resolution conversion process to be prioritized prior to the next process (step S512).

  If there is a resolution conversion process that should be prioritized, it waits until it disappears (step S512-YES). If it disappears (step S512-NO), the control is returned to the point A in FIG. Set and wait for the start of processing for the next frame. If the gradation correction process is not continued (NO in step S511), the gradation correction process is terminated (step S513).

FIG. 6 is a flowchart showing a processing flow of the CPU 81 in the sequence control unit 80 regarding the resolution conversion processing in the present embodiment.
When processing relating to resolution conversion processing is started (step S601), the sequence control unit first checks whether or not a request for resolution conversion processing has occurred (step S602). In response to the request from the user ahead of the network 4, the resolution conversion processing request is transmitted from the communication processing unit 60 to the sequence control unit 80.

  When a resolution conversion processing request is generated (step S602—YES), the sequence control unit 80 first checks whether there is a tone correction processing request for processing by sharing the resolution conversion unit 30 by time division multiplexing (step S602). Step S603). If it exists (step S603—YES), the process waits until the creation of the reduced image of the ODD field Y component of the current frame image is completed (step S604—NO). After the creation of the reduced image (step S604-YES) or when there is no tone correction processing request (step S603-NO), the sequence control unit 80 performs resolution conversion processing on the resolution conversion unit 30. To set parameters (step S605).

  The parameters set here include the address of the data transfer source for the data supply unit 31 and the total transfer size, the image size after conversion for the Y data resolution conversion unit 32 and the C data resolution conversion unit 33 (image reduction). Rate). Also included are filter coefficients such as edge enhancement processing performed using the post-processing filter 37, the transfer destination address of the reduced image with respect to the data output unit 34, and the total transfer size.

  Then, it waits until image data of an amount capable of starting the resolution conversion process is transferred to the memory (1) 40 (step S606-NO). If the image input interface 10 recognizes that the amount of image data that can start the resolution conversion process has been transferred to the memory (1) 40 (YES in step S606), the process proceeds to the next step. Then, the sequence controller 80 instructs the resolution converter 30 to perform the resolution conversion process (step S607), and the resolution converter 30 starts the resolution conversion process (t = 9 in FIG. 4).

  When the end of transfer of the resolution-converted data of the target image is recognized by a notification from the data output unit 34 (step S608, t = 12 in FIG. 4), the sequence control unit 80 also performs resolution conversion for the next frame. Check if the processing request is continuing. If it continues (step S609-YES), control is returned to immediately before step S603, and the start of processing for the next frame is awaited. If not continued (step S609-NO), the resolution conversion process is terminated (step S610).

  As described above, the control is performed so that the processing of both the resolution conversion and the reduced image creation is completed for each video period, which is the update interval of the data for one screen constituting the moving image. Thus, it is possible to perform resolution conversion on an image on which gradation correction processing has been performed and deliver it.

  In addition, the processing of the resolution conversion unit 30 is switched by time division multiplexing in this way. Thus, one resolution conversion unit 30 is used to generate a reduced image for generating a spatial filter processed image used for gradation correction processing and resolution conversion for the original image requested by the user required for the system. Can be made compatible. This makes it possible to reduce hardware resources.

  Also, the dedicated resolution conversion unit 30 is used for image reduction for creating a spatial filter processed image for gradation correction processing. This makes it possible to apply complex resolution conversion processing without using extra hardware resources as compared with the case where the gradation correction processing unit alone has a processing unit for creating a reduced image. As a result, the quality of the spatial filter processed image is improved, and an improvement in the image quality of the output image subjected to the gradation processing can be expected.

  In addition, a reduced image used for gradation correction processing is created using image data of one field for time division multiplexing. As a result, the delay of the output image for at least 0.5 frame can be reduced as compared with the case where a reduced image is created using image data for one frame. Further, by controlling the two processing units of gradation correction processing and resolution conversion processing with one sequence control unit 80, it is possible to suppress the entire processing delay within one frame.

(Second Embodiment)
In the first embodiment, a reduced image creation unit that creates a reduced image of the ODD field Y component, which is the original data for creating the spatially filtered image, is shared with the resolution conversion unit 30 shown in FIG. Wear resources were reduced.

  In the present embodiment, the pre-processing filter of the gradation correction processing unit is also shared with the resolution conversion unit to reduce hardware resources and change the processing procedure to reduce the time required for gradation correction processing. In this example, the output delay time is further reduced. Since there are many common parts with the first embodiment, the drawings and numbers are basically used, and only different parts will be described in detail.

FIG. 7 is a block diagram showing a configuration of the gradation correction processing unit 20 in the present embodiment. The configuration is almost the same as that in FIG. 2 of the first embodiment, but the preprocessing filter 22 is reduced as compared with FIG. The preprocessing filter 22 applies a predetermined low-pass filter to the reduced image of the ODD field Y component, which is the original data for creating the spatial filter processed image. This function is realized by a post-processing filter 37 of the resolution converter 30 as will be described later.
FIG. 8 shows the configuration of the resolution converter 30 in this embodiment. Although the configuration is almost the same as that of FIG. 3 of the first embodiment, a data path is added that returns from the resolution converter (1) 35a to the post-processing filter (1) 37a again via the line buffer (1) 36a. ing. This is because it is necessary to add a buffer amount when the resolution conversion unit 30 performs the processing performed by the pre-processing filter of the gradation correction processing, so that the line buffer can be rearranged and used as necessary. Is for the purpose.

  Hereinafter, the control flow and the like in this embodiment will be described with reference to the flowcharts of FIGS. 7, 8, 9, and 10 and FIG. 4 of the first embodiment. FIGS. 9 and 10 are flowcharts showing the flow of processing performed by the CPU 81 included in the sequence control unit 80 in controlling each module. FIG. 9 shows a flowchart relating to tone correction processing, and FIG. 10 shows a flowchart relating to resolution conversion processing. Since the basic flow is the same as that of the first embodiment, only the characteristic part of this embodiment will be described.

  In FIG. 9, the tone correction processing request is confirmed (step S902—YES). Then, the sequence control unit 80 sets the parameters for creating the preprocessed reduced image of the ODD field Y component, which is the original data of the spatial filter processed image used for the gradation correction processing, and the reconfiguration of the data path. (Step S903).

  At this time, as in the first embodiment, the total of the data transfer source address and transfer size for the data supply unit 31, and the path to the data supply destination (Y data resolution conversion unit 32) are set. Also, an image size (image reduction rate) after conversion for the Y data resolution conversion unit 32, a reduced image transfer destination address for the data output unit 34, a total transfer size, and the like are set. In the present embodiment, at the same time, the filter coefficient setting and the data path recombination setting are performed so that the post-processing filter 37 applies a low-pass filter corresponding to the pre-processing of the gradation correction processing.

  In some cases, the number of data required for the filtering process requires a larger number of data than when performing normal resolution conversion. In particular, as shown in FIG. 8, in this case, the data path to the line buffer 36 is rearranged so that the line buffer (2) 36b in the C data resolution converter 33 is also included in the Y data resolution converter 32. Make it available under control. This is because, for example, the processing performed by the post-processing filter 37 in normal resolution conversion is a 3 × 3 filter operation for edge enhancement, whereas the preprocessing required in the gradation correction processing is 5 ×. This is effective in the case of a low-pass filter or the like that performs an operation with the data of 5.

  In this way, after reconfiguring the configuration of the resolution conversion unit 30, it is recognized by a notification from the image input interface 10 that an amount of image data that can start generation of a reduced image has been transferred to the memory (1) ( Step S904—YES). Then, the sequence control unit 80 instructs the resolution conversion unit 30 to create a preprocessed reduced image of the ODD field Y component (step S905), and the resolution conversion unit 30 starts the reduction process (t = 3 in FIG. 4). ). At this time, the resolution conversion unit 30 creates a preprocessed reduced image of the ODD field Y component in a form preceding the filtering process corresponding to the preprocess in the gradation correction process, and stores it in the memory (1) 40. Will write.

  Then, the end of the creation of the preprocessed reduced image of the ODD field Y component is recognized by a notification from the data output unit 34 (step S906—YES, t = 4 in FIG. 4). Then, the sequence control unit 80 sets parameters necessary for executing the tone correction processing for the current frame to the tone correction processing unit (step S907).

  As in the first embodiment, the parameters set here include the transfer source address and transfer size of the preprocessed reduced image and gradation-corrected image data of the ODD field Y component for the data supply unit 21. It is. Also included are an enlargement ratio at the time of image enlargement by the spatial filter processed image creation unit 23, a lookup table and a correction amount used at the time of gradation correction calculation by the gradation correction calculation unit 24. Furthermore, the address of the transfer destination of the gradation corrected data for the data output unit 25, the total transfer size, and the like are included. However, in the present embodiment, as shown in FIG. 7, since the preprocessing filter 22 that existed in FIG. 2 does not exist, the setting relating to the coefficient of the filter is not performed.

  After that, it is recognized by a notification from the image input interface 10 that an amount of image data capable of starting the gradation correction processing has been transferred to the memory (1) (YES in step S908). Then, the sequence control unit 80 instructs the tone correction processing unit 20 to perform tone correction processing for the current frame (step S909), and the tone correction processing unit 20 starts the tone correction processing (t in FIG. 4). = 7).

  At this time, the preprocessed reduced image of the ODD field Y component read by the data supply unit 21 is directly transmitted to the spatial filter processed image creation unit 23 as shown in FIG. Therefore, the image is interpolated and enlarged to a necessary size to obtain a spatial filter processed image for gradation processing, and is sent to the gradation correction calculation unit 24. The subsequent processing is the same as in the first embodiment, but the processing in the preprocessing filter 22 in FIG. 2 in the phase of gradation correction processing after the EVEN field is input as compared with the first embodiment. As a result, the delay required for the gradation correction processing is reduced accordingly.

  FIG. 10 is a flowchart showing the processing flow of the CPU 81 in the sequence control unit 80 regarding the resolution conversion processing in the present embodiment. The basic flow is not different from FIG. 6 of the first embodiment. The difference is that in step S1005, the sequence control unit 80 instructs the resolution conversion unit 30 to set parameters for performing resolution conversion processing and to set a data path. It is.

  As described above, when there is a tone correction processing request, the data path to the line buffer 36 is rearranged, and the line buffer (2) 36b in the C data resolution converter 33 is also used for Y data. It can be used under the control of the resolution converter 32. Therefore, it is necessary to restore it and return the line buffer (2) 36b to the control of the C data resolution converter 33. Instructions for this are given at this step.

  As described above, the control is performed so that the processing of both the resolution conversion and the reduced image creation is completed for each video period, which is the update interval of the data for one screen constituting the moving image. Thus, it is possible to perform resolution conversion on an image on which gradation correction processing has been performed and deliver it.

  In addition, the processing and configuration of the resolution conversion unit 30 are switched in this way by time division multiplexing. Therefore, one resolution conversion unit 30 is used to generate a reduced image for generating a spatial filter processed image used for gradation correction processing, and to perform resolution conversion for an original image requested by a user required for the system. It is possible to achieve both. This makes it possible to reduce hardware resources. At the same time, the pre-processing for the reduced image necessary for the gradation correction processing can also realize further reduction of hardware resources by using the post-processing filter 22 of the resolution conversion unit 30.

  Also, a dedicated resolution conversion unit is used for image reduction for creating a spatial filter processed image for gradation correction processing. Therefore, it is possible to apply complex resolution conversion processing without applying extra hardware resources compared to a case where the gradation correction processing unit alone has a processing unit for creating a reduced image. As a result, the quality of the spatial filter processed image is improved, and an improvement in the image quality of the output image subjected to the gradation processing can be expected.

  In addition, a reduced image used for gradation correction processing is created using image data of one field for time division multiplexing, but compared with a case where a reduced image is created using image data for one frame. Thus, the delay of the output image can be reduced by at least 0.5 frames.

  Further, by controlling the two processing units of gradation correction processing and resolution conversion processing with one sequence control unit 80, it is possible to suppress the entire processing delay within one frame. Furthermore, the pre-processing for the reduced image necessary for the gradation correction processing can also be performed in advance by using the post-processing filter of the resolution conversion unit 30, thereby further reducing the delay of the output image. Can be expected to do.

(Third embodiment)
In order to perform the tone correction processing in the present proposal, a reduced image having the same angle of view as the processing target image is required to create a spatial filter processed image. In a system including a gradation correction processing unit having a conventional reduced image creation unit therein, the creation of the spatial filter processing image is sequential for each angle of view, and a plurality of angles of view are obtained within one video period. It was impossible to create a corresponding gradation correction processed image. In the present embodiment, a data extraction unit is incorporated in the data supply unit of the resolution conversion unit, and the fact that the resolution conversion unit has two resolution conversion units for Y and C is utilized. An example in which gradation correction processing can be performed on two images, one with an angle of view within one video period and another image cut out from the image, is shown.

  FIG. 11 is a block diagram illustrating a configuration of the resolution conversion unit 30 in the third embodiment. It is almost the same as FIG. 3 of the first embodiment, and is different in that a data cutout unit 38 is provided inside the data supply unit 31.

  The data cutout unit 38 is set with cutout region information from the sequence control unit 80 when the resolution conversion unit is used to create an ODD field Y component reduced image in the gradation correction processing. Then, according to the cut-out area information, a necessary portion is extracted from the image data read from the memory (1) 40 by the data supply unit 31. The extracted cut-out data is supplied to the C data resolution converter 33 in parallel with the original data being supplied to the Y data resolution converter 32. By using this mechanism, it is possible to create in parallel a reduced image of the ODD field Y component corresponding to two images, that is, an image with a certain angle of view and an image cut out from the image.

  Hereinafter, the control chart and the like in this embodiment will be described with reference to the timing charts of FIGS. 11 and 12, the flowchart of FIG. 13, and FIG. 1 of the first embodiment. FIG. 12 is a diagram illustrating a procedure in which the sequence control unit 80 controls the gradation correction processing unit 20, the resolution conversion unit 30, and the compression processing unit 50. FIG. 13 is a flowchart showing the flow of processing performed by the CPU 81 included in the sequence control unit 80 regarding the gradation correction processing in performing the control. In each figure, the same reference numerals are used for components common to the previous embodiment. Since the basic flow is the same as that of the first embodiment, only the characteristic part of this embodiment will be described.

  In FIG. 12, the basic flow is the same as FIG. 4 of the first embodiment. The difference is that a reduced image of the ODD field Y component corresponding to the clipped image is created at t = 3 to t = 4, and tone correction processing for the clipped image at t = 15 to 18 and compression of the output image The process is being performed. These differences will be described together in the description of FIG.

FIG. 13 is a flowchart showing a flow of processing in the CPU 81 in the sequence control unit 80 regarding the gradation correction processing in the present embodiment.
In FIG. 13, the processing is started (step S1301), and the tone correction processing request is confirmed (step S1302-YES). Then, the sequence control unit 80 first instructs the resolution conversion unit 30 to set parameters for creating a reduced image of the ODD field Y component that is the original data of the spatial filter processed image used for the gradation correction processing. (Step S1303).

  At this time, the tone correction processing request is a certain field angle size (for example, 1280 × 960, hereinafter, field angle 1) and an angle of view cut out from the image (for example, the center portion of the input image cut out at 640 × 480, the following image Assume that two corners 2) are generated. This request is a request transmitted from the user ahead of the network 4 through the communication processing unit 60 shown in FIG.

  In this case, as in the first embodiment, the total of the transfer source address and transfer size of the ODD field Y component image data of the angle of view 1 for the data supply unit 31 and the destination to which the data is supplied (Y data resolution conversion unit 32) The path setting for is set. Also, an image size (image reduction rate) after conversion for the Y data resolution conversion unit 32, a reduced image transfer destination address for the data output unit 34, a total transfer size, and the like are set. In the present embodiment, settings are made for creating reduced image data for angle of view 2 for performing gradation correction processing on an image of angle of view 2 at the same time.

  That is, as shown in FIG. 11, the data extraction unit 38 in the data supply unit 31 has the information about where to extract from the ODD field Y component image data of the angle of view 1 and the extracted image data as C A path to be supplied to the data resolution converter 33 is set. The sequence control unit 80 determines the image size (image reduction rate) after conversion for the C data resolution conversion unit 33, the transfer destination address of the reduced image for angle of view 2 for the data output unit 34, the total transfer size, and the like. Is set.

  After setting the resolution conversion unit 30 in this way, it is recognized by a notification from the image input interface 10 that image data of an amount capable of starting creation of a reduced image of the angle of view 1 has been transferred to the memory (1). (Step S1304-YES). Then, the sequence controller 80 instructs the resolution converter 30 to create a preprocessed reduced image of the ODD field Y component (step S1305), and the resolution converter 30 starts the reduction process (t = 3 in FIG. 12). ).

  At this time, in the resolution conversion unit 30, the reduced image of the ODD field Y component for the angle of view 1 is created by the Y data resolution conversion unit 32, and at the same time, the reduced image of the ODD field Y component for the angle of view 2 is converted to the C data resolution. Created by the conversion unit 33. These are written into the memory (1) 40. Since the image of the angle of view 2 is a partial image of the image of the angle of view 1, the creation of the reduced image for the angle of view 2 is completed in a shorter time than the time required for creating the reduced image for the angle of view 1.

  Then, the end of the preparation of the preprocessed reduced image for the ODD field Y component is recognized by a notification from the data output unit 34 (step S1306—YES, t = 4 in FIG. 12). Then, the sequence control unit 80 first sets parameters necessary for executing the gradation correction processing for the image with the angle of view 1 to the gradation correction processing unit 20 (step S1307). The subsequent processing up to step S1310 is the same as the processing up to step S510 in FIG. 5 of the first embodiment.

  When the tone correction processing for the image with the angle of view 1 ends (step S1310—YES), the sequence control unit 80 then determines whether or not tone correction with a different angle of view is necessary (step S1314). In the present embodiment, a request is generated to perform tone correction processing for an image with an angle of view 2 (step S1314—YES). Therefore, the sequence control unit 80 sets parameters necessary for executing the gradation correction processing for the image with the angle of view 2 to the gradation correction processing unit 20 (step S1307).

  Since the image data constituting the angle of view 1 has already been transferred to the memory (1) 40 and is held by the double buffer, the image data of the angle of view 2 created by cutting out from the image data has already been stored in the memory. Exists (step S1308-YES). Therefore, this time, the sequence control unit 80 instructs the gradation correction processing unit 20 to start the gradation correction processing for the image with the angle of view 2 (step S1309, t = 15 in FIG. 12).

  The gradation correction processing unit 20 performs gradation correction processing on the image data of the angle of view 2 by using the reduced image of the ODD field Y component for the angle of view 2 previously created, and t = 16 in FIG. From the point of time, the gradation correction processed data is sequentially stored on the memory (1) 40. The stored image angle 2 processed tone correction data is subjected to image compression processing by the compression processing unit 50 from that point in time when the compression processing unit has time to process (t = 17 in FIG. 12). (2) It is stored on 70 (FIG. 12 t = 18). Then, it is delivered to an external image user via the communication processing unit 60 and the network 4.

  On the other hand, the control related to the tone correction processing by the sequence control unit 80 also corresponds to the tone correction processing request related to the angle of view 2, and thus there is no pending tone correction processing (NO in step S1314). Therefore, the continuity of the processing request regarding the next frame is checked (step S1311). The subsequent control flow is the same as that of the first embodiment, and is therefore omitted here.

  In this way, only a necessary portion is extracted from the image data sequence obtained by one-time data reading by the data extraction unit, and the original data sequence and the extracted data sequence are provided in the resolution conversion unit, two series of Y and C, respectively. Are separately supplied to the resolution conversion unit and reduced. This makes it possible to create a spatial filter processing image used for tone correction processing corresponding to two different angles of view in parallel. As a result, it is possible to perform gradation correction processing on two images, an image having an angle of view within one video period and an image cut out from the image.

(Other embodiments according to the present invention)
Each means constituting the image processing apparatus according to the above-described embodiment of the present invention can be realized by operating a program stored in a RAM or a ROM of a computer. This program and a computer-readable recording medium recording the program are included in the present invention.

  Further, the present invention can be implemented as, for example, a system, apparatus, method, program, or recording medium. Specifically, the present invention may be applied to a system including a plurality of devices. The present invention may be applied to an apparatus composed of a single device.

  Note that the present invention supplies a software program that implements the functions of the above-described embodiments directly or remotely to a system or apparatus. This includes the case where the system or the computer of the apparatus is also achieved by reading and executing the supplied program code.

  Accordingly, since the functions of the present invention are implemented by computer, the program code installed in the computer also implements the present invention. In other words, the present invention includes a computer program itself for realizing the functional processing of the present invention.

  In that case, as long as it has the function of a program, it may be in the form of object code, a program executed by an interpreter, script data supplied to the OS, and the like.

  Examples of the recording medium for supplying the program include a floppy (registered trademark) disk, a hard disk, an optical disk, and a magneto-optical disk. Further, there are MO, CD-ROM, CD-R, CD-RW, magnetic tape, nonvolatile memory card, ROM, DVD (DVD-ROM, DVD-R) and the like.

  As another program supply method, there is a method of connecting to a homepage on the Internet using a browser of a client computer. The computer program itself of the present invention or a compressed file including an automatic installation function can be downloaded from the homepage by downloading it to a recording medium such as a hard disk.

  It can also be realized by dividing the program code constituting the program of the present invention into a plurality of files and downloading each file from a different homepage. That is, a WWW server that allows a plurality of users to download a program file for realizing the functional processing of the present invention on a computer is also included in the present invention.

  As another method, the program of the present invention is encrypted, stored in a recording medium such as a CD-ROM, distributed to users, and encrypted from a homepage via the Internet to users who have cleared predetermined conditions. Download the key information to be solved. It is also possible to execute the encrypted program by using the key information and install the program on a computer.

  Further, the functions of the above-described embodiments are realized by the computer executing the read program. Furthermore, based on the instructions of the program, an OS or the like running on the computer performs part or all of the actual processing, and the functions of the above-described embodiments can be realized by the processing.

  As another method, the program read from the recording medium is first written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Then, based on the instructions of the program, the CPU or the like provided in the function expansion board or function expansion unit performs part or all of the actual processing, and the functions of the above-described embodiments are also realized by the processing.

1 is a block diagram illustrating a configuration example of an image processing device according to a first embodiment of the present invention. It is a figure which shows the detail of the gradation correction process part in the 1st Embodiment of this invention. It is a figure which shows the detail of the resolution conversion part in the 1st Embodiment of this invention. It is a timing chart of the time relation of processing in a 1st embodiment of the present invention. It is a flowchart which shows the flow of a process in CPU in the sequence control part regarding the gradation correction process in the 1st Embodiment of this invention. It is a flowchart which shows the flow of a process in CPU in the sequence control part regarding the resolution conversion process in the 1st Embodiment of this invention. It is a figure which shows the detail of the gradation correction process part in the 2nd Embodiment of this invention. It is a figure which shows the detail of the resolution conversion part in the 2nd Embodiment of this invention. It is a flowchart which shows the flow of a process in CPU in the sequence control part regarding the gradation correction process in the 2nd Embodiment of this invention. It is a flowchart which shows the flow of a process in CPU in the sequence control part regarding the resolution conversion process in the 2nd Embodiment of this invention. It is a figure which shows the detail of the resolution conversion part in the 3rd Embodiment of this invention. It is a timing chart of the time relation of processing in a 3rd embodiment of the present invention. It is a flowchart which shows the flow of a process in CPU in the sequence control part regarding the gradation correction process in the 3rd Embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Image processing part 2 Image output process part 3 Image pick-up part 4 Network 10 Image input interface 20 Gradation correction process part 21 Data supply part 22 Preprocessing filter 23 Spatial filter process image creation part 24 Gradation correction calculation part 25 Data output part 30 Resolution conversion unit 31 Data supply unit 32 Y data resolution conversion unit 33 C data resolution conversion unit 34 Data output unit 35 Resolution converter 36 Line buffer 37 Post-processing filter 38 Data extraction unit 40 Memory (1)
50 Compression Processing Unit 60 Communication Processing Unit 70 Memory (2)
80 Sequence control unit 81 CPU

Claims (8)

And resolution converting means for converting the higher resolution of the input image input to the image input means,
Said image to create a spatial filtered image by interpolation for the filter image obtained by filtering more input image input to pair the reduced reduced image input means, based on the spatial filter processing image the created, and gradation correcting means for performing gradation correction for pairs in the input image,
Memory means for holding gradation-corrected image by the more input image input to the image input unit, image resolution has been converted by said resolution converting means, and said tone correction processing means,
Treatment and before Symbol spatial filtered image of the reduced image control Gosuru control means to perform a time division and a process of creating a use in creating convert more resolution of inputted input image image to the image input unit And
Wherein, in one picture period is the update interval of data for one screen constituting a moving picture, and the resolution conversion processing of for said one screen data and creation processing of the reduced image is complete An image processing apparatus that is controlled as described above. Before SL resolution conversion means, the image processing apparatus according to claim 1, characterized in that to create the reduced image using any image data of the odd and even lines of the input image. The image processing apparatus according to claim 1 , wherein the resolution conversion unit creates the reduced image using image data of odd lines of the input image . The resolution converting unit includes a plurality of processing means that converts the resolution to create a plurality of reduced images corresponding to a plurality of different angle from the input image in parallel, the tone correction processing means, wherein using a plurality of reduced images corresponding to different view angle according to any one of claims 1 to 3, characterized in that the gradation correction process against the image of the plurality of different angle Image processing device. The image processing apparatus according to claim 4, wherein the images having different angles of view are created by cutting out a part of the input image input by the image input unit.   Resolution conversion means for converting the resolution of the input image input by the image input means;
  A spatial filtered image is created by interpolation processing on a filter image obtained by filtering the reduced image obtained by reducing the input image input by the image input means, and the input image is based on the created spatial filtered image. Gradation correction processing means for performing gradation correction for
  Memory means for holding an input image input by the image input means, an image whose resolution has been converted by the resolution conversion means, and an image whose gradation has been corrected by the gradation correction processing means;
  Control means for controlling to perform time-division processing for converting the resolution of the tone-corrected image and processing for creating the reduced image used for creating the spatial filter processed image;
  The control means completes the resolution conversion processing and the reduced image creation processing for the data for one screen within one video period which is an update interval of the data for one screen constituting the moving image. An image processing apparatus that is controlled to   Resolution conversion means for converting the resolution of the input image input by the image input means;
  A spatial filtered image is created by interpolation processing on a filter image obtained by filtering the reduced image obtained by reducing the input image input by the image input means, and the input image is based on the created spatial filtered image. Gradation correction processing means for performing gradation correction for
  An image processing apparatus having an input image input by the image input means, an image whose resolution is converted by the resolution conversion means, and a memory means for holding an image whose gradation is corrected by the gradation correction processing means. An image processing method comprising:
  A control step of controlling to perform processing for converting the resolution of the input image input by the image input means and creation processing of the reduced image used for creating the spatial filter processing image in a time-sharing manner,
  In the control step, the resolution conversion process and the reduced image creation process for the data for one screen are completed within one video period that is an update interval of the data for one screen constituting the moving image. An image processing method characterized by performing control as described above.   Resolution conversion means for converting the resolution of the input image input by the image input means;
  A spatial filtered image is created by interpolation processing on a filter image obtained by filtering the reduced image obtained by reducing the input image input by the image input means, and the input image is based on the created spatial filtered image. Gradation correction processing means for performing gradation correction for
  An image processing apparatus having an input image input by the image input means, an image whose resolution is converted by the resolution conversion means, and a memory means for holding an image whose gradation is corrected by the gradation correction processing means. An image processing method comprising:
  A control step for controlling the time-division processing to convert the resolution of the tone-corrected image and the reduced image creation processing used to create the spatial filter processed image;
  In the control step, the resolution conversion process and the reduced image creation process for the data for one screen are completed within one video period that is an update interval of the data for one screen constituting the moving image. An image processing method characterized by performing control as described above.

Images