JP5541679B2 - Image processing apparatus and method, and program - Google Patents

Description

  The present invention relates to storing image data obtained by photographing with a photographing device. More specifically, the present invention relates to correcting and storing the shape and position of image data obtained by photographing with a camera in a state where the image is projected by a handwriting on a projection board (whiteboard) on which an image is projected by a projection apparatus.

  In recent years, with the spread of networks, the opportunity to distribute documents electronically has increased, and when a meeting is held, electronic data is projected as a document using a whiteboard as a projection board, and further with a water-based pen etc. on the whiteboard Often handwritten. In view of this, a technique for photographing a state in which handwritten information is added to a projection board (whiteboard) in a state where image data is projected by a projector with a digital camera or the like, and making it as electronically distributable data as minutes is considered. .

  Also, there is a technique for finding a distortion parameter by acquiring a known rectangular frame from an image photographed in a state where it is not directly facing a plane such as a projection plate, and correcting the image to a directly facing image (Patent Document 1). . If this technique is used, a known rectangular frame in the projection content, for example, a frame corresponding to the outer frame of the projection area or the frame of the projection plate itself, is detected, and distortion parameters are obtained based on the detected distortion of the frame. be able to. Then, with the correction using the calculated distortion parameter, an image without distortion can be obtained from an image captured without facing the projection plate.

  In addition, the outer frame of the projected image is detected by projecting and shooting the outer frame image, the position and size of the handwritten image written on the projection plate are corrected according to the detected outer frame, and the corrected writing There has been a technique of synthesizing an image with the original image data that has been projected (Patent Document 2).

Japanese Patent Laid-Open No. 03-094383 JP 2007-017543 A JP 2003-288588 A Japanese Patent No. 3026592

  When an unfixed portable projector is actually used, it is often operated in a state where the projection plate and the projector do not face each other accurately. In such a state, an image captured by a digital camera or the like includes distortion due to the positional relationship between the projection plate and the imaging position, and also includes distortion due to the positional relationship between the projection apparatus and the projection plate. That is, in the captured image, the portion of the image projected by the projection device is a combination of both distortions. On the other hand, since the user writes characters and the like directly on the projection plate (whiteboard), the writing portion (written image) does not include distortion due to the positional relationship between the projection device and the projection plate, and the projection plate and the shooting position. Only the influence of distortion due to the positional relationship of Therefore, there is a problem in that the shape of the handwritten part is distorted when the captured image is corrected with the outer frame of the projection area as a reference.

An image processing apparatus according to the present invention devised to solve the above problem is an input for inputting image data obtained by photographing a board on which a projection object is projected by a projection apparatus and a handwritten object is handwritten by a user. means and the input image data or al, the first frame and a frame extracting means for extracting a second frame constituting the four sides of the projection area to be projected by the projection device constituting the four sides of the plate If, to obtain a first distortion correction value based on the first frame, a correction value acquisition means for acquiring a second distortion correction value based on the second frame, from the input image data and object extracting means for extracting said handwriting object and the projection object, wherein for the extracted handwritten object performs distortion correction using the first distortion correction value, Distortion serial For extracted projection object using a distortion correction means for performing distortion seen correction using the second distortion correction value, the distortion-corrected handwriting objects and said distortion corrected projection object and an output means for generating the corrected image data, the.

  Even when the projection image and the handwritten image are photographed by the photographing device in a state where the projection data by the projection device is inclined, after correcting the image of the handwritten entry to the same shape as the original description shape, Image data superimposed at an appropriate position on the projection data can be generated.

It is a figure which shows the example of the input image acquisition environment of Embodiment 1. FIG. 6 is a diagram illustrating an example of an input image according to Embodiment 1. FIG. 2 is a diagram illustrating a configuration example of Embodiment 1. FIG. FIG. 3 is a block diagram illustrating an operation of the first embodiment. 3 is a flowchart of image processing in the first embodiment. It is a figure which shows the example of the frame extraction process in Embodiment 1. FIG. It is a figure which shows the example of the object division | segmentation process result in Embodiment 1. FIG. It is a figure which shows the example of the object correction process result in Embodiment 1. FIG. 6 is a diagram illustrating an example of an output image according to Embodiment 1. FIG. 10 is a flowchart of image processing in the second embodiment. FIG. 10 is a diagram illustrating an example of an input image in the second embodiment. It is a figure which shows the example of the handwritten part extraction in Embodiment 2. FIG. 10 is a diagram illustrating an example of an output image in Embodiment 2. FIG.

Embodiment 1
Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram illustrating an environment in which an image input to an image processing apparatus according to an embodiment of the present invention is acquired.

  A white board 101 having a quadrilateral shape serves both as a projection board and an entry board. A pen 102 is used to fill in the white board 101. An image is projected from the projection device (projector) 103 onto the white plate 101. The projection image is input to the projection apparatus 103 from a computer such as the PC 104. The white board 101 serves as a background for the projected image and the handwritten image. A photographing device (digital camera) 105 is used to photograph the white plate 101 including the projection area. In the above environment, the projection data 110 of the image captured by the imaging device 105 is projected from the projection device 103 onto the white plate 101, that is, the background. The projected image and the handwritten image can exist simultaneously by handwriting on the white plate on which the projected image is projected.

  FIG. 2 is a diagram for explaining an example of an input image input to the image processing apparatus of the present invention.

  The input image 200 photographed by the photographing device 105 includes a white board region frame 201 composed of four sides, a projection region frame 202 composed of four sides, an object 203 of the projected image, and a handwritten object 204 entered with the pen 102. When the projection device 203 projects from the oblique direction of the white plate 101, the projection area is distorted with respect to the white plate. In addition, an inclination, that is, linear distortion occurs from the positional relationship between the photographing position and the object. Therefore, in the photographed image 200, in particular, the projection area frame 202 and the projection image object 203 have a distortion different from the distortion of the white board area frame 201 of the white board 101 depending on the positional relationship between the projection device 103 and the white board 101. .

  FIG. 3 is a block diagram illustrating a configuration example of an image processing apparatus that implements the present invention.

  The image processing apparatus 300 includes an external storage medium I / F 301, a CPU 302, a memory 303, and a hard disk 304. The photographed image data is input from the external storage medium I / F 301, and the CPU 302 executes an image processing program for applying the processing of the present invention to the image data. The memory 303 is used for temporary storage of work memory and data when the program is executed, and the hard disk 304 is used for storing the program and data.

  The configuration of the image processing apparatus 300 shown in FIG. 3 is an example, and the image processing of the present invention may be executed by the PC 104 or may be executed on the electronic circuit or ASIC of the photographing apparatus 105.

  FIG. 4 is a block diagram illustrating image processing according to the present embodiment.

  An image photographed by the photographing device 105 is input to the frame extraction processing unit 401 and the object division processing unit 407 as an input image 400 from the external storage medium I / F 301.

  The frame extraction processing unit 401 extracts a frame composed of four straight lines from the input image 400, and obtains a white board area frame shape 402 corresponding to the outline of the white board area and a projection area frame shape 403 corresponding to the outline of the projection area. This is provided to the distortion correction value extraction processing unit 404. The distortion correction value extraction processing unit 404 performs a process of quantifying each distortion from the white board area frame shape 402 and the projection area frame shape 403, calculates a white board distortion correction value 405 from the white board area frame shape 402, and calculates the projection area frame. A projection distortion correction value 406 is calculated from the shape 403.

  On the other hand, the object division processing unit 407 provides the object correction processing unit 409 with each object data 408 obtained by dividing an object such as a type, symbol, or handwritten character from the input image 400. The object correction processing unit 409 corrects the object data based on the white plate distortion correction value 405 and the projection distortion correction value 406, and outputs an output image 410 formed by a set of corrected object data.

  FIG. 5 is a detailed flowchart for explaining the processing procedure of this embodiment according to the block diagram of FIG. Hereinafter, an example of generating an output image 410 having a width w and a height h when the image 200 of FIG. 2 is used as the input image 400 will be described with reference to the flowchart.

  In step S <b> 501, the frame extraction processing unit 401 extracts a quadrangle corresponding to the frame of the projection area from the input image 400.

  Extraction of a rectangle corresponding to a frame can be performed as follows using a known method. The edge enhancement method using Sobel, Laplacian filter, etc. makes the pixels corresponding to the boundary part of the white plate area and the boundary part of the projection area in the input image stand out. By using a known straight line extraction method such as a Hough transform or a minimum approximation method for this enhanced image, four sufficiently long sides constituting a quadrangle are extracted.

  An example of an image obtained by performing edge enhancement on the input image 200 of FIG. 2 is shown in FIG. A square 601 corresponds to the frame of the projection area. As shown in FIG. 6, a white plate region frame 602 described later is extracted from the edge-enhanced image outside the projection region frame 601. In step S501, the inner one of the two extracted frames is selected as the frame of the projection area.

  In step S502, the frame extraction unit extracts an unequal square corresponding to the frame of the white board area from the input image. This process is performed in association with the process of step S501, and the outer frame of the two frames extracted from the image obtained by edge-enhancing the input image may be selected as the white plate area frame. If only one frame is found, the edge enhancement parameters and the straight line detection method may be changed so that two frames are detected, and the processes in steps S501 to S502 may be performed again. Some types of white boards have rounded corners. In such a case, the straight lines that form the top, bottom, left, and right sides of the white board are detected from the edge-enhanced image. You may comprise so that a square position may be obtained on the basis of this position.

  In step S503, the correction value acquisition unit extracts and acquires a projection distortion correction value for correcting the distortion in the input image with reference to the projection region.

  Here, the projection distortion correction value is an operation for correcting an unequal-sided rectangular area generated when a rectangular area is projected onto a two-dimensional plane with a three-dimensional angle, so-called reverse perspective transformation calculation. Is a parameter of the transformation matrix used in. For example, as disclosed in Patent Document 3, by substituting the coordinates of the four vertices before conversion and the coordinates of the four vertices after conversion into an arithmetic expression for inverse perspective transformation, and solving the obtained simultaneous equations, Parameters are extracted.

  In the case of the process in step S503, the coordinates of the four vertices before the conversion correspond to the coordinates of the four vertices of the projection area frame extracted in step S501. The coordinates of the four vertices after conversion are the four corners of the image area of the output size w × h with respect to the projection area portion specified by the user in advance, that is, (0, 0), (w, 0), (w, h). , (0, h). Each parameter of the matrix obtained by solving the simultaneous equations created using both coordinates is stored as a projection distortion correction value.

  In step S504, the correction value acquisition unit extracts and acquires a white plate distortion correction value 405 for correcting the distortion in the input image with reference to the white plate region frame.

  The processing for obtaining the white plate distortion correction value 405 can be performed in the same manner as in step S503. Here, the four vertices of the white board region frame extracted in step S502 are substituted as the coordinates of the four vertices before conversion. The coordinates of the four vertices after conversion are the white plate aspect ratio a (horizontal / vertical) specified in advance by the user, the output size w × h for the projection area portion, and the height ratio r of the white board area to the projection area in the input image. Is obtained as follows.

(0,0), (W, 0), (W, H), (0, H) where H = h × r, W = a × H
A point where the line segment AB connecting the midpoint A of the upper side of the white plate area frame 602 and the midpoint B of the lower side of FIG. At this time, the above r is obtained by the ratio L1 / L2 of the length L1 of the line segment AB and the length L2 of the line segment CD.

  In step S505, the object divided out means divides and extracts the entry written on the white board in the input image and the projected projection into individual objects. The divided extracted object is stored as data having a shape and a position.

  The object division process is performed as follows. First, the input image is converted into a binary image so that the object pixel is black and the background pixel is white. For example, a technique can be used in which a luminance histogram of a multi-valued image is acquired and converted into a binarized image using a threshold value obtained adaptively from those peaks. Next, a labeling image is generated by a known method. For example, the following method can be used. The pixels of each line forming the binary image are scanned, and the start position and the end position are stored in a table for a section in which black pixels are continuous. When this process is completed for all lines, a labeling image is generated by giving the same label to the section in contact between the lines and forming a black pixel region. Each pixel block having the same label in the generated labeling image is regarded as an object.

  In step S505, initial values of shape information and position information stored in the object data are given. The shape information is information indicating a labeling image that can be specified by the object, and a label value of a corresponding pixel on the image. The position information is the lower left coordinates of a rectangle circumscribing the corresponding label pixel set on the labeling image.

  FIG. 7 shows the result of object division processing performed on the input image 200.

  As a result of labeling the continuous black pixels, the projection image 203 of the input image 200 is divided into 12 objects 7301 to 7312 indicated by circumscribed rectangles. The handwritten entry 204 of the input image 200 is divided into 13 objects 7401 to 7413 indicated by circumscribed rectangles.

  Step S506 is a step for controlling repetitive processing up to the repetitive termination step S512 for each of the objects extracted in step S505. That is, in the present embodiment, control is performed so that the processing is repeated for the 25 objects 7301 to 7312 and 7401 to 7413.

  First, the process proceeds to step S507 with the object 7301 as a processing target, and the object selection means determines whether the target object is a handwritten entry object or a projection object. If the object is a handwritten object, the process proceeds to step S508, and if it is another object, the process proceeds to step S509.

  The determination as to whether the object is a handwritten object or a projected object may be made, for example, by determining in advance that the projected object and the handwritten entry have different colors and using the color of each object acquired from the input image. Alternatively, it can be judged by evaluating handwriting based on the degree of fluctuation of the line or non-uniformity of shading. Alternatively, after correcting each object once using the projection distortion, it is possible to determine that the line segment in the object is horizontal / vertical as a projection. Alternatively, by using the determination result of the nearby object, for example, when there is an object of a projection with a uniform position and size, the target can be determined as the projection. Alternatively, the color of the handwritten entry is designated in advance, the color information of the target object is compared with the designated color information of the handwritten entry, and if it is determined that the color is different, it can be determined as a projection object. it can. Or at the time of imaging | photography of an input image, the projection by a projection device can be stopped and the state of only a handwritten entry can be image | photographed and judged for handwritten entry judgment. The classification result (judgment result) obtained in this way is held accompanying the data of each object.

  Here, since it is determined that the object 7301 is other than the handwritten object, the process proceeds to step S509, and the distortion correction unit corrects the target object shape with the projection distortion correction value.

  In this correction processing, a white plate distortion corrected labeling image is generated by rearranging the pixels of the labeling image generated in step S505 by inverse perspective transformation using the white plate distortion correction value 405. Then, the shape information of the target object data is rewritten so that the labeling image to be referred to becomes the whiteboard distortion corrected labeling image.

  In step S510, the distortion correction unit corrects the lower left coordinate, which is the position information of the target object, with the projection distortion correction value. This correction process is performed by setting a projection distortion correction parameter in the transformation matrix and performing a reverse perspective transformation calculation on the lower left coordinates before correction.

  Like the object 7301 determined to be other than the handwritten object, the same projection distortion value may be used for the position of the object whose shape is corrected by the projection distortion in step S509.

  In step S511, a corrected object pixel shape is extracted from the corrected labeling image associated with the object data and the label value information of the corresponding pixel in the image, and the output image is displayed according to the corrected position information of the object. To place. This may simply copy the target labeled pixels on the corrected labeling image into the output image. Or you may arrange | position as an independent object in an output image. For example, partial image data in which a pixel with a target label becomes an object color and the others are transparent may be generated, and graphic description data that is drawn at a specified position in the output image may be configured. Or you may comprise using the data which converted the outline, core line, etc. of the line part of this partial image into vector path data by the well-known vector conversion technique.

  For example, in the technique disclosed in Patent Document 4, horizontal and vertical inter-pixel vectors are detected based on the state of a target pixel and its neighboring pixels while raster scanning an image. Next, by extracting the contour of the image data based on the connection state between these inter-pixel vectors, information describing the circulation of connected pixel data called an outline vector as a set of inter-pixel vectors is generated.

  In step S512, if there is an unprocessed object, the process returns to step S506 to repeat the process, and if there is no unprocessed object, the process ends without repeating. Here, since 24 unprocessed objects remain, the process returns to step S506 and the process is repeated with the object 7302 as a processing target. Since the objects 7302 to 7312 are determined not to be handwritten objects in step 507 in the same manner as the object 7301, description thereof will be omitted.

  Subsequent to the processing of the object 7312, the object 7401 is set as a processing target. In step S507, it is determined that the object 7401 is a handwritten object, and thus the process proceeds to step S508.

  In step S508, the distortion correction unit corrects the shape of the target object with the whiteboard distortion correction value for the object determined to be handwritten. This correction process is the same as the correction process in step S509 except that the white plate distortion correction value 405 is used as a parameter instead of the projection distortion correction value. Subsequently, steps S510 to S512 are performed.

  If correction is made based on the white frame to eliminate distortion of the handwritten part, the correction amount of the handwritten part and the projected image part will be different, so if the corrected handwritten part image is directly combined with the projected image data, there is a risk of displacement. There is. On the other hand, in the present invention, while correcting the shape of a handwritten object, the white plate distortion correction value is used for shape correction, and the projection distortion correction value is used for position correction while correcting the shape distortion. Thus, the positional deviation between the handwritten object and the projection object can be reduced as much as possible.

  The objects 7402 to 7413 are also determined as handwritten objects in step S507, and the processing in steps S508 and S510 to S512 is performed in the same manner as the object 7401. When there are no unprocessed objects, the processing of this flowchart is ended.

  FIG. 8 is a diagram for explaining the process of each process in the flowchart of FIG.

  As a result of performing the shape correction in step S509 and the position correction in step S510 for the projection objects 7301 to 7312 that are not handwritten objects, that is, as a result of the reverse perspective transformation processing with the distortion of the projection area frame 202, a corrected image 801 is obtained ( FIG. 8 (A)). As a result of performing the shape correction in step S508 on the handwritten objects 7401 to 7413, that is, as a result of performing the reverse perspective transformation process with the distortion of the white board region frame 201, a corrected image 802 is obtained (FIG. 8B). The sizes of both images are shown based on the height ratio r of the white plate area to the projection area obtained in step S504.

  As a result of correcting the projection area frame position according to step S510 for the position information of the objects 7401 to 7413, that is, the rectangular lower left coordinates, the corrected 13 object lower left coordinate positions 803 are obtained (FIG. 8C). The coordinate position 803 is indicated by dots for the sake of explanation. A broken line 804 indicates the position of the projection object for reference.

  FIG. 9 is a diagram showing an output image according to the present embodiment in comparison with an example of the prior art.

  The output image 901 of this embodiment using two distortion correction values (FIG. 9A) is obtained by correcting the position and shape corrected projection object as shown in FIG. 8A and the correction shown in FIG. 8B. A handwritten object having a shape is arranged at the correction position shown in FIG.

  In the output image 902 according to the conventional technique, since all the objects are corrected using only the projection distortion correction value, the shape of the object handwritten by the writer is different from the original shape (FIG. 9B). In the output image 901, the shape of the handwritten object is not distorted.

  As described above, according to the present embodiment, an image obtained by photographing a white board in which an image is projected and information is added by handwriting can be corrected to a shape that is easy to see and combined.

[Embodiment 2]
In the first embodiment, correction processing is simply performed from a captured image. However, if image data input from the PC 104 to the projection apparatus can be identified and acquired, the correction may be performed by adding to the image data. Further, the correction value to be applied may be determined depending on the type of the handwritten object.

  FIG. 10 is a flowchart for explaining the processing procedure of the image processing method of the present embodiment. In the following, processing when the output image 410 is generated using the image 1100 shown in FIG. 11 as the input image 400 and added to and projected on the projected image data will be described with reference to the flowchart. The image 1100 is an image obtained by underlining a projected image by handwriting.

  In step S <b> 1001, the data specifying unit specifies the electronic data projected by the projection apparatus 103 when the imaging apparatus 105 captures the input image 1100. This data specification can be performed by a search technique such as a full text search using a character string on the assumption that the target file is stored in the PC 104. The key information for the search is obtained from an image obtained by correcting the projection area frame of the input image 1100 with projection distortion and a character string of the OCR result. Alternatively, the user may be prompted to input a file name of electronic data corresponding to the user, and may be specified using the input result. Alternatively, the projection data may be transmitted by communicating with the image processing apparatus 300 and the PC 104 that inputs an image to the projection apparatus 103.

  Subsequently, in steps S1002 to S1005, processing similar to that in steps S501 to S504 (FIG. 5) is performed. The coordinate values of the four vertices after conversion by the reverse perspective transformation in step S1004 are coordinate values (0, 0) and (w, 0) obtained from the width w and height h of the projection data specified in step S1001. , (W, h), (0, h) are used.

  When the projection distortion correction value and the white board distortion correction value are extracted for the projection area frame and the white board area frame by the processing of each step, the process proceeds to step S1006, and the contents of the specified projection data are arranged in the output image. For example, an image obtained by rendering the projection data and resized to the size of the output image may be pasted. Alternatively, the projection data may be separated into background and foreground objects and arranged at positions corresponding to each object.

  Subsequently, in step S1007, the object is divided in the same manner as in step S505 (FIG. 5), and then in step S1008, a process of repeating steps S1009 to S1014 for each divided or extracted object is controlled.

  In step S1009, it is determined whether there is an object that matches the target object in the projection data. The determination of coincidence or non-coincidence can be made based on the result of applying and correcting the projection distortion correction value for the object and the overlap of the projection data pixels identified in step S1005. Alternatively, a method of inferring from the position information of the object of the projection data can be used.

  FIG. 12 shows an example of detecting an object that matches the projection data. An image 1201 is obtained by correcting the input image 1100 with a projection distortion correction value, and an image 1202 is obtained by inverting the color of projection data. An image 1203 is obtained by adding the color information of the image 1201 and the image 1202, that is, by subtracting the color information of the projection data from the image 1201 to obtain a difference from the projection data.

  If it is determined in step S1009 that they match, since it has already been arranged on the output image in step S1006, the process proceeds to step S1015, which is a repeated termination. On the other hand, if it is determined that they do not coincide with each other, it is regarded as a handwritten object and the process proceeds to step S1010 to determine whether or not the object is a mark object such as an underline or a box written on the projected image. The determination as to whether or not the object is a mark object may be, for example, an object that is close to the projection object and that other than characters are mark objects. Specifically, an object having a size equal to or less than a constant and an aspect ratio close to 1 may be determined as a character, or a shape such as a straight line or an ellipse may be determined as a mark object.

  If it is determined that the object is a mark object, the process proceeds to step S1012, and if it is any other handwritten symbol, the process proceeds to step S1011. The image 1101 (FIG. 11) is determined to be a mark object.

  In steps S1011 to S1014, processing similar to that in steps S508 to S511 is performed. If there is an unprocessed object, the process returns to step S1008 to repeat the processing, and if there is no unprocessed object, the processing ends without repeating.

  FIG. 13 is a diagram showing an output image according to the present embodiment in comparison with an example of the prior art.

  Since the output image 1301 (FIG. 13A) according to the present embodiment uses the image data input to the projection apparatus 103 with respect to the projection data portion, it can be reproduced without deterioration with respect to the projection data portion.

  An output image 1302 (FIG. 13B) according to the conventional technique is an example of an output image as a result of correcting the shape of the mark object 1101 with the white plate distortion value, similarly to the character portion. In the output image 1302, the handwritten mark object 1101 overlaps the projection data portion.

  As described above, according to the present embodiment, there is no deterioration in the image quality of the projection data portion in a scene where the projection data can be specified. Also, by adding correction value determination processing for handwritten underlines and boxes, it is possible to add underline and boxes added by handwriting in an easy-to-see manner.

(Other examples)
The present invention can also be realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

Claims (9)

Input means for inputting image data projected object obtained by photographing the plate handwritten object is handwritten by the projected and the user by the projection device,
The inputted image data or, et al., A first frame constituting the four sides of the plate, a frame extracting means for extracting a second frame constituting the four sides of the projection area to be projected by the projection device,
To obtain a first distortion correction value based on the first frame, a correction value acquisition means for acquiring a second distortion correction value based on said second frame,
Wherein the input image data, an object extraction means for extracting said handwriting object and the projection object,
Performs distortion seen correction using the second distortion correction values with respect to the projection object performs distortion correction, which is the extraction with the first distortion correction values to the extracted handwritten objects and distortion correction means,
And output means for generating image data distortion correction using said distortion-corrected handwriting object and the distortion correction projection object,
An image processing apparatus comprising: The distortion correcting unit may further the handwritten object is determined whether or not the mark object attached to the projection object, if the handwritten object is determined as the mark object the second distortion correction value claim using the shape of the handwritten object is corrected, on the other hand, if the handwritten object is determined not to mark objects and corrects the shape of the handwritten object using the first distortion correction value The image processing apparatus according to 1.   The image processing apparatus according to claim 1, wherein the distortion correction unit corrects the positions of the handwritten object and the projection object using the second distortion correction value.   The object extraction means selects the handwritten object and the projection object using at least one of the color of each object, the fluctuation of the line of each object, and the unevenness of shading included in the input image data. The image processing apparatus according to any one of claims 1 to 3, wherein the image processing apparatus extracts the data.   The object extracting means selects and extracts the handwritten object and the projected object based on the position and size of an object existing in the vicinity of each object included in the input image data. The image processing apparatus according to any one of claims 1 to 3. The object extraction means specifies electronic data of the projection object projected by the projection device;
4. The image processing apparatus according to claim 1, wherein a handwritten object that does not exist in the electronic data is extracted from the input image data by comparison with the specified electronic data. And the output means, the image processing apparatus according to claim 6, characterized in that said distortion correcting means by generating the image data obtained by combining the distortion corrected electronic data and handwritten objects and the identified were output. An input step of inputting image data projected object obtained by photographing the plate handwritten object is handwritten by the projected and the user by the projection device,
The inputted image data or, et al., A first frame constituting the four sides of the plate, and a frame extraction step of extracting a second frame constituting the four sides of the projection area to be projected by the projection device,
To obtain a first distortion correction value based on the first frame, a correction value acquisition step of acquiring second distortion correction value based on said second frame,
Wherein the input image data, an object extraction step of extracting said handwriting object and the projection object,
Performs distortion seen correction using the second distortion correction values with respect to the projection object performs distortion correction, which is the extraction with the first distortion correction values to the extracted handwritten objects and distortion correction step,
An output step of generating image data distortion correction using said distortion-corrected handwriting object and the distortion correction projection object,
An image processing method comprising: Computer
Input means for inputting image data obtained by photographing a board on which a projection object is projected by a projection device and a handwritten object is handwritten by a user;
The inputted image data or al, the first frame and a frame extracting means for extracting a second frame constituting the four sides of the projection area to be projected by the projection device constituting the four sides of the plate,
The first the first distortion correction values obtained based on the frame, the correction value acquisition means for acquiring a second distortion correction value based on said second frame,
Object extraction means for extracting the projection object and the handwritten object from the input image data;
Performs distortion seen correction using the second distortion correction values with respect to the projection object performs distortion correction, which is the extraction with the first distortion correction values to the extracted handwritten objects Distortion correction means,
Output means for generating image data distortion correction using said distortion-corrected handwriting object and the distortion correction projection object,
A program characterized by functioning as

Images