JP2009060564A - Conference system - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a technology for composing a written image of an electronic blackboard device with a projection image of a projector, and to provide especially a conference system capable of designating a correct projection area even when a plurality of marks representing areas to be composed are detected. <P>SOLUTION: The system has: the electronic blackboard for reading an image written by a marker onto a writing sheet with a reader; a projector for projecting an image onto the writing sheet of the electronic blackboard; a projection area detecting means for calculating a projection area by detecting four write marks at an end part for representing an area of the projection image from the projector onto the writing sheet; and a composition means for composing the projection image with the written image written on the writing sheet to form a composite image. The projection area detecting means has a position estimating means for specifying four marks representing a position by calculating position relations among a plurality of detected written marks when four or more written marks are detected. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

The present invention relates to a system that combines and stores or prints a written image of an electronic blackboard device used in a meeting or the like and a projected image projected from a projector device.

The electronic blackboard device writes characters and figures with a marker on a white writing sheet, reads the written surface as image information with an optical sensor, reduces the read image information, prints it on paper, or records it. It has the function to perform.

The electronic blackboard device is originally written by the user with a marker, and it writes down the issues, solutions, concepts, schedules, plans, etc. reviewed at the conference. Is distributed as minutes, and this writing surface is read as image information and printed.

JP-A-4-181857 JP-A-10-297166

In a meeting that is performed while creating a minutes using an electronic blackboard device, a projector device may be used in combination.
For example, a document image created by a personal computer is enlarged and projected and referred to, and the conference proceeding is performed while writing the minutes of the conference on the electronic blackboard device.
In the case of such a meeting, conventionally, a projector device projects an image onto a dedicated projection screen, and writing of minutes and the like is performed on an electronic blackboard device or a whiteboard using a marker.

On the other hand, a technique for projecting a projector image on a writing surface of an electronic blackboard device and combining the projector image and the writing image for storage or printing has been studied for a long time.
In the above-mentioned patent document 1, a technique is shown in which an image written on an electronic blackboard device and a projected image are taken using a camera to create a composite image.
Patent Document 2 discloses a technique in which a projection area of a projector device is associated with a writing surface of an electronic blackboard device on a one-to-one basis, and writing data and projection data read by the electronic blackboard device are combined by a personal computer. ing.

However, in the case of creating a composite image using a camera as in Patent Document 1, the writer may overlap the shooting surface, and the composite image may be partially missing. In addition, it is necessary to secure the location of the camera, and a camera and its processing device are required, which increases the cost.

When the projection surface and the writing surface are in a one-to-one relationship as in Patent Document 2, the position where the projector device can be placed is limited, which causes a practical problem.

In addition, the applicant of the present application first detects the projection area by reading the position of the electronic blackboard by detecting the position mark by writing the four position marks on the writing surface. A patent application has been filed for a technique for combining an image and a projected image.
However, when a position mark is extracted from the writing surface in this way, for example, a writing that is not a position mark may be recognized as a position mark.
For example, there may be a case where simple circle writing or Roman letters a and o are detected as position marks.
For this reason, there is a case where the position for image synthesis cannot be calculated.

In order to solve the above problems, the present invention provides an electronic blackboard that reads an image written with a marker on a writing sheet by a reading device, a projector that projects an image on the writing sheet of the electronic blackboard, and the above-mentioned on the writing sheet. A projection area detecting means for calculating a projection area by detecting a writing mark at four end portions for indicating an area of the projection image from the projector, and the writing written on the projection image and the writing sheet; A conferencing system configured to synthesize an image and form a composite image, wherein the projection area detecting unit detects positions of a plurality of detected writing marks when four or more writing marks are detected. It is characterized by having position estimation means for specifying four points indicating positions by calculating the relationship.

According to the present invention, even when a plurality of writings are detected as position marks, it is possible to extract a correct position mark from the positional relationship of the detected marks.

Hereinafter, the best mode for carrying out the present invention will be described in detail with reference to the drawings.

An embodiment of the present invention will be described below with reference to the drawings.
FIG. 1 is a block diagram showing an embodiment of the present invention.
1 is an electronic blackboard main body, 2 is a writing sheet for writing on the surface of the main body with a marker, and 3 is an instruction for moving the writing sheet and scanning the writing surface of the writing sheet to read written image information. An operation panel for giving instructions or instructions for printing the read data with a printer (not shown), 4 is a projector, and 5 is a personal computer (hereinafter referred to as a personal computer).

The projector 4 is connected to the personal computer 5 and enlarges and projects the presentation image data created by the personal computer 5.
The personal computer 5 receives handwritten image data connected to the electronic blackboard 1 and read by the electronic blackboard.

Under such a configuration, the image data for projection displayed on the screen of the personal computer 5 is projected onto the writing sheet 2 of the electronic blackboard 1 via the projector 4.
The writing sheet 2 is written with a marker in a form to be added to the projected image.
When it is desired to synthesize the projected image and the written image that has been added, the writer writes the four points on the right and left and upper and lower edges of the region to be synthesized of the projected image with markers.
In the case of the present embodiment, green circles 6 are entered as position marks on the upper and lower ends of the left and right of the region where the projection image is to be synthesized.

The operation of creating a composite image will be described using the flowchart of FIG.
A composite image is created by starting a composite image creation program on a personal computer.
FIG. 7 shows an example of the operation window of the composite image creation program.
21 is a scan button for creating a composite image, 22 is a button for setting the color and shape of a position mark for setting the composite position of the projection image, and 23 is a setting for selecting a projection projector, for example. Button.
The initial position mark has a green round shape, but this can be changed to a red square shape by the mark button 22, for example.

Operation 1
Pressing the image capture button (hereinafter referred to as the scan button) on the operation panel of the composite image creation program displayed on the computer screen displays a window for selecting whether or not the projector is in use. .
When “Do not use projector” is selected in this window, it is not necessary to synthesize the projected image of the projector, so the electronic blackboard operates normally and the image is read after the writing surface is read by the reading device. Data is transferred to the computer.

Operation 2
When “using projector” is selected in the previous selection window, the bitmap image displayed on the personal computer screen is saved as a file.
Only the screen image with the operation panel displayed on the personal computer screen turned off is saved as a bitmap image file.
Such an operation is generally called screen capture.
The screen file may be saved in a format other than the bitmap.

Action 3
The electronic blackboard control signal is transmitted from the personal computer to the control unit of the electronic blackboard by the TWAIN software, the writing sheet of the electronic blackboard is transferred, and the written image is read by a reading device arranged on the back side of the electronic blackboard (not shown). It is.

Operation 4
The written image read on the electronic blackboard is transferred to the personal computer by the TWAIN software.
TWAIN is a technical specification for connecting a peripheral device such as an image input device and a personal computer.

Action 5
It is determined whether or not the handwritten image data from the electronic blackboard has been transferred correctly. If it fails, the bitmap image file of the personal computer screen saved in operation 2 is discarded, and the composite image creation operation is terminated.
An error message is displayed on the personal computer screen to prompt the writer to scan again.

Action 6
When correctly transmitted, if “Do not use projector” was selected in operation 1, a selection screen for selecting whether to save the read handwritten image data is displayed, and “Save” is selected. Ends the save process.

Action 7
When “use projector” is selected in the operation 1, four position marks are detected from the received handwritten image data.
The bitmap image saved in operation 1 is reduced and combined in the area surrounded by the four position marks, and this combined image is displayed in a display window as shown in FIG.
In the synthesized image display window shown in FIG. 5, the operator can correct the size and shape of the projected image, which will be described later.

Next, the position detection operation of the previous operation 7 will be described with reference to FIG.
Operation 71
When the position mark is written in a color other than green, the color is selected, and when the position mark is other than a round shape, for example, a square, the color and shape can be set by the operator using the setting window. .
Position detection is performed based on the set color and shape position marks.
This operation is referred to as color conversion filter setting.
If the position mark is written in a green circle shape with the initial setting, this operation is unnecessary.

Action 72
A pixel in which the position mark set in the color conversion filter setting is written is detected, and the pixel is stored in the image memory. The detected pixel is set to 1 and the other portions are stored as 0.
This operation is called a color conversion filter.

Action 73
In order to remove erroneously detected data due to dirt on the writing sheet or the like, a portion where the number of consecutive pixels is less than or equal to a predetermined number from the converted image data of the image memory is set to zero.
A rising edge and a falling edge of a predetermined number or more of green region images are extracted in the X-axis direction and the Y-axis direction orthogonal thereto.
This edge is filled with 1.
By this operation, the inside of the round shape is filled with 1 to form one region (hereinafter referred to as a point).
This operation is called an image filter.

The positional relationship of the plurality of position points detected in the previous operation 73 is analyzed.
This positional relationship analysis is referred to as image analysis.
A flowchart of this image analysis is shown in FIG.

Action 74
If the number of detected points is 4 points, it is determined whether the interval between the points and the angle of the line segment connecting the points are within a predetermined value, and if they are within the predetermined value, the 4 points TRUE is returned to make the region composed of
If the predetermined value is exceeded, a detection error is assumed and FALSE is returned.

Action 75
When the number of detected points is 4 points or more, the optimum 4 points are extracted by the position detecting means 8.
First, from the detected points, all combinations of four points that are at a distance greater than or equal to a predetermined value in the vertical and horizontal directions are acquired.
Subsequently, a priority order is calculated for all acquired combinations, and the combination with the highest priority order is used as position information.
The priority order is calculated based on the degree of approximation, aspect ratio, etc., of the shape composed of four points with the rectangle.

For example, as shown in FIG. 8, when the Roman letters a, b, and o are detected as the third, fourth, and seventh points, respectively, four points that are at a distance greater than or equal to a predetermined value in the vertical and horizontal directions. (1, 2, 5, 6), (1, 2, 5, 7), (1, 3, 5, 6), (1, 3, 5, 7) are acquired as combinations of.
The degree of approximation and aspect ratio of this combination with the rectangle are calculated.
The combination of point 1 and point 2 has a higher priority than point 1 and point 3 because of the difference in the vertical distance between point 1 and point 2, and point 1 and point 3.
Also, the combination of point 2 and point 6 has a higher priority than point 2 and point 7 due to the difference in lateral distance between point 2 and point 6 and point 2 and point 7.
Furthermore, the aspect ratio of (1, 2, 5, 6) has a higher priority than the aspect ratio of (1, 3, 5, 7).
By such an operation, (1, 2, 5, 6) is estimated as position information as the combination having the highest priority, and TRUE is returned.

Action 76
When the number of detected points is 3, the virtual fourth point is set from the positional relationship of the detected 3 points.
It is determined whether the interval between the four points including the virtual points and the angle of the line segment connecting the points are within a predetermined value. If the angle is within the predetermined value, a region composed of the four points is defined as a projected image synthesis region. Therefore, TRUE is returned.
If the predetermined value is exceeded, a detection error is assumed and FALSE is returned.

Based on the value returned from the image analysis program, TRUE, or FALSE, it is determined whether the image analysis is successful.
This operation is indicated by “Successful image analysis?” In FIG.

When an area composed of four points is detected by the image analysis program, this acquisition position is set as the display position of the projection image and is set as a projection image synthesis area.
If an area composed of four points cannot be specified, the initially set position is set as the display position of the projection image and is set as the projection image synthesis area.

FIG. 5 shows a state where the synthesized image is displayed.
In this display window, the operator can manually adjust the size of the projection image.
In the case of adjusting the size, adjustment marks are displayed on the four sides and the four corners of the projected image synthesis area as shown in FIG.
When the four side marks are moved, the area is enlarged or reduced in that direction.
When the marks at the four corners are moved, the shape is deformed in that direction by affine transformation.

In the present embodiment, the written position mark is synthesized as it is. However, this position mark is used only for specifying the synthesis area, and is an essentially unnecessary writing.
For this reason, it may be desired to delete this position mark from the composite image.
In order to erase the position mark, for example, it can be achieved by erasing all the images of the color of the designated position mark.
In this case, other writing is performed using the electronic blackboard apparatus in which writing is performed using a marker of a color other than that specified by the position mark.
Further, the deletion of the position mark can also be achieved by converting all the image data in the vicinity of the detected position to white based on the position information when the position mark is detected first.

The present invention is not limited to the above-described embodiments. For example, the image composition program, the image analysis, and the position detection method can be changed.
In addition, for example, in order to prevent an operator from erroneously writing a position mark other than the set shape and color, the set position mark is added to the image projected from the projector and projected. Also good.
When the writer enters the position mark on the writing sheet so as to trace the projected position mark, the position mark can be detected more stably.
Further, the operation screen and the window for composite display may have a different design from that shown above.
Further, the composite image may be created by a remote controller dedicated to the electronic blackboard instead of using a personal computer.

According to the present invention, it is possible to synthesize writing data of an electronic blackboard and projection data of a projector without adding new components to the electronic blackboard or projector, and a new conference system can be constructed. .

FIG. 1 is a front view showing a configuration according to an embodiment of the present invention. FIG. 2 is a flowchart showing an operation for creating a composite image according to the embodiment of the present invention. FIG. 3 is a flowchart showing the position detection operation of the embodiment of the present invention. FIG. 4 is a flowchart showing the image analysis operation of the embodiment of the present invention. FIG. 5 is a design diagram showing a composite image display window according to the embodiment of the present invention. FIG. 6 is a design diagram for explaining an operation for changing the display area of the composite image according to the embodiment of the present invention. FIG. 7 is a design diagram showing an operation window according to the embodiment of the present invention. FIG. 8 is a diagram of a writing surface showing an example when a plurality of writing marks are detected as position marks in the embodiment of the present invention.

Explanation of symbols

1: electronic blackboard 2: writing sheet 4: projector 5: personal computer 6: position mark 8: position estimating means

Claims (3)

An electronic blackboard that reads an image written by a marker on a writing sheet with a reading device, a projector that projects an image on the writing sheet of the electronic blackboard, and a region of a projected image from the projector on the writing sheet Projection area detection means for calculating a projection area is detected by detecting the writing marks at the four end portions, and the projection image and the writing image written on the writing sheet are synthesized to form a composite image. A projection system, wherein when the four or more writing marks are detected, the projection area detecting means calculates a positional relationship among the plurality of detected writing marks, thereby indicating a position. And a position estimation means for specifying four points. 2. The electronic blackboard apparatus according to claim 1, wherein the synthesizing unit arranges the projection image in a predetermined area and synthesizes the writing mark when the writing mark is not detected. 4. The conference system according to claim 1, wherein the synthesizing unit erases the writing mark to form a synthesized image.

Images