JP2021060781A - Image display system, image processing system and image processing program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide an image display system, an image processing device and an image processing program capable of realizing a display according to a speaker's explanation. An image forming apparatus of an image display system includes display data including a plurality of display images displayed by the image display device and a switching range for receiving an instruction to switch to another display image in each of the display images. The image display unit is provided with an image display unit that displays an image on the image display device based on In the transmitted captured image, when the indicator indicating the switching range is within the switching range in the currently displayed display image (YES in S142), the display image corresponding to this switching range is displayed on the image display device. It is characterized in that the display image to be displayed is switched (S144 or S145). [Selection diagram] Fig. 9

Description

The present invention relates to an image display system, an image processing device, and an image processing program for displaying an image.

As a conventional image display system, an electronic blackboard that displays the contents of handwritten input by a user by a projector is known (see, for example, Patent Document 1).

Japanese Unexamined Patent Publication No. 2012-240282

In the conventional image display system, it is conceivable to display the contents of the past board writing. However, when simply displaying the contents of the past board writing, there is a possibility that parts other than the part being explained by the speaker who is explaining using the displayed contents may be displayed, and the contents of the speaker's explanation may be displayed. There is a problem that the listener may not be able to concentrate.

Therefore, an object of the present invention is to provide an image display system, an image processing device, and an image processing program that can realize a display according to a speaker's explanation.

The image display system of the present invention includes an image display device for displaying an image, a photographing device for photographing an image display area in the image display device, and an image processing device for processing an image. An image display unit for displaying an image on the image display device based on display data for display by the image display device is provided, and the display data includes a plurality of display images displayed by the image display device and a plurality of display images. Each of the display images includes a switching range for receiving an instruction to switch to the other display image, and the image display unit can be used from the photographing device when the display image is displayed on the image display device. In the transmitted captured image, when the indicating unit indicating the switching range is within the switching range in the display image currently being displayed, the display image corresponding to the switching range is displayed on the image display device. It is characterized in that the display image to be caused is switched.

With this configuration, in the image display system of the present invention, when the display image is displayed on the image display device, the display image currently being displayed by the indicator indicating the switching range in the photographed image transmitted from the image display device. When the image is within the switching range in, the display image to be displayed on the image display device is switched to the display image corresponding to this switching range, so that the position of the indicator designated by the speaker for the display image to be displayed on the image display device. As a result, the display according to the speaker's explanation can be realized.

In the image display system of the present invention, the image processing device includes a display data generation unit that generates the display data, and the display data generation unit refers to an image of a board written by the photographing device. The display image and the designation of the switching range may be accepted, and the display data may be generated based on the accepted designation.

With this configuration, the image display system of the present invention accepts the designation of the display image and the switching range for the image of the board written by the photographing device, and generates the display data based on the accepted designation. It is possible to generate display data according to the designation by, and as a result, it is possible to realize the display according to the explanation of the speaker.

The image processing device of the present invention includes an image display unit for displaying an image on the image display device based on display data for display by the image display device, and the display data is displayed by the image display device. The image display unit displays the display image on the image display device, which includes a plurality of display images and a switching range in which each of the display images receives an instruction to switch to the other display image. In the case, when the indicating unit indicating the switching range is within the switching range of the displayed image currently being displayed in the captured image transmitted from the photographing device that captures the display area of the image in the image display device. The display image to be displayed on the image display device is switched to the display image corresponding to this switching range.

With this configuration, in the image processing device of the present invention, when the display image is displayed on the image display device, the indicator unit is set in the switching range of the currently displayed display image in the photographed image transmitted from the image display device. When it is on, the display image to be displayed on the image display device is switched to the display image corresponding to this switching range, so the display image to be displayed on the image display device is switched according to the position of the indicator specified by the speaker. As a result, the display according to the speaker's explanation can be realized.

The image processing program of the present invention causes a computer to realize an image display unit for displaying an image on the image display device based on display data for display by the image display device, and the display data is the image display device. The image display unit displays the display image on the image display device, including a plurality of display images displayed by the display image and a switching range in which each of the display images receives an instruction to switch to the other display image. In the captured image transmitted from the photographing device that captures the display area of the image in the image display device, the indicator indicating the switching range enters the switching range in the displayed image currently being displayed. At that time, the display image to be displayed on the image display device is switched to the display image corresponding to this switching range.

With this configuration, when the computer that executes the image processing program of the present invention displays the displayed image on the image display device, the switching range of the currently displayed display image in the captured image transmitted from the photographing device is changed. When the indicator is included in, the display image to be displayed on the image display device is switched to the display image corresponding to this switching range, so the position of the indicator specified by the speaker for the display image to be displayed on the image display device. As a result, the display according to the speaker's explanation can be realized.

The image display system, the image processing device, and the image processing program of the present invention can realize the display according to the explanation of the speaker.

It is a block diagram of the image display system which concerns on one Embodiment of this invention. It is a block diagram of the image forming apparatus shown in FIG. (A) It is a front view of an example of a board in a state where a board writing is written, which is photographed by the photographing apparatus shown in FIG. (B) It is a figure which shows an example of the board writing image generated by photographing the board shown in FIG. 3 (a). (A) It is a figure which shows an example of the display image which is displayed by the image display apparatus shown in FIG. 1, and the display order is 1st. (B) It is a figure which shows an example of the display image which is displayed by the image display apparatus shown in FIG. 1, and the display order is No. 2. (C) It is a figure which shows an example of the display image which is displayed by the image display apparatus shown in FIG. 1, and the display order is No. 3. (A) It is a figure which shows the area displayed by the display image shown in FIG. 4 (a) among the board writing images shown in FIG. 3 (b). (B) It is a figure which shows the area displayed by the display image shown in FIG. 4 (b) among the board writing images shown in FIG. 3 (b). (C) It is a figure which shows the area displayed by the display image shown in FIG. 4 (c) in the board writing image shown in FIG. 3 (b). It is a sequence diagram of the operation of the image display system shown in FIG. 1 when the board writing image is registered. It is a flowchart of the operation of the image forming apparatus shown in FIG. 2 in the case of generating display data. It is a front view of the blackboard on which the display image shown in FIG. 4A is displayed. It is a flowchart of the operation of the image forming apparatus which shows FIG. It is a figure which shows an example of the image taken by the photographing apparatus shown in FIG. (A) It is a figure which shows an example different from the example shown in FIG. 4A of the display image which the display order is No. 1. (B) It is a figure which shows an example which is different from the example shown in FIG. 4A and FIG. 11A of the display image in which the display order is No. 1.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.

First, the configuration of the image display system according to the embodiment of the present invention will be described.

FIG. 1 is a block diagram of the image display system 10 according to the present embodiment.

As shown in FIG. 1, the image display system 10 includes an image display device 11 for displaying an image, a photographing device 12 for photographing an image display area in the image display device 11, for example, a camera, and an MFP for processing an image. It is provided with an image forming apparatus 20 as an image processing apparatus such as (Multifaction Peripheral).

The image display device 11 is a projector that projects an image onto a screen such as a blackboard or a whiteboard.

The photographing device 12 may be mounted on the image display device 11.

FIG. 2 is a block diagram of the image forming apparatus 20.

As shown in FIG. 2, the image forming apparatus 20 includes an operation unit 21 which is an operation device such as a button into which various operations are input, and a display device such as an LCD (Fixed Crystal Display) which displays various information. Display unit 22, a printer 23 which is a printing device for printing an image on a recording medium such as paper, a scanner 24 which is a reading device for reading an image from a document, an external facsimile machine and a public telephone line (not shown) A fax communication unit 25, which is a fax device that performs fax communication via a communication line such as, and a network such as LAN (Local Area Network), the Internet, or directly by wire or wireless without a network, externally. The communication unit 26, which is a communication device that communicates with the device, the storage unit 27, which is a non-volatile storage device such as a semiconductor memory or HDD (Hard Disk Drive) that stores various information, and the entire image forming device 20. It includes a control unit 28 for controlling.

The storage unit 27 can store an image processing program 27a for processing an image. The image processing program 27a may be installed in the image forming apparatus 20 at the manufacturing stage of the image forming apparatus 20, for example, or the image forming apparatus 20 may be installed in the image forming apparatus 20 from an external storage medium such as a USB (Universal Serial Bus) memory or an SD card. It may be additionally installed in the image forming apparatus 20, or it may be additionally installed in the image forming apparatus 20 from the network.

The storage unit 27 can store the board writing image 27b as an image of the board writing written on a board such as a blackboard or a whiteboard. The storage unit 27 can store at least one board writing image in addition to the board writing image 27b.

FIG. 3A is a front view of an example of the board 30 in a state where the board writing 31 is written. FIG. 3B is a diagram showing an example of a board writing image.

The board 31 shown in FIG. 3 is just an example. Although the photographing area 32 by the photographing apparatus 12 is shown in FIG. 3A, the photographing area 32 is not drawn on the actual board 30. The board-written image shown in FIG. 3B is an image corresponding to the photographing region shown in FIG. 3A.

As shown in FIG. 2, the storage unit 27 can store display data 27c for display by the image display device 11. The storage unit 27 can store at least one display data in addition to the display data 27c. The display data 27c includes the board image correspondence information 27d indicating the board image corresponding to the display data 27c, and the display image 27e displayed by the image display device 11. The display data 27c can include at least one display image in addition to the display image 27e. The display data 27c is an instruction to switch to another display image in each of the display order information 27f indicating the display order of the display images included in the display data 27c and the display image included in the display data 27c. Includes 27 g of switching range information indicating a range (hereinafter referred to as “switching range”) for accepting.

FIG. 4A is a diagram showing an example of a display image in which the display order is No. 1. FIG. 4B is a diagram showing an example of a display image in which the display order is No. 2. FIG. 4C is a diagram showing an example of a display image in which the display order is No. 3. FIG. 5 (a) is a diagram showing a region displayed by the display image shown in FIG. 4 (a) among the board-written images shown in FIG. 3 (b). FIG. 5 (b) is a diagram showing a region displayed by the display image shown in FIG. 4 (b) among the board-written images shown in FIG. 3 (b). FIG. 5 (c) is a diagram showing a region displayed by the display image shown in FIG. 4 (c) among the board-written images shown in FIG. 3 (b).

As shown in FIGS. 4 and 5, the display image includes a region other than the region 51 displayed by the display image in the board writing image. As shown in FIG. 5, the switching range includes the previous page switching range 52a for switching to the display image one before the display order and switching to the display image one display order after. There is a subsequent page switching range 52b. However, as shown in FIG. 5A, the display image in which the display order is first, that is, first, does not include the switching range on the previous page. Further, as shown in FIG. 5C, the display image in which the display order is the last does not include the subsequent page switching range.

The control unit 28 shown in FIG. 2 includes, for example, a CPU (Central Processing Unit), a ROM (Read Only Memory) that stores programs and various data, and a memory used as a work area of the CPU of the control unit 28. RAM (Random Access Memory) is provided. The CPU of the control unit 28 executes a program stored in the storage unit 27 or the ROM of the control unit 28.

The control unit 28 includes a display data generation unit 28a that generates display data by executing the image processing program 27a, and an image display unit 28b that causes the image display device 11 to display an image based on the display data. Realize.

Next, the operation of the image display system 10 will be described.

First, the operation of the image display system 10 when registering a board writing image will be described.

FIG. 6 is a sequence diagram of the operation of the image display system 10 when registering a board writing image.

As shown in FIG. 6, the photographing device 12 generates a board writing image by photographing the board writing, for example, in response to an instruction by the user (S101).

Next, the photographing apparatus 12 transmits the board writing image generated in S101 to the image forming apparatus 20 (S102).

Next, the display data generation unit 28a of the image forming apparatus 20 stores the board writing image transmitted in S102 in the storage unit 27 (S103).

As described above, the board writing image is registered in the image forming apparatus 20.

Next, the operation of the image forming apparatus 20 when generating display data will be described.

FIG. 7 is a flowchart of the operation of the image forming apparatus 20 when generating display data.

The user can instruct the image forming apparatus 20 to generate display data, for example, via the operation unit 21. The instruction for generating display data includes designation of the board image to be generated for display data among the board images stored in the storage unit 27. When the control unit 28 of the image forming apparatus 20 is instructed to generate display data, the control unit 28 executes the operation shown in FIG. 7.

As shown in FIG. 7, the display data generation unit 28a displays the board writing image specified in the instruction for generating the display data on the display unit 22 (S121).

Next, the display data generation unit 28a inputs to, for example, the operation unit 21 the designation of the area displayed by the display image in the board writing image, the display order of the display image, and the switching range in the display image. Accepts each display image according to the instruction (S122).

Next, the display data generation unit 28a generates a display image based on the area received in S122 (S123).

For example, the display data generation unit 28a is based on the regions 51 shown in FIGS. 5 (a), 5 (b), and 5 (c), respectively, in FIGS. 4 (a), 4 (b), and FIG. The display image shown in 4 (c) is generated.

As shown in FIG. 7, the display data generation unit 28a displays the display order and switching range received in S122 after the processing in S123 as the attribute information of the display image generated in S123, which is the board image displayed in S121. The display data of (S124) is generated.

Next, the display data generation unit 28a stores the display data generated in S124 in the storage unit 27 (S125), and ends the operation shown in FIG. 7.

Next, the operation of the image display system 10 when displaying the display image will be described.

FIG. 8 is a front view of the blackboard on which the display image shown in FIG. 4A is displayed.

The user may instruct the image forming apparatus 20 to start displaying the displayed image via, for example, an operation unit 21, an image displaying device 11, or a computer such as a PC (Personal Computer) (not shown). it can. The instruction to start displaying the display image includes the designation of the board image to be displayed as the display image among the board images stored in the storage unit 27. When the image display unit 28b of the image forming apparatus 20 is instructed to start displaying the display image, the board image correspondence information indicates that the image display unit 28b corresponds to the board image specified in the instruction to start displaying the display image. Among the display images included in the display data (hereinafter referred to as “target display data”), the display image having the first display order is transmitted to the image display device 11. Therefore, as shown in FIG. 8, the image display device 11 displays the display image transmitted from the image forming device 20.

The photographing device 12 transmits an image obtained by photographing the screen on which the image is projected by the image display device 11 (hereinafter referred to as “photographed image”) to the image forming device 20 at regular time intervals.

FIG. 9 is a flowchart of the operation of the image forming apparatus 20 when switching the display image to be displayed on the image display apparatus 11.

The control unit 28 of the image forming apparatus 20 performs the operation shown in FIG. 9 every time the captured image is transmitted from the photographing device 12 after transmitting the display image having the first display order to the image display device 11. Execute.

As shown in FIG. 9, the image display unit 28b determines the position of the user's hand as an instruction unit indicating the switching range by executing image processing on the captured image transmitted from the photographing device 12. It is specified in the captured image (S141).

FIG. 10 is a diagram showing an example of an image 60 captured by the photographing device 12.

The captured image 60 shown in FIG. 10 is an example of an image captured by the photographing device 12 on the blackboard on which the display image shown in FIG. 4A is displayed. The captured image 60 includes the user's hand 61.

As shown in FIG. 9, after the processing of S141, the image display unit 28b sets the position specified in S141 to the switching range in the currently displayed display image based on the position of the currently displayed display image in the captured image. It is determined whether or not it is contained (S142).

When the image display unit 28b determines in S142 that the position specified in S141 is within the switching range in the currently displayed display image, the position specified in S141 is the previous page switching range and the rear in the currently displayed display image. It is determined which of the page switching positions it is in (S143).

When the image display unit 28b determines in S143 that the position specified in S141 is within the switching range of the previous page in the display image currently being displayed, the display image among the display images included in the target display data is displayed. The display image whose order is immediately before the display image currently being displayed is transmitted to the image display device 11 (S144). Therefore, the image display device 11 displays the display image transmitted from the image forming device 20.

When the image display unit 28b determines in S143 that the position specified in S141 is in the subsequent page switching position in the display image currently being displayed, the display image among the display images included in the target display data is displayed. A display image whose order is one after the display image currently being displayed is transmitted to the image display device 11 (S145). Therefore, the image display device 11 displays the display image transmitted from the image forming device 20.

When the image display unit 28b determines in S142 that the position specified in S141 is not within the switching range in the display image currently being displayed, or ends the processing of S144 or S145, the operation shown in FIG. 9 ends.

As described above, in the image display system 10, when the display image is displayed on the image display device 11, the hand 61 indicating the switching range is currently being displayed in the captured image transmitted from the photographing device 12. When the display image is within the switching range (YES in S142), the display image to be displayed on the image display device 11 is switched to the display image corresponding to this switching range (S144 or S145). The display image to be displayed can be switched according to the position of the hand 61 designated by the speaker, and as a result, the display according to the speaker's explanation can be realized.

Since the image display system 10 accepts the designation of the display image and the switching range for the image of the board written by the photographing device 12 (S122), and generates the display data based on the accepted designation (S123 to S124). , Display data can be generated according to the designation by the speaker, and as a result, the display according to the speaker's explanation can be realized.

The image display system 10 accepts the designation of the switching range in the present embodiment, but when the board writing is vertical writing, the specific range on the upper right in the area displayed by the displayed image in the board writing image is switched to the previous page. It may be automatically specified as a range, or a specific range at the lower left in the area displayed by the displayed image in the board writing image may be automatically specified as a subsequent page switching range. Similarly, when the board writing is horizontal writing, the image display system 10 may automatically specify a specific range on the upper left of the area displayed by the displayed image in the board writing image as the previous page switching range. A specific range at the lower right in the area displayed by the display image of the board image may be automatically specified as the subsequent page switching range. The image display system 10 may automatically specify the switching range and then respecify the switching range by the user.

FIG. 11A is a diagram showing an example of the display image in which the display order is No. 1 different from the example shown in FIG. 4A. FIG. 11B is a diagram showing an example of the display image in which the display order is No. 1 different from the examples shown in FIGS. 4 (a) and 11 (a).

As shown in FIG. 4, the display image is an image obtained by hollowing out a region other than the region to be displayed in the board-written image in the present embodiment. However, as shown in FIG. 11A, the display image may be an image in which a region other than the display target region of the board writing image is filled with the background color of the board writing image. Further, as shown in FIG. 11B, the display image may be an image obtained by hollowing out an area other than the board writing in the area to be displayed in the board writing image.

When displaying the display image on the image display device 11, the image display unit 28b acquires the screen color of the image display device 11 based on the image captured by the photographing device 12, and displays the display image according to the acquired color. You may change the color of. For example, when the display image is generated based on the blackboard board image, the image display unit 28b may invert the color of the display image when the whiteboard is adopted as the screen by the image display device 11.

The indicator of the present invention is a hand in the present embodiment, but it may be something other than a hand, such as the tip of a pointer.

The image display device 11 is a projector in the present embodiment, but a display device other than the projector, such as a large LCD, may be used.

The image processing apparatus of the present invention is the image forming apparatus 20 in the present embodiment, but a computer other than the image forming apparatus such as a PC may be used.

10 Image display system 11 Image display device 12 Imaging device 20 Image forming device (image processing device, computer)
27a Image processing program 27b Board writing image (board writing image)
27c Display data 27e Display image 27g Switching range information (information indicating the switching range)
28a Display data generation unit 28b Image display unit 31 Board writing 52a Previous page Switching range (switching range)
52b Subsequent page switching range (switching range)
60 Captured image 61 Hand (indicator)

Claims (4)

An image display device that displays images and
An imaging device that captures an image display area in the image display device, and
Equipped with an image processing device that processes images
The image processing device includes an image display unit that displays an image on the image display device based on display data for display by the image display device.
The display data is
A plurality of display images displayed by the image display device, and
Each of the display images includes a switching range for receiving an instruction to switch to the other display image.
When the display image is displayed on the image display device, the image display unit is in the display image currently being displayed by an instruction unit indicating the switching range in the photographed image transmitted from the image display device. An image display system comprising switching the display image to be displayed on the image display device to the display image corresponding to the switching range when the switching range is entered. The image processing device includes a display data generation unit that generates the display data.
The display data generation unit is characterized in that it accepts the designation of the display image and the switching range for the image of the board written by the photographing apparatus, and generates the display data based on the accepted designation. The image display system according to claim 1. An image display unit for displaying an image on the image display device based on display data for display by the image display device is provided.
The display data is
A plurality of display images displayed by the image display device, and
Each of the display images includes a switching range for receiving an instruction to switch to the other display image.
When the display image is displayed on the image display device, the image display unit indicates the switching range in the captured image transmitted from the photographing device that captures the display area of the image in the image display device. An image process characterized by switching the display image to be displayed on the image display device to the display image corresponding to the switching range when the instruction unit is within the switching range of the display image currently being displayed. apparatus. A computer is realized with an image display unit that displays an image on the image display device based on display data for display by the image display device.
The display data is
A plurality of display images displayed by the image display device, and
Each of the display images includes a switching range for receiving an instruction to switch to the other display image.
When the display image is displayed on the image display device, the image display unit indicates the switching range in the captured image transmitted from the photographing device that captures the display area of the image in the image display device. An image process characterized by switching the display image to be displayed on the image display device to the display image corresponding to the switching range when the instruction unit is within the switching range of the display image currently being displayed. program.

Images