JP7599001B2 - Augmented reality display system - Google Patents

Description

The present invention relates to an augmented reality image display system that can create and display a synthetic image in which at least a portion of a designed object shown in an image is replaced with a virtual design.

With recent developments in information technology, systems have been proposed that display images in which virtual images are overlaid on actual images (augmented reality images).

For example, Patent Document 1 discloses a periscope that appropriately processes images of various viewing objects such as landscapes and provides them to users, and that includes an imaging means for capturing images of the viewing object, a storage means for storing virtual images of objects whose locations are specified within the viewing object in association with the viewing object, and an image synthesis means for extracting from the storage means a virtual image corresponding to the captured image of the viewing object and synthesizing and displaying it, thereby enabling the periscope to quickly and reliably display images of objects such as ruins within the actual view.

On the other hand, a sales method has been proposed in which a large amount of image data of pictures, etc. is stored and distributed via a communication network such as the Internet, the distributed image data is printed on clothing such as T-shirts, and the printed clothing is then sold.

For example, Patent Document 2 discloses a clothing sales method in which clothes capable of displaying images are made available for trying on by loaning them out in a sales floor that has a full-length mirror or other such mirror, and a large number of image data are provided in that sales floor or a wider area for selection of items to be tried on using electronic data processing, and one or more image data items selected as items to be tried on are used to display images of the clothes to be tried on, and the image data selected as items to be purchased is printed on clothing such as T-shirts, and the printed clothing is sold.

JP 2003-264740 A JP 2003-41420 A

However, conventional systems for displaying augmented reality images, such as the periscope mentioned above, had the problem that the virtual reality images superimposed on the real images were fixed and could not be easily changed.

However, the above-mentioned clothing sales method has the problem that it is necessary to prepare try-on clothes capable of displaying images. In addition, there is a problem that, since people have various body shapes, it is practically difficult to prepare try-on clothes capable of displaying images to fit these body shapes. Furthermore, there is a problem that the designs printed on the clothing, etc. are limited to those prepared in advance. In addition, there is a problem that the designs cannot be changed. Another problem is that such a system is large and cannot be installed in a space-saving manner.

In consideration of the above-mentioned circumstances, the present invention aims to provide an augmented reality image display system that can easily change the virtual reality design of an object whose design is to be changed, and can display an augmented reality image using the same.

As a result of extensive research and development into the problems mentioned above, the inventors of the present invention have discovered the following groundbreaking augmented reality image display system.

The first aspect of the present invention for solving the above problem is an augmented reality image display system comprising a photographing unit for photographing an object to be designed, an augmented reality image synthesis unit for replacing at least a part of the image of the object to be designed photographed by the photographing unit with a virtual design to synthesize an augmented reality image, and a display unit for displaying the augmented reality image, in which the object to be designed has, on at least a part of its surface, a virtual design replacement unit which is replaced with the virtual design when displayed on the display unit, and an identification unit which indicates that the image includes the object to be designed when photographed by the photographing unit, and the augmented reality image synthesis unit synthesizes the virtual design a specific part confirmation unit that confirms whether a specific part is included in an image of the designed object photographed by the photographing unit; a specific part judgment unit that judges whether the specific part confirmed by the specific part confirmation unit has been photographed so as to satisfy a predetermined judgment condition; a virtual design extraction unit that extracts a virtual design from the virtual design storage unit based on the judgment of the specific part judgment unit; and a synthetic image creation unit that creates an augmented reality image by replacing the virtual design replacement unit with the virtual design extracted by the virtual design extraction unit.

Here, "augmented reality image" refers to an augmented reality image obtained by overlaying a virtual reality image on a real image.

According to this first aspect, the virtual design of the object to be designed can be easily changed, and an augmented reality image using the same can be displayed.

A second aspect of the present invention is an augmented reality image display system as described in the first aspect, characterized in that the specified judgment condition is that the specified part is not confirmed by the specified part confirmation unit for a specified period of time, and the virtual design extraction unit extracts a virtual design from the virtual design storage unit in accordance with the specified extraction condition.

According to this second aspect, the virtual design of the object to be designed can be changed more easily, and an augmented reality image using the same can be easily displayed.

A third aspect of the present invention is an augmented reality image display system described in the first or second aspect, characterized in that the composite image creation unit has a center position calculation unit that calculates the position of the center of the virtual design replacement part in the image of the designed object captured by the capture unit, an arrangement unit that arranges the center of the virtual design at the center position of the virtual design replacement part calculated by the center position calculation unit, a length detection unit that detects the length of a specified part of the virtual design replacement part in the image of the designed object captured by the capture unit, an adjusted virtual design creation unit that creates an adjusted virtual design by adjusting the length of the virtual design corresponding to the specified part based on the length of the specified part of the virtual design replacement part, and a synthesis unit that replaces the virtual design replacement part with the adjusted virtual design and synthesizes an augmented reality image.

According to the third aspect, it is possible to easily create an augmented reality image that matches the shape and size of the virtual design replacement part of the design subject photographed by the photographing unit. As a result, by viewing the augmented reality image, for example, a person wearing or holding up the T-shirt can clearly imagine what it would be like to wear the T-shirt with the virtual design.

A fourth aspect of the present invention is an augmented reality image display system described in any one of the first to third aspects, further comprising an invisible light irradiating unit capable of irradiating invisible light, and the specific part confirmation unit detects invisible light reflected by the specific part and determines whether a specific part is included based on the area of the part that reflects the invisible light.

According to this fourth aspect, it is possible to detect specific parts more accurately and without being affected by the external environment than when detecting specific parts based on color, shape, etc.

A fifth aspect of the present invention is an augmented reality image display system described in any one of the first to fourth aspects, characterized in that it has a clarification processing unit that clarifies the difference between the virtual design replacement portion of the designed object contained in the image captured by the capture unit and the portion other than the virtual design replacement portion.

In the present invention, the virtual design replacement portion and the portion other than the virtual design replacement portion are distinguished from the image captured by the image capture unit. However, the image captured by the image capture unit is affected by the external environment (external light wavelength, intensity, etc.), and depending on the state of the external environment, it may become difficult to distinguish between the virtual design replacement portion and the portion other than the virtual design replacement portion.

According to the fifth aspect, even in such a situation, the image (video) processing by the clarification processing unit makes it possible to accurately distinguish between the virtual design replacement portion and the portion other than the virtual design replacement portion. As a result, the virtual design of the object to be designed can be easily changed regardless of the external environment, and an augmented reality image using the virtual design can be reliably displayed.

A sixth aspect of the present invention is an augmented reality image display system described in any one of the first to fifth aspects, characterized in having an input unit for inputting information such as characters and pictures, and a virtual design addition unit for adding the information input from the input unit to the virtual design.
According to the sixth aspect, a virtual design to which new information has been added and an augmented reality image using the virtual design can be created and displayed.

A seventh aspect of the present invention is an augmented reality image display system described in any one of the first to sixth aspects, characterized in that it has an image data creation unit that creates image data from the augmented reality image created by the synthetic image creation unit.

According to the seventh aspect, it is possible to create image data of the created augmented reality video (predetermined image data from a series of augmented reality videos). This image data is image data (augmented reality image) of a certain moment in the augmented reality video, and is image data that captures a certain moment in augmented reality that differs from reality. This image data can be printed out using a printer or the like, or used as a background image for a personal computer or the like.

An eighth aspect of the present invention is an augmented reality video display system described in any one of the first to seventh aspects, characterized in having a sound output unit that outputs sound based on sound data, a sound data storage unit that stores the sound data, and a sound data extraction unit that extracts sound data from the sound data storage unit based on a virtual design.

According to the eighth aspect, it is possible to output sound that matches the augmented reality image displayed on the display unit.

A ninth aspect of the present invention is a virtual design object composite image display system described in any one of the first to eighth aspects, characterized in that it has a printing unit that prints a virtual design on an object of the same type or a different type as the object to be designed.

According to the ninth aspect, by transmitting the virtual design to a printing unit, it is possible to produce an item of the same type or a different type as the designed item, printed with the same design as the augmented reality image.

A tenth aspect of the present invention is an augmented reality image display system according to any one of the first to ninth aspects, further comprising an information terminal connected via a network, and capable of controlling at least one of the following from the information terminal: power on/off, the angle of the shooting unit relative to the object to be designed, the brightness, luminance and contrast of the display unit, and the specified judgment conditions of the identification unit judgment unit.

According to this tenth aspect, the behavior of the augmented reality image display system (power on/off, the angle of the shooting unit relative to the designed object, the brightness and contrast of the display unit, and at least one of the specified judgment conditions of the identification unit judgment unit) can be controlled from a remote location.

An eleventh aspect of the present invention is an augmented reality image display system characterized in that a virtual design object synthetic image display system described in any one of the first to tenth aspects is stored in a housing.

According to the 11th aspect, since the virtual design object composite image display system is built into the housing, the size is compact, so that it can be installed in various environments and can be easily transported.

A twelfth aspect of the present invention is an augmented reality image display system described in any one of the first to eleventh aspects, characterized in that the housing has a door, a door detection unit that detects whether the door is open or closed, and an effect improvement unit that displays a predetermined image on the display unit depending on the state of the door detected by the door detection unit.

According to the twelfth aspect, an effective image can be displayed on the display unit when the door is opened or closed. As a result, the amusement of the augmented reality image display system can be improved.

A thirteenth aspect of the present invention is an augmented reality image display system described in any one of the first to twelfth aspects, characterized in that the designed item is clothing, a towel, a mask or a smartphone case.

In this thirteenth aspect, the virtual design of clothing, towels, masks or smartphone cases can be easily changed and augmented reality images using the same can be displayed.

In this invention, "part" and "system" do not simply mean physical means, but also include cases where the functions of the "part" or "system" are realized by software. Furthermore, the functions of one "part" or "system" may be realized by two or more physical means or devices, and the functions of two or more "parts" or "systems" may be realized by one physical means or device.

FIG. 1 is a schematic diagram of a T-shirt used in the first embodiment. FIG. 2 is a schematic conceptual diagram of the augmented reality image display system according to the first embodiment. FIG. 3 is a schematic diagram of a main body according to the first embodiment. FIG. 4 is a schematic conceptual diagram of the augmented reality image synthesis unit according to the first embodiment. FIG. 5 is a schematic conceptual diagram of the composite video creation unit according to the first embodiment. FIG. 6 is a flowchart showing the operation of the augmented reality image display system according to the first embodiment.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of an augmented reality image display system according to the present invention will be described below with reference to the accompanying drawings. Note that the present invention is not limited to the following embodiment.
(Embodiment 1)

First, the object to be designed that is photographed by the augmented reality image display system of this embodiment described later will be described. The object to be designed is not particularly limited as long as it has a virtual design replacement section 301 (301a, 301b) on at least a part of its surface that is replaced with a virtual design 410 when displayed on the display section 30 of the augmented reality image display system 1 described later, and a specification section 302 that indicates that the image includes the object to be designed when photographed by the photographing section 20 of the augmented reality image display system 1 described later. The object to be designed is not particularly limited as long as it can be printed. Examples of the object to be designed include clothing, towels, masks, and smartphone cases.

The virtual design replacement unit 301 is not particularly limited as long as it can be replaced with a virtual design when displayed on the display unit 30 of the augmented reality image display system 1. Examples of the virtual design replacement unit 301 include a part colored green or blue, which is used for chromakey.

The identifying portion 302 is not particularly limited as long as it indicates that the image contains the object to be designed when captured by the capturing unit 20 of the augmented reality image display system 1. Examples of the identifying portion 302 include a portion colored with paint that has a higher light reflectance than other portions, a portion with a specific shape that allows image analysis to recognize that the object to be designed is included in the image, and the like.

In this embodiment, as shown in Figure 1, a T-shirt is used as the design object 300. A specific portion 302 is formed in the center of the T-shirt, which is colored with paint that has a higher light reflectance than other portions. The surface 301a (virtual design replacement portion) other than the specific portion 302 is dyed green, and the back surface 301b (virtual design replacement portion) is dyed blue.

The shape, size, and material of the T-shirt are not particularly limited as long as they can form the above-mentioned specific portion 302 and virtual design replacement portions 301a and 301b. The T-shirt may be actually worn by a person, or may be held up in front of or behind the person with respect to the imaging unit 20.

Next, the augmented reality image display system in this embodiment will be described. Figure 2 is a schematic conceptual diagram of the augmented reality image display system in this embodiment. As shown in this figure, the augmented reality image display system 1 is composed of a main body 10 having two doors 11 attached so as to open from the center to both the left and right sides, and a printing unit 100 connected to the main body 10 via a network 200. Although not shown, in this embodiment, multiple main bodies 10 are connected to the printing unit 100 via the network 200.

First, the main body 10 will be described. Figure 3 is a schematic diagram of the main body 10 with the door open. As shown in this figure, the main body 10 has a rectangular parallelepiped housing 12, and inside it are a shooting unit 20, a display unit 30, and an augmented reality image synthesis unit 40. Although not shown, the shooting unit 20, the display unit 30, and the augmented reality image synthesis unit 40 are connected by wire or wirelessly.

The photographing unit 20 is attached to the center of the door side (front side) of the top panel inside the housing 12, and is capable of photographing people or the object to be designed located in front of the main body 10. The photographing unit 20 is not particularly limited as long as it can photograph the specific portion 302 and the virtual design replacement portions 301a, 301b of the object to be designed 300. Examples of the photographing unit 20 include a commercially available CCD camera.

The display unit 30 is attached to the inner back panel of the housing 12 and is capable of displaying the augmented reality image described below. There are no particular limitations on the display unit 30 as long as it is capable of displaying the augmented reality image. Examples of the display unit 30 include a liquid crystal display and an organic EL display.

Furthermore, the augmented reality image synthesis unit 40 will be described. The augmented reality image synthesis unit 40 is attached to the lower part inside the housing 12, and is capable of synthesizing an augmented reality image by replacing at least a part of the image of the designed object 300 captured by the capture unit 20 with a virtual design 410 described later.

Figure 4 is a schematic conceptual diagram of the augmented reality image synthesis unit 40. As shown in this figure, the augmented reality image synthesis unit 40 has a virtual design storage unit 41, a specific part confirmation unit 42, a specific part judgment unit 43, a virtual design extraction unit 44, and a synthetic image creation unit 45.

The virtual design storage unit 41 is not particularly limited as long as it is capable of storing multiple virtual designs 410. Examples of the virtual design storage unit 41 include commercially available database software capable of storing multiple virtual designs 410.

Here, the virtual design 410 refers to image data that is replaced by the virtual design replacement parts 301a and 301b of the design subject 300 when displayed on the display unit 30. The shape of the virtual design 410 is preferably similar (including the same) to the shape of the virtual design replacement parts 301a and 301b of the design subject 300, but does not have to be the same. The virtual design 410 in this embodiment may be, for example, video (image) data of a character appearing in a manga or animation from the neck to the waist. In this embodiment, the virtual design is composed of two image data: a virtual design on the front side of a T-shirt and a virtual design on the back side of the T-shirt. The virtual design may include virtual design-related information about the virtual design (image data of the entire image of the virtual design, the creator of the virtual design, and information about the price and size of the T-shirt to be described later) and image data related to the virtual design (image data of a "frame" displayed around the display unit, etc.).

The specific part confirmation unit 42 is not particularly limited as long as it can confirm, by color, shape, etc., whether the specific part 302 is included in the image of the designed object 300 captured by the imaging unit 20. Examples of the specific part confirmation unit 42 include a program having such a function.

In addition, an invisible light irradiating unit capable of irradiating invisible light such as an infrared irradiator is attached to the photographing unit 20, and when invisible light such as infrared light is irradiated, the specific part confirmation unit 42 may include a program that detects the invisible light reflected by the specific part 302 and determines whether the specific part 302 is included based on the area of the part that reflects the invisible light (determines whether the area of the part that reflects the invisible light can be considered to be the same as the area of the specific part 302 that has been set in advance). By configuring the specific part confirmation unit 42 in this manner, the specific part can be detected more accurately without being affected by the external environment than when the specific part 302 is detected based on color, shape, etc. In this embodiment, a program is used that recognizes a part that is brighter than other parts as the specific part 302.

The identification part judgment unit 43 is not particularly limited as long as it can judge whether the identification part 302 confirmed by the identification part confirmation unit 42 has been photographed so as to satisfy a specified judgment condition. Examples of the identification part judgment unit 43 include a program having such a function. The specified condition is not particularly limited as long as it is a state (condition) in which the identification part 302 can be confirmed by the identification part confirmation unit 42. In this embodiment, the specified condition is that the identification part 302 is continuously recognized (photographed) by the identification part confirmation unit 42.

The virtual design extraction unit 44 is not particularly limited as long as it can extract the virtual design 410 from the virtual design storage unit 41 based on the judgment of the identification part judgment unit 43. Examples of the virtual design extraction unit 44 include a program having such a function. In this embodiment, a program is used that does not extract a new virtual design 410 when a predetermined condition (the identification part 302 is continuously recognized (photographed)) is met, but extracts the virtual design 410 from the virtual design storage unit 41 in accordance with the predetermined extraction condition when the identification part 302 is not recognized for a certain period of time (e.g., 0.5 seconds).

The composite image creation unit 45 is not particularly limited as long as it can create an augmented reality image by replacing the virtual design replacement units 301a and 301b with the virtual design 410 extracted by the virtual design extraction unit 44. Examples of the composite image creation unit 45 include programs having such functions. In this embodiment, it is configured as follows.

Figure 5 is a schematic conceptual diagram of the composite image creation unit 45. As shown in this figure, the composite image creation unit 45 has a center position calculation unit 450, a placement unit 451, a length detection unit 452, an adjusted virtual design creation unit 453, and a synthesis unit 454.

The center position calculation unit 450 is not particularly limited as long as it can calculate the center position of the virtual design replacement parts 301a and 301b in the image of the design subject 300 captured by the shooting unit 20. Examples of the center position calculation unit 450 include programs having such functions. In this embodiment, a program is used that calculates the center position in the vertical direction from the upper and lower ends of the virtual design replacement parts 301a and 301b captured by the shooting unit 20, calculates the center position in the horizontal direction from the right and left ends of the virtual design replacement parts 301a and 301b, and finally calculates the center position of the virtual design replacement parts 301a and 301b in the image of the design subject 300 captured by the shooting unit 20.

The placement unit 451 is not particularly limited as long as it can place the center of the virtual design at the center position of the virtual design replacement parts 301a, 301b calculated by the center position calculation unit 450. Examples of the placement unit 451 include programs having such functions. In this embodiment, a program is used that similarly calculates the position of the center of the virtual design and aligns this position with the center positions of the virtual design replacement parts 301a, 301b calculated by the center position calculation unit 450.

The length detection unit 452 is not particularly limited as long as it can detect the length of a specified portion of the virtual design replacement portions 301a, 301b in the image of the design subject 300 captured by the capture unit 20. Examples of the length detection unit 452 include programs having such functions. In this embodiment, a program is used that divides the virtual design replacement portions 301a, 301b evenly in the horizontal direction by a specified number (e.g., 20 portions) and calculates the width (number of dots, etc.) of each divided portion.

The adjusted virtual design creation unit 453 is not particularly limited as long as it can create an adjusted virtual design obtained by adjusting the length of the virtual design 410 corresponding to a predetermined part based on the length of the predetermined part of the virtual design replacement parts 301a and 301b. Examples of the adjusted virtual design creation unit 453 include programs having such functions. In this embodiment, a program is used that divides the virtual design into a predetermined number (e.g., 20) of equal parts in the horizontal direction, adjusts the width (number of dots, etc.) of each divided part to the length of each part calculated by the length detection unit 452, and then synthesizes them to create an adjusted virtual design adjusted to the same shape and size as the virtual design replacement parts 301a and 301b in the image of the design target object 300 captured by the capture unit 20, as done by the length detection unit 452.

The synthesis unit 454 is not particularly limited as long as it can replace the virtual design replacement units 301a and 301b with the adjusted virtual designs and synthesize the augmented reality image. Examples of the synthesis unit 454 include programs having such functions. A similar program is used in this embodiment. Note that in this embodiment, each of the front and back of the T-shirt can be replaced with a corresponding virtual design based on the color of the front of the T-shirt (the color of the front determines whether it is the front or back).

The main body 10 is not particularly limited as long as it has the above-mentioned functions. The main body 10 may be, for example, a commercially available personal computer or a dedicated video processing device.

The printing unit 100 shown in FIG. 2 is not particularly limited as long as it can print the virtual design 410 on an item of the same type as or different from the design target object 300. Examples of the printing unit 100 include a printing machine and a printing company. In this embodiment, a printing machine that can print the virtual design 410 on the front (part corresponding to the virtual design replacement part 301a) and back (part corresponding to the virtual design replacement part 301b) of a T-shirt is included.

Furthermore, the network 200 is not particularly limited as long as it can connect the main body 10 and the printing unit 100, regardless of whether it is wired or wireless. Examples of the network 200 include the Internet and an intranet.

Next, we will explain the operation of the augmented reality image display system 1. Figure 6 is a flowchart showing the operation of the augmented reality image display system 1.

First, when the augmented reality image display system 1 is started, the image of the object to be designed 300 is captured by the image capture unit 20 (S1). Then, the identification unit confirmation unit 42 checks whether the image of the identification unit 302 is included in the image captured by the image capture unit 20 (S2).

Then, if the predetermined conditions are met (S3), the center position calculation unit 450 calculates the positions of the centers of the virtual design replacement parts 301a and 301b in the image of the design subject 300 (S4). After that, the placement unit 451 calculates the position of the center of the virtual design 410 (S5), and the length detection unit 452 detects the length of the predetermined part (S6).

Next, an adjusted virtual design is created by the adjusted virtual design creation unit 453 (S7), and then an augmented reality image is created by the synthesis unit 454 (S8). Then, the augmented reality image is finally displayed on the display unit 30 (S9). Here, virtual design-related information about the virtual design displayed on the display unit 30 may be displayed, together with the augmented reality image, at the bottom of the display unit 30, etc.

On the other hand, if the predetermined conditions are not met, the virtual design extraction unit 44 extracts a virtual design 410 from the virtual design storage unit 41 in accordance with the predetermined extraction conditions (S11). Thereafter, the same process is carried out from S4.
These operations are then repeated until the final operation is performed (S10).

By configuring the augmented reality image display system 1 as described above, it is possible to easily change the virtual reality design of an object whose design is to be changed, and to display an augmented reality image using the design. Also, by configuring the composite image creation unit 45 as described above, it is possible to easily create an augmented reality image that matches the shape and size of the virtual design replacement units 301a, 301b of the object to be designed 300 photographed by the photographing unit 20. As a result, by viewing the augmented reality image, a person wearing or holding up the T-shirt can clearly imagine what it would look like wearing the T-shirt with the virtual design.

In addition, in this embodiment, the virtual design extraction unit 44 is configured not to extract a new virtual design when a predetermined condition (the specific part 302 is continuously recognized (photographed)) is satisfied, and to extract a virtual design from the virtual design storage unit 41 according to the predetermined extraction condition when the specific part 302 is not recognized for a certain period of time (for example, 0.5 seconds). Therefore, the virtual design can be easily changed by holding a hand, arm, or obstacle between the specific part 302 and the photographing unit 20, or by flapping a T-shirt so that the specific part 302 is not photographed by the photographing unit 20. In this embodiment, such an excellent user interface is provided. The predetermined extraction condition is not particularly limited, and for example, the virtual design may be extracted according to a predetermined order, or may be extracted randomly.

Furthermore, by sending a favorite virtual design 410 to the printing unit 100, it is possible to produce an article of the same or different type as the object to be designed, with the same design (virtual design 410) as the design displayed in the augmented reality video printed on it. Note that the article of the same type as the object to be designed 300 may be of the same type as the one photographed by the photographing unit 20, or may be of a type that is partially different.

In addition, since the augmented reality image display system 1 is built into the housing 12, it has a compact size and can be installed in various environments and can be easily transported. In other words, the augmented reality image display system 1 is also the world's smallest custom T-shirt store where customers can virtually try on T-shirts.
(Embodiment 2)

In embodiment 1, the virtual design could not be changed, but the present invention is not limited to this.

For example, the augmented reality video display system according to embodiment 1 may be configured to further include an input unit for inputting information such as characters and pictures, and a virtual design adding unit for adding the information input from the input unit to the virtual design. Note that there is no particular limitation on the locations where the input unit and the virtual design adding unit are attached.

Here, the input unit is not particularly limited as long as it can input information such as characters and pictures. Examples of the input unit include a keyboard, a smartphone, a laptop computer, a dedicated input device, etc.

The virtual design adding unit is not particularly limited as long as it is capable of adding information input from the input unit to the virtual design. Examples of the virtual design adding unit include programs having such functions. The virtual design adding unit can be incorporated into the augmented reality video display system of embodiment 1 by adding the information input from the input unit to the virtual design extracted by the virtual design extraction unit and making it a new virtual design.

By configuring the augmented reality image display system in this way, it is possible to create a virtual design with new information added and an augmented reality image using the virtual design. In this embodiment, it is possible to display an augmented reality image of an original T-shirt with new information added to the virtual design, and also to produce a T-shirt with the original design by transmitting the design to a printing department.
(Embodiment 3)

In the above-described embodiment, an augmented reality image can be displayed and a design object can be created on which a virtual design is printed, but the present invention is not limited to this.

The augmented reality video display system according to the embodiment described above may be configured to further include a sound output unit that outputs sound based on sound data, a sound data storage unit that stores the sound data, and a sound data extraction unit that extracts sound data from the sound data storage unit based on a virtual design.

Here, the sound output unit is not particularly limited as long as it can output sound based on sound data. Examples of the sound output unit include speakers. Here, the sound data is not particularly limited as long as it is data related to sound. Examples of the sound data include sound effect data and performance data of a specified song. The location where the sound output unit is attached is not particularly limited, but it is desirable for it to be attached to the back surface of the display unit. By attaching the sound output unit in this position, the augmented reality image displayed on the display unit and the sound work together to improve the amusement factor.

The sound data storage unit is not particularly limited as long as it is capable of storing sound data. Examples of sound data storage units include commercially available database software.

The sound data extraction unit is not particularly limited as long as it can extract sound data from the sound data storage unit based on the virtual design. For example, the sound data extraction unit may be a program in which the virtual design and the sound data are uniquely (one-to-one) associated with each other, and which extracts sound data associated with the virtual design extracted by the virtual design extraction unit.

By configuring the augmented reality image display system in this way, it is possible to output sound that matches the augmented reality image displayed on the display unit. Specifically, for example, if an original T-shirt of a certain band is displayed on the display unit as the augmented reality image, it is possible to output a song by that band. As a result, it is possible to increase people's desire to purchase a T-shirt with the same design (virtual design) as the design displayed on the augmented reality image.

It goes without saying that the virtual design storage unit 41 may also function as the sound data storage unit, and the virtual design extraction unit may also function as the sound data extraction unit.
(Embodiment 4)

In the above-described embodiment, no processing was performed on the image captured by the capture unit before the augmented reality image was synthesized by the synthetic image creation unit of the augmented reality image synthesis unit, but the present invention is not limited to this.

The image captured by the capture unit is affected by the external environment (external light wavelength, intensity, etc.), and depending on the conditions of the external environment, it may become difficult to distinguish between the virtual design replacement part and parts other than the virtual design replacement part.

Therefore, for example, the augmented reality image display system may be configured to have a clarification processing unit that clarifies the difference between the virtual design replacement portion of the designed object contained in the image captured by the capture unit and the portion other than the virtual design replacement portion.

Here, the clarification processing unit is not particularly limited as long as it can clarify the difference between the virtual design replacement part of the designed object and the part other than the virtual design replacement part. Specifically, it can be a program that can change the contrast, brightness, luminance, etc. of the image captured by the capture unit. The clarification processing unit may be built into the capture unit, or the augmented reality image display system may be configured to have a personal computer or the like that has the above-mentioned program or the like built in.

By configuring the augmented reality image display system in this way, it is possible to distinguish between the virtual design replacement part and the part other than the virtual design replacement part of the object to be designed based on the data in which the difference between the virtual design replacement part and the part other than the virtual design replacement part is clarified by the clarification processing part. As a result, the augmented reality image display system can easily change the virtual design of the object to be designed regardless of the external environment, and can reliably display an augmented reality image using the virtual design.
(Embodiment 5)

In the above-described embodiment, the augmented reality image created by the synthetic image creation unit is displayed on the display unit, but the present invention is not limited to this.

For example, the augmented reality video display system may be configured to have an image data creation unit that creates image data from an augmented reality video. The image data created by the image data creation unit is image data (augmented reality image) of a certain moment in the augmented reality video, and is image data that captures a certain moment in augmented reality that is different from reality.

Here, the image data creation unit is not particularly limited as long as it is capable of creating image data from augmented reality video, and examples include a program having such a function or a personal computer having such a program built in.

By configuring the augmented reality image display system in this manner, the image data created by the image data creation unit can be printed by a printer or the like, or can be used as a background image for a personal computer or the like.
(Embodiment 6)

In the above-described embodiment, the method of controlling the power on/off, the angle of the shooting unit relative to the designed object, the brightness, luminance and contrast of the display unit, and the specified judgment conditions of the identification unit judgment unit is not particularly limited, but the augmented reality image display system may be configured to be controllable from a remote location.

For example, it may be possible to connect to an information terminal located in a remote location via a network and control from that information terminal at least one of the following: power on/off, the angle of the shooting unit relative to the object to be designed, the brightness, luminance and contrast of the display unit, and the specified judgment conditions of the identification unit judgment unit.

To turn the power to the augmented reality image display system on and off, simply install a switch that can turn the power on and off via the network.

In addition, to adjust the angle of the camera unit relative to the object being designed, an angle adjustment device incorporating a motor that can be controlled from the outside can be attached, and this can be controlled from an information terminal while viewing the image captured by the camera unit.

To adjust the brightness, luminance and contrast of the display unit, as well as the specified judgment conditions of the specific part judgment unit, a personal computer or the like having such a program installed can be attached and controlled from an information terminal.

By configuring the augmented reality image display system in this manner, it is possible to control the behavior of the augmented reality image display system (power on/off, the angle of the shooting unit relative to the designed object, the brightness and contrast of the display unit, and at least one of the specified judgment conditions of the identification unit judgment unit) from a remote location.
Other Embodiments

The augmented reality video display system of embodiment 3 may be configured to further include a door detection unit that detects whether the door is open or closed, and an effect improvement unit that causes the sound output unit to output sound effects and the display unit to display a predetermined image (effect image) according to the state of the door detected by the door detection unit. Here, the sound effects and effect images are not particularly limited.

The door detection unit is not particularly limited as long as it can detect whether a door inside the housing is opened or closed. Examples of the door detection unit include commercially available door sensors.

The effect improvement unit is not particularly limited as long as it can cause the sound output unit to output a sound effect and the display unit to display a predetermined image (effect image) in accordance with the state of the door detected by the door detection unit. Examples of the effect improvement unit include programs having such functions.

By configuring the augmented reality video display system in this way, it is possible to output effective sounds and display effective images on the display unit when the door is opened or closed. As a result, it is possible to improve the entertainment value of the augmented reality video display system.

In addition, in the first embodiment, the augmented reality image synthesis unit is configured to have a virtual design storage unit, a specific part confirmation unit, a specific part judgment unit, a virtual design extraction unit, and a synthetic image creation unit, but the present invention is not limited to this. The configuration of the augmented reality image synthesis unit is not particularly limited as long as it can replace at least a part of the image of the designed object captured by the capture unit with a virtual design to synthesize an augmented reality image.

REFERENCE SIGNS LIST 1 Augmented reality image display system 10 Main body 11 Door 12 Housing 20 Shooting unit 30 Display unit 40 Augmented reality image synthesis unit 41 Virtual design storage unit 42 Specific part confirmation unit 43 Specific part judgment unit 44 Virtual design extraction unit 45 Synthetic image creation unit 100 Printing unit 200 Network 300 Designed object 301 Virtual design replacement unit 301a T-shirt surface (virtual design replacement unit)
301b Back of T-shirt (virtual design replacement part)
302 Identification unit 410 Virtual design 450 Center position calculation unit 451 Placement unit 452 Length detection unit 453 Adjusted virtual design creation unit 454 Synthesis unit

Claims (12)

A photography unit that photographs the designed object;
an augmented reality image synthesis unit that replaces at least a part of the image of the object to be designed captured by the capture unit with a virtual design to synthesize an augmented reality image;
a display unit that displays the augmented reality image;
An augmented reality image display system comprising:
The object to be designed is
a virtual design replacement portion on at least a portion of the front surface, the virtual design replacement portion being replaced with the virtual design when displayed on the display portion;
A specification unit that indicates that the image includes the design object when photographed by the photographing unit,
The augmented reality image synthesis unit includes:
a virtual design storage unit for storing a plurality of the virtual designs;
A specific part confirmation unit that confirms whether the specific part is included in the image of the designed object photographed by the photographing unit;
a specific part determination unit that determines whether the specific part confirmed by the specific part confirmation unit is photographed so as to satisfy a predetermined determination condition;
a virtual design extraction unit that extracts the virtual design from the virtual design storage unit based on a determination by the identification unit determination unit;
a synthetic image creation unit that creates the augmented reality image by replacing the virtual design replacement unit with the virtual design extracted by the virtual design extraction unit,
the predetermined determination condition is that the identification unit is not confirmed by the identification unit confirmation unit for a predetermined period of time,
the virtual design extraction unit extracts the virtual design from the virtual design storage unit in accordance with a predetermined extraction condition;
An augmented reality image display system. The synthetic image creation unit includes:
a center position calculation unit that calculates the position of the center of the virtual design replacement part in the image of the design subject captured by the image capture unit;
an arrangement unit that arranges a center portion of the virtual design at a position of a center portion of the virtual design replacement portion calculated by the center position calculation unit;
a length detection unit that detects a length of a predetermined portion of the virtual design replacement unit in the image of the design subject captured by the image capture unit;
an adjusted virtual design creation unit that creates an adjusted virtual design by adjusting a length of the virtual design corresponding to a predetermined portion based on a length of the predetermined portion of the virtual design replacement unit;
a synthesis unit that synthesizes the augmented reality image by replacing the virtual design replacement unit with the adjusted virtual design;
2. The augmented reality image display system according to claim 1, further comprising: Further comprising an invisible light irradiating unit capable of irradiating invisible light,
the specific part confirmation unit detects the invisible light reflected by the specific part, and determines whether the specific part is included based on an area of a portion that reflects the invisible light.
3. The augmented reality image display system according to claim 1 or 2. A clarification processing unit that clarifies the difference between the virtual design replacement part of the object to be designed included in the image captured by the imaging unit and a part other than the virtual design replacement part,
4. The augmented reality image display system according to claim 1, wherein the augmented reality image display system comprises: an input section for inputting character and picture information;
a virtual design adding unit that adds the information input from the input unit to the virtual design;
5. The augmented reality image display system according to claim 1, further comprising: an image data creation unit that creates image data from the augmented reality image created by the synthetic image creation unit;
6. The augmented reality image display system according to claim 1, wherein the augmented reality image display system comprises: a sound output unit that outputs sound based on the sound data;
a sound data storage unit for storing the sound data;
a sound data extracting unit that extracts the sound data from the sound data storage unit based on the virtual design;
7. The augmented reality image display system according to claim 1, further comprising: 8. The augmented reality image display system according to claim 1, further comprising a printing unit that prints the virtual design on an item of the same type as or different from the item to be designed. Further comprising an information terminal connected via a network,
The information terminal can control at least one of the power on/off, the angle of the photographing unit relative to the design object, the brightness, luminance and contrast of the display unit, and the predetermined judgment condition of the identification unit judgment unit.
9. The augmented reality image display system according to claim 1, wherein the augmented reality image display system comprises: 10. An augmented reality image display system comprising the augmented reality image display system according to claim 1 housed in a housing. The housing has a door,
A door detection unit that detects whether the door is open or closed;
an effect improving unit that displays a predetermined image on the display unit according to the state of the door detected by the door detection unit;
The augmented reality image display system according to claim 10, further comprising: The augmented reality image display system according to any one of claims 1 to 11, characterized in that the designed object is clothing, a towel, a mask or a smartphone case.