KR102404729B1 - Method for applying artificial flavors in situation of virtual reality, augmented reality and mixed reality - Google Patents

Abstract

Disclosed is a method for applying artificial taste using gloves and a sensory content display device. The method for applying artificial taste of the present invention includes receiving and storing set values of temperature (℃), current intensity (μA), and frequency (Hz) corresponding to the taste expressed in the image or picture and the image or picture and storing the image or picture, sensory content A step of reproducing an image or picture through a display device, and a temperature stimulation by temperature corresponding to the taste expressed in the reproduced image or picture is transmitted through a temperature stimulation unit provided at the distal end of each finger of the glove, and the reproduced image or Electric stimulation by electric current of intensity and frequency corresponding to the taste expressed in the picture is transmitted through electrodes provided at the distal end of each finger of the glove, and through a haptic module provided in the palm of the glove to the palm of the user. and delivering a pressure stimulus corresponding to the reproduced image or picture.

Description

How to apply artificial taste in virtual reality, augmented reality and mixed reality situations { METHOD FOR APPLYING ARTIFICIAL FLAVORS IN SITUATION OF VIRTUAL REALITY, AUGMENTED REALITY AND MIXED REALITY }

The present invention relates to a method for applying artificial taste in virtual reality, augmented reality, and mixed reality situations, and more particularly, to an artificial taste realization and delivery method for implementing various tastes by combining current, temperature and frequency.

Metaverse is a compound word of meta and universe, meaning virtual space and real world, respectively, and refers to a virtual world like the real world. In order to realize an experience that satisfies the senses of taste and touch), research is being conducted in various fields.

In the current metaverse, all experiences through an avatar can be felt only visually and aurally, but by expanding this, the virtual world in which all senses of touch, smell, and taste can all be felt the same as the avatar is called the real metaverse. In order to implement this, high-level artificial sensory technology that can be applied in the metaverse must be applied.

In 2011, Dr. Nimesha Ranasinghe of the National University of Singapore (NUS) published a thesis on the Digital Taste Interface, which realizes the taste of virtual food by stimuli such as electricity, frequency, and heat. did From this, it was found that by stimulating the tongue with various levels of current and temperature, it was possible to realize the sense of various tastes, including salty taste, sweet taste, bitter taste, and sour taste.

When a taste molecule binds to a taste receptor protein on the tongue, the current flowing through the cell changes and a signal is transmitted to the brain. At this time, it is possible to feel the taste without eating food by artificially creating an electrical signal generated when the taste molecules combine. In addition, since it is well known that the sweet receptor protein is sensitive to temperature, the sweet taste can be felt more strongly by increasing the temperature.

However, there was still a lack of discussion on how to implement more diverse and subdivided taste or how to provide an artificial taste experience to humans, and the present invention is concerned with how these research results can be put into practice and utilized. started with the idea of

The present invention is to satisfy the above-mentioned needs, and an object of the present invention is to experience artificial senses for various tastes realized by combining current, temperature and frequency, and to use images, pictures, and haptic stimulation suitable for these to maximize the experience.

The artificial taste application method using gloves and a sensory content display device according to an embodiment of the present invention for achieving the above object is an image or picture and a temperature (℃) corresponding to the taste expressed in the image or picture, current receiving and storing set values of intensity (μA) and frequency (Hz) of The temperature stimulation by temperature is transmitted through the temperature stimulation unit provided at the distal end of each finger of the glove, and electrical stimulation by electric current of intensity and frequency corresponding to the taste expressed in the reproduced image or picture is applied to each finger of the glove. and transmitting the pressure stimulus corresponding to the reproduced image or picture to the user's palm through a haptic module provided in the palm portion of the glove and transmitting through an electrode provided at the distal end.

In this case, the storing may include a temperature (°C) corresponding to a taste expressed at at least one preset point in a timeline of an image reproduced through the sensory content display device, a current strength (μA), and Storing the set value of the frequency (Hz) and transmitting the temperature stimulation and the expression by the temperature corresponding to the taste expressed at the preset time point when the preset time point is reached during the reproduction of the image The operation of the artificial taste transmitting device and the sensory content display device may be synchronized to transmit the electrical stimulation by a current having an intensity and a frequency corresponding to the taste.

In addition, the artificial taste application method further includes the step of sensing the motion or position of the sensory content display device through a motion sensor or a position sensor attached to a sensory content display device implemented with a head mounted display (HMD) or AR glasses. The transmitting of the temperature of the sensory content display device may include a manipulation through a user interface provided outside the sensory content display device, or the delivered temperature of the sensory content display device based on the sensed motion information or location information of the sensory content display device. It is possible to adjust the intensity and frequency of the current of electrical stimulation.

In addition, in the delivering step, the finger distal portion to which the stimulus is to be delivered is determined according to the taste expressed in the reproduced image or picture among the finger distal portions of the glove, and the temperature is applied to the temperature stimulation unit and the electrode provided at the determined finger distal portion. Stimulation and electrical stimulation can be delivered.

In addition, in the delivering step, in the taste expressed in the reproduced image or picture, the temperature may deliver a temperature stimulus that varies with time within a preset range.

In addition, in the delivering step, in the taste expressed in the reproduced image or picture, at least one of temperature or current intensity may deliver a temperature stimulus or electrical stimulus that varies with time within a preset range.

In addition, the temperature, the intensity and frequency of the current corresponding to the taste expressed in the reproduced image or picture may be in the range of 0 to 40 ℃, the range of 0 to 250 μA, and the range of 0 to 800 Hz, respectively.

On the other hand, the artificial taste application method using gloves and AR glasses according to another embodiment of the present invention corresponds to an image or picture and a taste expressed in the image or picture (℃), current strength (μA), and frequency receiving and storing a set value of (Hz), sensing a specific food by a camera provided on a part of the AR glasses, and applying temperature stimulation by a temperature corresponding to the sensed specific food to the glove and transmitting through the temperature stimulation unit provided at the distal end of each finger, and transmitting the electrical stimulation by the current having an intensity and frequency corresponding to the sensed specific food through electrodes provided at the distal end of each finger of the glove.

In addition, the artificial taste delivery device using the glove and the sensory content display device according to an embodiment of the present invention provides an image or picture and a temperature (°C) corresponding to the taste expressed in the image or picture, and the intensity of current (μA) and a storage unit for receiving and storing the set value of the frequency (Hz), a temperature stimulation unit provided at the distal end of each finger of the glove, and electrical stimulation through an electrode provided at the distal end of each finger of the glove. When the image or picture is reproduced through a transmission unit that transmits the pressure stimulus through a haptic module provided in the palm of the glove and the sensory content display device, the image or picture is expressed in the reproduced image or picture. Transmits a temperature stimulus by a temperature corresponding to the taste being played, delivers an electrical stimulus by a current of intensity and frequency corresponding to the taste expressed in the reproduced image or picture, and a pressure corresponding to the reproduced image or picture and a processor for controlling the transmitting unit to transmit the stimulus.

According to various embodiments of the present invention as described above, it is possible to transmit a stimulus in which the intensity, frequency, and temperature of electric current are fused to the user so that the user can feel and adjust the artificial senses for various tastes.

In addition, by displaying an image capable of maximizing the artificial taste experience or providing a haptic stimulus, a user's need for pleasure or convenience obtained from the artificial taste experience may be satisfied.

In order to more fully understand the drawings recited in the Detailed Description, a brief description of each drawing is provided.
1 is a block diagram schematically illustrating the configuration of an artificial taste delivery device according to an embodiment of the present invention;
2 is a diagram illustrating a flow chart for explaining an artificial taste delivery process according to an embodiment of the present invention;
3 is a view showing a haptic glove to which an artificial taste transmission device is attached according to an embodiment of the present invention;
4 is a view showing a haptic module and a position sensor according to an embodiment of the present invention;
5 is a view showing the shape of an electrode according to an embodiment of the present invention;
6 is a view for explaining a method of synchronizing the operations of the artificial taste transmitting apparatus and the sensory content display apparatus according to an embodiment of the present invention;
7 is a view showing the appearance and structure of the HMD in an example in which the sensory content display device of the present invention is implemented as an HMD;
8 is a view for explaining a process of indirectly experiencing the taste of food in front of a user's eyes using AR glasses according to an embodiment of the present invention;
9 is a flowchart for briefly explaining a method for applying an artificial taste according to an embodiment of the present invention;
10 is a flowchart for briefly explaining a method for applying an artificial taste according to another embodiment of the present invention.

First, terms used in the present specification and claims have been selected in consideration of functions in various embodiments of the present invention. However, these terms may vary depending on the intention of a person skilled in the art, legal or technical interpretation, and emergence of new technology. Also, some terms may be arbitrarily selected by the applicant. These terms may be interpreted in the meanings defined herein, and in the absence of specific definitions, they may be interpreted based on the general content of the present specification and common technical common sense in the art.

Also, the same reference numerals or reference numerals in each drawing appended hereto indicate parts or components that perform substantially the same functions. For convenience of description and understanding, the same reference numbers or reference numerals are used in different embodiments to be described. That is, even though all the components having the same reference number are shown in the plurality of drawings, the plurality of drawings do not mean one embodiment.

Also, in this specification and claims, terms including ordinal numbers such as 'first' and 'second' may be used to distinguish between elements. This ordinal number is used to distinguish the same or similar components from each other, and the meaning of the term should not be limitedly interpreted due to the use of the ordinal number. As an example, for components combined with such an ordinal number, the order of use or arrangement of the elements should not be construed as being limited by the number. If necessary, each ordinal number may be used interchangeably.

In this specification, the singular expression includes the plural expression unless the context clearly dictates otherwise. In the present application, terms such as 'comprise' or 'comprise' are intended to designate that a feature, number, step, operation, component, part, or combination thereof described in the specification exists, but one or more other It should be understood that this does not preclude the possibility of addition or presence of features or numbers, steps, operations, components, parts, or combinations thereof.

Also, in an embodiment of the present invention, when it is said that a part is connected to another part, this includes not only direct connection but also indirect connection through another medium. In addition, the meaning that a certain part includes a certain component means that other components may be further included, rather than excluding other components, unless specifically stated to the contrary.

Hereinafter, the present invention will be described in more detail with reference to the accompanying drawings.

Hereinafter, the present invention will be described in more detail with reference to the accompanying drawings.

1 is a block diagram schematically illustrating the configuration of an artificial taste delivery device according to an embodiment of the present invention.

The artificial taste transmission device 100 of the present invention includes a storage unit 110 , a transmission unit 120 , a communication unit 130 , and a processor 140 .

The storage unit 110 serves to store programs and data necessary for the operation of the artificial taste delivery device 100 . In particular, the storage unit 110 may store the temperature (°C), the current strength (μA), and the frequency (Hz) corresponding to a plurality of different flavors.

As shown in Table 1 below, a mapping table is stored in which the intensity of current (μA), frequency (Hz), and temperature (℃), which are the conditions for the corresponding taste to be expressed, are mapped according to the type of taste and the type of subdivided taste. there may be

kind of taste taste Current magnitude (μA) Frequency (Hz) Temperature (℃)

Sour taste Tuscan Orange Flavor 180 800 30 Brazilian Mango Grapefruit Flavor 180 800 25 Malibu Lemon Flower Flavor 180 800 18 lime flavor 60-180 800 20-30 spicy Madagascar spice flavor 20 800 20-30 mint flavor Malibu Lemon Flower Flavor 20 400 36 mint flavor Peppermint Candy Flavor 40 800 18 sweetness chocolate flavor 180-60 descent 800 35-25 descent salty salt to taste 50 100 20-30 rise bitter Chinese medicine taste 60-140 400 20-30 rise

At this time, it is preferable that the temperature corresponding to each taste is set within the range of 0 to 40° C., the intensity of the current is in the range of 0 to 250 μA, and the frequency is set within the range of 0 to 800 Hz.

In addition to the stored mapping table, the storage unit 110 may store a specific taste input from a user through a user interface (not shown), and setting information on temperature, current strength, and frequency to be mapped to the specific taste.

The transmitting unit 120 is configured to transmit electrical stimulation by temperature stimulation and current.

Specifically, the delivery unit 120 may deliver a temperature stimulation by a temperature corresponding to the selected taste and an electrical stimulation by a current having an intensity and a frequency corresponding to the selected taste. The material of the delivery unit 120 may be any material as long as it can deliver the temperature stimulation and the electrical stimulation to a specific part of the user. However, it is preferable that the delivery unit 120 has an arrangement and structure that allows the user's tongue to come into contact with the artificial taste delivery device 100. In the present invention, the delivery unit 120 is provided at the distal end of each finger of the haptic glove 1 . 120 is arranged so that when the user touches each finger to the tongue while wearing the haptic glove 1, the temperature stimulation and electrical stimulation are transmitted from the transmission unit 120 to the tongue, so that the user can feel the taste.

The haptic glove 1 of the present invention may be basically made of an insulating material having a space inside which a hand can be inserted.

The transfer unit 120 may include a temperature stimulation unit 121 for temperature stimulation and an electrode 122 for electrical stimulation.

The temperature stimulation unit 121 may be implemented as a heating coil or a thermoelectric element.

A thermoelectric element uses heat absorption or heat generation due to the Peltier effect, and an n-type semiconductor and a p-type semiconductor are repeatedly connected in series, and heat absorption or heat generation is determined according to the direction in which current flows. When a heat source is applied, holes in the p-type semiconductor and electrons in the n-type semiconductor take the energy and move to the heat sink, where heat energy is converted into electrical energy while working on an external load. can

The electrode 122 may include a pair of conductive parts, a power connector, and a lead wire, and the pair of conductive parts may include any one of metals of silver, silver chloride, titanium, gold, and platinum for conductivity.

The conductive part may be electrically connected to the power supply to which power is supplied through the lead wire and the power connector, and each lead wire connecting the conductive part and the power supply may be covered with a separate insulating material so as not to be exposed to the outside.

A specific arrangement method of the electrode 122 in the haptic glove 1 will be described later with reference to FIG. 5 .

The communication unit 130 is configured to be connected to the sensory content display apparatus 200 or an external device through a network to perform communication. In more detail, the communication unit 130 may exchange a synchronization signal including a reproduction time of a currently reproduced image for operation in synchronization with the sensory content display apparatus 200 .

The communication method of the network is not limited, and may include not only a communication method using a communication network (eg, a mobile communication network, wired online, wireless online, broadcasting network) that the network may include, but also short-range wireless communication. For example, the network may include wireless Internet technologies such as Wireless LAN (WLAN) (WiFi), Wireless broadband (Wibro) and/or World Interoperability for Microwave Access (Wimax), Bluetooth, Near Field Communication (NFC) and/or or short-range communication technologies such as Radio Frequency Identification (RFID), Code Division Multiple Access (CDMA), Global System for Mobile communication (GSM), Long Tern Evolution (LTE), and/or mobile communication technologies such as LTE-Advanced, and/or It can be implemented as a network based on V2X (Vehicle to Everything) communication technology, such as V2I (Vehicle to Infra).

The processor 140 is a component that controls the overall operation of the artificial taste delivery device 100 .

According to an embodiment, the processor 140 is a digital signal processor (DSP), a microprocessor (microprocessor), a time controller (TCON), a central processing unit (CPU), a microcontroller (MCU). unit), micro processing unit (MPU), controller, application processor (AP), or communication processor (CP), ARM processor, or one or more of the following terms. Also, the processor 140 may be implemented as a system on chip (SoC), large scale integration (LSI), or a field programmable gate array (FPGA) type having a built-in processing algorithm.

The processor 140 may control the transmitting unit 120 to deliver a temperature stimulation by a temperature corresponding to the selected taste and an electrical stimulation by a current having an intensity and frequency corresponding to the selected taste.

At this time, the processor 140 may control the transmitter 120 to adjust the temperature of the temperature stimulus delivered from the transmitter 120, the intensity and frequency of the electrical stimulus, according to the stimulus control command input through the user interface. can Accordingly, the processor 140 may include a control module capable of controlling the temperature and the current, respectively.

On the other hand, as shown in Table 1, in a preset taste among a plurality of different tastes, the processor 140 delivers a temperature stimulus or electrical stimulus that varies with time within a preset range, in which the intensity of temperature or current is set. can

For example, in order to realize the chocolate taste among sweetness, the intensity of the current may be decreased from 180 μA to 60 μA, and the temperature may be decreased from 35° C. to 25° C. In order to realize the taste of Chinese medicine among bitterness, the temperature may be increased from 20°C to 30°C within a current range of 60 μA to 140 μA.

In this way, when the temperature stimulation and the electrical stimulation are transmitted from the transmission unit 120 , the user puts his or her tongue on the transmission unit 120 to receive the temperature stimulation and the electrical stimulation. Temperature stimulation and electrical stimulation transmitted through the taste buds are perceived as a specific taste through the brain.

Meanwhile, FIG. 2 is a diagram illustrating a flow chart for explaining an artificial taste delivery process according to an embodiment of the present invention.

First, the setting value corresponding to the taste is stored (S210). Thereafter, upon receiving a selection command for any one of a plurality of stored different flavors (S220), the temperature, current strength and frequency corresponding to the received selection command may be adjusted (S240). In addition, a stimulus control command for directly adjusting the temperature, current intensity and frequency may be input ( S230 ), and in this case, temperature, current intensity and frequency may be adjusted according to the stimulus control command.

The temperature, current strength, and frequency adjusted in this way are transmitted to the user as temperature stimulation and electrical stimulation through the temperature stimulation unit 121 and the electrode 122 of the transmission unit 120 (S250).

When the user brings the temperature stimulation unit 121 and the electrode 122 constituting the transmission unit 120 to the tongue, the temperature stimulation and electrical stimulation transmitted in this way are applied to the user's tongue, thereby generating and transmitting artificial taste. can be

3 is a diagram illustrating a haptic glove to which an artificial taste transmission device is attached according to an embodiment of the present invention.

As shown in FIG. 3 , in the artificial taste transmission device of the present invention, the control module including the storage unit 110 and the processor 140 is located on the wrist of the haptic glove 1, and transmits the temperature stimulation and the electrical stimulation. The transfer unit 120 for the haptic glove 1 may be provided at the distal end of each finger.

The transfer unit 120 may be composed of five transfer modules 120-1 to 120-5 for each finger. Each of the transfer modules 120-1 to 120-5 and the processor 140 may be connected through a conductive electrode plate, and a connecting member for connecting an external conductive wire may be formed on one side of the conductive electrode plate.

Each of the delivery modules 120-1 to 120-5 may include a temperature stimulation unit 121 and an electrode 122, respectively.

In the case where the temperature stimulation unit 121 is implemented as a thermoelectric element, the control module positioned on the wrist of the haptic glove 1 may include a temperature control PCB for controlling the thermoelectric element. A thermoelectric element for transmitting a temperature stimulus is provided together with the electrode 122 at the finger distal end of the haptic glove 1, and may be connected to the temperature control PCB with each conductive metal of the anode and the cathode.

The electrode 122 may be implemented as a flexible printed circuit board (FPCB) board for electrodes provided at the distal end of each finger of the haptic glove 1 or as a ring type that can be fitted on each finger of the haptic glove 1 can

4 is a diagram illustrating a haptic module and a position sensor according to an embodiment of the present invention.

As shown in FIG. 4 , a haptic module 123 for delivering a pressure stimulus by pressure corresponding to a selected taste may be included on the palm side of the haptic glove 1 , and haptic modules may be included in the back of the hand or the finger region. At least one position sensor 124 for sensing the position of each finger portion of the glove 1 may be included.

The haptic module 123 may include a piezoelectric element and a plurality of actuators to transmit realistic tactile sensations under the control of the processor 140 . The haptic module 123 may be operated such that a pressure stimulus corresponding to the selected taste is applied to the palm portion.

As another embodiment, the haptic module 123 may include a pneumatic actuator that generates pressure using air pressure and stimulates the kinesthetic sense of the user's palm.

The position sensor 124 may include at least one of a 3-axis gyroscope, a 3-axis acceleration sensor, a 3-axis geomagnetic sensor, and a piezoelectric sensor to detect a motion or position of each finger of the haptic glove 1 .

Accordingly, the processor 140 detects a position or motion of a finger according to the position information of each finger portion sensed by the position sensor 124 , and the distal end of each finger of the haptic glove 1 according to the sensed position or motion. It is possible to control the taste delivered to the user to be different by adjusting the temperature of the temperature transmitted to the stimulation, the strength and frequency of the electric current of the electrical stimulation.

In this case, a mapping table in which a taste corresponding to a preset position or motion of a finger, respectively, and a temperature for realizing the taste, an intensity of a current, and a frequency for realizing the taste are mapped may be stored.

Meanwhile, FIG. 5 shows an example in which the glove is implemented as a finger glove in the form of a thimble including electrodes 122-1 and 122-2 in contact with the distal end of the finger.

The finger glove 50 is designed to have an internal space that can be inserted into each finger of the user, and may be made of a flexible plastic or rubber material so that an elastic force can act according to the thickness of the finger.

Each of the electrodes 122-1 and 122-2 may be connected to a control module provided on the wrist through a 4P wire.

In this case, the processor 140 may control to deliver the electrical stimulation and the temperature stimulation to the finger glove 50 selected by the user from among the plurality of finger gloves 50 corresponding to the plurality of fingers. For example, when the finger glove 50 corresponding to the index finger is selected by the user through the user interface among the finger glove 50 corresponding to the index finger and the finger glove 50 corresponding to the thumb, the processor ( 140 may deliver electrical stimulation and temperature stimulation only to the finger glove 50 corresponding to the index finger.

In addition, when two or more tastes are selected from among a plurality of different tastes, the processor 140 transmits an electrical stimulus and a temperature stimulus corresponding to the selected taste to two or more finger gloves selected by the user from among the plurality of finger gloves, respectively. can For example, when a specific sweet taste and a specific salty taste are selected from among a plurality of different tastes, and the finger glove 50 corresponding to the index finger and the finger glove 50 corresponding to the thumb are selected by the user, the index finger Electrical stimulation and temperature stimulation corresponding to the selected sweet taste may be transmitted to the finger glove 50 corresponding to the finger, and electrical stimulation and temperature stimulation corresponding to the salty taste may be transmitted to the finger glove 50 corresponding to the thumb. have.

On the other hand, the finger glove 50 corresponding to each finger can be flowed to deliver temperature stimulation and electrical stimulation corresponding to each category according to the type of taste, such as salty taste, sweet taste, bitter taste, spicy taste, and sour taste. . For example, when a taste belonging to the sour taste category is selected, the finger glove 50 corresponding to the index finger is selected, and when a taste belonging to the mint or mint flavor category is selected, the finger glove 50 corresponding to the middle finger is selected. Temperature stimulation and electrical stimulation corresponding to each taste may be delivered through the.

6 is a diagram for explaining synchronization of an artificial taste transmitting apparatus and a sensory content display apparatus according to an embodiment of the present invention.

According to an embodiment of the present invention, when an image or picture is reproduced through the sensory content display apparatus 200 , the processor 140 corresponds to a taste expressed in the image or picture reproduced on the sensory content display apparatus 200 . The temperature stimulation caused by the temperature is transmitted through the temperature stimulation unit 121 provided at the distal end of each finger of the haptic glove 1, and electricity is generated by electric current of intensity and frequency corresponding to the taste expressed in the reproduced image or picture. The stimulus is transmitted through the electrodes 122 provided at the distal end of each finger of the haptic glove 1, and the image reproduced on the user's palm through the haptic module 123 provided in the palm of the haptic glove 1, or A pressure stimulus corresponding to the picture can be transmitted.

For example, when a scene of consuming a specific food is shown in the reproduced image of the sensory content display apparatus 200 , a temperature stimulus, an electrical stimulus, and a pressure stimulus corresponding to the taste of the specific food may be delivered to the user. Here, the pressure stimulus may be applied so that a texture similar to the texture of the specific food is felt.

Referring to FIG. 6 , a temperature corresponding to a taste expressed at at least one preset point in the timeline 63 of an image 61 to be reproduced through software provided in connection with the artificial taste delivery device 100 . , the strength of the current and the value of the frequency may be set.

At a point in time when a specific taste is expressed in the timeline 63 of the image 61 to be played back, the specific taste among “Sour”, “Sweet”, “Salty”, “Spicy” and “Bitter” on the setting menu 62 . By selecting any one taste corresponding to , the temperature stimulation and electrical stimulation by the current of temperature, intensity, and frequency corresponding to the selected taste may be set to be delivered at a time point at which the specific taste is expressed.

In addition, at a time point at which a specific taste is expressed in the timeline 63 of the image 61 to be reproduced, the temperature, the intensity of the current, and the frequency corresponding to the specific taste may be directly set.

This setting information is transmitted to the sensory content display apparatus 200, and the sensory content display apparatus 200 exchanges synchronization signals with the artificial taste transmitting apparatus 100 to express a preset taste in the timeline 63. At this point, the temperature stimulation and the electrical stimulation may be transmitted to the finger distal end of the haptic glove 1 .

For example, when an image in which an orange is displayed is displayed through the sensory content display apparatus 200 at the playback time of 15 minutes and 20 seconds among the entire timeline of a two-hour video, temperature stimulation by temperature corresponding to the taste of orange; Electrical stimulation by the current of the intensity and frequency corresponding to the taste of orange may be transmitted to the finger distal end of the haptic glove 1 .

On the other hand, the setting menu 62 is a menu for selecting an electrode to which a temperature stimulus and an electrical stimulus are to be transmitted and an intensity thereof among the electrodes of the distal end of each finger of the haptic glove 1, and a pressure stimulus by the haptic module 123 and its intensity. It may include a menu for selecting .

In addition, the processor 140 displays a list of bookmarks for tastes expressed in the reproduced image on the sensory content display apparatus 200, and when any one of the displayed bookmark lists is selected, the image is reproduced at the time point at which the selected taste is expressed. position can be moved.

For example, the processor 140 separately lists and stores the taste expressed at each of a plurality of time points in which the user sets the temperature stimulus and the electrical stimulus to be transmitted in the timeline of the image, and adds the list to a part of the reproduced image. You can control the display. When any one of the tastes displayed in the corresponding list is selected, the playback position of the image may be moved directly to the point in time at which the selected taste is expressed.

Furthermore, the processor 140 may display a list of restaurants that sell a recommended food menu corresponding to the selected taste and a recommended food menu within a preset radius from the user's location.

Accordingly, when the user selects any one restaurant from the displayed restaurant list, information on the corresponding restaurant and map information for guiding a road to the corresponding restaurant may be displayed on the display apparatus 200 .

To this end, the storage unit 110 may pre-store an image of food corresponding to a plurality of different tastes, information on restaurants selling the corresponding food, and map information in advance. Alternatively, images of food corresponding to a plurality of different tastes, information on restaurants selling the corresponding food, and map information may be received in real time from an external server (not shown) connected to a network through the communication unit 130 .

On the other hand, the processor 140 determines a finger distal end to which a stimulus is to be delivered according to a taste expressed in an image or a picture reproduced on the sensory content display device 200, among the finger distal portions of the haptic glove 1, and provides the determined finger distal end portion. The temperature stimulation unit 121 and the electrode 122 may be controlled to transmit the temperature stimulation and the electrical stimulation.

For example, the processor 140 may control the temperature stimulation and electrical stimulation to be transmitted to the index finger when the taste expressed in the image reproduced on the sensory content display apparatus 200 is sour and to the thumb when the taste is bitter. have.

7 is a view showing the appearance and structure of the HMD in an example in which the sensory content display apparatus of the present invention is implemented as an HMD.

The HMD 200 ′ of the present invention has a body 75 including a headband 71 , a vertical length adjustment unit 72 , a circumference length adjustment unit 73 , an audio output unit 74 , and a display unit 210 . ), an auxiliary device 76 in the form of a mask including the transfer unit 120 , and a protrusion 37 on which the transfer unit 220 is disposed.

The headband 71 is connected to the body part 75 and the length can be adjusted by the vertical length adjusting unit 72 and the circumferential length adjusting unit 73 . In this case, the headband 71 may be connected to the body part 75 so that it can be rotated while being worn on the user's head.

The audio output unit 74 is configured to output voice or sound. The audio output unit 74 may be implemented in the form of a headphone of a bone conduction method that transmits sound through a bone rather than the eardrum.

The body part 75 may include various buttons for controlling the display part 210 and the HMD 200'.

An auxiliary device 76 in the form of a mask may be disposed under the body portion 75 . The auxiliary device 76 is designed to cover the mouth and its surroundings of the user wearing the HMD 200 ′, and may include a transmission unit 120 and a microphone (not shown). As shown in FIG. 7 , the delivery unit 120 may be disposed on the protrusion 77 formed to protrude at an inner position corresponding to the user's mouth in the auxiliary device 76 .

Therefore, when the user sticks out his tongue and contacts the protrusion 77 or bites the protrusion 77 in his mouth, the temperature stimulation unit 121 and the electrode 122 of the delivery unit 120 touch the tongue to feel the taste. can

Meanwhile, the HMD 200 ′ may include a motion sensor (not shown) or a position sensor (not shown) in any one of components including the body part 75 .

The motion sensor may detect a user's motion by embedding at least one of a 3-axis gyroscope, a 3-axis acceleration sensor, and a 3-axis geomagnetic sensor.

The location sensor may position a location based on various methods using a Global Positioning System (GPS), a base station (Cell-ID), Wi-Fi, and the like.

The processor 140 senses a user's motion or location through a motion sensor or a location sensor of the HMD 200', and any one of a plurality of different tastes based on the sensed motion information or location information of the HMD 200' You can choose one.

For example, if the processor 140 recognizes that the sensed user's motion is performing a specific exercise, energy is insufficient when the physical strength of the body is consumed. , 800Hz) and temperature stimulation (falling from 35°C to 25°C) can be controlled to make the user feel the sweet taste.

In addition, when the sensed user's location is within a preset distance from a specific restaurant, the processor 140 selects a pre-stored taste corresponding to a representative menu sold at the corresponding restaurant, and electrical stimulation and temperature stimulation corresponding to the selected taste It can also be controlled to transmit . The taste corresponding to such a representative menu and set values of the intensity, frequency, and temperature of the current corresponding thereto may be provided by the corresponding restaurant.

Meanwhile, the processor 140 corresponds to a pre-selected taste according to the motion information of the HMD 200 ′ sensed by the motion sensor of the HMD 200 ′ or the position information of the HMD 200 ′ sensed by the position sensor. It is also possible to control the temperature of the temperature stimulation, the strength and frequency of the electric stimulation current in detail.

8 is a diagram for explaining a process of indirectly experiencing the taste of food in front of a user's eyes by using AR glasses in an example in which the sensory content display device of the present invention is implemented with AR glasses.

AR glasses 200'' are glasses-type electronic devices that implement Augmented Reality content on a transparent lens. Unlike the HMD 200', the AR glasses 200'' allow viewing of the real world beyond the lens with a virtual image displayed on the lens. The AR glasses 200'' may include a camera 82 capable of photographing the environment in front of the user in order to recognize the real world, analyze the recognized real world information, and provide appropriate information or content to the user.

Referring to FIG. 8 , the AR glasses 200 ″ may recognize the orange 81 placed in front of the user's eyes wearing the AR glasses 200 ″ through the camera 82 .

The AR glasses 200 ″ may transmit information on the recognized orange 81 to the artificial taste delivery device 100 , and the processor 140 generates a temperature stimulus by a temperature corresponding to the recognized orange 81 . It is transmitted through the temperature stimulation unit 121 provided at the distal end of each finger of the haptic glove 1 , and electrical stimulation by current of intensity and frequency corresponding to the recognized orange 81 is applied to each finger of the haptic glove 1 . It can be transmitted through the electrode 122 provided at the distal end.

In this case, various graphics related to taste expression such as the juice of the orange 81 bursting may be displayed on the display unit 83 of the AR glasses 200 ″.

At this time, like the orange 81 , a mapping table in which the intensity (μA), frequency (Hz), and temperature (°C) of the current, which are the conditions for the corresponding taste to be expressed according to the taste that can express the corresponding food for each type of food, are mapped. It may be pre-stored in the storage unit 110 .

The storage unit 110 may store food input from the user through the user interface and setting information on temperature, current strength, and frequency to be mapped to the taste of the food.

9 is a flowchart for briefly explaining a method for applying an artificial taste according to an embodiment of the present invention.

First, set values of temperature (℃), current strength (μA), and frequency (Hz) corresponding to the image or picture and the taste expressed in the image or picture are received and stored (S910).

In this case, the temperature (℃), the current intensity (μA), and the frequency (Hz) corresponding to the taste expressed at at least one preset point in the timeline of the image reproduced through the sensory content display device are set value can be stored.

Thereafter, an image or picture is reproduced through the sensory content display device (S920).

Thereafter, the temperature stimulation by temperature corresponding to the taste expressed in the reproduced image or picture is transmitted through the temperature stimulation unit provided at the tip of each finger of the glove, and the intensity and frequency corresponding to the taste expressed in the reproduced image or picture The electrical stimulation by the electric current of It transmits (S930).

At this time, when a preset time point is reached during image reproduction, artificial taste to deliver temperature stimulation by temperature corresponding to the taste expressed at the preset time point and electrical stimulation by electric current of intensity and frequency corresponding to the expressed taste Operations of the delivery device and the sensory content display device may be synchronized.

In addition, the motion or position of the sensory content display device is sensed through a motion sensor or a position sensor attached to a sensory content display device implemented with a head mounted display (HMD) or AR glasses, and a user interface provided outside the sensory content display device It is possible to adjust the temperature of the transmitted temperature stimulus and the intensity and frequency of the electric stimulus transmitted based on the operation or the sensed motion information or location information of the sensory content display device.

In addition, it is possible to determine the finger distal portion to which the stimulus is to be delivered according to the taste expressed in the reproduced image or picture among the finger distal portions of the glove, and to deliver the temperature stimulation and electrical stimulation to the temperature stimulation unit and electrodes provided at the determined finger distal portion.

In addition, in the taste expressed in the reproduced image or picture, it is possible to deliver a temperature stimulus in which the temperature varies with time within a preset range.

In addition, in the taste expressed in the reproduced image or picture, at least one of the temperature or the intensity of the current may deliver a temperature stimulus or an electric stimulus that varies with time within a preset range.

10 is a flowchart for briefly explaining a method for applying an artificial taste according to another embodiment of the present invention.

First, set values of temperature (℃), current strength (μA), and frequency (Hz) corresponding to the image or picture and the taste expressed in the image or picture are received and stored (S1010).

Thereafter, a specific food is sensed by a camera provided on a portion of the AR glass (S1020).

Thereafter, the temperature stimulation by the temperature corresponding to the sensed specific food is transmitted through the temperature stimulation unit provided at the distal end of each finger of the glove, and the electrical stimulation by the current of the intensity and frequency corresponding to the sensed specific food is applied to each of the gloves. It is transmitted through an electrode provided at the distal end of the finger (S1030).

According to various embodiments of the present invention as described above, it is possible to transmit a stimulus in which the intensity, frequency, and temperature of electric current are fused to the user so that the user can feel and adjust the artificial senses for various tastes.

In addition, by displaying an image capable of maximizing the artificial taste experience or providing a haptic stimulus, a user's need for pleasure or convenience obtained from the artificial taste experience may be satisfied.

Meanwhile, the artificial taste realization method according to various embodiments described above may be implemented as a program and stored in various recording media. That is, a computer program that is processed by various processors to execute the method for implementing the artificial taste described above may be used in a state stored in a recording medium.

As an example, i) receiving and storing set values of temperature (℃), current intensity (μA) and frequency (Hz) corresponding to the image or picture and the taste expressed in the image or picture, ii) sensory content reproducing an image or picture through a display device; iii) transmitting a temperature stimulation by a temperature corresponding to the taste expressed in the reproduced image or picture through a temperature stimulation unit provided at the distal end of each finger of the glove, and the reproduced image Alternatively, electrical stimulation by electric current of intensity and frequency corresponding to the taste expressed in the picture is transmitted through electrodes provided at the distal end of each finger of the glove, and the user's palm is passed through a haptic module provided in the palm of the glove. A non-transitory computer readable medium in which a program for performing a step of delivering a pressure stimulus corresponding to an image or a picture reproduced on the screen is stored (non-transitory computer readable medium) may be provided.

The non-transitory readable medium refers to a medium that stores data semi-permanently, rather than a medium that stores data for a short moment, such as a register, cache, memory, and the like, and can be read by a device. Specifically, the various applications or programs described above may be provided by being stored in a non-transitory readable medium such as a CD, DVD, hard disk, Blu-ray disk, USB, memory card, ROM, and the like.

In the above, preferred embodiments of the present invention have been illustrated and described, but the present invention is not limited to the specific embodiments described above, and it is common in the technical field to which the present invention pertains without departing from the gist of the present invention as claimed in the claims. Various modifications are possible by those having the knowledge of, of course, and these modifications should not be individually understood from the technical spirit or prospect of the present invention.

100: artificial taste delivery device 110: storage unit
120: transfer unit 130: communication unit
140: processor 200: immersive content display device
200': HMD 200'': AR Glass

Claims (10)

A method for applying artificial taste using gloves and a sensory content display device, the method comprising:
receiving and storing set values of temperature (℃), current strength (μA) and frequency (Hz) corresponding to the image or picture and the taste expressed in the image or picture;
reproducing the image or picture through the sensory content display device; and
The temperature stimulation by temperature corresponding to the taste expressed in the reproduced image or picture is transmitted through the temperature stimulation unit provided at the distal end of each finger of the glove, and the intensity corresponding to the taste expressed in the reproduced image or picture, and Transmits electrical stimulation by a frequency current through electrodes provided at the distal end of each finger of the glove, and corresponds to the image or picture reproduced on the user's palm through a haptic module provided at the palm of the glove Including; delivering a pressure stimulus to be
The storing step is
Setting values of temperature (°C), current strength (μA), and frequency (Hz) corresponding to the taste expressed at at least one preset point in the timeline of the image reproduced through the sensory content display device save the
The delivering step is
When the preset time point is reached during the reproduction of the image, the temperature stimulation by the temperature corresponding to the taste expressed at the preset time point and the electrical stimulation by the current of the intensity and frequency corresponding to the expressed taste synchronize the operations of the artificial taste delivery device and the sensory content display device to deliver,
The method of applying the artificial taste,
displaying a list of bookmarks for tastes expressed in the reproduced image on the sensory content display device;
when any one of the displayed bookmark list is selected, moving the playback position of the image to a time point at which the selected taste is expressed; and
and displaying a recommended food menu corresponding to the selected taste and a list of restaurants selling the recommended food menu within a preset radius from the user's location. delete According to claim 1,
The method further comprises: sensing the motion or position of the sensory content display device through a motion sensor or a position sensor attached to a sensory content display device implemented with a head mounted display (HMD) or AR glasses;
The delivering step is
Based on manipulation through a user interface provided outside the sensory content display device or the sensed motion information or location information of the sensory content display device, the temperature of the transmitted temperature stimulus, the intensity of the electric current delivered, and A method of applying artificial taste, characterized in that the frequency is adjusted. According to claim 1,
The delivering step is
Among the finger tips of the glove, a finger tip to which a stimulus is to be delivered is determined according to the taste expressed in the reproduced image or picture, and the temperature stimulation and electrical stimulation are delivered to the temperature stimulation part and electrodes provided at the determined finger tip part. Artificial taste application method, characterized in that. According to claim 1,
The delivering step is
In the taste expressed in the reproduced image or picture, the artificial taste application method, characterized in that the temperature is within a preset range and a temperature stimulus that varies with time is delivered. According to claim 1,
The delivering step is
In the taste expressed in the reproduced image or picture, at least one of temperature or current strength is within a preset range, and temperature stimulation or electrical stimulation that varies with time is delivered. According to claim 1,
The method for applying artificial taste, characterized in that the temperature, the intensity of the current, and the frequency corresponding to the taste expressed in the reproduced image or picture are in the range of 0 to 40 °C, 0 to 250 μA, and 0 to 800 Hz, respectively. delete In an artificial taste delivery device using gloves and a sensory content display device,
a storage unit for receiving and storing set values of temperature (℃), current strength (μA) and frequency (Hz) corresponding to the image or picture and the taste expressed in the image or picture;
A haptic module that transmits a temperature stimulus through a temperature stimulation unit provided at the distal end of each finger of the glove, transmits an electrical stimulation through an electrode provided at the distal end of each finger of the glove, and is provided in the palm of the glove a transmission unit that transmits pressure stimulation through; and
When the image or picture is reproduced through the sensory content display device, a temperature stimulus corresponding to a taste expressed in the reproduced image or picture is transmitted, and a temperature corresponding to the taste expressed in the reproduced image or picture is transmitted. A processor that transmits electrical stimulation by current of intensity and frequency, and controls the transmitting unit to deliver pressure stimulation corresponding to the reproduced image or picture;
The processor is
Setting values of temperature (°C), current strength (μA), and frequency (Hz) corresponding to the taste expressed at at least one preset point in the timeline of the image reproduced through the sensory content display device store in the storage unit,
When the preset time point is reached during the reproduction of the image, the temperature stimulation by the temperature corresponding to the taste expressed at the preset time point and the electrical stimulation by the current of the intensity and frequency corresponding to the expressed taste synchronize the operations of the artificial taste delivery device and the sensory content display device to deliver,
A bookmark list for a taste expressed in the reproduced image is displayed on the display device, and when any one of the displayed bookmark list is selected, the playback position of the image is moved to a time point at which the selected taste is expressed, and the selected The sensory content display apparatus, characterized in that the sensory content display device is controlled to display a recommended food menu corresponding to the taste and a list of restaurants selling the recommended food menu within a preset radius from the user's location. A computer program stored in a recording medium for executing the artificial taste application method of any one of claims 1, 3 to 7.

Images