WO2018194227A1 - Three-dimensional touch recognition device using deep learning and three-dimensional touch recognition method using same - Google Patents

Abstract

One embodiment of the present invention provides: a three-dimensional touch recognition device for enabling a user input to be performed on a soft elastic material, and various user inputs to be processed by being recognized and determined using deep learning; and a three-dimensional touch recognition method using same. The three-dimensional touch recognition device using deep learning, according to one embodiment of the present invention, comprises: an input unit provided with a sheet having a user input applied to the outer surface thereof, and having a function of being restored to the initial shape thereof when the user input is released; a plurality of markers arranged along the inner surface of the sheet; a capturing unit for collecting marker images which are images of markers changing according to the user input; a lighting unit for irradiating light towards the inner surface of the sheet; and an analysis unit for generating a three-dimensional pattern by analyzing the marker images, and outputting a deep learning result value by means of a repetitive operation of inputting data associated with the three-dimensional pattern in a deep learning algorithm.

Description

3D touch recognition device using deep learning and 3D touch recognition method using same

The present invention relates to a three-dimensional touch recognition device using deep learning and a three-dimensional touch recognition method using the same, and more particularly, to allow a user input to be performed on a flexible soft material, a variety of users using deep learning A three-dimensional touch recognition apparatus capable of recognizing, determining, and processing an input, and a three-dimensional touch recognition method using the same.

Commonly used input devices include a mouse, a keyboard, a touch pad, a trackball, and the like, which may be used by the user to grab or touch the casing and the main part of the device with a proper force and move and click. Sophisticated operation is required.

However, in the case of a disabled person, a child, or an elderly person, there is a problem in that it may be difficult or impossible to form an elaborate operation required to use the input device having the above configuration.

In addition, such devices are being developed to facilitate the use of such users, but it is common to use a high-level technology such as voice recognition or to adopt a special structure, and thus, most of the structures are complicated and production costs are high. have.

Recently, machine learning, including deep learning, has contributed greatly to the activation and accuracy of the pattern recognition field with various hardware developments including GPUs that can support the development of IoT technology and big data processing. This can be used to improve the diversity and accuracy of perception.

In Korean Patent No. 10-1719278 (name of the invention: deep learning framework and image recognition method for visual content-based image recognition), deep learning technology is modularized, in / out parameter property extraction and training data for each module Equipped with a content-based deep learning analysis tool to automate set analysis and deep learning scenarios, and a parameter property interworking module for interlocking parameter properties of IN / OUT between modules through the content-based deep learning analysis tool, and interworking between modules is possible. An analysis result repository that stores the analysis results through the dynamic call interface interworking module, the standard API interface integration module between the modules, the one-pass integration module and the deep learning analysis tool integrating the task analysis, results and confirmation of the modules into one. An integrated GUI framework is disclosed.

(Preceding Patent Document) Republic of Korea Patent Registration 10-1719278

An object of the present invention for solving the above problems is to implement a user input such as pressing, moving, etc. without the need for complicated operation.

An object of the present invention is to analyze a three-dimensional pattern input to a device using a deep learning algorithm.

The technical problem to be achieved by the present invention is not limited to the technical problem mentioned above, and other technical problems not mentioned above may be clearly understood by those skilled in the art from the following description. There will be.

The configuration of the present invention for achieving the above object, the input unit having a sheet having a function to restore the initial shape when the user input acts on the outer surface and the user input is resolved; A plurality of markers arranged and arranged along an inner side surface of the sheet; An imaging unit which collects a marker image as an image of the marker that changes in response to the user input; An illumination unit irradiating light toward an inner surface of the sheet; And an analysis unit configured to generate a 3D pattern by analyzing the marker image and to output a deep learning result through an iterative operation of inputting data of the 3D pattern into a deep learning algorithm.

In an embodiment of the present invention, the deep learning algorithm may be any one of a deep neural network, a convolutional neural network, or a cyclic neural network.

In an embodiment of the present disclosure, the analysis unit may determine the three-dimensional pattern by a position change or a brightness change of a size point marker, which is a marker that is recognized as the largest size among the plurality of markers. .

In an embodiment of the present disclosure, the analysis unit may be configured by changing the size or shape of the size auxiliary marker positioned within a predetermined range around the size point marker, or changing the brightness of the size point marker. The dimensional pattern can be determined.

In an embodiment of the present disclosure, the analysis unit may determine the three-dimensional pattern by changing the position or changing the brightness of the brightness point marker, which is a marker recognized among the plurality of the markers with the highest brightness. .

In an embodiment of the present disclosure, the analysis unit may be configured by changing the position or brightness of the brightness point marker, and changing the size or shape of the brightness assist marker positioned within a predetermined range around the brightness point marker. The dimensional pattern can be determined.

In an embodiment of the present invention, the marker may be formed in a circular shape.

According to an embodiment of the present disclosure, a start notification unit may further include a start notification unit configured to perform visual notification when a start input, which is the first user input among the user inputs, acts on the outer surface of the sheet.

In an embodiment of the present disclosure, the deep learning result value may be received from the analysis unit, and the control unit may further include a control unit for transmitting a control signal corresponding to the deep learning result value to an external device.

The configuration of the present invention for achieving the above object, (i) the step of the user input acting on the sheet; (ii) transferring the marker image photographed by photographing the inner surface of the sheet per unit time to the analyzer; (iii) the analysis unit analyzing the marker image to generate a three-dimensional pattern by changing the position or brightness of the marker; And (iv) inputting data of the three-dimensional pattern into a deep learning algorithm and outputting a deep learning result value.

In the embodiment of the present invention, in the step (iii), the analysis unit, the three-dimensional pattern by changing the position of the size point marker that is the marker that is recognized the largest size among the plurality of the markers; You can judge.

In an embodiment of the present disclosure, in the step (iii), the analysis unit may generate the three-dimensional pattern by changing the position of the brightness point marker, which is a marker recognized by the imaging unit among the plurality of markers having the highest brightness. You can judge.

In an embodiment of the present invention, in the step (iii), it is possible to measure the horizontal displacement of the three-dimensional displacement of the marker by changing the position of the marker.

In an embodiment of the present invention, in the step (iii), it is possible to measure the vertical displacement of the three-dimensional displacement of the marker by changing the brightness of the marker.

The effect of the present invention according to the configuration as described above, so that the user input is performed on a soft material with elasticity, through which a variety of user input can be recognized, determined and processed, so that sophisticated behavior such as the disabled, patients, infants It is convenient for difficult users.

In addition, the effect of the present invention is that by analyzing and determining the three-dimensional pattern input to the device by the user input using a deep learning algorithm, it is possible to improve the accuracy for various user patterns.

The effects of the present invention are not limited to the above-described effects, but should be understood to include all the effects deduced from the configuration of the invention described in the detailed description or claims of the present invention.

1 is a perspective view of a three-dimensional touch recognition device according to an embodiment of the present invention.

2 is a block diagram of a 3D touch recognition device according to an embodiment of the present invention.

3 is a schematic diagram of the matter that the first user input acts on the sheet according to an embodiment of the present invention.

4 is a schematic diagram of the matter that the pressing operation acts on the sheet according to an embodiment of the present invention.

5 is a schematic diagram of the matter that the movement operation in one direction for the sheet according to an embodiment of the present invention.

6 is a schematic diagram of the matter that the movement operation in the other direction with respect to the sheet according to an embodiment of the present invention.

7 is an image in which a three-dimensional pattern is imaged according to an exemplary embodiment.

The most preferred embodiment of the present invention, the input unit having a sheet having a function to restore the initial shape when the user input is applied to the outer surface and the user input is resolved; A plurality of markers arranged and arranged along an inner side surface of the sheet; An imaging unit which collects a marker image as an image of the marker that changes in response to the user input; An illumination unit irradiating light toward an inner surface of the sheet; And an analysis unit configured to generate a 3D pattern by analyzing the marker image and to output a deep learning result through an iterative operation of inputting data of the 3D pattern into a deep learning algorithm.

Hereinafter, with reference to the accompanying drawings will be described the present invention. As those skilled in the art would realize, the described embodiments may be modified in various different ways, all without departing from the spirit or scope of the present invention. In the drawings, parts irrelevant to the description are omitted in order to clearly describe the present invention, and like reference numerals designate like parts throughout the specification.

Throughout the specification, when a part is said to be "connected (connected, contacted, coupled)" with another part, it is not only "directly connected" but also "indirectly connected" with another member in between. "Includes the case. In addition, when a part is said to "include" a certain component, this means that it may further include other components, without excluding the other components unless otherwise stated.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. As used herein, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, action, component, part, or combination thereof described on the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a perspective view of a three-dimensional touch recognition device according to an embodiment of the present invention, Figure 2 is a block diagram of a three-dimensional touch recognition device according to an embodiment of the present invention.

As shown in FIGS. 1 and 2, the 3D touch recognition apparatus using the deep learning of the present invention includes a sheet 110 having a function of restoring to an initial shape when a user input is applied to an outer surface and the user input is cancelled. An input unit 100 provided; Markers 120 are arranged in a plurality arranged along the inner surface of the sheet 110; An imaging unit 200 collecting a marker image as an image of the marker 120 that changes in response to a user input; An illumination unit 300 for irradiating light to the inner surface of the sheet 110; And an analyzing unit 400 for generating a 3D pattern by analyzing the marker image and outputting a deep learning result through an iterative operation of inputting data of the 3D pattern to the deep learning algorithm.

The marker 120 may be formed in a circular shape.

In an embodiment of the present invention, the marker 120 is described as being circular, but is not necessarily limited thereto, and may be formed in various shapes such as an ellipse, a square, and a polygon. In addition, the marker 120 may be a figure formed by a line, or may be a form in which a color is filled in the figure.

The installation pattern of the plurality of markers 120 may have a predetermined number of rows and columns, and various patterns such as the same spacing between rows and rows, columns and columns, or an arrangement of concentric circles may be considered.

Preferably, the plurality of markers 120 may form a square array.

In this case, the change in the marker image may be generated by a change in the spacing between the columns or rows of the array formed by the marker 120 or a change in the shape of the array. In addition, the change in the marker image may be generated by a change in the size or shape of the marker 120 itself.

The user input may be performed by stopping or operating a body part of the user while the body part of the user is in contact with the outer surface of the sheet 110.

A part of the user's entity may mean any body part capable of pressing, releasing, moving, etc. in contact with one surface of the seat 110 by using a hand or a finger, a foot or a toe, an elbow or a knee.

In addition, the user who generates the user input may include not only a person but also a machine, a robot, and other devices.

In the case of pressing, the pressing pressure and the time for holding the pressing are not limited. In the case of pressing, the time required for releasing the pressing is not limited to a specific range. Furthermore, when a press occurs for a predetermined time and then a press release immediately occurs, this may be referred to as a click. Movement means that the user moves from one point of one surface of the sheet 110 to another while maintaining the pressing state. The moving path is not limited to a specific one, and may include a straight line, a curved line, a curved line, Circles, ellipses, arcs, splines and the like, but may not be limited thereto.

The imaging unit 200 may include an element such as a CCD, but is not limited thereto.

In addition, the imaging unit 200 may include a wide-angle lens to enable photographing of the entire photographing surface.

The sheet 110 may be made of an elastic material.

The remaining portion of the input unit 100 except for the sheet 110 may be formed of a flexible material, such as the sheet 110, or may be formed of a material having no elasticity, unlike the sheet 110.

In addition, the input unit 100 may have a shape in which the internal space is opened from the outside, and as shown in FIG. 2, the input space 100 may have a shape in which the internal space is closed from the outside.

The lighting unit 300 may include an LED lamp.

And, if the user input does not occur for more than a predetermined time, a configuration such as automatically turned off may be adopted.

3 is a schematic diagram of the matter that the first user input to the sheet 110 according to an embodiment of the present invention, Figure 4 is a pressing operation for the seat 110 according to an embodiment of the present invention Figure 5 is a schematic diagram of the action, Figure 5 is a schematic diagram of the action of the movement action in one direction with respect to the sheet 110 according to an embodiment of the present invention. And, Figure 6 is a schematic diagram of the matter that the movement action in a different direction with respect to the sheet 110 according to an embodiment of the present invention, Figure 7 is a three-dimensional pattern is imaged according to an embodiment of the present invention Image.

3 (a), 4 (a), 5 (a) and 6 (a) may be cross-sectional views of the input unit to which the respective user inputs act. 3 (b), 4 (b), 5 (b) and 6 (b) may be plan views of the inner surface of the sheet on which the respective user inputs are acting.

3 to 6, reference numeral 121 may indicate a size point marker 121 or a brightness point marker 121. In addition, reference numeral 122 may indicate a size assist marker 122 or a brightness assist marker 122.

The analysis unit 400 may determine the 3D pattern by changing the position or changing the brightness of the size point marker 121, which is a marker that is recognized as the largest size among the plurality of markers 120 by the imaging unit 200. have. Alternatively, the analysis unit 400 determines the 3D pattern by changing the position or changing the brightness of the brightness point marker 121, which is a marker recognized among the plurality of markers 120 by the imaging unit 200. can do.

The size point marker 121 and the brightness point marker 121 may represent the largest marker 120 and the brightest marker 120 based on the marker image recognized by the imaging unit 200. Particularly, in the case of the brightness point marker 121, the imaging unit 200 and the lighting unit 300 are located in the same direction, and thus the brightness increases when the position of the marker 120 approaches the imaging unit 200. It may be.

The three-dimensional pattern may be a change in position by two-dimensional horizontal displacement and a vertical displacement perpendicular to this horizontal displacement. In an embodiment of the present invention, as shown in FIGS. 3 to 6, the two-dimensional horizontal displacement may be expressed as a displacement on the x-y plane, and the vertical displacement may be expressed as a displacement with respect to the z axis perpendicular to the x-y plane.

As an embodiment, the matter of determining the 3D pattern by changing the position or changing the brightness of the marker 120 having the largest size will be described.

As shown in FIG. 3, when a user input is applied to the sheet 110 for the first time, the size point marker 121 may be recognized. 4 to 6, when the pressing operation is performed after the pressing operation is performed on the outer surface of the sheet 110 by using the hand, the imaging unit 200 may be pressed by pressing and moving. The position of the size point marker 121 that is recognized as the largest size may be changed. Here, the movement of the size point marker 121 may be recognized by changing the position of the marker 120 having the largest size recognized by the imaging unit 200 among the plurality of markers 120. That is, the marker 120 itself may not be moved but may recognize the position change of the size point marker 121 to determine the movement of the size point marker 121.

And, the analysis unit 400, the position change or brightness change of the size point marker 121, and the change in the size or shape of the size auxiliary marker 122 located within a predetermined range around the size point marker 121. The three-dimensional pattern can be judged by.

The predetermined range is L ranges (L is an arbitrary integer) around the matrix range or the size point markers 121 forming an array of NXMs (N and M are arbitrary integers) around the size point markers 121. The marker may be in the range of a circle included. However, the present invention is not limited thereto. In addition, it is not excluded that the predetermined range is the entire inner surface of the sheet 110. (The predetermined range is shown by the dotted line in FIGS. 5 and 6.)

When the size point marker 121 is repositioned, the size or shape of the size auxiliary marker 122 located within a predetermined range is changed, and data about the size marker 12 is previously stored in the analysis unit 400 or is learned by deep learning. It may be stored in the analysis unit 400.

In addition, the analysis unit 400 analyzes not only an image of a position change or a brightness change of the size point marker 121, but also an image of a change of the size or shape of each of the plurality of individual size auxiliary markers 122 and three-dimensionally. The pattern can be determined.

Specifically, as shown in FIGS. 5 and 6, when the size point marker 121 performs the horizontal displacement movement, when the position change of the size point marker 121 is performed, the size assistance within a predetermined range of the 3 × 3 array is performed. The size or shape of the marker 122 is changed, and the three-dimensional pattern may be determined by comprehensively determining the movement of the size point marker 121 and the size or shape change of the size auxiliary marker 122.

The two-dimensional horizontal displacement of the hand can be measured by changing the position of the size point marker 121 as described above, and the vertical displacement of the hand can be measured by changing the brightness of the size point marker 121. In conclusion, three-dimensional patterns can be determined for user input by hand.

As another embodiment, the matter of determining the 3D pattern by changing the position or changing the brightness of the marker 120 recognized as the highest brightness will be described.

As shown in FIG. 3, when the user input is applied to the sheet 110 for the first time, the brightness point marker 121 may be recognized. 4 to 6, when the pressing operation is performed after the pressing operation is performed on the outer surface of the sheet 110 by using the hand, the imaging unit 200 may be pressed by pressing and moving. The position of the brightness point marker 121 recognized as the highest brightness may be changed. Here, the movement of the brightness point marker 121 may be recognized by changing the position of the marker 120 that is recognized as the highest brightness among the plurality of markers 120 by the imaging unit 200. That is, the marker 120 itself does not move but recognizes the positional change of the brightness point marker 121 to determine the movement of the brightness point marker 121.

In addition, the analysis unit 400 changes the position or the brightness of the brightness point marker 121 and the change in the size or shape of the brightness assist marker 122 positioned within a predetermined range around the brightness point marker 121. The three-dimensional pattern can be judged by.

The predetermined range is L ranges (L is an arbitrary integer) around the matrix range or the brightness point markers 121 forming an array of NXMs (N and M are arbitrary integers) around the brightness point markers 121. The marker may be in the range of a circle included. However, the present invention is not limited thereto. In addition, it is not excluded that the predetermined range is the entire inner surface of the sheet 110. (The predetermined range is shown by the dotted line in FIGS. 5 and 6.)

When the brightness point marker 121 is repositioned, the size or shape of the brightness subsidiary marker 122 positioned within a predetermined range is changed, and data about the brightness subsidiary marker 122 is stored in advance in the analysis unit 400 or is learned by deep learning. It may be stored in the analysis unit 400.

In addition, the analysis unit 400 analyzes not only an image of a position change or a brightness change of the brightness point marker 121, but also an image of a change in size or shape of each of the plurality of individual brightness assistant markers 122 to be three-dimensional. The pattern can be determined.

Specifically, as shown in FIGS. 5 and 6, when the brightness point marker 121 performs the horizontal displacement movement, when the position change of the brightness point marker 121 is performed, brightness assistance within a predetermined range of the 3 × 3 array is performed. The size or shape of the marker 122 changes, and the three-dimensional pattern may be determined by comprehensively determining the movement of the brightness point marker 121 and the change in the size or shape of the brightness assist marker 122.

The two-dimensional horizontal displacement of the hand can be measured by changing the position of the brightness point marker 121 as described above, and the vertical displacement of the hand can be measured by changing the brightness of the brightness point marker 121. In conclusion, three-dimensional patterns can be determined for user input by hand.

Specifically, the matter of determining the horizontal displacement and the vertical displacement will be described. (Hereinafter, the brightness point marker 121 will be described as a reference.)

As shown in FIG. 3, when the user performs an initial pressing operation on the outer surface of the sheet 110, any one marker 120 is picked up at a portion where the pressing operation of the inner surface of the sheet 110 is performed. The brightness may increase while moving closer to the unit 200. In addition, the marker 120 having increased brightness may be recognized as the brightness point marker 121, and the first point P1 may be set as a start coordinate.

And, as shown in Figure 4, when the user performs a pressing operation with a stronger force than the initial pressing operation on the outer surface of the sheet 110, the brightness point marker 121 by the pressing operation of a stronger force It may be recognized that the brightness is further increased, and the vertical displacement due to the user input may be measured by the brightness change of the brightness point marker 121. That is, it can be measured by recognizing that the three-dimensional coordinates are changed from the first point (P1) to the second point (P2).

When the brightness is not increased but the brightness is decreased, that is, when the brightness point marker 121 is spaced apart from the image pickup unit 200 and the brightness decreases, a vertical displacement may be formed in a direction opposite to that in FIG. 4. Can be.

As shown in FIG. 5, when the user performs a movement operation in one direction with respect to the outer surface of the sheet 110, the marker 120 having the greatest brightness may be changed, and such a marker 120 may be changed. ), It may be recognized that the brightness point marker 121 moves from the second point P2 to the third point P3.

In addition, as shown in FIG. 6, when the user performs a movement operation in a different direction with respect to the outer surface of the sheet 110, the marker 120 having the greatest brightness may be changed. As a result of the change of 120, the brightness point marker 121 may be recognized to move from the third point P3 to the fourth point P4.

In addition, a three-dimensional pattern as shown in FIG. 7 may be formed by the complex displacement of the horizontal displacement and the vertical displacement as described above.

The three-dimensional touch recognition apparatus using the deep learning of the present invention can detect not only a three-dimensional pattern but also a force in a vertical direction or a horizontal direction.

As shown in FIG. 4, when the pressing operation is performed, a vertical displacement is formed, and the vertical force may be measured by multiplying the vertical displacement by the elastic modulus of the sheet 110.

Therefore, in order to detect the force in the vertical direction, the sheet 110 may be manufactured so that the elastic modulus of the sheet 110 is the same at each point of the sheet 110.

As shown in FIG. 5, when the moving operation is performed, a horizontal displacement is formed. In this case, the horizontal force is a vertical force N and the sheet 110 applied to the seat 110 by the pressing operation of the user's hand. It can be measured as a force corresponding to the frictional force calculated by the product of the friction coefficient (μ) of).

Here, the friction coefficient of the sheet 110 may be determined by referring to the friction coefficient reference data table stored in the analysis unit 400.

As shown in FIG. 5, since the pressing operation is performed on the sheet 110 to generate a horizontal displacement by moving a hand (user input) in a vertical displacement state, the coefficient of friction of the sheet 110 is determined by the sheet 110. It may be different from the surface friction coefficient as a property of the surface of.

The coefficient of friction of the sheet 110 used in the horizontal force detection method of the present invention is determined in consideration of the state in which the shape of the sheet 110 is bent by the pressing operation. It can be determined by the data obtained through mechanical experiments on the sides.

The force in the horizontal direction may be calculated by multiplying the friction coefficient of the sheet 110 determined by the above process with the vertical force, which is the vertical force.

The deep learning algorithm may be one of a deep neural network, a convolutional neural network, or a cyclic neural network.

The deep learning algorithm used in the 3D touch recognition device using the deep learning of the present invention may be a known technique.

In the exemplary embodiment of the present invention, the neural network described above is used as the deep learning algorithm, but the present invention is not limited thereto.

As described above, each three-dimensional pattern may be represented by a change in three-dimensional coordinates, and as shown in FIG. 7, each three-dimensional pattern may be represented and stored as a three-dimensional image, respectively.

The analysis unit 400 performs a learning using a deep learning algorithm for each 3D pattern, and analyzes and determines a 3D pattern by a user input based on the learned data to derive a deep learning result value. Can be.

Specifically, even though the same user performs user input for input of the same 3D pattern, the position of each coordinate forming the 3D pattern may be different, but the analysis unit 400 inputs the 3D pattern input by the user. By analyzing and determining based on the learned date, the user can recognize the user input of the intended 3D pattern. Accordingly, even if the 3D pattern input by the user is determined to be the same pattern and there is a difference in the 3D coordinates, the analyzer 400 determines that the pattern generated by the user input is the same as the 3D pattern stored in advance. It may be determined as a pattern, and a deep learning result value thereof may be output.

The 3D touch recognition apparatus using the deep learning of the present invention may further include a start notification unit that performs visual notification when a start input, which is the first user input among the user inputs, acts on the outer surface of the sheet 110.

When the first user input is applied to the outer surface of the sheet 110 as described above, the formation of the three-dimensional pattern may be started, the user may need a method for checking whether the first user input is recognized.

Therefore, when the start input, which is the first user input, acts on the outer surface of the sheet 110, the start notification unit emits light so that the user can confirm the start of formation of the three-dimensional pattern.

In an embodiment of the present disclosure, the start notification unit emits light so that the 3D pattern formation may be confirmed. However, the start notification unit is not limited thereto, and the user may confirm the start of formation of the 3D pattern using sound or vibration. You can do that.

The 3D touch recognition apparatus using the deep learning of the present invention further includes a controller 500 that receives the deep learning result from the analysis unit 400 and transmits a control signal corresponding to the deep learning result to external equipment. can do.

Each three-dimensional pattern can be matched to a specific instruction. Specifically, when the 3D pattern as shown in FIG. 7 is formed, and if it is matched that the start of the operation of the external equipment is performed, the controller 500 analyzes and determines the input of the 3D pattern as shown in FIG. 7 to the controller 500. The deep learning result value is output, and the controller 500 may transmit a control signal matching the deep learning result value to the external device to start the operation of the external device.

A computer having a three-dimensional touch recognition device using the deep learning of the present invention can be manufactured.

The deep learning result value for the 3D pattern processed by the analyzer 400 may be transmitted to the computer through the control unit 500, and a command corresponding to the 3D pattern may be transmitted to the computer so that the computer may execute the command. .

The robot having the 3D touch recognition device using the deep learning of the present invention can be manufactured.

As a specific embodiment, the end of the robot performing the work corresponding to the three-dimensional pattern can be moved, the three-dimensional touch recognition device using the deep learning of the present invention can function as a steering device for controlling the robot.

Hereinafter, a three-dimensional touch recognition method using a three-dimensional touch recognition device using the deep learning of the present invention will be described.

In a first step, user input may act on the sheet 110.

In the second step, the imaging unit 200 may transfer the photographed marker image to the analysis unit 400 by photographing the inner surface of the sheet 110 per unit time.

Here, the unit time may be in milliseconds (ms), and the photographing may be performed in units smaller than milliseconds (ms) in order to improve accuracy.

In a third step, the analysis unit 400 may analyze the marker image to generate a three-dimensional pattern by changing the position or brightness of the marker 120.

Here, the analysis unit 400 may determine the 3D pattern by changing the position of the size point marker 121, which is a marker that is recognized as the largest size among the plurality of markers 120 by the imaging unit 200. . Alternatively, the analyzer 400 may determine the 3D pattern by changing the position of the brightness point marker 121, which is a marker recognized among the plurality of markers 120 by the imaging unit 200 with the highest brightness. .

In a third step, the horizontal displacement of the three-dimensional displacement of the marker 120 can be measured by changing the position of the marker 120. The vertical displacement of the three-dimensional displacement of the marker 120 may be measured by changing the brightness of the marker 120.

In a fourth step, the deep learning result may be output by inputting data of the 3D pattern to the deep learning algorithm.

The foregoing description of the present invention is intended for illustration, and it will be understood by those skilled in the art that the present invention may be easily modified in other specific forms without changing the technical spirit or essential features of the present invention. will be. Therefore, it should be understood that the embodiments described above are exemplary in all respects and not restrictive. For example, each component described as a single type may be implemented in a distributed manner, and similarly, components described as distributed may be implemented in a combined form.

The scope of the present invention is represented by the following claims, and it should be construed that all changes or modifications derived from the meaning and scope of the claims and their equivalents are included in the scope of the present invention.

Claims (16)

An input unit including a sheet having a function of restoring an initial shape when a user input acts on an outer surface and the user input is cancelled; A plurality of markers arranged and arranged along an inner side surface of the sheet; An imaging unit which collects a marker image as an image of the marker that changes in response to the user input; An illumination unit irradiating light toward an inner surface of the sheet; And A deep learning unit for generating a 3D pattern by analyzing the marker image and outputting a deep learning result through an iterative operation of inputting data of the 3D pattern into a deep learning algorithm. 3D touch recognition device using a. The method according to claim 1, The deep learning algorithm is a three-dimensional touch recognition device using deep learning, characterized in that any one of a deep neural network, a composite product neural network or a cyclic neural network. The method according to claim 1, The analysis unit 3D using deep learning, characterized in that for determining the three-dimensional pattern by changing the position or the brightness of the size point marker that is the marker that is recognized the largest size among the plurality of markers. Touch recognition device. The method according to claim 3, The analysis unit may determine the three-dimensional pattern by changing the position or brightness of the size point marker, and the size or shape of the size sub-marker located within a predetermined range around the size point marker. 3D touch recognition device using deep learning. The method according to claim 1, The analysis unit may determine the three-dimensional pattern by changing the position or brightness of the brightness point marker, which is a marker that is recognized as the highest brightness among the plurality of markers, the three-dimensional using deep learning Touch recognition device. The method according to claim 5, The analyzing unit may determine the three-dimensional pattern by changing the position or brightness of the brightness point marker, and changing the size or shape of the brightness subsidiary marker located within a predetermined range around the brightness point marker. 3D touch recognition device using deep learning. The method according to claim 1, The marker is a three-dimensional touch recognition apparatus using deep learning, characterized in that formed in a circular shape. The method according to claim 1, 3. The apparatus of claim 3, further comprising a start notification unit configured to perform visual notification when a start input, which is the first user input among the user inputs, acts on the outer surface of the sheet. The method according to claim 1, And a controller configured to receive the deep learning result value from the analysis unit and to transmit a control signal corresponding to the deep learning result value to an external device. A computer comprising a three-dimensional touch recognition device using deep learning according to any one of claims 1 to 8. A robot having a three-dimensional touch recognition device using deep learning according to any one of claims 1 to 8. In the three-dimensional touch recognition method using a three-dimensional touch recognition device using a deep learning, (i) the user input acting on the sheet; (ii) transferring the marker image photographed by photographing the inner surface of the sheet per unit time to the analyzer; (iii) the analysis unit analyzing the marker image to generate a three-dimensional pattern by changing the position or brightness of the marker; And and (iv) inputting data of the three-dimensional pattern into a deep learning algorithm and outputting a deep learning result value. The method according to claim 12, In the step (iii), the analysis unit, the three-dimensional pattern, characterized in that for determining the three-dimensional pattern by changing the position of the size point marker that is the marker that is recognized the largest size among the plurality of markers Touch recognition method. The method according to claim 12, In the step (iii), the analysis unit, the three-dimensional pattern, characterized in that by determining the position of the brightness point marker, which is a marker that is recognized as the highest brightness among the plurality of the markers, the three-dimensional pattern Touch recognition method. The method according to claim 12, In the step (iii), the horizontal displacement of the three-dimensional displacement of the marker by measuring the position of the marker, characterized in that for measuring the horizontal displacement. The method according to claim 12, In the step (iii), the vertical displacement of the three-dimensional displacement of the marker by changing the brightness of the marker, characterized in that the three-dimensional touch recognition method.

Images