KR102726966B1 - Manufacturing facility control method through voice recognition - Google Patents

Abstract

The present invention relates to a method for controlling manufacturing facilities through voice recognition, which improves bottlenecks and errors by distributing and processing control commands for controlling manufacturing facilities.

Description

{Manufacturing facility control method through voice recognition}

The present invention relates to a method for controlling manufacturing facilities through voice recognition, and more specifically, to a method for controlling manufacturing facilities through voice recognition that improves bottlenecks and errors by distributing and processing control commands for controlling manufacturing facilities.

As voice recognition technology advances, voice recognition technology is being applied to various electronic devices, such as TVs, speakers, and mobile phones. The conventional method of voice recognition control is to convert the user's utterance into data, recognize it, analyze the content of the recognized utterance, and execute it. At this time, the meaning of the recognized utterance is confirmed through a query to a database built in advance.

For example, if a user utters "Tell me the weather for tomorrow", the user's command is "Tell me" and the target of this command is "tomorrow's weather". The database has a category called <Weather>, and this category includes various words such as "tomorrow's weather", "today's weather", "New York weather", "Washington weather", "weather forecast", and "weather conditions". In addition, the database has various words such as "Tell me", "Tell me", and "How is it?" pre-registered as commands that the user might want for the <Weather> category, meaning to tell the weather. Therefore, when the user utters "Tell me the weather for tomorrow", the voice recognition system recognizes this as meaning to provide information about tomorrow's weather, and performs the corresponding action.

There are increasing cases where such conventional voice recognition systems are applied to control manufacturing facilities such as factories. As such conventional technology, Korean Patent Publication No. 10-2021-0086020 (July 8, 2021) published "Method and system for controlling factory facilities using deep learning-based voice recognition", which comprises the steps of: receiving a sound signal, separating the sound signal into a voice signal and a noise signal using a deep neural network learning algorithm; converting the voice signal into a text signal; transmitting the text signal; And the factory equipment control unit that receives the text signal analyzes the text signal into an intent and an entity, and controls the entity to perform an action corresponding to the intent, wherein the deep neural network is a neural network learned to extract only a voice signal from a sound source signal including a noise signal. The present invention relates to a method for controlling factory equipment using speech recognition based on deep learning.

However, the above conventional "method and system for controlling factory equipment using voice recognition based on deep learning" has had difficulty processing a large number of input voices in the field of large-scale manufacturing equipment control where control commands come from multiple users. Accordingly, there is a growing need for a process that can smoothly process control commands from multiple users to be suitable for the field of large-scale manufacturing equipment control.

Korean Patent Publication No. 10-2021-0086020 (published on July 8, 2021), "Method and system for controlling factory equipment using deep learning-based voice recognition"

In order to solve the above problems, the present invention aims to provide a method for controlling manufacturing facilities through voice recognition, which improves bottlenecks and errors by distributing and processing control commands for controlling manufacturing facilities.

In order to achieve the above-mentioned purpose, the present invention provides a manufacturing facility control method through voice recognition, which comprises a speaker recognition and STT performance step (S10); an intention inference performance step (S20); and a command transmission step (S30) for transmitting a command to a linked facility, wherein the command transmission step (S30) comprises a command intention and control authority confirmation step (S31); a control command distributed processing step (S32); and a control command transmission step (S33).

At this time, the command intent in the command intent and control authority confirmation step (S31) is characterized by including at least one of the target facility, an action to be performed by the facility, a target to which the command to be performed is to be applied, or a target to which the command to be performed is to be moved or applied.

At this time, the control command distributed processing step (S32) is characterized by including at least one of a control command parallel processing step (S321) or a control command sequential processing step (S322).

At this time, the control command parallel processing step (S321) is characterized by processing control commands less than a preset number in parallel to prevent delay in the operation of manufacturing equipment.

At this time, in the control command sequential processing step (S322), in order to prevent malfunction in the case where duplicate or conflicting commands are simultaneously transmitted to one facility, commands newly transmitted in excess of a preset number are applied in a standby state to prevent malfunction of the manufacturing facility.

At this time, when performing the control command sequential processing step (S322), it is characterized in that the control commands waiting in the control command sequential processing step (S322) can be canceled in batches through a ‘stop’ command.

The method for controlling manufacturing equipment using voice recognition according to the present invention improves bottlenecks and errors in equipment control commands by distributing and processing control commands for controlling manufacturing equipment.

FIG. 1 is an exemplary diagram of a system for performing a method for controlling manufacturing equipment through voice recognition according to an embodiment of the present disclosure.
Figure 2 is a flow chart of a method for controlling manufacturing equipment through voice recognition according to an embodiment of the present invention.
FIG. 3 is a detailed flowchart of a method for controlling manufacturing equipment through voice recognition according to an embodiment of the present invention.
FIG. 4 is an exemplary diagram showing a process of performing speaker recognition in a method for controlling manufacturing equipment through voice recognition according to an embodiment of the present invention.
FIG. 5 is an exemplary diagram showing a process of performing intention inference in a method for controlling manufacturing equipment through voice recognition according to an embodiment of the present invention.
FIG. 6 is a general schematic diagram of an exemplary computing environment in which embodiments of the present invention may be implemented.

Various embodiments and/or aspects are now disclosed with reference to the drawings. In the following description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of one or more of the aspects. It will be apparent to one skilled in the art, however, that such aspect(s) may be practiced without these specific details. The following description and the annexed drawings set forth specific exemplary aspects of one or more of the aspects in detail. It should be understood, however, that these aspects are exemplary and that any of the various methods of the principles of the various aspects may be utilized, and the disclosed teachings are intended to encompass all such aspects and their equivalents. In particular, the terms “embodiment,” “example,” “aspect,” “example,” and “example” as used herein are not to be construed as being preferred or advantageous over other aspects or designs.

Hereinafter, regardless of the drawing symbols, identical or similar components are given the same reference numerals and redundant descriptions thereof are omitted. In addition, when describing the embodiments disclosed in this specification, if it is determined that a detailed description of a related known technology may obscure the gist of the embodiments disclosed in this specification, the detailed description thereof is omitted. In addition, the attached drawings are only intended to facilitate easy understanding of the embodiments disclosed in this specification, and the technical ideas disclosed in this specification are not limited by the attached drawings.

Although the terms first, second, etc. are used to describe various elements or components, it is to be understood that these elements or components are not limited by these terms. These terms are merely used to distinguish one element or component from another element or component. Accordingly, it is to be understood that a first element or component referred to below may also be a second element or component within the technical spirit of the present invention.

Unless otherwise defined, all terms (including technical and scientific terms) used in this specification may be used with a meaning that can be commonly understood by a person of ordinary skill in the art to which the present invention belongs. In addition, terms defined in commonly used dictionaries shall not be ideally or excessively interpreted unless explicitly specifically defined.

Additionally, the term "or" is intended to mean an inclusive "or" rather than an exclusive "or." That is, unless otherwise specified or clear from the context, "X employs A or B" is intended to mean either of the natural inclusive permutations. That is, if X employs A; X employs B; or X employs both A and B, "X employs A or B" can apply to any of these cases. Furthermore, the term "and/or" as used herein should be understood to refer to and include all possible combinations of one or more of the associated items listed.

Also, it should be understood that the terms "comprises" and/or "comprising" mean the presence of the features and/or components, but do not preclude the presence or addition of one or more other features, components, and/or groups thereof. Also, unless otherwise specified or clear from the context to refer to the singular form, the singular form as used in the specification and claims should generally be construed to mean "one or more."

Additionally, the terms “information” and “data” as used herein may often be used interchangeably.

When it is said that a component is "connected" or "connected" to another component, it should be understood that it may be directly connected or connected to that other component, but that there may be other components in between. On the other hand, when it is said that a component is "directly connected" or "directly connected" to another component, it should be understood that there are no other components in between.

The suffixes "module" and "part" used for components in the following description are given or used interchangeably only for the convenience of writing the specification, and do not in themselves have distinct meanings or roles.

The purpose and effect of the present disclosure, and the technical configurations for achieving them, will become clear with reference to the embodiments described in detail below together with the attached drawings. In explaining the present disclosure, if it is judged that a specific description of a known function or configuration may unnecessarily obscure the gist of the present disclosure, the detailed description thereof will be omitted. In addition, the terms described below are terms defined in consideration of the functions in the present disclosure, and these may vary depending on the intention or custom of the user or operator.

However, the present disclosure is not limited to the embodiments disclosed below, and may be implemented in various different forms. These embodiments are provided only to make the present disclosure complete and to fully inform those skilled in the art of the scope of the disclosure, and the present disclosure is defined only by the scope of the claims. Therefore, the definition should be made based on the contents throughout this specification.

In the present disclosure, the computing device can perform speaker verification and intent inference according to a request received from a user terminal, and transmit a facility control command. Specifically, the computing device can call a model that performs speaker recognition, a model that performs intent inference, a model that transmits a control command, and the like. In addition, the computing device can perform speaker verification and intent inference and transmit a facility control command to a manufacturing facility.

Hereinafter, the equipment control process according to the present invention will be described in detail with reference to the attached drawings.

FIG. 1 illustrates an exemplary system for performing a method for controlling manufacturing equipment through voice recognition according to some embodiments of the present disclosure.

Referring to FIG. 1, the equipment control process may include a computing device (100), a user terminal (200), a server (300), a manufacturing facility (400), and a network (N). However, the above-described components are not essential for implementing a system for performing speaker verification, performing intent inference, and transmitting an equipment control command, and thus, a system for performing speaker verification, performing intent inference, and transmitting an equipment control command may have more or fewer components than the components listed above.

The computing device (100) may include any type of computer system or computer device, such as, for example, a microprocessor, a mainframe computer, a digital processor, a handheld device, or a device controller.

The processor (110) can typically process the overall operation of the computing device (100). The processor (110) can process signals, data, information, etc. input or output through components included in the computing device (100) or run application programs stored in the storage unit (120), thereby providing or processing appropriate information or functions to the user.

In the present disclosure, the processor (110) may request speaker verification performance information and intent inference performance information to the server (300) upon receiving a request signal for performing speaker verification and performing intent inference from the user terminal (200) to transmit a facility control command. Here, the speaker verification performance information and the intent inference performance information may be an interface that is created to control a function provided by an operating system or a programming language. For example, the speaker verification performance information and the intent inference performance information may infer speaker verification and intent of a voice input by a user through a pre-learned model. The speaker verification performance information and the intent inference performance information may be information for performing speaker verification and intent inference of a voice input by the processor (110). According to an embodiment, the speaker verification performance information and the intent inference performance information may include at least one of an IP (Internet Protocol) address, a host address, and port information. The processor (110) can perform speaker verification and intent inference of an input voice included in a request signal by calling a learning model determined based on speaker verification performance information and intent inference performance information.

The storage (120) may include memory and/or a permanent storage medium. The memory may include at least one type of storage medium among a flash memory type, a hard disk type, a multimedia card micro type, a card type memory (e.g., an SD or XD memory, etc.), a random access memory (RAM), a static random access memory (SRAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), a programmable read-only memory (PROM), a magnetic memory, a magnetic disk, and an optical disk.

The communication unit (130) may include one or more modules that enable communication between a computing device (100) and a communication system, between a computing device (100) and a user terminal (200), between a computing device (100) and a server (300), or between a computing device (100) and a network (N). The communication unit (130) may include at least one of a mobile communication module, a wired Internet module, and a wireless Internet module.

The user terminal (200) may include a personal computer (PC), notebook, mobile terminal, smart phone, tablet PC, etc. owned by the user, and may include all types of terminals that can connect to wired/wireless networks.

In the present disclosure, the user terminal (200) can receive a voice input from a user. The user terminal (200) can transmit a command to the manufacturing facility (400) by transmitting speaker verification performance information and intention inference performance information to the computing device (100). As the result is received, the user terminal (200) can also display the received result. Meanwhile, the user terminal (200) may be configured to include an administrator terminal (210) or an operator terminal (220).

According to some embodiments of the present disclosure, the computing device (100) and the user terminal (200) may be implemented in a single configuration.

The server (300) may include any type of computer system or computer device, such as, for example, a microprocessor, a mainframe computer, a digital processor, a handheld device, and a device controller.

According to some embodiments of the present disclosure, the server (300) may store speaker recognition models, intent inference models, etc. For example, the speaker recognition model and the intent inference model may be dynamically expanded. Here, dynamically expanded may mean that they are automatically generated/added without intervention of an administrator. For example, the speaker recognition model and the intent inference model may be additionally generated based on a request signal for data speaker verification and intent inference. The additionally generated speaker recognition model and intent inference model may transmit speaker verification request and intent inference request information to the server (300). Accordingly, the server (300) may store connection information for each of all speaker recognition models and intent inference models.

According to some embodiments of the present disclosure, the computing device (100) and the server (300) may be implemented as a single entity. Accordingly, the computing device (100), the server (300), and the speaker recognition model and the intent inference model may be implemented as a single configuration. For example, the computing device (100) may be included in the server (300) and operate as a single configuration. As another example, the computing device (100) and the speaker recognition model and the intent inference model may be included in the server (300) and operate as a single configuration.

The above manufacturing facility (400) is connected to a computing device (100), a user terminal (200) or a server (300) through a network (N), and performs operations according to commands transmitted from the computing device (100), the user terminal (200) or the server (300). The above manufacturing facility (400) may include identical or similar components corresponding to the computing device (100) in order to perform operations according to the commands. However, the above manufacturing facility (400) does not include all components of the computing device (100), and it would be appropriate to have components sufficient to perform operations according to control commands.

The network (N) can be configured regardless of its communication mode, such as wired or wireless, and can be configured with various communication networks, such as a personal area network (PAN) and a wide area network (WAN). In addition, the network can be the well-known World Wide Web (WWW), and can also use a wireless transmission technology used for short-distance communication, such as infrared (IrDA: Infrared Data Association) or Bluetooth. The technologies described in this specification can be used not only in the networks mentioned above, but also in other networks.

The manufacturing equipment control method using voice recognition according to the present invention includes a speaker recognition and STT performance step (S10) of verifying whether an input voice is a registered speaker and performing STT on the input voice to convert it into text, as illustrated in FIG. 2; an intent inference performance step (S20) of inferring the intent of the converted text; and a command transmission step (S30) of transmitting a command to a linked equipment.

The above step may be performed through a computing device (100), a user terminal (200), a server (300), or a manufacturing facility (400). Some of the processes performed in the above process, such as speaker recognition, intention inference, and command transmission, may be performed through a pre-learned model by applying a network model such as a convolutional neural network (CNN), a recurrent neural network (RNN), an auto encoder, a generative adversarial network (GAN), a restricted boltzmann machine (RBM), a deep belief network (DBN), a Q network, a U network, a Siamese network, and a generative adversarial network (GAN).

In the speaker recognition and STT performance step (S10), as in the example illustrated in Fig. 4, a series of processes are performed to verify whether the input voice is a registered speaker and to perform STT on the input voice to convert it into text. Accordingly, the speaker recognition and STT performance step (S10) may include a process of inputting a sound source of a target speaker to be recognized, a process of extracting and storing features of the target speaker sound source, a process of inputting a user's voice, a process of distributedly processing the input user's voice, a process of extracting features of the input user's voice, a process of calculating a similarity between the target speaker sound source to be recognized and the input user's voice, and a process of performing STT.

The process of inputting the audio source of the target speaker includes a series of processes of receiving the audio source of the target speaker, that is, the user with the control authority of the equipment, and constructing a database for performing speaker recognition. It is preferable that the audio sources of the target speaker be input in the process of inputting the audio source of the target speaker. This is to improve the accuracy of speaker verification later by receiving the audio sources of the target speaker in multiple inputs. The audio sources received in the process of inputting the audio source of the target speaker may be in the formats of wav, mp4, mp3, Mpc, msv, ogg, AAC, FLAC, ALAC, AIFF, DSD, etc., and it is preferable that the format be wav. In the process of inputting the audio source of the target speaker, the control authority for each equipment may be set differently for the target speaker and input. Meanwhile, the process of extracting and storing the features of the target speaker audio source refers to a series of processes of extracting and storing features using a numerical value (feature) representing the unique features of the sound extracted from the input audio source of the target speaker. The feature of the process of extracting and storing the features of the target speaker sound source refers to the feature that discards unnecessary information including noise and leaves only the important characteristics including frequency. In relation to the feature that extracts and stores the features of the target speaker sound source in the process of extracting and storing the features, the input target speaker sound source is divided into short sections of about 25ms, and the finely divided voice can be called a frame. The Fourier transform is performed on each frame to extract the frequency information contained in the corresponding section of the voice (frame). Meanwhile, the result of performing the Fourier transform on each frame can be called a spectrum. In the above spectrum, a filter (Mel Filter Bank) that closely examines the frequency range that is sensitive to human speech recognition and analyzes the remaining range relatively less densely can be applied. This is called the Mel spectrum. Here, the log taken is called the log-Mel spectrum. The numerical value (feature) according to the step (S12) of extracting and storing the features of the target consonant sound source of the present invention may be the information of the frequency domain changed into a new time domain by applying the inverse Fourier transform to the log-Mel spectrum. In the process of extracting and storing the features of the target consonant sound source of the recognition, the control authority for each facility may be set differently for each target consonant and input and stored. Meanwhile, the process of receiving the user's voice refers to a series of processes of receiving the user's voice. The user's voice received in the process of receiving the user's voice is the voice of the user who wants to control the current large-scale manufacturing facility, and may be the target consonant who input the sound source in the process of inputting the sound source of the target consonant or another user. Whether the voice received in the process of receiving the user's voice is the target consonant who input the sound source in the process of inputting the sound source of the target consonant or another user will be verified in the process described below. In the process of receiving a user's voice, the input user's voice can be in the formats of wav, mp4, mp3, Mpc, msv, ogg, AAC, FLAC, ALAC, AIFF, DSD, etc., and it is preferable that it be in the wav format. Meanwhile, in the process of distributing and processing the input user's voice, in the process of receiving the user's voice, the voices input simultaneously for speaker verification are distributedly processed to prevent overload or errors from occurring in speaker verification. This is to minimize bottlenecks by applying the semaphore concept, which performs parallel processing up to a certain number of requests and waits for and then processes requests exceeding that number in order to process voices transmitted simultaneously from multiple users under the control of large-scale manufacturing facilities. The process of distributing and processing the input user's voice is performed using a semaphore, which is a method of restricting access to shared resources in a multiprogramming environment. This is based on a mutual exclusion algorithm. For example, in a multiprogramming environment, access to shared resources is controlled by an integer variable controlled by two atomic functions, so that only a limited number of processes or threads can access n shared resources. At this time, the count of the semaphore is 1 or more, and the number of processes/threads that can enter can be controlled by adjusting the count. Accordingly, the process of distributing the input user's voice may include at least one of the process of parallel processing the input user's voice or the process of sequentially processing the input user's voice. The process of parallel processing the user's voice may process input voices less than a preset number in parallel. In the above step, a certain number of input voices are processed simultaneously. At this time, a critical section may be set in the process of parallel processing the user's voice, and the section that is processed simultaneously as much as the limit is set may be called a critical section. For example, if the processing number set in the process of distributing the input user's voice is '5', '5' input voices are processed simultaneously to facilitate the speaker recognition speed in the future. The process of sequentially processing the input user's voice can prevent a bottleneck phenomenon by applying a waiting state to newly transmitted input voices exceeding a preset number. The input voices exceeding a preset number can be applied to continuously loop in an empty loop and enter a critical zone if possible. For example, if the processing number set in the process of distributedly processing the input user's voice is '5', '5' input voices are processed simultaneously and a waiting state is applied from the 6th input voice to prevent a bottleneck phenomenon. Meanwhile, the process of extracting the input user's voice features includes a series of processes of extracting and storing features using a numerical value (feature) representing the unique characteristics of the sound extracted from the input user's voice in the process of inputting the user's voice. The numerical value (feature) refers to a numerical value (feature) that discards unnecessary information including noise and leaves only important characteristics including frequency. In the process of extracting the voice features of the input user, the input sound source of the target speaker is divided into short sections of about 25 ms, and the finely divided voice can be called a frame. A Fourier transform is performed on each frame to extract the frequency information contained in the corresponding section of voice (frame). Meanwhile, the result of performing a Fourier transform on each frame can be called a spectrum. A filter (Mel Filter Bank) that examines the frequency range that is sensitive to human speech recognition in detail and analyzes the remaining ranges relatively less densely can be applied to the above spectrum. This is called a Mel Spectrum. The logarithm of this is called a log-Mel Spectrum. The feature according to the step (S12) of extracting and storing features of the target consonant sound source of the present invention may be information in the frequency domain converted into a new time domain by applying an inverse Fourier transform to the log-mel spectrum. Meanwhile, the process of calculating the similarity between the target consonant sound source of recognition and the input user voice may calculate the similarity between the target consonant sound source of recognition and the input voice, thereby determining whether the user who input the voice in the process of receiving the user's voice is a user with facility control authority. In the process of calculating the similarity between the target consonant sound source of recognition and the input user voice, the similarity may be calculated by comparing a weighted sum model of N normal distributions of the features of the target consonant sound source of recognition with a weighted sum model of N normal distributions of the features of the input voice. Meanwhile, the process of performing STT includes a series of processes including converting the input voice into text through the process of receiving the user's voice, the process of distributing and processing the input user's voice, the process of extracting the features of the input user's voice, and the process of calculating the similarity between the target consonant sound source and the input user's voice, and passing the converted text to the intention inference execution step (S20). Converting the input voice into text in the process of performing STT can be performed by applying a known STT model. STT (Speech-to-Text) refers to the process of a computer interpreting a spoken language spoken by a person and converting the content into text data. The voice converted in the process of performing STT may include various foreign languages in addition to Korean.

In the above intention inference execution step (S20), as illustrated in Fig. 5, a series of processes for inferring the intention of the text converted in the speaker recognition and STT execution step (S10) are performed. Intention inference can utilize methods such as patterns, morphological analysis, machine learning, and external linkage such as ChatGPT, and among these, whether to apply and the priority can be specified.

According to one embodiment, the intention inference execution step (S20) may include a process of requesting API connection information from a server, a process of inferring the intention by calling an API determined based on the API connection information when the API connection information is received from the server, and a process of passing the result of inferring the intention to the command transmission step (S30).

The process of inferring intent can be performed by applying an intent inference model, and the intent inference model can be a network model such as a convolutional neural network (CNN), a recurrent neural network (RNN), an auto encoder, a generative adversarial network (GAN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a Q network, a U network, a Siamese network, or a generative adversarial network (GAN).

In the above command transmission step (S30), a series of processes for transmitting commands to linked equipment are performed. Accordingly, the above command transmission step (S30) may include a command intent and control authority confirmation step (S31), a control command distribution processing step (S32), and a control command transmission step (S33).

In the above command intent and control authority confirmation step (S31), a process is performed to confirm whether the user's command intent is to control the equipment and whether the user has control authority based on the intent inference result. The command intent in the above command intent and control authority confirmation step (S31) may include at least one of a: target equipment, b: action to be performed on the equipment, c: target to which the execution command is to be applied, or d: target to be moved or applied. For example, the command intent in the command intent and control authority confirmation step (S31) may be commands such as ['stop' 'equipment 1'], ['pick up' 'object 1' with 'robot arm 1'], ['rotate' 'robot arm 1' '90 degrees to the right'], and ['put down' 'object A' at 'location B' with 'robot arm 1']. In this case, 'robot arm 1' may be the target facility, 'object A' may be the target to which the execution command is to be applied, 'location B' may be the target to be moved or applied, and 'put down' may be an action to perform the facility. Meanwhile, in the command intent and control authority confirmation step (S31), whether the user has the command intent and whether the user has the control authority for the operation of the corresponding part of the corresponding equipment are simultaneously satisfied is determined, and only when both conditions of the user having the command intent and the user having the control authority are simultaneously satisfied, the control command distribution processing step (S32) described later is performed.

In the above control command distribution processing step (S32), control commands are distributed and processed to prevent overload or errors in manufacturing equipment from occurring. Accordingly, the control command distribution processing step (S32) may include at least one of the control command parallel processing step (S321) and the control command sequential processing step (S322).

In the above control command parallel processing step (S321), control commands less than a preset number are processed in parallel to prevent delays in the operation of the manufacturing equipment. For example, if the processing number set in the control command distributed processing step (S32) is '5', '5' control commands are processed simultaneously to ensure smooth transmission of the control commands to the manufacturing equipment.

In the above control command sequential processing step (S322), in order to prevent malfunction when duplicate or conflicting commands are simultaneously transmitted to one facility, commands newly transmitted in excess of a preset number are applied in a standby state to prevent malfunction of the manufacturing facility. For example, when the processing number set in the control command sequential processing step (S322) is '5', '5' control commands are processed simultaneously, and from the 6th control command, they are applied in a standby state to prevent a bottleneck phenomenon. In performing the above control command sequential processing step (S322), the processing limit may be limited to a known model or a pre-learned model.

Meanwhile, when performing the control command sequential processing step (S322), only the control commands that are waiting, excluding the control commands that are currently being processed, can be canceled in bulk through a 'stop' command in the control command sequential processing step (S322). For example, when the processing number set in the control command sequential processing step (S322) is '5', '5' control commands are processed simultaneously, but only the 6th or more control commands that are applied in a waiting state are canceled in bulk through a command such as 'stop', thereby facilitating control of the equipment when a malfunction occurs.

In the above control command transmission step (S33), the command intent and control authority are confirmed in the command intent and control authority confirmation step (S31), and in the control command distribution processing step (S32), the distributedly processed control command is transmitted to the manufacturing equipment to operate the manufacturing equipment.

FIG. 6 illustrates a general schematic diagram of an exemplary computing environment in which embodiments of the present disclosure may be implemented.

Although the present disclosure has been generally described above in the context of computer-executable instructions that may be executed on one or more computers, those skilled in the art will appreciate that the present disclosure may also be implemented in combination with other program modules and/or as a combination of hardware and software.

In general, modules herein include routines, procedures, programs, components, data structures, and the like that perform particular tasks or implement particular abstract data types. Furthermore, those skilled in the art will appreciate that the methods of the present disclosure can be practiced with other computer system configurations, including single-processor or multiprocessor computer systems, minicomputers, mainframe computers, as well as personal computers, handheld computing devices, microprocessor-based or programmable consumer electronics, and the like, each of which may be operatively connected to one or more associated devices.

The described embodiments of the present disclosure can also be practiced in distributed computing environments where certain tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules may be located in both local and remote memory storage devices.

Computers typically include a variety of computer-readable media. Media that can be accessed by a computer include volatile and nonvolatile media, transitory and non-transitory media, removable and non-removable media. By way of example and not limitation, computer-readable media can include computer-readable storage media and computer-readable transmission media.

Computer-readable storage media includes volatile and nonvolatile media, transitory and non-transitory media, removable and non-removable media implemented in any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Computer-readable storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROMs, digital video disks (DVDs) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be accessed by a computer and which can be used to store the desired information.

Computer-readable transmission media typically includes any information delivery media that embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal, such as a carrier wave or other transport mechanism. The term modulated data signal means a signal that has one or more of its characteristics set or changed so as to encode information in the signal. By way of example, and not limitation, computer-readable transmission media includes wired media, such as a wired network or direct-wired connection, and wireless media, such as acoustic, RF, infrared, and other wireless media. Combinations of any of the above are also intended to be included within the scope of computer-readable transmission media.

An exemplary environment (1100) implementing various aspects of the present disclosure is illustrated, including a computer (1102) including a processing unit (1104), a system memory (1106), and a system bus (1108). The system bus (1108) couples system components, including but not limited to the system memory (1106), to the processing unit (1104). The processing unit (1104) may be any of a variety of commercially available processors. Dual processors and other multiprocessor architectures may also be utilized as the processing unit (1104).

The system bus (1108) may be any of several types of bus structures that may additionally be interconnected to a memory bus, a peripheral bus, and a local bus using any of a variety of commercial bus architectures. The system memory (1106) includes read-only memory (ROM) (1110) and random access memory (RAM) (1112). A basic input/output system (BIOS) is stored in nonvolatile memory (1110), such as ROM, EPROM, EEPROM, and the BIOS contains basic routines that help transfer information between components within the computer (1102), such as during start-up. The RAM (1112) may also include high-speed RAM, such as static RAM, for caching data.

The computer (1102) also includes an internal hard disk drive (HDD) (1114) (e.g., EIDE, SATA)—which may also be configured for external use within a suitable chassis (not shown), a magnetic floppy disk drive (FDD) (1116) (e.g., for reading from or writing to a removable diskette (1118)), and an optical disk drive (1120) (e.g., for reading from or writing to a CD-ROM disk (1122) or other high capacity optical media such as a DVD). The hard disk drive (1114), the magnetic disk drive (1116), and the optical disk drive (1120) may be connected to the system bus (1108) by a hard disk drive interface (1124), a magnetic disk drive interface (1126), and an optical drive interface (1128), respectively. An interface (1124) for implementing an external drive includes, for example, at least one or both of Universal Serial Bus (USB) and IEEE 1394 interface technologies.

These drives and their associated computer-readable media provide nonvolatile storage of data, data structures, computer-executable instructions, and the like. In the case of a computer (1102), the drives and media correspond to storing any data in a suitable digital format. While the description of computer-readable storage media above has referred to HDDs, removable magnetic disks, and removable optical media such as CDs or DVDs, those skilled in the art will appreciate that other types of computer-readable storage media, such as zip drives, magnetic cassettes, flash memory cards, cartridges, and the like, may also be used in the exemplary operating environment, and that any such media may contain computer-executable instructions for performing the methods of the present disclosure.

A number of program modules, including an operating system (1130), one or more application programs (1132), other program modules (1134), and program data (1136), may be stored in the drive and RAM (1112). All or portions of the operating system, applications, modules, and/or data may also be cached in RAM (1112). It will be appreciated that the present disclosure may be implemented in a variety of commercially available operating systems or combinations of operating systems.

A user may enter commands and information into the computer (1102) via one or more wired/wireless input devices, such as a keyboard (1138) and a pointing device such as a mouse (1140). Other input devices (not shown) may include a microphone, an IR remote control, a joystick, a game pad, a stylus pen, a touch screen, and the like. These and other input devices are often connected to the processing unit (1104) via an input device interface (1142) that is coupled to the system bus (1108), but may be connected by other interfaces such as a parallel port, an IEEE 1394 serial port, a game port, a USB port, an IR interface, and the like.

A monitor (1144) or other type of display device is also connected to the system bus (1108) via an interface, such as a video adapter (1146). In addition to the monitor (1144), the computer typically includes other peripheral output devices (not shown), such as speakers, a printer, and so on.

The computer (1102) may operate in a networked environment using wired and/or wireless communications to the manufacturing facility (400). The manufacturing facility (400) may be connected to a conventional network node, and the manufacturing facility (400) may perform operations with the same or similar configuration to the computer (1102). The logical connections depicted include wired/wireless connections to a local area network (LAN) (1152) and/or a larger network, such as a wide area network (WAN) (1154). Additionally, any means capable of communicating, whether a local area network (LAN) (1152) or a wide area network (WAN) (1154), may be included in the logical connections.

When used in a LAN networking environment, the computer (1102) is connected to the local network (1152) via a wired and/or wireless communications network interface or adapter (1156). The adapter (1156) may facilitate wired or wireless communications to the LAN (1152), which may also include a wireless access point installed therein for communicating with the wireless adapter (1156). When used in a WAN networking environment, the computer (1102) may include a modem (1158), be connected to a communications server on the WAN (1154), or have other means for establishing communications over the WAN (1154), such as via the Internet. The modem (1158), which may be internal or external and wired or wireless, is connected to the system bus (1108) via a serial port interface (1142). In a networked environment, program modules described for the computer (1102) or portions thereof may be stored in a remote memory/storage device (1150). It will be appreciated that the network connections depicted are exemplary and other means of establishing a communications link between the computers may be used.

The computer (1102) is configured to communicate with any wireless device or object that is configured and operates in wireless communication, such as a printer, a scanner, a desktop and/or portable computer, a portable data assistant (PDA), a communication satellite, any equipment or location associated with a wireless detectable tag, and a telephone. This includes at least Wi-Fi and Bluetooth wireless technologies. Accordingly, the communication may be a predefined structure as in a conventional network, or may simply be an ad hoc communication between at least two devices.

Wi-Fi (Wireless Fidelity) allows you to connect to the Internet and other things without wires. Wi-Fi is a wireless technology that allows devices, such as computers, to send and receive data, indoors and outdoors, anywhere within the coverage area of a base station, similar to a cell phone. Wi-Fi networks use a wireless technology called IEEE 802.11 (a,b,g, etc.) to provide secure, reliable, and high-speed wireless connections. Wi-Fi can be used to connect computers to each other, to the Internet, and to wired networks (using IEEE 802.3 or Ethernet). Wi-Fi networks can operate in the unlicensed 2.4 and 5 GHz radio bands, at data rates of, for example, 11 Mbps (802.11a) or 54 Mbps (802.11b), or in products that include both bands (dual band).

Those skilled in the art will appreciate that the various illustrative logical blocks, modules, processors, means, circuits, and algorithm steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, various forms of programs or design code (referred to herein, for convenience, as “software”), or a combination of both. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as hardware or software depends upon the particular application and design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in varying ways for each particular application, but such implementation decisions should not be interpreted as causing a departure from the scope of the present disclosure.

The various embodiments presented herein can be implemented as a method, apparatus, or article of manufacture using standard programming and/or engineering techniques. The term "article of manufacture" includes a computer program or media accessible from any computer-readable device. For example, computer-readable storage media include, but are not limited to, magnetic storage devices (e.g., hard disks, floppy disks, magnetic strips, etc.), optical disks (e.g., CDs, DVDs, etc.), smart cards, and flash memory devices (e.g., EEPROMs, cards, sticks, key drives, etc.). The term "machine-readable medium" includes, but is not limited to, wireless channels and various other media capable of storing, retaining, and/or transmitting instructions(s) and/or data.

The description of the disclosed embodiments is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to these embodiments will be apparent to those skilled in the art, and the generic principles defined herein may be applied to other embodiments without departing from the scope of the disclosure. Thus, the disclosure is not intended to be limited to the embodiments disclosed herein, but is to be construed in the widest scope consistent with the principles and novel features disclosed herein.

(S10): Speaker recognition and STT execution stage
(S20): Intention inference execution step
(S30): Command transmission stage
(S31): Command intent and control authority confirmation step
(S32): Control command distribution processing stage
(S33): Control command transmission stage

Claims (6)

In a method for controlling manufacturing equipment through voice recognition, which is performed by including a computing device (100), a user terminal (200), a server (300), a manufacturing equipment (400), and a network (N),
A computing device (100) includes a speaker recognition and STT performance step (S10) in which the computing device (100) verifies whether the user's input voice input through the user terminal (200) is a registered speaker through connection with the server (300) and performs STT (Speech-to-Text) on the input voice to convert it into text; an intent inference performance step (S20) in which the computing device (100) infers the intent of the text converted in the speaker recognition and STT performance step (S10) through connection with the server (300); and a command transmission step (S30) in which the computing device (100) transmits a command to the linked equipment to transmit the command to the linked equipment.
The above command transmission step (S30) includes a command intent and control authority confirmation step (S31) in which the result of the intent inference performed step (S20) through the computing device (100) is a command intent and whether the user has control authority; a control command distribution processing step (S32) in which the control command is distributedly processed through the computing device (100); and a control command transmission step (S33) in which the computing device (100) transmits the control command to the manufacturing equipment (400).
The above control command distributed processing step (S32) includes at least one of the control command parallel processing step (S321) and the control command sequential processing step (S322), wherein in the control command sequential processing step (S322), in order to prevent malfunction in the case where duplicate or conflicting commands are simultaneously transmitted to one facility, the computing device (100) applies a command newly transmitted in excess of a preset number to a standby state, thereby preventing malfunction of the manufacturing facility. A method for controlling manufacturing facilities through voice recognition.
In the first paragraph,
A method for controlling manufacturing equipment through voice recognition, characterized in that the command intent in the above command intent and control authority confirmation step (S31) includes at least one of target equipment, an action to be performed by the equipment, a target to which the command to be performed is to be applied, or a target to be moved or applied.
delete In the first paragraph,
A method for controlling manufacturing equipment through voice recognition, characterized in that in the above control command parallel processing step (S321), the computing device (100) processes control commands less than a preset number in parallel to prevent delay in the operation of the manufacturing equipment.
delete In the first paragraph,
A method for controlling manufacturing equipment through voice recognition, characterized in that, when performing the above control command sequential processing step (S322), control commands waiting in the control command sequential processing step (S322) can be canceled in bulk through a 'stop' command via the user terminal (200) or computing device (100).