KR20240121445A - Apparatus and method for measuring user location based on a cnn model using markov transition field - Google Patents

Abstract

The present specification relates to a method for measuring a user's location using a convolutional neural network model, comprising: a step of receiving wireless signals from a plurality of access points; a step of measuring RSSI of the received wireless signals; and a step of measuring the user's location based on the RSSI of the measured wireless signals using a learned convolutional neural network model and a fingerprinting technique.

Description

{APPARATUS AND METHOD FOR MEASURING USER LOCATION BASED ON A CNN MODEL USING MARKOV TRANSITION FIELD}

The present specification relates to a method for measuring a user's location, and more specifically, to a method for measuring a user's location based on a convolutional neural network model using a Markov transfer field and a device for supporting the same.

Recently, with the development of wireless communication technology, research on positioning technology has been actively conducted. In addition, with the popularization of IoT (Internet of Things) services, the importance of indoor positioning technology that estimates the location of objects in indoor spaces is being highlighted. Among positioning technologies, the most widely used GPS (Global Positioning System) is a technology that measures location in an outdoor environment. However, indoors, GPS signals are interfered with or attenuated by obstacles, making reception difficult, resulting in low positioning accuracy. Therefore, various technologies such as AoA (Angle of Arrival), ToA (Time of Arrival), and TDoA (Time Difference of Arrival) have been studied to ensure the accuracy of indoor positioning technology. These traditionally known indoor positioning algorithms still have limitations in terms of indoor accuracy.

To solve the limitations of traditional wireless indoor positioning methods, indoor positioning research based on CNN (Convolutional Neural Network), an artificial neural network mainly used for image processing, is receiving attention, and among them, excellent performance is being shown by using RSSI (Received Signal Strength Indicator) of BLE (Blue-Tooth Low Energy) for artificial neural network learning. Indoor positioning technology based on fingerprinting through Wi-Fi and BLE has the characteristics of low cost, low power, and popularization of smartphones, and has the advantage of being able to perform positioning without a separate sensor even on mobile devices.

However, RSSI has an error of several meters due to unstable values caused by surrounding obstacles or multipath effects. Therefore, there are cases where it is difficult to estimate accurate location.

In order to learn time series data, which is a one-dimensional vector, as a two-dimensional vector image, the time-dependent characteristics of the time series data must be maintained even after dimensional transformation.

If one-dimensional vectors are converted into simple two-dimensional vectors, an edge problem may occur where time dependency is damaged. To compensate for this shortcoming, research on CNNs that apply MTF (Markov Transition Field) to convert time series data into images has been conducted, and it is showing high accuracy in indoor positioning technology.

The purpose of this specification is to provide an MTF data generation algorithm that synthesizes noise into RSSI, which is time-series data, to complement the shortcomings of existing RSSI-based fingerprint positioning techniques, and a deep learning indoor positioning model trained with the data.

In addition, this specification aims to provide a method for learning RSSI and noise with variability in an offline test environment by synthesizing time series data and noise with the proposed method, and to perform model learning and performance comparison using the Indoor-unified-wifi-ds dataset.

The technical problems to be achieved in the present invention are not limited to the technical problems mentioned above, and other technical problems not mentioned will be clearly understood by a person having ordinary skill in the technical field to which the present invention belongs from the description below.

The present specification relates to a method for measuring a user's location using a convolutional neural network model, comprising: a step of receiving a wireless signal from a plurality of access points; a step of measuring a Received Signal Strength Indication (RSSI) of the received wireless signal; a step of measuring the user's location based on the RSSI of the measured wireless signal using a learned convolutional neural network model and a fingerprinting technique, wherein the step of learning the convolutional neural network model comprises: a step of performing preprocessing to divide the RSSI of the measured wireless signal into type slots of a specific time unit; a step of adding random noise to the preprocessed RSSI to generate one-dimensional time series data; a step of applying a Markov Transition Field (MTF) to the one-dimensional time series data to convert it into two-dimensional image data; and a step of using the two-dimensional image data as learning data for the convolutional neural network model.

In addition, the step of converting into two-dimensional image data by applying the MTF in the present specification is characterized by including the steps of: dividing the one-dimensional time series data into a specific number (Q) of time intervals; constructing a weight matrix having a size of a specific number (Q) x a specific number (Q); generating a Markov transition matrix (W) by normalizing the sum of each column in the constructed weight matrix to 1; and generating a Markov transition field (M) based on the generated transition matrix.

Additionally, in this specification, the random noise is characterized as being Gaussian noise.

In addition, the present specification provides a terminal for measuring a user's location using a convolutional neural network model, comprising: a wireless communication unit for receiving wireless signals from a plurality of access points; an AI device for learning the convolutional neural network model; And a processor functionally connected to the wireless communication unit and the AI device, and controlling the overall operation of the terminal, wherein the processor measures the Received Signal Strength Indication (RSSI) of the received wireless signal, and controls the measured RSSI of the wireless signal to measure the location of the user based on a learned convolutional neural network model and a fingerprinting technique, wherein the AI device performs preprocessing to divide the RSSI of the measured wireless signal into type slots of specific time units, generates one-dimensional time series data by adding random noise to the preprocessed RSSI, converts the one-dimensional time series data into two-dimensional image data by applying a Markov Transition Field (MTF), and uses the two-dimensional image data as learning data of the convolutional neural network model to learn the convolutional neural network model.

This specification has the effect of more accurately measuring the user's location indoors by converting RSSI values, which are time series data, into two-dimensional image data through MTF transformation, learning through a CNN algorithm, and applying a fingerprinting technique.

The effects obtainable from the present invention are not limited to the effects mentioned above, and other effects not mentioned will be clearly understood by those skilled in the art to which the present invention pertains from the description below.

The accompanying drawings, which are incorporated in and are intended to aid in the understanding of the present invention and are intended to be a part of the detailed description, provide embodiments of the present invention and, together with the detailed description, serve to explain the technical features of the present invention.
Figure 1 is a diagram showing an example of a perceptron structure to which the method proposed in this specification can be applied.
Figure 2 is a diagram showing an example of a multilayer perceptron structure to which the method proposed in this specification can be applied.
Figure 3 is a diagram showing an example of a deep neural network to which the method proposed in this specification can be applied.
Figure 4 is a diagram showing an example of a convolutional neural network to which the method proposed in this specification can be applied.
Figure 5 is a diagram showing an example of a filter operation in a convolutional neural network to which the method proposed in this specification can be applied.
Figure 6 is a diagram showing an example of a neural network structure having a cyclic loop to which the method proposed in this specification can be applied.
Figure 7 is a diagram showing an example of the operational structure of a recurrent neural network to which the method proposed in this specification can be applied.
Figure 8 is an example of a DNN model to which the present invention can be applied.
Figure 9 is a block diagram of an AI device according to one embodiment of the present invention.
Figure 10 shows an example of a design diagram of a noise synthesis MTF model experimental system proposed in this specification.
FIG. 11 is a diagram showing an example of a fingerprinting-based indoor positioning method to which the method proposed in this specification can be applied.
Figure 12 is a diagram showing the RSSI measured for each beacon.
Figure 13 is a diagram showing the distribution of RSSI values by AP.
Figures 14 and 15 are diagrams showing examples of noise synthesis MTF conversion images proposed in this specification.
Fig. 16 is a flowchart showing an example of a terminal operation method for measuring a user's location using a Markov transition field-based CNN proposed in this specification.
Figure 17 is an internal block diagram illustrating a terminal to which the method proposed in this specification can be applied.

It should be noted that the technical terms used in this specification are only used to describe specific embodiments and are not intended to limit the scope of the technology disclosed in this specification. In addition, unless specifically defined otherwise in this specification, the technical terms used in this specification should be interpreted as having a meaning generally understood by a person having ordinary skill in the art to which the technology disclosed in this specification belongs, and should not be interpreted in an excessively comprehensive or excessively narrow sense. In addition, when a technical term used in this specification is an incorrect technical term that does not accurately express the scope of the technology disclosed in this specification, it should be replaced with a technical term that can be correctly understood by a person having ordinary skill in the art to which the technology disclosed in this specification belongs. In addition, general terms used in this specification should be interpreted as defined in the dictionary or according to the context, and should not be interpreted in an excessively narrow sense.

Terms including ordinal numbers such as first, second, etc. used in this specification may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component.

Hereinafter, embodiments disclosed in this specification will be described in detail with reference to the attached drawings. Regardless of the drawing symbols, identical or similar components are given the same reference numerals and redundant descriptions thereof will be omitted.

In addition, when describing the technology disclosed in this specification, if it is judged that a detailed description of a related known technology may obscure the gist of the technology disclosed in this specification, the detailed description thereof will be omitted. In addition, it should be noted that the attached drawings are only intended to facilitate easy understanding of the concept of the technology disclosed in this specification, and should not be construed as limiting the concept of the technology by the attached drawings.

Artificial Intelligence (AI)

The most important and newly introduced technology in the 6G system is AI. The 4G system did not involve AI. The 5G system will support partial or very limited AI. However, the 6G system will be fully AI-supported for automation. Advances in machine learning will create more intelligent networks for real-time communications in 6G. Introducing AI in communications can simplify and improve real-time data transmission. AI can use a lot of analytics to determine how complex target tasks are performed. In other words, AI can increase efficiency and reduce processing delays.

AI can also play a significant role in M2M, machine-to-human, and human-to-machine communication. AI can also be a rapid communication in BCI (Brain Computer Interface). AI-based communication systems can be supported by metamaterials, intelligent structures, intelligent networks, intelligent devices, intelligent cognitive radio, self-sustaining wireless networks, and machine learning.

Below, we will look at machine learning in more detail.

Machine learning is a series of operations that teach machines to create machines that can perform tasks that people can or cannot do. Machine learning requires data and a learning model. In machine learning, data learning methods can be broadly divided into three: supervised learning, unsupervised learning, and reinforcement learning.

Neural network learning is to minimize the error of the output. Neural network learning is a process of repeatedly inputting learning data into the neural network, calculating the neural network output and target error for the learning data, and backpropagating the neural network error from the output layer of the neural network to the input layer in the direction of reducing the error, thereby updating the weights of each node of the neural network.

Supervised learning uses training data with correct answers labeled in the training data, while unsupervised learning may not have correct answers labeled in the training data. That is, for example, in the case of supervised learning for data classification, the training data may be data in which each category is labeled in the training data. The labeled training data is input to the neural network, and the error can be calculated by comparing the output (category) of the neural network with the label of the training data. The calculated error is backpropagated in the neural network in the reverse direction (i.e., from the output layer to the input layer), and the connection weights of each node in each layer of the neural network can be updated according to the backpropagation. The amount of change in the connection weights of each node that is updated can be determined according to the learning rate. The neural network's calculation of the input data and the backpropagation of the error can constitute a learning cycle (epoch). The learning rate can be applied differently depending on the number of repetitions of the learning cycle of the neural network. For example, in the early stages of learning a neural network, a high learning rate can be used to allow the network to quickly achieve a certain level of performance, thereby increasing efficiency, while in the later stages of learning, a low learning rate can be used to increase accuracy.

Depending on the characteristics of the data, the learning method may vary. For example, if the goal is to accurately predict data transmitted from the transmitter to the receiver in a communication system, it is preferable to perform learning using supervised learning rather than unsupervised learning or reinforcement learning.

The learning model corresponds to the human brain, and the most basic linear model can be thought of, but the machine learning paradigm that uses highly complex neural network structures, such as artificial neural networks, as learning models is called deep learning.

The neural network cores used in learning methods are largely divided into deep neural networks (DNNs), convolutional deep neural networks (CNNs), and recurrent boltzmann machines (RNNs).

An artificial neural network is an example of multiple perceptrons connected together.

Figure 1 is a diagram showing an example of a perceptron structure to which the method proposed in this specification can be applied.

Referring to Fig. 1, when an input vector x=(x1,x2, ...,xd) is input, the entire process of multiplying each component by a weight (W1,W2, ...,Wd), adding up all the results, and then applying the activation function σ(·) is called a perceptron. A large-scale artificial neural network structure can extend the simplified perceptron structure illustrated in Fig. 1 to apply the input vector to perceptrons of different dimensions. For convenience of explanation, input values or output values are called nodes.

Meanwhile, the perceptron structure illustrated in Fig. 1 can be explained as consisting of a total of three layers based on input and output values. An artificial neural network in which there are H perceptrons of (d+1) dimensions between the 1st layer and the 2nd layer, and K perceptrons of (H+1) dimensions between the 2nd layer and the 3rd layer can be expressed as in Fig. 2.

Figure 2 is a diagram showing an example of a multilayer perceptron structure to which the method proposed in this specification can be applied.

The layer where the input vector is located is called the input layer, the layer where the final output value is located is called the output layer, and all layers located between the input layer and the output layer are called hidden layers. The example in Fig. 2 shows three layers, but when counting the number of actual artificial neural network layers, the input layer is excluded, so it can be viewed as a total of two layers. The artificial neural network is composed of perceptrons of basic blocks connected in two dimensions.

The above-mentioned input layer, hidden layer, and output layer can be applied jointly not only to multilayer perceptron but also to various artificial neural network structures such as CNN and RNN, which will be described later. The more hidden layers there are, the deeper the artificial neural network becomes, and the machine learning paradigm that uses a sufficiently deep artificial neural network as a learning model is called deep learning. In addition, the artificial neural network used for deep learning is called a deep neural network (DNN: Deep neural network).

Figure 3 is a diagram showing an example of a deep neural network to which the method proposed in this specification can be applied.

The deep neural network illustrated in Fig. 3 is a multilayer perceptron composed of eight hidden layers and eight output layers. The multilayer perceptron structure is expressed as a fully-connected neural network. In a fully-connected neural network, there is no connection relationship between nodes located in the same layer, and there is a connection relationship only between nodes located in adjacent layers. DNN has a fully-connected neural network structure and is composed of a combination of multiple hidden layers and activation functions, and can be usefully applied to identify correlation characteristics between inputs and outputs. Here, the correlation characteristic can mean the joint probability of inputs and outputs.

Meanwhile, depending on how multiple perceptrons are connected to each other, various artificial neural network structures different from the aforementioned DNN can be formed.

Figure 4 is a diagram showing an example of a convolutional neural network to which the method proposed in this specification can be applied.

In DNN, nodes located within a single layer are arranged in a one-dimensional vertical direction. However, Fig. 4 can assume a case where nodes are arranged two-dimensionally, with w nodes in width and h nodes in height (convolutional neural network structure of Fig. 4). In this case, since a weight is added to each connection in the connection process from one input node to the hidden layer, a total of h×w weights must be considered. Since there are h×w nodes in the input layer, a total of h2w2 weights are required between two adjacent layers.

The convolutional neural network of Fig. 4 has a problem in that the number of weights increases exponentially according to the number of connections. Therefore, instead of considering the connections of all modes between adjacent layers, it assumes that there is a small filter, and performs weighted sum and activation function operations on the overlapping portions of the filters, as in Fig. 5.

Figure 5 is a diagram showing an example of a filter operation in a convolutional neural network to which the method proposed in this specification can be applied.

One filter has a weight corresponding to the number of its size, and learning of the weight can be performed so that a specific feature on the image can be extracted as a factor and output. In Fig. 5, a 3×3 sized filter is applied to the 3×3 area at the upper left of the input layer, and the output value resulting from performing weighted sum and activation function operations on the corresponding node is stored in z22.

The above filter performs weighted sum and activation function operations while moving a certain horizontal and vertical interval while scanning the input layer, and places the output value at the current filter position. This operation method is similar to the convolution operation for images in the field of computer vision, so a deep neural network with this structure is called a convolutional neural network (CNN), and the hidden layer generated as a result of the convolution operation is called a convolutional layer. In addition, a neural network with multiple convolutional layers is called a deep convolutional neural network (DCNN).

In the convolution layer, the number of weights can be reduced by calculating the weighted sum by including only the nodes located in the area covered by the filter at the node where the current filter is located. As a result, one filter can be used to focus on features for a local area. Accordingly, CNN can be effectively applied to image data processing where physical distance in a two-dimensional area is an important judgment criterion. Meanwhile, CNN can apply multiple filters immediately before the convolution layer, and can also generate multiple output results through the convolution operation of each filter.

On the other hand, there may be data for which sequence characteristics are important depending on the data properties. Considering the length variability and chronological relationship of these sequence data, the structure that applies the method of inputting one element of the data sequence at each time step and inputting the output vector (hidden vector) of the hidden layer output at a specific time together with the next element in the sequence to an artificial neural network is called a recurrent neural network structure.

Figure 6 is a diagram showing an example of a neural network structure having a cyclic loop to which the method proposed in this specification can be applied.

Referring to Fig. 6, a recurrent neural network (RNN) is a structure that inputs elements (x1(t), x2(t), , ..., xd(t)) of a certain time point t in a data sequence into a fully connected neural network, and applies a weighted sum and an activation function along with the hidden vector (z1(t-1), z2(t-1), ..., zH(t-1)) of the immediately previous time point t-1. The reason for transmitting the hidden vector to the next time point in this way is because the information in the input vectors of the previous time points is considered to be accumulated in the hidden vector of the current time point.

Figure 7 is a diagram showing an example of the operational structure of a recurrent neural network to which the method proposed in this specification can be applied.

Referring to Figure 7, the recurrent neural network operates in a predetermined time order with respect to the input data sequence.

When the input vector (x1(t), x2(t), ,..., xd(t)) at time point 1 is input to the recurrent neural network, the hidden vector (z1(1), z2(1), ..., zH(1)) is input together with the input vector (x1(2), x2(2), ..., xd(2)) at time point 2, and the hidden layer vector (z1(2), z2(2), ..., zH(2)) is determined through the weighted sum and activation function. This process is repeatedly performed until time points 2, 3, ,,, and T.

Meanwhile, when multiple hidden layers are arranged in a recurrent neural network, it is called a deep recurrent neural network (DRNN). Recurrent neural networks are designed to be usefully applied to sequence data (e.g. natural language processing).

It is a neural network core used in a learning manner, and includes various deep learning techniques such as DNN, CNN, RNN, Restricted Boltzmann Machine (RBM), Deep Belief Network (DBN), and Deep Q-Network, and can be applied to fields such as computer vision, speech recognition, natural language processing, and speech/signal processing.

Recently, there have been attempts to integrate AI with wireless communication systems, but these have been focused on the application layer, network layer, and especially deep learning in the field of wireless resource management and allocation. However, these studies are gradually developing into the MAC layer and the Physical layer, and in particular, there are attempts to combine deep learning with wireless transmission in the Physical layer. AI-based physical layer transmission means applying signal processing and communication mechanisms based on AI drivers, rather than traditional communication frameworks, in terms of fundamental signal processing and communication mechanisms. For example, it can include deep learning-based channel coding and decoding, deep learning-based signal estimation and detection, deep learning-based MIMO mechanism, and AI-based resource scheduling and allocation.

DNN(Deep Neural Network) Model

Figure 8 is an example of a DNN model to which the present invention can be applied.

A deep neural network (DNN) is an artificial neural network (ANN) that consists of multiple hidden layers between the input layer and the output layer. Like a typical artificial neural network, a deep neural network can model complex non-linear relationships.

For example, in a deep neural network structure for an object identification model, each object can be represented as a hierarchical composition of image basic elements. At this time, additional layers can gradually combine the features of the lower layers. This feature of deep neural networks allows modeling of complex data with a smaller number of units (nodes) compared to similarly performed artificial neural networks.

As the number of hidden layers increases, the artificial neural network is said to be 'deep', and the machine learning paradigm that uses such a sufficiently deep artificial neural network as a learning model is called deep learning. And, the sufficiently deep artificial neural network used for such deep learning is collectively called a deep neural network (DNN: Deep neural network).

In the present invention, data required to train a POI data generation model can be input into the input layer of the DNN, and while passing through hidden layers, meaningful data that can be used by a user can be generated through the output layer.

In the specification of the present invention, the artificial neural network used for this deep learning method is collectively referred to as DNN, but it is obvious that other deep learning methods can be applied if they can output meaningful data in a similar manner.

AI device block diagram

Figure 9 is a block diagram of an AI device according to one embodiment of the present invention.

The AI device (20) may include an electronic device including an AI module capable of performing AI processing, or a server including the AI module. In addition, the AI device (20) may be included as a component of at least a portion of the electronic device and may be provided to perform at least a portion of the AI processing.

The above AI device (20) may include an AI processor (21), a memory (25) and/or a communication unit (27).

The above AI device (20) is a computing device capable of learning a neural network and can be implemented as various electronic devices such as a server, desktop PC, notebook PC, tablet PC, etc.

The AI processor (21) can learn a neural network using a program stored in the memory (25). In particular, the AI processor (21) can learn a neural network for recognizing image-related data. Here, the neural network for recognizing image-related data can be designed to simulate the human brain structure on a computer, and can include a plurality of network nodes having weights that simulate neurons of a human neural network. The plurality of network modes can each exchange data according to a connection relationship so as to simulate the synaptic activity of neurons in which neurons exchange signals through synapses. Here, the neural network can include a deep learning model developed from a neural network model. In the deep learning model, a plurality of network nodes can be located in different layers and exchange data according to a convolution connection relationship. Examples of neural network models include various deep learning techniques such as deep neural networks (DNNs), convolutional deep neural networks (CNNs), recurrent Boltzmann machines (RNNs), restricted Boltzmann machines (RBMs), deep belief networks (DBNs), and deep Q-networks, which can be applied to fields such as computer vision, speech recognition, natural language processing, and speech/signal processing.

Meanwhile, the processor performing the function as described above may be a general-purpose processor (e.g., CPU), but may be an AI-specific processor for artificial intelligence learning (e.g., GPU).

The memory (25) can store various programs and data required for the operation of the AI device (20). The memory (25) can be implemented as a non-volatile memory, a volatile memory, a flash memory, a hard disk drive (HDD), a solid state drive (SDD), etc. The memory (25) is accessed by the AI processor (21), and data reading/recording/modifying/deleting/updating, etc. can be performed by the AI processor (21). In addition, the memory (25) can store a neural network model (e.g., a deep learning model (26)) generated through a learning algorithm for data classification/recognition according to one embodiment of the present invention.

Meanwhile, the AI processor (21) may include a data learning unit (22) that learns a neural network for data classification/recognition. The data learning unit (22) may learn criteria regarding which learning data to use to determine data classification/recognition and how to classify and recognize data using the learning data. The data learning unit (22) may acquire learning data to be used for learning and learn a deep learning model by applying the acquired learning data to the deep learning model.

The data learning unit (22) may be manufactured in the form of at least one hardware chip and mounted on the AI device (20). For example, the data learning unit (22) may be manufactured in the form of a dedicated hardware chip for artificial intelligence (AI), or may be manufactured as a part of a general-purpose processor (CPU) or a graphics processor (GPU) and mounted on the AI device (20). In addition, the data learning unit (22) may be implemented as a software module. When implemented as a software module (or a program module including instructions), the software module may be stored in a non-transitory computer readable media that can be read by a computer. In this case, at least one software module may be provided by an OS (Operating System) or provided by an application.

The data learning unit (22) may include a learning data acquisition unit (23) and a model learning unit (24).

The learning data acquisition unit (23) can acquire learning data required for a neural network model for classifying and recognizing data. For example, the learning data acquisition unit (23) can acquire original data and/or sample data for input into a neural network model as learning data.

The model learning unit (24) can learn to have a judgment criterion on how to classify a given data using the acquired learning data. At this time, the model learning unit (24) can learn the neural network model through supervised learning that uses at least some of the learning data as a judgment criterion. Alternatively, the model learning unit (24) can learn the neural network model through unsupervised learning that discovers a judgment criterion by learning on its own using the learning data without guidance. In addition, the model learning unit (24) can learn the neural network model through reinforcement learning using feedback on whether the result of the situation judgment according to the learning is correct. In addition, the model learning unit (24) can learn the neural network model using a learning algorithm including error back-propagation or gradient descent.

Once the neural network model is learned, the model learning unit (24) can store the learned neural network model in memory. The model learning unit (24) can also store the learned neural network model in the memory of a server connected to the AI device (20) via a wired or wireless network.

The data learning unit (22) may further include a learning data preprocessing unit (not shown) and a learning data selection unit (not shown) to improve the analysis results of the recognition model or to save resources or time required for creating the recognition model.

The learning data preprocessing unit can preprocess the acquired data so that the acquired data can be used for learning for situation judgment. For example, the learning data preprocessing unit can process the acquired data into a preset format so that the model learning unit (24) can use the acquired learning data for learning for image recognition.

In addition, the learning data selection unit can select data required for learning from among the learning data acquired from the learning data acquisition unit (23) or the learning data preprocessed from the preprocessing unit. The selected learning data can be provided to the model learning unit (24). For example, the learning data selection unit can select only data for objects included in a specific area as learning data by detecting a specific area from the acquired image.

In addition, the data learning unit (22) may further include a model evaluation unit (not shown) to improve the analysis results of the neural network model.

The model evaluation unit inputs evaluation data into the neural network model, and if the analysis result output from the evaluation data does not satisfy a predetermined criterion, it can cause the model learning unit (22) to learn again. In this case, the evaluation data may be predefined data for evaluating the recognition model. For example, the model evaluation unit can evaluate that the predetermined criterion is not satisfied if the number or ratio of evaluation data for which the analysis result of the learned recognition model for the evaluation data is inaccurate exceeds a preset threshold.

The communication unit (27) can transmit the AI processing result by the AI processor (21) to an external electronic device.

Here, the external electronic device is a device that can communicate with the AI device via wired/wireless communication, and may include a broadcasting device or a viewer device, which will be described later. In this case, the AI device may be implemented in the administrator device.

Meanwhile, the AI device (20) illustrated in Fig. 8 is described as being functionally divided into an AI processor (21), memory (25), and communication unit (27), but it should be noted that the aforementioned components may be integrated into one module and referred to as an AI module.

In addition, the contents described in FIGS. 1 to 9 discussed above can be implemented in various embodiments by combining with the method proposed in the present specification described below.

In the following, we will examine an indoor positioning method based on BLE RSSI fingerprinting using a CNN model that learns visualized Markov Transition Field (MTF) images proposed in this specification.

First, the indoor positioning technology based on RSSI fingerprinting can be divided into two stages: (1) RSSI signal collection (offline stage) and (2) position estimation (online stage), as shown in Fig. 10. Fig. 10 shows an example of a design diagram of a noise synthesis MTF model experiment system proposed in this specification.

Referring to Figure 10, in the RSSI signal collection step, RSSI is collected from each AP to build a radio map. Then, in the location estimation step, the user's location is estimated based on the actually measured RSSI.

We briefly review RSSI and fingerprint-based indoor positioning methods, and then specifically review the Markov transition field-based indoor positioning method proposed in this specification.

RSSI has very different characteristics depending on the complex indoor space. Noise occurs due to reflections, interference, and signal attenuation caused by the structure and materials of the building.

Most indoor positioning technologies have a stable RSSI through a noise filtering process such as Kalman filtering to remove this fluctuating noise.

However, since RSSI easily changes depending on changes in the measurement environment such as time, temperature, and humidity, positioning accuracy may decrease when RSSI values change. Noise synthesis is one of the time series data augmentation techniques, and by synthesizing noise into preprocessed RSSI, RSSI that is more robust to changes in the surrounding environment can be generated as learning data.

Therefore, this specification applies noise synthesis as a means to enhance RSSI and performs a process of converting it into MTF.

FIG. 11 is a diagram showing an example of a fingerprinting-based indoor positioning method to which the method proposed in this specification can be applied.

Next, we examine the algorithm proposed in this specification for imaging time series data.

In order to use one-dimensional time series data as input to CNN, it needs to be converted into two-dimensional image data. Algorithms for visualizing one-dimensional time series data include Recurrence Plot (RP), Gramian Anguler Field (GAF), and Markov Transfer Field (MTF). RP is an algorithm that expresses time series data as an m-dimensional spatial trajectory and visualizes the distance between points located on each spatial trajectory. It compares a threshold value and the time series data value, and if the time series data value is less than or equal to the threshold value, it is expressed as 1, and if it is greater than the threshold value, it is expressed as 0. Therefore, if the threshold value is not appropriately adjusted, the time series data cannot be converted into an image with temporal dependence. Here, ε is a threshold value, and the RP of point (i, j) can be expressed by the following mathematical expression 1.

GAF is an algorithm that uses a non-Cartesian coordinate system by converting time series data into polar coordinate values instead of using the existing Cartesian coordinate system. It uses the sum or difference of the polar coordinate angles of time series data to create an image. The image after converting time series data to GAF cannot be converted back to time series data, which can cause loss of time series information.

According to the data comparing CNN outlier detection learning by applying imaging algorithms, it can be seen that among the three imaging algorithms, the F1 score and True Positive Rate (TPR) of Markov Transition Field were measured to be the highest. This may differ depending on the characteristics of time series data, but among the three algorithms, Markov Transition Field showed the best performance, so it can be considered suitable for the image transformation method.

Markov transition field represents the transition probability field of discretized time series data. One-dimensional time series data X is divided into Q intervals, and a weight matrix W' of size Q×Q is constructed along the time axis. By normalizing the sum of each column (interval) in W' to 1, the Markov transition matrix W is generated as shown in the following mathematical expression 2.

The Markov transition matrix represents the frequency of the interval qj {1<j<Q} corresponding to the time series data xi at timestamp ti.

At this time, W loses the distribution and temporal dependency of X, which is the original time series data.

In order to preserve information in the time domain, each probability can be sorted in time order and transformed into a Markov transition field. Mathematical expression 3 below represents the Markov transition field M. Mij, the value of row i and column j of M, represents the transition probability from qi, the interval to which data belongs when timestamp ti, to qj, the interval to which data belongs when timestamp tj.

Next, we examine the Markov transition field-based indoor positioning model proposed in this specification.

The experimental environment is conducted in a specific indoor space. The RSSI collected from multiple beacons contains missing or outlier values. After the RSSI data collection and preprocessing discussed above, Gaussian noise to be used as learning data is synthesized into the collected RSSI. The noise in the area with the highest probability of occurrence can be selected and synthesized into the RSSI. After that, the synthesized RSSI is converted into a Markov transition field to build learning data.

Here, the process of preprocessing RSSI refers to the process of dividing the sequence data of RSSI collected from multiple beacons into specific time units (or time slots).

Afterwards, MTF transformation is performed on the one-dimensional time series data divided by the specific time unit.

Let's take a closer look at the RSSI collection and noise synthesis process.

An experimental space was set up in a 3m x 2m area within the above indoor specific space, and divided into 15 x 10 square cells of 20cm x 20cm. RSSI was collected using fingerprinting from 8 beacons surrounding the area. 500 data were collected for each cell at 100msec intervals. A total of 8 days' worth of data were acquired over a period of 3 months: January 5, February 15, February 22, March 13, March 14, March 27, and April 3, 2022.

However, on March 27 and April 3, 300 RSSIs were collected. Fig. 12 shows the RSSI measured for each beacon, and Fig. 13 shows the distribution of RSSI values for each AP. More specifically, Fig. 13 shows the RSSI distribution before and after noise synthesis. Referring to Fig. 12 and Fig. 13, it can be seen that the instability of the RSSI measured by date, time, surrounding devices, temperature, humidity, etc. is revealed even in the same environment.

The distance d estimation formula between a transmitter and a receiver using RSSI can be expressed as mathematical formula 4 below. n is the path loss index, which is usually set to 4 for indoor environments. A is the RSSI value constant at the reference distance from the receiver.

Gaussian noise is synthesized on the preprocessed RSSI to alleviate inaccurate predictions when the RSSI value fluctuates due to disturbances such as rapid environmental changes. In other words, it artificially processes and learns uneven data. The probability density function f(x) for X can be expressed by the following mathematical expression 5. μ represents the mean, and σ ² represents the standard deviation. The noise in the area with the highest probability of occurrence is selected and synthesized into the RSSI. The synthesized data is expressed by the following mathematical expression 6.

MinMaxScaling is applied to the synthesized data and the original data respectively to normalize them. The synthesized dataset is trained to make the positioning model more robust to noise.

Next, we look at the Markov transition field transform process.

The RSSI data set that has completed the preprocessing process must be converted into a one-dimensional vector input before it can be converted into an MTF. As shown in FIGS. 14 and 15, when a signal with a constant period is converted into an MTF, its features are clearly revealed as an image. FIGS. 14 and 15 are diagrams showing examples of noise-synthesized MTF converted images proposed in this specification.

This is related to the temporal dependency of time series data. The sliding window technique is applied to generate the time series data array required for MTF generation. Converting the array to MTF completes the image generation required for learning.

This specification generated an MTF with a size of 224 by concatenating 28 RSSI time series data collected from 8 APs for each AP. The RSSI images of APs 1 to 8 are as shown in Fig. 14. The CNN model used for learning is DensNet, which takes images of the size of 224×224 as input. Therefore, the image size of the Markov transition field was adjusted to 224 in both width and height.

The model to be used for learning proposed in this specification is DenseNet, which is a model that improves the flow of information by directly connecting the previous layer in the neural network to all the next layers. The 1st layer of DensNet performs the activation function H operation by combining all the x0, x1, ...x 1-1th layers. The activation function of DensNet consists of a batch normalization layer and a 3 × 3 convolution layer. The mathematical expression 7 below represents the Dense Connectivity corresponding to the 1st layer of DensNet.

Table 1 below shows an example of learning parameters for the model used for learning.

As previously discussed, we examined a learned model that synthesized Gaussian noise into MTF to improve positioning accuracy according to RSSI fluctuations in an indoor environment.

By applying the MTF algorithm to convert RSSI, which is time series data, into image format, we were able to confirm improvement in positioning accuracy due to the instability of RSSI.

By synthesizing noise in the process of converting RSSI to MTF, we can obtain the effect of augmenting the dataset, and it can be seen that the proposed model is reliably trained in noise. Therefore, if the technique of synthesizing RSSI noise rather than removing it is used in MTF, the positioning accuracy can be further improved. If the mean or standard deviation of the Gaussian distribution is adjusted, the optimal noise required for training can be found.

Fig. 16 is a flowchart showing an example of a terminal operation method for measuring a user's location using a Markov transition field-based CNN proposed in this specification.

First, the terminal receives wireless signals from multiple access points (S1610).

And, the terminal measures the RSSI (Received Signal Strength Indication) of the received wireless signal (S1620).

And, the terminal measures the location of the user based on the RSSI of the measured wireless signal using a learned convolutional neural network model and a fingerprinting technique (S1630).

More specifically, the terminal can perform the following procedure to learn a convolutional neural network model.

That is, the terminal preprocesses the RSSI of the measured wireless signal.

And, the terminal adds random noise to the preprocessed RSSI to generate one-dimensional time series data.

And, the terminal applies a Markov Transition Field (MTF) to the one-dimensional time series data to convert it into two-dimensional image data.

Additionally, the terminal can use the two-dimensional image data as learning data for the convolutional neural network model.

In addition, the step of converting into two-dimensional image data by applying the MTF may include the steps of (1) dividing the one-dimensional time series data into a specific number (Q) of time intervals, (2) constructing a weight matrix having a size of a specific number (Q) × a specific number (Q), (3) normalizing the sum of each column in the constructed weight matrix to 1 to generate a Markov transition matrix (W), and (4) generating a Markov transition field (M) based on the generated transition matrix.

Here, the Markov transition matrix (W) can be defined as follows.

Here, W is a Markov transition matrix and represents the frequency of the interval qj {1<j<Q} corresponding to the time series data xi of timestamp ti, and Q represents the specific number.

Additionally, the Markov transition field (M) can be defined as follows.

Additionally, the random noise may be Gaussian noise, and the Gaussian noise ( p ( x )) may be defined as follows.

Terminal internal block diagram

Figure 17 is an internal block diagram illustrating a terminal to which the method proposed in this specification can be applied.

The terminal (100) may include a wireless communication unit (110), an input unit (120), an output unit (130), a memory (140), and a control unit (or processor, 150). The components illustrated in FIG. 17 are not essential for implementing the terminal, and thus the terminal described in this specification may have more or fewer components than the components listed above. Although not illustrated in FIG. 17, the AI device of FIG. 9 may be included in the terminal.

More specifically, among the above components, the wireless communication unit may include one or more modules that enable wireless communication between a terminal and a wireless communication system, between a terminal and another terminal, or between a terminal and a server. In addition, the wireless communication unit may include one or more modules that connect the terminal to one or more networks.

The wireless communication unit may include at least one of a mobile communication module, a short-range communication module, and a location information module.

The input unit may include a user input unit (e.g., a touch key, a mechanical key, etc.) for receiving information from a user. The input unit may additionally include a camera or a video input unit for video signal input, a microphone for audio signal input, or an audio input unit. Voice data or image data collected from the input unit may be analyzed and processed into a user's control command.

The output section is for generating output related to vision, hearing, or tactile sensations, and may include at least one of a display section, an acoustic output section, a haptic module, and an optical output section. The display section may be formed as a layer structure with a touch sensor or formed as an integral part, thereby implementing a touch screen. This touch screen may function as a user input section that provides an input interface between a terminal and a user, and may provide an output interface between the terminal and the user.

In addition, the memory stores data that supports various functions of the terminal. The memory can store a plurality of application programs (or applications) running on the terminal, data for the operation of the terminal, and commands. At least some of these application programs can be downloaded from an external server via wireless communication. In addition, at least some of these application programs can exist on the terminal from the time of shipment for the basic functions of the terminal (e.g., incoming and outgoing call functions, message reception and outgoing functions). Meanwhile, the application program can be stored in the memory, installed on the device, and driven by the control unit to perform the operation (or function) of the terminal.

In addition to the operations related to the above application programs, the control unit typically controls the overall operations of the terminal. The control unit can process signals, data, information, etc. input or output through the components discussed above, or can operate application programs stored in memory, thereby providing or processing appropriate information or functions to the user.

In addition, the control unit can control at least some of the components examined with reference to FIG. 13 in order to drive an application program stored in the memory. Furthermore, the control unit can operate at least two or more of the components included in the terminal in combination with each other in order to drive the application program.

At least some of the above components may cooperate with each other to implement the operation, control, or control method of the terminal according to various embodiments described below. In addition, the operation, control, or control method of the terminal may be implemented on the terminal by driving at least one application program stored in the memory.

We will examine the contents of the terminal of Fig. 17 implementing the method of Fig. 16, that is, the method for measuring the user's location using a convolutional neural network model.

To this end, the terminal includes a wireless communication unit, an AI device, and a processor (or control unit), and the AI device may include the components mentioned in FIG. 9.

The above wireless communication unit receives wireless signals from multiple access points.

And, the AI device learns the convolutional neural network model.

And, the processor is functionally connected to the wireless communication unit and the AI device, and controls the overall operation of the terminal.

Specifically, the processor measures the Received Signal Strength Indication (RSSI) of the received wireless signal, and controls the measured RSSI of the wireless signal to measure the location of the user based on a learned convolutional neural network model and a fingerprinting technique.

In addition, the AI device preprocesses RSSI of the measured wireless signal, adds random noise to the preprocessed RSSI to generate one-dimensional time series data, applies a Markov Transition Field (MTF) to the one-dimensional time series data to convert it into two-dimensional image data, and uses the two-dimensional image data as learning data of the convolutional neural network model to learn the convolutional neural network model.

In addition, the AI device can convert the one-dimensional time series data into two-dimensional image data by applying the MTF by dividing the one-dimensional time series data into a specific number (Q) of time intervals, constructing a weight matrix having a size of a specific number (Q) × a specific number (Q), normalizing the sum of each column in the constructed weight matrix to 1 to generate a Markov transition matrix (W), and generating a Markov transition field (M) based on the generated transition matrix.

The embodiments described above are combinations of components and features of the present invention in a given form. Each component or feature should be considered optional unless explicitly stated otherwise. Each component or feature may be implemented in a form that is not combined with other components or features. In addition, it is also possible to combine some components and/or features to form an embodiment of the present invention. The order of operations described in the embodiments of the present invention may be changed. Some components or features of one embodiment may be included in another embodiment, or may be replaced with corresponding components or features of another embodiment. It is obvious that claims that do not have an explicit citation relationship in the scope of the patent may be combined to form an embodiment or may be included as a new claim by post-application amendment.

Embodiments of the present invention may be implemented by various means, for example, hardware, firmware, software, or a combination thereof. In case of implementation by hardware, an embodiment of the present invention may be implemented by one or more ASICs (application specific integrated circuits), DSPs (digital signal processors), DSPDs (digital signal processing devices), PLDs (programmable logic devices), FPGAs (field programmable gate arrays), processors, controllers, microcontrollers, microprocessors, and the like.

In the case of implementation by firmware or software, one embodiment of the present invention may be implemented in the form of a module, procedure, function, etc. that performs the functions or operations described above. The software code may be stored in a memory and may be driven by a processor. The memory may be located inside or outside the processor and may exchange data with the processor by various means already known.

It will be apparent to those skilled in the art that the present invention may be embodied in other specific forms without departing from the essential characteristics of the present invention. Accordingly, the above detailed description should not be construed as limiting in all respects but should be considered as illustrative. The scope of the present invention should be determined by a reasonable interpretation of the appended claims, and all changes within the equivalent scope of the present invention are included in the scope of the present invention.

100: Terminal

Claims (12)

A method for measuring a user's location using a convolutional neural network model,
A step of receiving wireless signals from multiple access points;
A step of measuring the RSSI (Received Signal Strength Indication) of the received wireless signal;
A step of measuring the location of the user based on the RSSI of the measured wireless signal using a learned convolutional neural network model and a fingerprinting technique,
The steps for learning the above convolutional neural network model are:
A step of performing preprocessing to divide the RSSI of the measured wireless signal into type slots of specific time units;
A step of generating one-dimensional time series data by adding random noise to the above preprocessed RSSI;
A step of converting the above one-dimensional time series data into two-dimensional image data by applying a Markov Transition Field (MTF); and
A method characterized by comprising a step of using the above two-dimensional image data as learning data of the above convolutional neural network model. In the first paragraph,
The step of converting into two-dimensional image data by applying the above MTF is:
A step of dividing the above one-dimensional time series data into a specific number (Q) of time intervals;
A step of constructing a weight matrix of a specific number (Q) × specific number (Q) size;
A step of generating a Markov transition matrix (W) by normalizing the sum of each column in the above-configured weight matrix to 1; and
A method characterized by comprising a step of generating a Markov transition field (M) based on the generated transition matrix. In the second paragraph,
A method characterized in that the above Markov transition matrix (W) is defined as follows.

The above W is a Markov transition matrix and represents the frequency of the interval qj {1<j<Q} corresponding to the time series data xi of timestamp ti, and the above Q represents the specific number. In the third paragraph,
A method characterized in that the above Markov transition field (M) is defined as follows.
In the fourth paragraph,
A method characterized in that the above random noise is Gaussian noise. In clause 5,
A method characterized in that the above Gaussian noise is defined as follows.

The above p ( x ) is Gaussian noise. In a terminal for measuring the user's location using a convolutional neural network model,
A wireless communication unit that receives wireless signals from multiple access points;
An AI device that learns the above convolutional neural network model; and
A processor functionally connected to the wireless communication unit and the AI device and controlling the overall operation of the terminal, wherein the processor comprises:
Measure the RSSI (Received Signal Strength Indication) of the received wireless signal,
Controlling the RSSI of the measured wireless signal to measure the location of the user based on the learned convolutional neural network model and fingerprinting technique,
The above AI device,
Preprocessing is performed to divide the RSSI of the measured wireless signal into type slots of specific time units.
Random noise is added to the above preprocessed RSSI to generate one-dimensional time series data.
The Markov Transition Field (MTF) is applied to the above one-dimensional time series data to convert it into two-dimensional image data.
A terminal characterized in that the two-dimensional image data is used as learning data for the convolutional neural network model to learn the convolutional neural network model. In the seventh paragraph, the AI device,
Divide the above one-dimensional time series data into a specific number (Q) of time intervals,
A weight matrix of a specific number (Q) × specific number (Q) in size is constructed,
In the weight matrix constructed above, the sum of each column is normalized to 1 to generate a Markov transition matrix (W).
A terminal characterized in that it converts into two-dimensional image data by applying the MTF by generating a Markov transition field (M) based on the generated transition matrix. In Article 8,
A terminal characterized in that the Markov transition matrix (W) is defined as follows.

The above W is a Markov transition matrix and represents the frequency of the interval qj {1<j<Q} corresponding to the time series data xi of timestamp ti, and the above Q represents the specific number. In Article 9,
A terminal characterized in that the Markov transition field (M) is defined as follows.
In Article 10,
A terminal characterized in that the above random noise is Gaussian noise. In Article 11,
A terminal characterized in that the above Gaussian noise is defined as follows.

The above p ( x ) is Gaussian noise.