JP2017215669A - Probability density function estimation device, continuous value prediction device, method, and program - Google Patents

Abstract

A correct probability density function representing continuous value data generated by an individual can be estimated. A topic estimation unit estimates a topic to which each continuous value data belongs, and represents a distribution of weight parameters for each topic of each individual and a distribution of shape parameters characterizing a histogram for each topic. Estimate hyper parameters. The individual probability density function estimating unit 26 estimates the shape parameter for each topic and the weight for each topic of each individual according to the posterior probability of the shape parameter and the weight posterior probability, and for each individual, for each estimated topic. A probability density function of continuous value data generated by the individual, which is represented by a linear sum using a weight for each topic of the estimated individual of the histogram for each topic based on the shape parameter, is output. [Selection] Figure 1

Description

  The present invention relates to a probability density function estimation device, a continuous value prediction device, a method, and a program.

  In general, we consider a situation in which an individual who behaves stochastically generates a continuous value and observes that continuous data as data. For example, a consumer (= individual) purchases a certain amount (= continuous value), a consumer (= individual) purchases at a certain time interval (= continuous value), a traveler (= An individual moves to the next place of stay at a certain time interval (= continuous value), a traveler (= individual) stays at a certain position coordinate (= continuous value), and a word in a document (= individual) There are various things such as appearing at a certain time (= continuous value), occurring at a time (= continuous value) when there is a failure of the device (= individual).

  If a probabilistic rule that an individual follows when generating a continuous value, that is, a probability density function can be estimated from the data, a continuous value that the individual will generate in the future can be predicted. In the above example, the consumer's future purchase time and the traveler's future place of stay can be predicted. Probability density function estimation methods can be broadly divided into two types: parametric density estimation and nonparametric density estimation. Parametric density estimation is a method in which a probability density function is expressed by a distribution function having parameters, and the values of the parameters are determined from data. On the other hand, nonparametric density estimation is a method of estimating from data without making a strong assumption on the shape of the probability density function. Because no strong assumptions are made, non-parametric density estimation requires more data for accurate estimation. A representative example of nonparametric density estimation is a histogram method, and the present invention relates to this histogram method. Note that the histogram method divides the domain of a variable that takes continuous values into a finite number of sections, assumes that the probability density function takes a constant value in each section, and estimates the value of each section from the observed data This is a method for estimating the probability density function. The probability density function estimated based on the histogram method is particularly called a histogram. Non-Patent Document 1 is an apparatus that represents a probability density function of an individual using a histogram.

J. Rissanen, "Density estimation by stochastic complexity", IEEE Transactions on Information Theory, Vol. 38, pp.315-323, 1992.

  When the data observed from an individual is sufficiently available, a histogram method that can reproduce more various shapes is suitable instead of parametric density estimation in which the shape of the density function is limited (low degree of freedom). However, in many cases, since there is little data of individuals available, the histogram method cannot be applied and parametric density estimation is used. At this time, when the shape of the distribution function assumed in the parametric density estimation is different from the true shape, a correct probability density function cannot be estimated, and the result prediction accuracy is lowered. Table 1 summarizes the advantages and disadvantages of the histogram method and parametric density estimation.

  Since we cannot know the true probability density function inherent in an individual, we should use the histogram method whenever possible.

  The present invention has been made in view of the above circumstances, and provides a probability density function estimation device, method, and program capable of estimating a correct probability density function representing continuous value data generated by an individual. Objective.

  It is another object of the present invention to provide a continuous value prediction apparatus, method, and program capable of accurately predicting continuous value data generated by an individual.

  In order to achieve the above object, a probability density function estimation device according to a first aspect of the present invention receives, as an input, a set of continuous value data generated by the individual and observed by a plurality of individuals. For each continuous value data included, the histogram for each topic, which represents the probability density of the continuous value, is expressed using the number of divisions and bin widths defined for each topic, and the topic to which the continuous value data belongs. According to the posterior probability, the topic to which the continuous value data belongs is estimated, and based on the estimation result of the topic for each continuous value data, a hyperparameter that represents a distribution of weights for each topic of each individual, and a histogram for each topic A topic estimation unit that estimates hyperparameters representing a distribution of shape parameters that characterize the topic, and the topic estimation unit Therefore, the estimated hyperparameter representing the weight distribution, the hyperparameter representing the shape parameter distribution, and the posterior probability of the shape parameter based on the estimation result of the topic for each continuous value data, and the weight According to the posterior probability, the shape parameter for each topic and the weight of each individual for each topic are estimated, and for each individual, the estimated individual of the histogram for each topic based on the shape parameter for each estimated topic And an individual probability density function estimating unit that outputs a probability density function of continuous value data generated by the individual expressed by a linear sum using a weight for each topic.

  Further, in the probability density function estimation method according to the second invention, the topic estimation unit receives a set of continuous value data generated by the individual and observed in a plurality of individuals, and is included in the continuous value data set. For each continuous value data, the posterior of the topic to which the continuous value data belongs, expressed using the division number and bin width determined for each topic in the histogram for each topic, representing the probability density of the continuous value According to the probability, the topic to which the continuous value data belongs is estimated, and based on the estimation result of the topic for each continuous value data, a hyper parameter representing the distribution of weights for each topic of each individual, and a histogram for each topic Hyperparameters representing the distribution of the shape parameters to be characterized, and the individual probability density function estimation unit The posterior probability of the shape parameter based on the hyperparameter representing the distribution of the weight, the hyperparameter representing the distribution of the shape parameter, and the estimation result of the topic for each continuous value data, Estimating the shape parameter for each topic and the weight for each topic for each individual according to the posterior probability of weight, and for each individual, the estimated histogram of each topic based on the shape parameter for each estimated topic A probability density function of continuous value data generated by the individual, which is expressed by a linear sum using a weight for each topic of the individual, is output.

  The probability density function estimation device according to a third aspect of the present invention is an input of a set of continuous value data generated by the individual, observed by a plurality of individuals, and for each continuous value data included in the set of continuous value data, The continuous value data belongs according to the posterior probability of the topic to which the continuous value data belongs, expressed using the number of divisions and bin width of each topic of the histogram for each topic, which represents the probability density of the continuous value. Estimating a topic and, for each topic, according to the a posteriori probability of the number of divisions of the histogram for the topic, expressed using the estimation result of the topic for each continuous value data and the bin width of the histogram for the topic, Estimate the number of histogram divisions for a topic, estimate the topic for each continuous value data, and Histogram bin width for each topic is estimated according to the histogram bin width posterior probability for each topic, expressed using the number of histogram divisions for the pick, and based on the topic estimation results for each continuous value data A division number bin width topic estimation unit for estimating a hyperparameter representing a distribution of weights for each topic of each individual and a hyperparameter representing a distribution of shape parameters characterizing a histogram for each topic; and the division number bin width topic The posterior probability of the shape parameter based on the hyperparameter representing the distribution of the weight, the hyperparameter representing the distribution of the shape parameter, and the estimation result of the topic for each continuous value data, estimated by the estimation unit, and To the posterior probability of weight The shape parameter for each topic and the weight for each topic for each individual, and the histogram for each topic based on the shape parameter for each estimated topic, the number of divisions, and the bin width, An individual probability density function estimator that outputs a probability density function of continuous value data generated by the individual represented by a linear sum using the weight of each estimated individual for each topic.

  Also, in the probability density function estimation method according to the fourth aspect of the invention, the division number bin width topic estimation unit receives a set of continuous value data generated by the individual and observed by a plurality of individuals, and the continuous value data For each continuous value data included in the set, the histogram for each topic that represents the probability density of the continuous value is represented using the number of divisions and bin width for each topic, and the topic to which the continuous value data belongs. The topic to which the continuous value data belongs is estimated according to the posterior probability, and the topic is expressed using the estimation result of the topic for each continuous value data and the bin width of the histogram for the topic for each topic. In accordance with the posterior probability of the number of histogram divisions for the topic, the number of histogram divisions for the topic is estimated and The histogram bin width for each topic is estimated according to the a posteriori probability of the histogram bin width for each topic expressed using the estimation result of the topic and the number of histogram divisions for each topic, and each continuous value Based on the estimation result of the topic for the data, the hyperparameter representing the distribution of the weight for each topic of each individual and the hyperparameter representing the distribution of the shape parameter characterizing the histogram for each topic are estimated, and the individual probability density function Based on the hyperparameter representing the distribution of the weight, the hyperparameter representing the distribution of the shape parameter, and the estimation result of the topic for each continuous value data estimated by the division number bin width topic estimation unit The shape parameters Estimating the shape parameter for each topic and the weight for each topic of each individual according to the probability and the posterior probability of the weight, and based on the estimated shape parameter for each topic, the number of divisions, and the bin width A probability density function of continuous value data generated by the individual, which is expressed by a linear sum using a weight for each topic of the estimated individual of the histogram for each topic, is output.

  A continuous value prediction apparatus according to a fifth aspect of the present invention is a continuous value prediction apparatus that predicts continuous value data generated by an individual to be predicted, and is a continuous value obtained in advance for the individual to be predicted. Using the shape parameter characterizing the histogram for each topic, the number of divisions of the histogram for each topic, the histogram for each topic based on the bin width of the histogram for each topic, and the weight for each topic of the individual A continuous value predicting unit that predicts continuous value data generated by the individual to be predicted is represented in accordance with a probability density function of continuous value data generated by the individual represented by a linear sum.

  A continuous value prediction method according to a sixth aspect of the present invention is a continuous value prediction method in a continuous value prediction apparatus that predicts continuous value data generated by an individual to be predicted, wherein the continuous value prediction unit determines the individual to be predicted. The shape parameter characterizing the histogram for each topic, representing the probability density of a continuous value determined in advance, the number of divisions of the histogram for each topic, the histogram for each topic based on the bin width of the histogram for each topic, The continuous value data generated by the individual to be predicted is predicted according to the probability density function of the continuous value data generated by the individual represented by a linear sum using the weight of each individual for each topic.

  A program according to a seventh invention is a program for causing a computer to function as each part constituting the probability density function estimation device or the continuous value prediction device.

  According to the probability density function estimation apparatus, method, and program of the present invention, the shape parameter for each topic and the weight of each individual for each topic are estimated, and the shape parameter for each estimated topic for each individual. An individual is generated by outputting a probability density function of continuous value data generated by the individual represented by a linear sum using a weight for each topic of the estimated individual of the histogram for each topic based on An effect is obtained that a correct probability density function representing continuous value data can be estimated.

  Further, according to the continuous value prediction apparatus, method, and program of the present invention, the shape parameter characterizing the histogram for each topic, the number of divisions of the histogram for each topic, the histogram for each topic based on the bin width of the histogram for each topic By predicting the continuous value data generated by the individual to be predicted according to the probability density function of the continuous value data generated by the individual represented by a linear sum using the weights for each topic of the individual, The continuous value data generated by can be predicted with high accuracy.

It is a block diagram which shows the structure of the probability density function estimation apparatus which concerns on the 1st Embodiment of this invention. It is a block diagram which shows the structure of the continuous value prediction apparatus which concerns on the 1st, 2nd embodiment of this invention. It is a flowchart which shows the probability density function estimation process routine in the probability density function estimation apparatus which concerns on the 1st Embodiment of this invention. It is a flowchart which shows the continuous value prediction process routine in the continuous value prediction apparatus which concerns on the 1st Embodiment of this invention. It is a figure which shows an example of a histogram. It is a block diagram which shows the structure of the probability density function estimation apparatus which concerns on the 2nd Embodiment of this invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

<Outline according to Embodiment of the Present Invention>
First, an outline of the embodiment of the present invention will be described.

  In many cases, there is little data obtained from each individual, but on the other hand, a small number of data can be obtained from a plurality of individuals. For example, in an e-commerce site, most customers make purchases only a few times, but the number of registered customers is extremely large. Each individual has its own probability density function, but there should be individuals having similar density functions. In this case, (i) the probability density function of all individuals is expressed by a linear sum of a small number of histograms, and (ii) the difference of probability density functions for each individual is expressed by a difference in weighting of the linear sum. Assuming that, the high degree of freedom that is an advantage of the histogram method can be secured, and the sparsity of data, which is its weakness, can be avoided. There is already a device that expresses the probability density function of an individual as a linear sum of histograms (Non-Patent Document 1 above), but it is for a single individual and avoids the sparsity of data that is a weak point of the histogram method. I can't do it.

  In addition to the above, (iii) the idea was to construct a variable bin width histogram. The width of the section determined by the histogram method is called a bin width. However, when there is a restriction that the bin width is equal, the following problems arise.

  Since the division position in the histogram, that is, the discontinuity point of the probability density function cannot be set to an arbitrary position, there is a possibility that a deviation occurs between the true probability density function and the estimated probability density function.

  In the probability density function of the estimation target, when the region where the function form is localized and the widely distributed region coexist, the bin width should be set narrow in the former, and the bin width should be set wide in the latter, Since only a single value can be taken for the bin width, a narrow bin width (or a half-width bin width) is determined from the data in the entire region. As a result, the estimation accuracy of the probability density function may deteriorate. In addition, the fact that the bin width narrower than necessary is selected means that the domain is divided into more sections than necessary.At this time, the number of parameters to be estimated from the data in the histogram method increases, and the estimation / The calculation cost at the time of prediction will increase.

  Although there is a prior art of a histogram method (hereinafter referred to as variable bin width histogram) that estimates a non-constant bin width from data (for example, Non-Patent Document 2), the probability density function of all individuals is expressed by a linear sum of a small number of histograms. However, there is still no device that realizes a variable bin width histogram after expressing the difference in probability density function for each individual by the difference in weighting of the linear sum.

  The embodiment of the present invention has been made in view of the above points. (I) The probability density function of all individuals is expressed by a linear sum of a small number of histograms. (Ii) The difference of probability density functions for each individual. (Iii) It is intended to provide a technology that makes it possible to estimate the probability density function with high accuracy and in individual units by constructing a variable bin width histogram. And

[Non-Patent Document 2]: P. Kontkanen and P. Myllymaki, "MDL Histogram Density Estimation", International Conference on Artificial Intelligence and Statistics, pp.219-226, 2007.

  In the embodiment of the present invention, a case where the present invention is applied to a probability density function estimation device that estimates a probability density function from a set of continuous value data of each individual and a continuous value prediction device will be described as an example. Specifically, first, the probability density function estimation apparatus estimates the probability density function of an individual expressed by a histogram. Next, in the continuous value prediction apparatus, a continuous value to be observed in the future is predicted based on the estimated probability density function of the individual. Details of each device will be described below.

[First Embodiment]
<Configuration of probability density function estimation device according to first embodiment of the present invention>
Next, the configuration of the probability density function estimation apparatus according to the first embodiment of the present invention will be described. As shown in FIG. 1, the probability density function estimation apparatus 100 according to the first embodiment of the present invention includes a CPU, a RAM, a program for executing a probability density function estimation processing routine described later, and various data. It can be composed of a computer including a stored ROM. Functionally, the probability density function estimation apparatus 100 includes an input unit 10, a calculation unit 20, and an output unit 40 as shown in FIG.

  The input unit 10 receives input of a set of continuous value data of the individual observed by a plurality of individuals.

Continuous value data is generated from an individual to be analyzed. The continuous value data includes an individual ID (hereinafter referred to as u), the number of continuous value data observed in the individual u (hereinafter referred to as _Nu ), and the total number of continuous value data observed in all individuals. (Hereinafter referred to as N), a set of continuous value data observed in all individuals (represented as {t _j } ≡ (t ₁ , t ₂ ,... T _N )), and the individual that generated each continuous value data A set of IDs (represented as {u _j } ≡ (u ₁ , u ₂ ,... U _N )).

In addition, the input unit 10 includes the number K of histograms for expressing the probability density function of an individual, and a definition area of the probability density function (a range of values that can be taken by continuous values).

Is accepted as input. However, hereinafter, the K histograms are denoted by k = 1, 2, 3,..., K, respectively, and the k-th histogram is referred to as a topic k.

Further, the input unit 10 receives as input the number of divisions W _{k of the} domain of each topic and the width (= bin width) Δ _lk (T ₁ −T ₀ ) of each divided section of each topic (1 ≦ k ≦). K, 1 ≦ l ≦ W _k ). Where Δ _lk is the unitized bin width,

Holds. Hereinafter, as shown in the following expression, a set of parameters is represented when no index is added.

  Here, the principle of the generation model used in this embodiment will be described.

First, it is assumed that continuous value data is generated from each individual based on the following generation model. j-th individual u _j that generated the data t _j of the probability density function p expressed by the linear sum of K histogram (t | φ, W, Δ ) generates the data t _j from.

... (1)

However, z _j is a latent variable representing the topic (1-K) to which the j-th data belongs,

Is the weight of individual u _j to topic k, and p (t _j | z _j = k, φ _.k ) is the histogram

... (2)

The distribution of the topic k expressed as _follows , φ _lk is a parameter that determines the shape of the histogram, and w _j ^k is

... (3)

An integer l satisfying 1 (1 <w _j ^k <W _k ). The probability density function expressed by a histogram is obtained by equally dividing the domain [T ₀ , T ₁ ] into W _k sections and setting the probability density value in each section l ( ₁ to W _k ) to a constant value φ _lk. W _j ^k indicates in which section the continuous value data t _j is included.

Next, two types of parameters appearing in the generation models of the above formulas (1) to (3)

_And Dirichlet distribution that is a conjugate prior distribution with respect to φ _lk .

... (4)

Next, from the above formulas (1) to (4), the data belonging topic z≡ (z ₁ , z ₂ ,... Z _N ) and the observed continuous value data t≡ (t ₁ , t ₂ ,... T _N ).

... (5)

Here, N _ku is the number of data whose belonging topic is k among the data of individual u, N _kl is the number of data included in the l-th section of the histogram among all data whose belonging topic is k, N _k represents the total number of data whose topic is k, and in order to simplify the model, all elements of the parameter β of the Dirichlet distribution are set equal (β ₁ = β ₂ =... Β).

  The generation model in the present embodiment can be calculated by the above equations (1) to (5). The above is the principle of the generation model in the present embodiment.

  The calculation unit 20 includes a continuous value data storage unit 22, a topic estimation unit 24, and an individual probability density function estimation unit 26.

  The continuous value data storage unit 22 stores a set of continuous value data received by the input unit 10.

For each continuous value data included in the set of continuous value data stored in the continuous value data storage unit 22, the topic estimation unit 24 determines the number of divisions W _k and bin width determined for each topic k in the histogram for each topic. The topic k to which the continuous value data belongs is estimated according to the posterior probability p (z | t, α, β, W) of the topic z to which the continuous value data belongs, which is expressed using _Δlk . Also, the topic estimation unit 24 weights each individual u with respect to each topic k based on the estimation result of topic k for each continuous value data.

A hyper parameter α representing the distribution, estimates a hyper parameter β representative of the distribution of the shape parameter phi. _K characterizing histogram for each topic. The topic estimation unit 24 repeatedly performs the above topic estimation and hyperparameter estimation.

A specific calculation method performed by the topic estimation unit 24 will be described in detail below. First, by calculating the posterior probability p (z | t, α, β, W) of the belonging topic z≡ (z ₁ , z ₂ ,... Z _N ) for each continuous value data, the unknown parameters α, β,

, Φ can all be estimated. Since it is difficult to analytically handle the posterior probability, the topic estimation unit 24 estimates the topic to be assigned to the topic z _j to which each continuous value data t _j obtained from the simultaneous probability of the above equation (5) belongs. Formula for Gibbs sampling

... (6)

Are used to generate and hold ^P samples (z ⁽¹⁾ , z ⁽²⁾ ,..., Z ^(P) ) of z≡ (z ₁ , z ₂ ,... Z _N ). However, −j appearing in the above equation (6) represents a subset obtained by excluding the jth data from the total number N of data sets. Actually it is not a sample of z itself, but a sufficient statistic

Will hold.

In addition, the topic estimation unit 24 generates P samples, and at the same time, based on the probabilistic EM method (see, for example, Non-Patent Document 3), according to the following equation (7), the unknown hyperparameter α, Update the value of β P times. Then, the topic estimation unit 24 sets the hyper parameter values α ^(P) and β ^(P) after the P updates as the respective estimated values.

... (7)

  A specific update formula is given by the following formula (8).

... (8)

[Non-Patent Document 3]: C. Bishop. “Pattern Recognition and Machine Learning”, Springer, New York, 2006.

The individual probability density function estimator 26 estimates the hyperparameters α and β estimated by the topic estimator 24 and the topic estimation results (z ⁽¹⁾ , z ⁽²⁾ ,..., Z ^{(P )} Based on the posterior probability of the shape parameter and the posterior probability of the weight based on ⁾ ), the shape parameter φ _lk for each topic k and the weight for each topic k of each individual u

Is estimated.

Specifically, the individual probability density function estimation unit 26 calculates α, β, sufficient statistics obtained by the topic estimation unit 24.

, The shape parameter φ _lk for each topic k and the weight for each topic k for each individual u according to the following equations (9), (10)

Is estimated.

In the present embodiment, the shape parameter φ _lk for each topic k and the weight of each individual u for each topic k

The average of each posterior probability is adopted as the estimated value of.

... (9)

... (10)

Further, the individual probability density function estimation unit 26 uses the results obtained by the above formulas (9) and (10), and for each individual u, a histogram for each topic based on the estimated shape parameter φ _lk for each topic. Of the probability density function of the continuous value data generated by the individual u represented by a linear sum using the weights of each estimated individual for each topic

Is output from the output unit 40.

(11)

However, the histogram

Is defined by equation (2).

Where the probability density function of an individual is

It is expressed by For each individual u

(1 ≦ k ≦ K) is output, the number of divisions W _k , the unitized bin width Δ _lk , and φ _lk (1 ≦ l ≦ Wk) are output for each topic k, and as a common domain T≡ [T ₀ , T ₁ ] is output. An output example is shown in Table 2. Table 2 shows an example in which the number of topics K = 3.

<Configuration of continuous value prediction apparatus according to first embodiment of the present invention>
Next, the configuration of the continuous value prediction apparatus according to the first embodiment of the present invention will be described. As shown in FIG. 2, the continuous value prediction apparatus 200 according to the first embodiment of the present invention stores a CPU, a RAM, a program for executing a continuous value prediction processing routine described later, and various data. It can be composed of a computer including a ROM. Functionally, the continuous value prediction apparatus 200 includes an input unit 50, a calculation unit 60, and an output unit 70 as shown in FIG. The continuous value prediction apparatus 200 predicts continuous value data that the individual to be predicted will generate in the future.

The input unit 50 uses the shape parameter φ _lk that characterizes the histogram for each topic k, which is estimated by the probability density function estimation apparatus 100 in advance for the individual u to be predicted, and the weight for each topic.

And the histogram division number W _k for each topic and the histogram bin width Δ _lk for each topic.

  The calculation unit 60 includes a continuous value prediction unit 62.

The continuous value prediction unit 62 receives the shape parameter φ _lk that characterizes the histogram for each topic k, and the weight for each topic, received by the input unit 50.

, The weight of the histogram for each topic for each topic of the individual u based on the number of histogram divisions W _k for each topic and the bin width Δ _lk of the histogram for each topic

The continuous value data generated in the future by the individual u to be predicted is predicted according to the probability density function of the continuous value data generated by the individual u _j shown in the above equation (11), which is expressed by a linear sum using

Specifically, the continuous value prediction unit 62 outputs the probability density function of the individual u to be predicted output by the probability density function estimation device 100.

Is used to calculate the expected value E [t ^u ] of the continuous value data to be generated in the future, and is output by the output unit 70 as the predicted value of the continuous value data.

<Operation of the probability density function estimation apparatus according to the first embodiment of the present invention>
Next, the operation of the probability density function estimation apparatus 100 according to the first embodiment of the present invention will be described. When receiving the set of continuous value data to which the individual ID of the individual is assigned in the input unit 10, the probability density function estimation apparatus 100 stores the set of continuous value data in the continuous value data storage unit 22. Further, when the input unit 10 _{receives the} number of topics K, the domain of continuous values, the number of divisions W _k and the bin width Δ _lk of each topic k, the probability density function estimation apparatus 100 receives FIG. The probability density function estimation processing routine shown in FIG.

First, in step S100, the set of continuous value data received by the input unit 10, the number of topics K, the domain of continuous values, the number of divisions W _k and the bin width Δ _lk of each histogram for topic k are acquired. To do.

Next, in step S102, the topic estimation unit 24 estimates the initial values of α and β or the previous step S104 for each continuous value data included in the set of continuous value data stored in the continuous value data storage unit 22. And the sufficient statistics N _ku , N _kl , and N _k obtained from the initial value of topic k or the previous estimation result for each continuous value data, The topic k to which the continuous value data belongs is estimated. In addition, sufficient statistics N _ku , N _kl , N _k , and N _l are calculated based on the estimation result of topic k for each continuous value data.

In step S104, the topic estimation unit 24, based on sufficient statistics N _ku , N _kl , N _k , and N _l obtained from the estimation result of topic k for each continuous value data obtained in step S102 above, Weight of individual u for each topic k

A hyper parameter α representing the distribution, the hyper parameter β representative of the distribution of the shape parameter phi. _K characterizing histogram for each topic, is updated according to the equation (8).

  In step S106, it is determined whether the processes in steps S102 to S104 have been repeated P times. If not repeated, the process returns to step S102 and the processes in steps S102 to S104 are repeated. The process proceeds to S108.

In step S108, the individual probability density function estimation unit 26 calculates the topic for each of the continuous value data calculated in step S102 obtained for each of the hyperparameters α and β updated in step S104. Sufficient statistics corresponding to the results (z ⁽¹⁾ , z ⁽²⁾ , ..., z ^(P) )

The average of the posterior probabilities of the shape parameter φ _lk for each topic k and the weight for each topic k of each individual u shown in the above formulas (9) and (10) based on

From the average of the posterior probabilities of, the shape parameter φ _lk for each topic k and the weight of each individual u for each topic k

Is estimated.

In step S110, the estimated shape parameter φ _lk for each topic k and the weight of each individual u for each topic k

Estimated probability density function of continuous value data generated by each individual u using

Is output from the output unit 40, and the probability density function estimation processing routine is terminated.

<Operation of Continuous Value Prediction Device According to First Embodiment of the Present Invention>
Next, the operation of the continuous value prediction apparatus 200 according to the first embodiment of the present invention will be described. When the input unit 50 receives the probability density function of the continuous value data generated by the individual u estimated by the probability density function estimation device 100 for the individual u to be predicted, the continuous value prediction device 200 performs the prediction shown in FIG. Execute the processing routine.

  In step S200, a probability density function of continuous value data generated by the individual u is acquired.

  In step S202, the continuous value prediction unit 62 calculates the expected value of the continuous value data based on the probability density function of the continuous value data generated by the individual u received by the input unit 50, and sets it as the predicted value.

  In step S204, the output unit 70 outputs the predicted value of the continuous value data obtained in step S202 as a result, and ends the prediction processing routine.

  As described above, according to the probability density function estimation apparatus according to the first embodiment, the shape parameter for each topic and the weight of each individual for each topic are estimated, and each estimated topic for each individual is estimated. By outputting a probability density function of continuous value data generated by an individual represented by a linear sum using a weight for each topic of the estimated individual of the histogram for each topic based on the shape parameter for A correct probability density function representing the continuous value data to be generated can be estimated.

  Further, according to the first embodiment, the probability density function of all individuals is expressed by a linear sum of a small number of histograms, the difference in probability density function for each individual is expressed by the weight of the linear sum, and the variable bin By constructing the width histogram, the probability density function can be estimated with high accuracy and in individual units. For example, the probability density function can be estimated using a variable bin width histogram as shown in FIG.

  In addition, according to the continuous value predicting apparatus according to the first embodiment, the shape parameter characterizing the histogram for each topic, the number of divisions of the histogram for each topic, the histogram for each topic based on the bin width of the histogram for each topic The individual is generated by predicting the continuous value data generated by the individual to be predicted according to the probability density function of the continuous value data generated by the individual, expressed as a linear sum using the weight of each individual topic. It is possible to accurately predict continuous value data.

[Second Embodiment]
explain about. In addition, about the part which becomes the structure similar to 1st Embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  The second embodiment is different from the first embodiment in that the affiliated topic is estimated, and at the same time, the histogram division number and bin width are estimated for each topic.

<Configuration of probability density function estimation apparatus according to second embodiment of the present invention>
As illustrated in FIG. 6, the probability density function estimation apparatus 300 according to the second embodiment includes an input unit 10, a calculation unit 320, and an output unit 40.

The input unit 10 receives input of a set of continuous value data of the individual observed by a plurality of individuals. In addition, the input unit 10 includes the number K of histograms for expressing the probability density function of an individual, and a definition area of the probability density function (a range of values that can be taken by continuous values).

Is accepted as input.

  The calculation unit 320 includes a continuous value data storage unit 22, a division number bin width topic estimation unit 324, and an individual probability density function estimation unit 26.

Similarly to the topic estimation unit 24 of the first embodiment, the division number bin width topic estimation unit 324 continuously outputs each continuous value data included in the set of continuous value data stored in the continuous value data storage unit 22. The posterior probability p (z) of the topic z to which the continuous value data belongs, expressed using the number of divisions W _k and the bin width Δ _lk determined for each topic k in the histogram for each topic, representing the probability density of the values | T, α, β, W, Δ), the topic k to which the continuous value data belongs is estimated.

  In addition, the division number bin width topic estimation unit 324, for each topic, uses the topic estimation result for each continuous value data and the histogram bin width for the topic, and the subsequent number of histogram divisions for the topic. The number of histogram divisions for the topic is estimated according to the probability p (W | z, t, α, β, W, Δ). Further, the division number bin width topic estimation unit 324 uses the topic estimation result for each continuous value data and the histogram division number for each topic, and the histogram bin width posterior probability p for each topic. Estimate the bin width of the histogram for each topic according to (Δ | z, t, α, β, W, Δ).

Similarly to the topic estimation unit 24 according to the first embodiment, the division number bin width topic estimation unit 324 applies each topic u to each topic k based on the estimation result of the topic k for each continuous value data. weight

A hyper parameter α representing the distribution, estimates a hyper parameter β representative of the distribution of the shape parameter phi. _K characterizing histogram for each topic. The topic estimation unit 24 repeatedly performs the above-described topic estimation, division number estimation, bin width estimation, and hyperparameter estimation.

  A specific calculation method performed by the division number bin width topic estimation unit 324 will be described in detail below.

  First, similarly to the topic estimation unit 24 of the first embodiment, the Gibbs sampling of the affiliated topic is executed once. Next, in order to obtain sampling of the number of divisions W, Dirichlet distribution as prior distribution

(12)

And the marginal distribution of the simultaneous distribution of the above equation (5) is performed with respect to the bin width Δ. γ is a parameter of the Dirichlet distribution. Peripheralization is approximately performed based on Bayesian Information Criteria (Non-Patent Document 3 above).

... (13)

Where Δ ^* is the maximum posterior probability estimate of bin width.

(14)

  Furthermore, uniform distribution as no information prior distribution about the number of divisions

(15)

And Gibbs sampling formula for the number of divisions W

... (16)

Get. Sampling of the division number W is executed according to the above equation (16). At the same time, the bin width is given by equation (14). Empirically, the distribution of the number of divisions W is sharp, so the number of divisions obtained in the last sampling

Is an estimated value. The update formulas for the unknown hyperparameters α and β are given by the following formula (17) in the same manner as the formula (8).

... (17)

The individual probability density function estimation unit 26 calculates α, β, and sufficient statistics obtained by the division number bin width topic estimation unit 324.

, W, and Δ, the shape parameter φ _lk for each topic k and the weight for each topic k of each individual u according to the above equations (9) and (10)

Is estimated.

Further, the individual probability density function estimation unit 26 uses the results obtained by the above formulas (9) and (10), and for each individual u, a histogram for each topic based on the estimated shape parameter φ _lk for each topic. Of the probability density function of the continuous value data generated by the individual u represented by a linear sum using the weights of each estimated individual for each topic

Is output from the output unit 40.

  In addition, about the other structure and effect | action of the probability density function estimation apparatus 300 which concern on 2nd Embodiment, since it is the same as that of 1st Embodiment, description is abbreviate | omitted.

<Configuration of continuous value prediction apparatus according to second embodiment of the present invention>
The continuous value prediction apparatus according to the second embodiment is similar to the first embodiment in that the continuous value data generated by the individual u, which is estimated by the probability density function estimation apparatus 300 for the individual u to be predicted. Based on the probability density function, the expected value of the continuous value data is calculated and used as a predicted value.

  As described above, according to the probability density function estimation apparatus according to the second embodiment, the number of histogram divisions for each topic and the bin width of the histogram for each topic are estimated, the shape parameter for each topic, and Estimate the weight of each individual for each topic, and for each individual, each estimated topic for each topic in the shape parameter for each estimated topic, the number of divisions for each topic, and the histogram for each topic based on the bin width for each topic By outputting a probability density function of continuous value data generated by an individual represented by a linear sum using a weight for, a correct probability density function representing continuous value data generated by the individual can be estimated.

  The present invention is not limited to the above-described embodiment, and various modifications and applications can be made without departing from the gist of the present invention.

  For example, the probability density function estimation apparatuses 100 and 300 according to the above embodiment have been described with respect to the case where the continuous value data storage unit 22 is provided. For example, the continuous value data storage unit 22 includes the probability density function estimation apparatuses 100 and 300. The probability density function estimation apparatuses 100 and 300 may be configured to refer to the continuous value data storage unit 22 by communicating with the external apparatus using a communication unit.

  Moreover, although the case where the probability density function estimation apparatuses 100 and 300 and the continuous value prediction apparatus 200 are configured as separate apparatuses has been described as an example in the above embodiment, the probability density function estimation apparatuses 100 and 300 and the continuous value prediction are described. The device 200 may be configured as one device.

  In addition, the probability density function estimation device and the continuous value prediction device described above have a computer system inside, but if the “computer system” uses a WWW system, a homepage providing environment (or Display environment).

  In the present specification, the embodiment has been described in which the program is installed in advance. However, the program can be provided by being stored in a computer-readable recording medium.

10, 50 Input unit 20, 60, 320 Operation unit 22 Continuous value data storage unit 24 Topic estimation unit 26 Individual probability density function estimation unit 40, 70 Output unit 62 Continuous value prediction unit 100, 300 Probability density function estimation device 200 Continuous value Prediction device 324 Division number bin width topic estimation unit

Claims (7)

A set of continuous value data generated by the individual observed by a plurality of individuals is input, and for each continuous value data included in the continuous value data set, a histogram of each topic representing the probability density of the continuous value. , The topic to which the continuous value data belongs is represented according to the posterior probability of the topic to which the continuous value data belongs, expressed using the number of divisions and bin width determined for each topic,
Topic estimation that estimates hyperparameters representing the distribution of weights for each topic of each individual and hyperparameters representing the distribution of shape parameters that characterize the histogram for each topic based on the topic estimation results for each continuous value data And
A posterior probability of the shape parameter based on the hyperparameter representing the distribution of the weight, the hyperparameter representing the distribution of the shape parameter, and the estimation result of the topic for each continuous value data, estimated by the topic estimation unit, And estimating the shape parameter for each topic and the weight for each topic for each individual according to the posterior probability of the weight, and for each individual, the histogram for each topic based on the shape parameter for each estimated topic, An individual probability density function estimator that outputs a probability density function of continuous value data generated by the individual represented by a linear sum using a weight for each topic of the estimated individual;
Probability density function estimation device. A set of continuous value data generated by the individual observed by a plurality of individuals is input, and for each continuous value data included in the continuous value data set, a histogram of each topic representing the probability density of the continuous value. , The topic to which the continuous value data belongs is estimated according to the posterior probability of the topic to which the continuous value data belongs, expressed using the number of divisions and bin width for each topic,
For each topic, the division of the histogram for the topic according to the posterior probability of the number of divisions of the histogram for the topic, expressed using the estimation result of the topic for each continuous value data and the bin width of the histogram for the topic Estimate the number,
Estimate the histogram bin width for each topic according to the a posteriori probability of the histogram bin width for each topic, expressed using the topic estimation results for each continuous value data and the number of histogram divisions for each topic. ,
Based on the estimation result of the topic for each continuous value data, the number of divisions for estimating the hyperparameter representing the distribution of the weight for each topic of each individual and the hyperparameter representing the distribution of the shape parameter characterizing the histogram for each topic A bin width topic estimator;
The shape parameter based on the hyperparameter representing the distribution of the weight, the hyperparameter representing the distribution of the shape parameter, and the estimation result of the topic for each continuous value data estimated by the division number bin width topic estimation unit. The shape parameter for each topic and the weight for each topic of each individual in accordance with the posterior probability of the weight and the weight posterior probability, and the shape parameter for each of the estimated topics, the number of divisions, and the bin width An individual probability density function estimator that outputs a probability density function of continuous value data generated by the individual represented by a linear sum using a weight for each topic of the estimated individual of the histogram for each topic based on ,
Probability density function estimation device. A continuous value prediction apparatus for predicting continuous value data generated by an individual to be predicted,
Representing the probability density of continuous values obtained in advance for the individual to be predicted, shape parameters characterizing the histogram for each topic, the number of divisions of the histogram for each topic, and each histogram based on the bin width of the histogram for each topic Continuous that predicts continuous value data generated by the individual to be predicted according to a probability density function of continuous value data generated by the individual represented by a linear sum using a weight for each topic of the individual of the histogram for the topic A continuous value prediction device including a value prediction unit. The topic estimation unit receives a set of continuous value data generated by the individual observed by a plurality of individuals, and represents the probability density of continuous values for each continuous value data included in the set of continuous value data. According to the posterior probability of the topic to which the continuous value data belongs, represented by using the division number and bin width determined for each topic of the histogram for each topic, estimate the topic to which the continuous value data belongs,
Based on the estimation result of the topic for each continuous value data, a hyperparameter representing a distribution of weights for each topic of each individual and a hyperparameter representing a distribution of shape parameters characterizing a histogram for each topic are estimated,
The individual probability density function estimation unit is based on the hyper parameter representing the distribution of the weight, the hyper parameter representing the distribution of the shape parameter, and the estimation result of the topic for each continuous value data estimated by the topic estimation unit. In accordance with the posterior probability of the shape parameter and the posterior probability of the weight, the shape parameter for each topic and the weight of each individual for each topic are estimated, and for each individual, the estimated shape parameter for each topic is A probability density function estimation method for outputting a probability density function of continuous value data generated by the individual, expressed by a linear sum using a weight for each topic of the estimated individual of a histogram for each topic based thereon. The division number bin width topic estimation unit receives a set of continuous value data generated by the individual observed by a plurality of individuals, and for each continuous value data included in the continuous value data set, the probability of the continuous value The topic to which the continuous value data belongs is estimated according to the posterior probability of the topic to which the continuous value data belongs, which is expressed using the number of divisions and the bin width for each topic in the histogram for each topic that represents density. ,
For each topic, the division of the histogram for the topic according to the posterior probability of the number of divisions of the histogram for the topic, expressed using the estimation result of the topic for each continuous value data and the bin width of the histogram for the topic Estimate the number,
Estimate the histogram bin width for each topic according to the a posteriori probability of the histogram bin width for each topic, expressed using the topic estimation results for each continuous value data and the number of histogram divisions for each topic. ,
Based on the estimation result of the topic for each continuous value data, a hyperparameter representing a distribution of weights for each topic of each individual and a hyperparameter representing a distribution of shape parameters characterizing a histogram for each topic are estimated,
An individual probability density function estimation unit estimated by the division number bin width topic estimation unit is a hyperparameter representing the weight distribution, a hyperparameter representing the shape parameter distribution, and the topic of each continuous value data. Based on the estimation result, according to the posterior probability of the shape parameter and the posterior probability of the weight, the shape parameter for each topic and the weight for each topic of each individual are estimated, and the shape parameter for each estimated topic, Output a probability density function of continuous value data generated by the individual represented by a linear sum using a weight for each topic of the estimated individual of the histogram for each topic based on the number of divisions and the bin width Probability density function estimation method. A continuous value prediction method in a continuous value prediction apparatus for predicting continuous value data generated by an individual to be predicted,
A shape parameter characterizing a histogram for each topic, which is obtained in advance by the continuous value prediction unit for the individual to be predicted, characterizing the probability density of the continuous value, the number of divisions of the histogram for each topic, and the histogram for each topic A continuous generated by the individual to be predicted according to a probability density function of continuous value data generated by the individual represented by a linear sum using a weight for each topic of the individual of the histogram for each topic based on the bin width of A continuous value prediction method that predicts value data.   The program for functioning a computer as each part which comprises the probability density function estimation apparatus of Claim 1 or Claim 2, or the continuous value prediction apparatus of Claim 3.