L. K. Saul and M. I. Jordan. Mixed memory Markov models. In D. Madigan and P. Smyth, editors, Proceedings of the 1997 Conference on Artificial Intelligence and Statistics. Ft. Lauderdale, FL, 1997.  Home/Search   Document Not in Database   Summary   Related Articles   Check

....based on mutual information is used to select features automatically from training data in a manner that encourages discrimination between states. In Coupled HMMs, each observation stream is modeled by a hidden state sequence and the state transitions are coupled between all the state sequences [116]. All these models can also be viewed as di erent instances of a class of models called graphical models, which are being explored as another framework for ASR [10, 135] Another alternative acoustic model is the Stochastic Segment Model or trajectory model (SSM) 43, 98] While each hidden state ....

Lawrence Saul and Michael Jordan. Mixed memory Markov models. Machine Learning, 37:75-87, 1999.

....the U.S. dollar [8] In both cases the real valued XR returns were quantized into 9 symbols. Papageorgiou predicts the directions of changes in Swiss franc U.S. dollar XRs using a second order Markov model (MM) and analyses the correlational structure in the ve major XRs through a mixed memory MM [11]. Giles, Lawrence and Tsoi [2] 3] considered the same set of ve major XRs and predicted the XR directional changes by applying recurrent neural networks to symbolic streams obtained by quantizing the historic real valued directional change values using the self organizing map [12] Generally, ....

Saul LK, and M.I. Jordan MI. Mixed memory markov models. In Proceedings of the 6th International Workshop on Arti cial Intelligence and Statistics, Fort Lauderdale, Florida. 1998.

....problems with robustly estimating model parameters in the large complex systems typically used in speech recognition. Furthermore, it may result in large memory and runtime costs for the system. 2. 4 Loosely coupled streams Loosely coupled models [22] one form of which is the mixed memory model [26], may be viewed as a compromise between the independent stream system and and the discrete stream system. Here the distribution associated with each state of a stream is in uenced by the states of the other streams. There are various possibilities for the nature of this in uence. The extremes of ....

....determined by the state of all the streams, with only the stream transitions independent of one another. Loosely coupled streams allow a compromise between the two systems to be made (and, in a more general model than the factorial HMM, coupling for the stream transitions) The mixed memory model [26] uses the following method for describing the in uence of the emission probabilities of one stream on another p(o t jq t ) S Y s=1 S X u=1 (s) u p(o (s) t jq (u) t ) 12) where the stream weights, s) u , satisfy S X u=1 (s) u = 1; s) u 0 (13) 8 It is ....

L K Saul and M I Jordan. Mixed memory Markov models. Machine Learning, 37:75-87, 1999.

....U.S. dollar [8] In both cases the real valued XR returns were quantized into 9 symbols. Papageorgiou predicts the directions of changes in Swiss franc U.S. dollar XRs using a second order Markov model (MM) and analyses the correlational structure in the five major XRs through a mixed memory MM [11]. Giles, Lawrence and Tsoi [2] 3] considered the same set of five major XRs and predicted the XR directional changes by applying recurrent neural networks to symbolic streams obtained by quantizing the historic real valued directional change values using the self organizing map [12] Generally, ....

L.K. Saul and M.I. Jordan, "Mixed memory markov models," in Proceedings of the 6th International Workshop on Artificial Intelligence and Statistics, Fort Lauderdale, Florida, 1998.

.... difficulties through additional conditional independence assumptions and approximations which exploit the internal, combinatorial structure of the metastates and observations both to reduce the number of parameters and as the basis for efficient, approximate training and decoding algorithms (e.g. [10], 3] The form of model simplification most appropriate for speech is an open research question. In this paper, we simply adopt the scheme proposed in [10] in order to gain insight into the issues involved. This Mixed Memory Approximation for reducing the number of parameters is described in ....

....and observations both to reduce the number of parameters and as the basis for efficient, approximate training and decoding algorithms (e.g. 10] 3] The form of model simplification most appropriate for speech is an open research question. In this paper, we simply adopt the scheme proposed in [10] in order to gain insight into the issues involved. This Mixed Memory Approximation for reducing the number of parameters is described in Section 2.1; some (more generally applicable) decoding and estimation schemes of differing computational complexity are discussed in Section 2.4. 2.1. The ....

[Article contains additional citation context not shown here]

LK Saul and MI Jordan. Mixed Memory Markov Models. Machine Learning, 37:75--87, 1999.

....U.S. dollar [16] In both cases the real valued XR returns were quantized into 9 symbols. Papageorgiou predicts the directions of changes in Swiss franc U.S. dollar XRs using a second order Markov model (MM) and analyses the correlational structure in the five major XRs through a mixed memory MM [18]. Giles, Lawrence and Tsoi [6] considered the same set of five major XRs and predicted the XR directional changes by applying recurrent neural networks to symbolic streams obtained by quantizing the historic real valued directional change values using the self organizing map [11] Generally, it ....

L.K. Saul and M.I. Jordan. Mixed memory markov models. In Proceedings of the 6th International Workshop on Artificial Intelligence and Statistics, Fort Lauderdale, Florida, 1998.

....U.S. dollar [16] In both cases the real valued XR returns were quantized into 9 symbols. Papageorgiou predicts the directions of changes in Swiss franc U.S. dollar XRs using a second order Markov model (MM) and analyses the correlational structure in the five major XRs through a mixed memory MM [18]. Giles, Lawrence and Tsoi [6] considered the same set of five major XRs and predicted the XR directional changes by applying recurrent neural networks to symbolic streams obtained by quantizing the historic real valued directional change values using the self organizing map [11] Generally, it ....

L.K. Saul and M.I. Jordan. Mixed memory markov models. In Proceedings of the 6th International Workshop on Artificial Intelligence and Statistics, Fort Lauderdale, Florida, 1998.

....U.S. dollar [16] In both cases the real valued XR returns were quantized into 9 symbols. Papageorgiou predicts the directions of changes in Swiss franc U.S. dollar XRs using a second order Markov model (MM) and analyses the correlational structure in the five major XRs through a mixed memory MM [18]. Giles, Lawrence and Tsoi [6] considered the same set of five major XRs and predicted the XR directional changes by applying recurrent neural networks to symbolic streams obtained by quantizing the historic real valued directional change values using the self organizing map [11] Generally, it ....

L.K. Saul and M.I. Jordan. Mixed memory markov models. In Proceedings of the 6th International Workshop on Artificial Intelligence and Statistics, Fort Lauderdale, Florida, 1998.

....U.S. dollar [16] In both cases the real valued XR returns were quantized into 9 symbols. Papageorgiou predicts the directions of changes in Swiss franc U.S. dollar XRs using a second order Markov model (MM) and analyses the correlational structure in the five major XRs through a mixed memory MM [18]. Giles, Lawrence and Tsoi [6] considered the same set of five major XRs and predicted the XR directional changes by applying recurrent neural networks to symbolic streams obtained by quantizing the historic real valued directional change values using the self organizing map [11] Generally, it ....

L.K. Saul and M.I. Jordan. Mixed memory markov models. In Proceedings of the 6th International Workshop on Artificial Intelligence and Statistics, Fort Lauderdale, Florida, 1998.

....U.S. dollar [17] In both cases the real valued XR returns were quantized into 9 symbols. Papageorgiou predicts the directions of changes in Swiss franc U.S. dollar XRs using a second order Markov model (MM) and analyses the correlational structure in the five major XRs through a mixed memory MM [19]. Giles, Lawrence and Tsoi [6] considered the same set of five major XRs and predicted the XR directional changes by applying recurrent neural networks to symbolic streams obtained by quantizing the historic real valued directional change values using the self organizing map [11] Generally, it ....

L.K. Saul and M.I. Jordan. Mixed memory markov models. In Proceedings of the 6th International Workshop on Artificial Intelligence and Statistics, Fort Lauderdale, Florida, 1998.

....paper presents a framework for analyzing coupled time series using Markov models that reduces the burden of estimating the (usually large number of) parameters involved in all but the most trivial of models. For this, we appeal to work in factorial Markov models [2] and mixed memory Markov models [4]. 2 Markov models Markov models offer a stochastic interpretation of time series; the next event has a probabilistic dependency on the past k events. The most trivial Markov model is a Markov chain, a simple integer time process composed of a set of n states. The current state is temporally ....

....explosion in the size of the state space leads to large increases in the number of parameters that need to be estimated in the transition matrix. Figure 1: A coupled time series; I t denotes the state at time t and i t denotes the th component of I t . Mixed memory Markov models [4] overcome this problem by representing the transition matrix as a convex combination of the elementary transition matrices of each underlying component; the description presented here closely follows that presented in [4] Let I t denote the tth element of a coupled time series and i t denote ....

[Article contains additional citation context not shown here]

L.K. Saul and M.I. Jordan. Mixed memorymarkov models. In Proceedings of the 1997 Conference on Artificial Intelligence and Statistics, 1997.

....4: Tree structured HMMs In factorial HMMs, the state variables at one time step are assumed to be a priori independent given the state variables at the previous time step. This assumption can be relaxed in many ways by introducing coupling between the state variables in a single time step [45]. One interesting way to couple the variables is to order them, such that S (m) t depends on S (n) t for 1 n m. Furthermore, if all the state variables and the output also depend on an observable input variable, X t , we obtain the Bayesian network shown in Figure 6. S (1) t 1 S (2) t 1 ....

L. K. Saul and M. I. Jordan. Mixed memory Markov models. In D. Madigan and P. Smyth, editors, Proceedings of the 1997 Conference on Artificial Intelligence and Statistics. Ft. Lauderdale, FL, 1997.

....Triangulation ffl Unfortunately, the following graph is not triangulated: ffl Here is a triangulation: ffl We have created cliques of size N 4 . The junction tree algorithm is not efficient for factorial HMMs. 63 Hidden Markov decision trees (Jordan, Ghahramani, Saul, 1997) ffl We can combine decision trees with factorial HMMs ffl This gives a command structure to the factorial representation U 1 Y 1 U 2 Y 2 U 3 Y 3 ffl Appropriate for multiresolution time series ffl Again, the exact calculation is intractable and we must use variational methods 64 ....

.... 8 7 6 5 4 Size of state space Validation set log likelihood HMM model of Bach Chorales 100 Fitting a factorial HMM to the Bach chorale data 10 1 10 2 10 3 10 9 8 7 6 5 4 Factorial HMM Model of Bach Chorales 101 Example hidden Markov decision tree (Jordan, Ghahramani, Saul, 1997) ffl Recall the hidden Markov decision tree, which also yielded intractably large cliques when triangulated: U 1 Y 1 U 2 Y 2 U 3 Y 3 ffl We can variationally transform this model into one of two simplified models: or x z 1 z 1 2 z 1 3 y 1 1 x z 2 1 z 2 2 z 2 3 y 2 x z 3 1 z 3 2 z ....

Saul, L. K., & Jordan, M. I. (in press). Mixed memory Markov models. In D. Madigan & P. Smyth (Eds.), Proceedings of the 1997 Conference on Artificial Intelligence and Statistics, Ft. Lauderdale, FL.

....to learning in sigmoid networks) 5.2. Introducing couplings The architecture for factorial HMMs presented in Section 2 assumes that the underlying Markov chains interact only through the observations. This constraint can be relaxed by introducing couplings between the hidden state variables (cf. Saul Jordan, 1997). For example, if S (m) t depends on S (m) t Gamma1 and S (m Gamma1) t Gamma1 , equation (3) is replaced by the following factorization P (S t jS t Gamma1 ) P (S (1) t jS (1) t Gamma1 ) M Y m=1 P (S (m) t jS (m) t Gamma1 ; S (m Gamma1) t Gamma1 ) 13) Similar exact, ....

....1994) is generalized to include Markovian dynamics for the decisions. Hidden Markov decision trees provide a useful starting point for modeling time series with both temporal and spatial structure at multiple resolutions. We explore this generalization of factorial HMMs in Jordan, Ghahramani, and Saul (1997). 6. Conclusion In this paper we have examined the problem of learning for a class of generalized hidden Markov models with distributed state representations. This generalization provides both a richer modeling tool and a method for incorporating prior structural information about the state ....

Saul, L. K., & Jordan, M. I. (1997). Mixed memory Markov models. In D. Madigan , & P.

No context found.

L. K. Saul and M. I. Jordan. Mixed memory Markov models. In D. Madigan and P. Smyth, editors, Proceedings of the 1997 Conference on Artificial Intelligence and Statistics. Ft. Lauderdale, FL, 1997.

No context found.

L. Saul and M. Jordan, "Mixed memory Markov models, " Machine Learning, vol. 37, pp. 37--87, 1999.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC