J. Schmidhuber, M. C. Mozer, and D. Prelinger. Continuous history compression. In H. Huning, S. Neuhauser, M. Raus, and W. Ritschel, editors, Proc. of Intl. Workshop on Neural Networks, RWTH Aachen, pages 87-95. Augustinus, 1993.  Home/Search   Document Details and Download   Summary   Related Articles   Check

....100 papers on diverse topics including fine arts [96] and the nature of surprises [97] Apparently he even founded a religion [94] Most of his articles, however, are about machines that learn from experience. I have started to compile an incomplete list of references to work by him and his lab [117, 116, 39, 50, 40, 42, 43, 41, 52, 49, 56, 44, 54, 47, 48, 51, 53, 57, 46, 68, 45, 55, 69, 64, 65, 59, 66, 58, 67, 60, 63, 61, 73, 71, 79, 70, 74, 62, 72, 75, 78, 82, 80, 76, 81, 77, 84, 89, 88, 94, 87, 85, 96, 83, 100, 86, 90, 99, 91, 93, 105, 119, 95, 92, 97, 120, 118, 98, 125, 130, 129, 126, 128, 124, 123, 122, 131, 127, 35, 34, 36, 38, 32, 33, 37, 27, 28, 25, 24, 22, 23, 15, 9, 21, 10, 16, 26, 17, 18, 6, 7, 8, 13, 11, 20, 19, 14, 12, 115, 114, 121, 30, 106, 108, 107, 29, 31, 109, 110, 111, 112, 113, 5, 101, 103, 104, 4, 3, 2, 1, 102]. Hopefully I ll be able to add missing entries soon. Future work will concentrate on categorizing related papers and establishing common threads. ....

J. Schmidhuber, M. C. Mozer, and D. Prelinger. Continuous history compression. In H. Huning, S. Neuhauser, M. Raus, and W. Ritschel, editors, Proc. of Intl. Workshop on Neural Networks, RWTH Aachen, pages 87--95. Augustinus, 1993.

No context found.

J. Schmidhuber, M. C. Mozer, and D. Prelinger. Continuous history compression. In H. Huning, S. Neuhauser, M. Raus, and W. Ritschel, editors, Proc. of Intl. Workshop on Neural Networks, RWTH Aachen, pages 87-95. Augustinus, 1993.

....this pattern. Across phrases, however, a more global view of the organization is necessary. The difficult problem of learning coarse as well as fine structure has been addressed recently by connectionist researchers (Mozer, 1992; Mozer Das, 1993; Myers, 1990; Ring, 1992; Schmidhuber, 1992; Schmidhuber, Mozer, Prelinger, 1993). The basic idea in many of these approaches involves building a reduced description (Hinton, 1988) of the sequence that makes global aspects more explicit or more readily detectable. In the case of the AABA structure, this might involve taking the sequence of notes composing A and redescribing ....

....like for the system to discover the decomposition itself. Ideally, each level of the hierarchy should encode meaningful chunks or events in the sequence the nature of a chunk being determined by the statistics of the environment and the next higher level should operate on these chunks. Schmidhuber, Mozer, and Prelinger (1993) describe an architecture of this sort in which the higher level analyzes only components of the input that cannot be interpreted by the lower level, yielding an automatic decomposition of sequences. This architecture has not been tested on musical sequences. Mozer and Das (1993) present an ....

Schmidhuber, J. H., Mozer, M. C., & Prelinger, D. (1993). Continuous history compression. In H. Huening , S. Neuhauser, M. Raus, & W. Ritschel (Eds.), Proceedings of the International Workshop on Neural Networks, RWTH Aachen (pp. 87-95). Augustinus.

....that the code components are statistically independent. This can be useful in conjunction with 1 J. Schmidhuber, TUM, 80290 Munchen, Germany (schmidhu informatik.tu muenchen.de) supervised statistical classifiers that assume statistical independence of their input variables. And see [5] [8], and especially [7] for adaptive networks that try to measure the information conveyed by components of input sequences. The results of these measurements modulate the sequence processing strategy of a recurrent network R in a way that greatly improves R s ability to detect correlations between ....

J. H. Schmidhuber, M. C. Mozer, and D. Prelinger. Continuous history compression. In H. Huning, S. Neuhauser, M. Raus, and W. Ritschel, editors, Proc. of Intl. Workshop on Neural Networks, RWTH Aachen, pages 87--95. Augustinus, 1993.

No context found.

Schmidhuber, J., Mozer, M. C., and Prelinger, D. (1993). Continuous history compression. In Huning, H., Neuhauser, S., Raus, M., and Ritschel, W., editors, Proc. of Intl. Workshop on Neural Networks, RWTH Aachen, pages 87--95. Augustinus.

....The net input, however, tends to perturb the stored information, which again makes long term storage impracticable. Schmidhuber s chunker systems do have a capability to bridge very long time lags, but only if the input sequence exhibits locally predictable regularities (see Schmidhuber, 1992b; Schmidhuber et al. 1993; and Mozer, 1992) Long Short Term Memory (LSTM) the new approach presented in this paper, overcomes the problems above. Unlike chunking systems, even in noisy, highly unpredictable environments, LSTM can learn loss free information storage spanning arbitrary time periods. A major LSTM ....

Schmidhuber, J. H., Mozer, M. C., and Prelinger, D. (1993). Continuous history compression. In Huning, H., Neuhauser, S., Raus, M., and Ritschel, W., editors, Proc. of Intl. Workshop on Neural Networks, RWTH Aachen, pages 87--95. Augustinus.

....sequences. Recall that conventional recurrent networks failed to learn the task within 10 7 training sequences. In this special case, the speed up factor obtained by adaptive redundancy reduction is at least 10 3 . More details and extensions of the principle above can be found in [21] 23] [29], and especially in [24] The next section describes a somewhat different kind of neural predictor for compressing natural text (as opposed to artificial symbol strings) 3 EXAMPLE 2: Text Compression The example from the previous section was based on artificial data from a stochastic automaton. ....

J. Schmidhuber, M. C. Mozer, and D. Prelinger. Continuous history compression. In H. Huning, S. Neuhauser, M. Raus, and W. Ritschel, editors, Proc. of Intl. Workshop on Neural Networks, RWTH Aachen, pages 87--95. Augustinus, 1993.

....of memory models, it is not comprehensive and does not address all issues in the design of a memory model. Some models in the literature, although they can be pigeonholed into a certain cell in the taxonomy, have critical properties that are ignored by the taxonomy. For example, Schmidhuber (1992; Schmidhuber, Prelinger, Mozer, 1993) has proposed a type of I delay memory in which the memory size and the i delay parameters are dynamically determined based on the input sequence as it is Mozer 11 processed. The idea underlying this approach is to discard redundant input elements in a sequence. See Myers, 1990, and Ring, ....

Schmidhuber, J., Prelinger, D., & Mozer, M. C. (1993). Continuous history compression. Manuscript in preparation.

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC