J. Jackson. The Harmonic Sieve: A Novel Application of Fourier Analysis to Machine Learning Theory and Practice. PhD thesis, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, 1995. NDFT Resampling/DSS  Home/Search   Document Not in Database   Summary   Related Articles   Check

....Such transformations can construct a Walsh representation with only a polynomial number of low order terms that are exponentially more signi cant than the rest when tter chromosomes are given more copies through a redundant, equivalent representation. This is a very desirable property [48, 29] for ecient function induction from data which is a fundamental problem in learning, data mining, and optimization. This also means that in such representations higher order interactions among the variables are negligible. We still need to identify the class of functions for which such ....

J. Jackson. The Harmonic Sieve: A Novel Application of Fourier Analysis to Machine Learning Theory and Practice. PhD thesis, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, 1995.

....a function and points out how the general problem of function induction relates to this representation in particular. 2. 3 The Multidimensional Fourier Transform and the Nonlinearity of Functions In this section, we recapitulate the Multidimensional Fourier Transform (MFT) or the Walsh transform [13, 14, 6, 19, 35, 43], which is a useful tool for detailed study of the functions we will be considering as well as the GCTs themselves. We use the MFT to study real valued functions de ned on X. Let F be the set of all such functions. F forms a ( Q n k i ) dimensional vector space over R. The MFT of a function f ....

J. Jackson. The Harmonic Sieve: A Novel Application of Fourier Analysis to Machine Learning Theory and Practice. PhD thesis, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, 1995.

....to work well in nature, the class of such functions may not be trivial and we should explore it further. Key Words: Gene expression, genetic code, function induction. 1 The analysis is identical to that using Walsh basis [4, 45] however, the the term Fourier is chosen because of its historical [32, 18] use in function approximation literature. Complex Systems, 11 (2001) 1 1 ; c 2001 Complex Systems Publications, Inc. 2 1. Introduction Learning functions from data is important in many elds such as inductive learning, statistics, and data mining. It may also be important for ....

....or non overlapping subfunctions where each of the sub functions can depend on at most k variables. This condition guarantees a polynomial size description of the target function in Fourier representation. An alternate technique for estimating the Fourier representations is proposed elsewhere [18, 32]. An extension of this technique for detecting function structure in genetic algorithms is reported in [41] Although there exists many functions with a polynomial size canonical representation, it is not clear why the natural genetic tness function should have such a property. This paper ....

[Article contains additional citation context not shown here]

J. Jackson. The Harmonic Sieve: A Novel Application of Fourier Analysis to Machine Learning Theory and Practice. PhD thesis, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, 1995.

....a redundant, equivalent representation of tter members. Key Words: Gene expression, genetic code, Fourier basis representation, polynomial complexity function induction. 1 The analysis is identical to that using Walsh basis [4] however, I choose the term Fourier because of its historical [32, 18] use in function approximation literature. 2 Although the class of such functions is yet to be precisely characterized, this paper develops a general understanding. 1 1 Introduction Learning functions from data is important in many elds such as inductive learning, statistics, and data mining. ....

....or non overlapping sub functions where each of the sub functions can depend on at most k variables. This condition guarantees a polynomial size description of the target function in Fourier representation. An alternate technique for estimating the Fourier representations is proposed elsewhere [18, 32]. An extension of this technique for detecting function structure in genetic algorithms is reported in [41] Although there exists many functions with a polynomial size canonical representation, it is not clear why the natural genetic tness function should have such a property. This paper ....

[Article contains additional citation context not shown here]

J. Jackson. The Harmonic Sieve: A Novel Application of Fourier Analysis to Machine Learning Theory and Practice. PhD thesis, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, 1995.

....boosting algorithms to achieve a new hardcore set construction which subsumes and improves results of Impagliazzo and Nisan in the circuit size parameter and has a slightly worse measure size parameter. We first consider Freund s boost by majority algorithm from [10] which, following Jackson [18], we refer to as F1. Algorithm F1 simulates at most k = O( 2 log(1= distributions D i and combines its k hypotheses using the majority function. Jackson s analysis ( 18] pp. 57 59) yields the following fact about F1: Fact 14 If F1 is given inputs ; EX(f; D) and a (1=2 ....

....worse measure size parameter. We first consider Freund s boost by majority algorithm from [10] which, following Jackson [18] we refer to as F1. Algorithm F1 simulates at most k = O( 2 log(1= distributions D i and combines its k hypotheses using the majority function. Jackson s analysis ([18], pp. 57 59) yields the following fact about F1: Fact 14 If F1 is given inputs ; EX(f; D) and a (1=2 ) approximate weak learner WL for f; then with high probability each distribution D 0 which F1 simulates for WL satisfies L1 (D 0 ) O(1= 2 ) L1 (D) This immediately implies ....

J. Jackson. "The Harmonic Sieve: A Novel Application of Fourier Analysis to Machine Learning Theory and Practice," Ph.D. thesis, Carnegie Mellon University, Aug. 1995; Available as technical report CMUCS -95-183.

....and Bruck and Smolensky [BS 92] use abstract harmonic analysis to derive necessary and sufficient conditions for a Boolean function to be a polynomial threshold. Finally, in recent years, several results in Learning Theory have been obtained by applying Fourier based techniques (see [Man 94] and [Jac 95] for a comprehensive and exhaustive survey on this topic) 1.2 Definitions Unless otherwise specified, the indexing of vectors and matrices starts from 0 rather than 1. The symbol e 1 denotes the first column of the identity matrix. A T denotes the transpose of a matrix A. ae(B) denotes the ....

J. Jackson. The Harmonic Sieve: A Novel Application of Fourier Analysis to Machine Learning Theory and Practice. PhD Thesis, Carnagie Mellon University (1995).

...., how well can we predict (The difference from the question addressed by this research is that now the number of labeled bit vectors can be polynomial in the number of terms. If the algorithm is allowed to choose the bit vectors, then the chance of correctness can be made arbitrarily close to 1 [Jac]. But how well can we do if the bit vectors are random The algorithms for monotone Boolean formulas are the best known, and so the simple algorithm described above is the state of the art. There is no hardness result known for this question. ....

J Jackson. The Harmonic Sieve: A Novel Application of Fourier Analysis to Machine Learning Theory and Practice. PhD thesis, CMU, 1995. CMU-CS-95-183.

....poly(n; t; log ) where t is the number of boxes. In this case only equivalence queries are used, i.e. we learn this class in the on line model [20] This result generalizes the learnability of O(1) DNF and of boxes in O(1) dimensions. The class of k degenerate boxes was previously considered in [17, 18, 24]. Our algorithm for this class also learns the class of unions of O(1) boxes from equivalence queries only. The first two results are obtained in two steps: First, in Section 3, we show how to learn these classes but with complexity which is polynomial in (and the other parameters of the ....

J. C. Jackson. The Harmonic Sieve: A Novel Application of Fourier Analysis to Machine Learning Theory and Practice. PhD thesis, Technical Report CMUCS -95-184, School of Computer Science, Carnegie Mellon University, 1995.

....set of instances x chosen at random from D, each one labeled with whether it is a positive or negative instance. We chose the PAC learning model for demonstrating that ARVPs are polynomially learnable because the PAC model is a distribution free approach (in contrast to more restricted approaches [6, 7, 8, 9] that make specific assumptions about the distributions of input examples) In the PAC model, so long as the distribution D is the same during learning and testing, the algorithm should give a good result, regardless of the distribution type. Another alternative is the exact learning model ....

J. Jackson. The Harmonic Sieve: A Novel Application of Fourier Analysis to Machine Learning Theory and Practice. PhD thesis, Carnegie Mellon University, 1995.

....with a given target function. The second algorithm is the Harmonic Sieve (HS) an algorithm that eciently learns the class of DNF expressions with respect to the uniform distribution [11] Our analysis corrects a de ciency in an earlier attempt at producing a noisetolerant version of HS [11,12]. As both algorithms utilize Fourier analysis, we brie y discuss the multi dimensional discrete Fourier transform before analyzing these learning algorithms. 12 5.1 The Fourier transform For each set A f1; ng we de ne the function A : f0; 1g n f1; 1g as A (x) 1) P i2A ....

....on both the variance of g and on the number of subsets at each level of the WP recursion. The other component of the Harmonic Sieve is a modi cation of a hypothesisboosting algorithm due to Freund. We give only the basic ideas here; the reader interested in a detailed discussion is referred to [12]. Given a weak learning algorithm for a function class and the task of producing a strong hypothesis with respect to uniform, the hypothesis booster rst uses the weak learner to produce a weak hypothesis with respect to uniform. It then determines a number k of stages that will be performed (how ....

[Article contains additional citation context not shown here]

J. C. Jackson. The Harmonic Sieve: A Novel Application of Fourier Analysis to Machine Learning Theory and Practice. PhD thesis, Carnegie Mellon University, Aug. 1995. Available as technical report CMU-CS-95-183.

....that distribution free learnability of 1 DL requires access to at least half of the bits in each example. Our study of k TOP is motivated in part by the fact that it is known to have useful Fourier properties [17] furthermore, it has also been studied in the context of empirical machine learning [18]. We exploit the Fourier properties of k TOP to show rst that k TOP is weakly k RFA learnable and that this learning is ecient for constant k. Second, we show that with respect to the uniform distribution, k TOP is strongly k RFA learnable with polynomial (in the usual learning parameters, and ....

.... a ] 1 2 j 1 2s : We use the notation O( in the following theorem and elsewhere to represent the standard big O notation with log factors suppressed. Theorem 9 k TOP is weakly k RFA learnable in time O(n k 1 s 2 ) Proof Sketch: By standard Cherno bound arguments (see, e.g. [18]) given an example oracle for f and a xed a we can produce an estimate of Pr x2D [f = a ] that, with probability at least 1 over the random draws by the example oracle, is within 1= 8s) of the true value. Furthermore, the algorithm producing this estimate runs in time O(ns 2 log 1 ) ....

[Article contains additional citation context not shown here]

Jackson, J. C. (1995). The Harmonic Sieve: A Novel Application of Fourier Analysis to Machine Learning Theory and Practice. PhD thesis, Carnegie Mellon University. Available as technical report CMU-CS-95-183.

....sketched in somewhat more detail in Figures 4.1 and 4.2. We will assume here and elsewhere that the number of terms s in the target function s representation as a DNF is known. This assumption can be relaxed by placing a standard guess and double loop around the body of the HS program (see, e.g. [24] for details) this increases the running time of the algorithm by at most a factor of log s. The HS algorithm runs for O(s 2 log(1= stages. At each stage i, r i (x) line 8) represents the number of weak hypotheses w j among those hypotheses produced before stage i that are right on x. For ....

.... USING A QUANTUM EXAMPLE ORACLE 15 Using this fact and the earlier description of the algorithm, it is straightforward to show that the algorithm as given has a time bound of O(ns 6 = 12 ) This compares with a bound on the Harmonic Sieve of O(ns 8 = 18 ) Furthermore, as noted in [24], the 3 factor that appears in the Harmonic Sieve can be brought arbitrarily close to 2 ; with this improvement the bounds improve to approximately O(ns 6 = 8 ) and ( O(ns 8 = 12 ) respectively. While neither bound is a small polynomial, the quantum algorithm is ....

J. C. Jackson, The Harmonic Sieve: A Novel Application of Fourier Analysis to Machine Learning Theory and Practice, PhD thesis, Carnegie Mellon University, Aug. 1995. Available as technical report CMU-CS-95-183.

....An implementation of F1 that incorporates these ideas and takes into account several smaller issues is given in Figure 1. The following lemma concerning the functionality and running time of F1 is due to Freund [21] the previously unpublished proof of one part of Freund s argument is presented in [32]. Lemma 6 (Freund) Algorithm F1, given positive ffl, ffi, and fl, a ( 1 2 Gamma fl) approximate PAC learner for representation class F , and example oracle EX(f;D) for some f 2 F and any distribution D, runs in time polynomial in n, s, fl Gamma1 , ffl Gamma1 , and log(ffi Gamma1 ) and ....

....the same as the big O notation but suppresses logarithmic factors, and T (WL) denotes the maximum time required by any call to WL. By changing the bound on fl in line 1 of the algorithm, it is possible to move the contribution of ffl to the algorithm s time bound arbitrarily close to ffl Gamma4 [32], but we have chosen simplicity over efficiency here for expositional purposes. F1 can also be used to boost a weak membership query learner into a strong membership query learner. The 15 Invocation: h F1(EX(f; D) fl; WL; ffl; ffi) Input: Example oracle EX(f;D) for target f and distribution ....

[Article contains additional citation context not shown here]

J. C. Jackson, The Harmonic Sieve: A Novel Application of Fourier Analysis to Machine Learning Theory and Practice, PhD thesis, Carnegie Mellon University, Aug. 1995. Available as technical report CMU-CS-95183.

....sketched in somewhat more detail in Figures 4.1 and 4.2. We will assume here and elsewhere that the number of terms s in the target function s representation as a DNF is known. This assumption can be relaxed by placing a standard guess and double loop around the body of the HS program (see, e.g. [24] for details) this increases the running time of the algorithm by at most a factor of log s. The HS algorithm runs for O(s 2 log(1 #) stages. At each stage i, r i (x) line 8) represents the number of weak hypotheses w j among those hypotheses produced before stage i that are right on x. For ....

.... does not appear in the bound) Using this fact and the earlier description of the algorithm, we can straightforwardly show that the algorithm as given has a time bound of O(ns 6 # 12 ) This compares with a bound on the harmonic sieve of O(ns 8 # 18 ) Furthermore, as noted in [24], the # 3 factor that appears in the harmonic sieve can be brought arbitrarily close to # 2 ; with this improvement the bounds improve to approximately O(ns 6 # 8 ) and ( O(ns 8 # 12 ) respectively. While neither bound is a small polynomial, the quantum algorithm is somewhat ....

<F3.797e+05> J. C.<F3.838e+05> Jackson,<F3.808e+05> The Harmonic Sieve: A Novel Application of Fourier Analysis to Machine Learning Theory and<F3.838e+05> Practice, Ph.D. thesis, Carnegie Mellon University, Pittsburgh, 1995. Available as Technical report CMU-CS-95-183.

No context found.

J. Jackson. The Harmonic Sieve: A Novel Application of Fourier Analysis to Machine Learning Theory and Practice. PhD thesis, School of Computer Science, Carnegie Mellon University, Pittsburgh, PA, 1995. NDFT Resampling/DSS

Online articles have much greater impact   More about CiteSeer.IST   Add search form to your site   Submit documents   Feedback  

CiteSeer.IST - Copyright Penn State and NEC