Citation Context

...eaky rectified linear function (Maas et al., 2013): f(x) = max(x, 0) + 0.01min(x, 0). Other strong tree-structured models have been proposed in past work (Socher et al., 2014; Irsoy and Cardie, 2014; =-=Tai et al., 2015-=-), but we believe that these two provide a valuable case study, and that positive results on here are likely to generalize well to stronger models. To run the model forward, we assemble the two tree-s...

Citation Context

...pped if the norm of the parameter 4 Method r ρ MSE Illinois-LH [18] 0.7993 0.7538 0.3692 UNAL-NLP [19] 0.8070 0.7489 0.3550 Meaning Factory [20] 0.8268 0.7721 0.3224 ECNU [21] 0.8414 – – Mean vectors =-=[22]-=- 0.7577 0.6738 0.4557 DT-RNN [23] 0.7923 0.7319 0.3822 SDT-RNN [23] 0.7900 0.7304 0.3848 LSTM [22] 0.8528 0.7911 0.2831 Bidirectional LSTM [22] 0.8567 0.7966 0.2736 Dependency Tree-LSTM [22] 0.8676 0....

Citation Context

...t depend on external dependency or constituency parse results. One could also use tree-based composition method such as Recursive Neural Tensor Network (Socher et al., 2013b) or Tree-Structured LSTM (=-=Tai et al., 2015-=-; Zhu et al., 2015) as alternatives. Specifically, we try CNN with multiple convolutional filters of different widths (Tang et al., 2015) to produce sentence representation. Figure 2 displays the meth...

Citation Context

...es. We show the proposed S-LSTM model does utilize the structures and achieve results better than those ignoring such priori structures. In addition, in parallel to our efforts, independent research (=-=Tai et al., 2015-=-; Le & Zuidema, 2015) has been conducted to explore ideas close to ours. We refer the readers to these coming interesting works as well. 3. The Model Model brief In this paper, we extend LSTM to struc...

Citation Context

...acy and training cost. Our model achieves 51.2% accuracy on finegrained classification and 88.6% on binary classification, outperforming the best published numbers obtained by a deep recursive model (=-=Tai et al., 2015-=-) and a convolutional model (Kim, 2014). On the Chinese news categorization task, our model achieves 80.0% accuracy, while the closest baseline achieves 79.2%. 2 Related Work Deep neural networks have...

Citation Context

...notations (pairwise labels between spans of text, such as condition in the above example) can be straightforwardly incorporated into the RNN framework presented above: an event’s representation can be augmented to indicate that it bears a certain labeled discourse relation with another event, similar to how coreference information between entities was encoded in section 4. Second, the deeper tree-structured discourse structures output by an RST parser can be used to define the topology of a neural net. That is, instead of a chain-structured LSTM modeling the discourse, a tree-structured LSTM (Tai et al., 2015; Zhu et al., 2015) could be used, with the network’s tree structure dictated by the RST parser’s output. This is similar to the approach used by Bhatia et al. (2015) for sentiment analysis, but they use a simpler RNN and propagated a single scalar latent variable, instead of a vector-valued latent state. 5.3.2 Incorporating Learned Discourse Structure Off-the-shelf discourse parsers capture one notion of discourse structure, namely, the discourse structure which the designers of the treebank annotation conventions decided was generally important, without a particular end task in mind. However...

Citation Context

...ons between internal parts of text. We can use RST relations to parse a text. On the basis of the fact, we propose RST-LSTM model which conduct LSTM on the RST parse tree. RST-LSTM uses the forget components of segments to automatically enhance the nucleus information and filter the satellite information, so semantic feature representations of text will be more sufficient and more accurate. This model is trained on Stanford Sentiment Treebank dataset Socher et al. (2013). Experimental results show that this method not only has a higher classification accuracy than the state-of-the-art methods Tai et al. (2015); Socher et al. (2013, 2012, 2011b), but also trains quickly. 2. Related Works 2.1. Sentiment Analysis The improvement of sentiment analysis tasks is mainly at the step of text feature learning. Recently, neural network methods to learn continuous text representations have been widely used because of their high performance. These methods can be divided into two directions. One is devoted to the research of word embedding. High quality word embedding is the basis of natural language processing task. Mikolov et al. (2013b) first discovers that RNN can learn the analogy relationship between words...

Citation Context

... gain in performance. Recursive neural networks (RecNNs) are syntactic functions that rely on natural language’s inherent structure to achieve state-of-the-art accuracies on sentiment analysis tasks (=-=Tai et al., 2015-=-). As in NBOW, each word type has an associated embedding. However, the composition function g now depends on a parse tree of the input sequence. The representation for any internal node in a binary p...

Citation Context

...et al., 2011; Mikolov et al., 2013; Pennington et al., 2014), and these have been used effectively for many downstream tasks (Turian et al., 2010; Socher et al., 2011; Kim, 2014; Bansal et al., 2014; =-=Tai et al., 2015-=-). There has also been recent work on creating representations for word sequences such as phrases or sentences. Many functional architectures have been proposed to model compositionality in language, ...