Recursive neural models, which use syntactic parse trees to recursively generate representations bottom-up from parse children, are a popular new architecture, promising to capture structural properties like long-distance semantic dependencies. But understanding exactly which tasks this parse-based method is appropriate for remains an open question. In this paper we benchmark {\bf recursive} neural models against sequential {\bf recurrent} neural models, which are structured solely on word sequences. We investigate 4 tasks: (1) sentence-level sentiment classification; (2) matching questions to answer-phrases; (3) discourse parsing; (4) computing semantic relations (e.g., {\em component-whole} between nouns). We implement apply basic, general versions of recursive and recurrent models and apply to each task. Our analysis suggests that syntactic tree-based recursive models are very helpful for tasks that require representing long-distance relations between words (e.g., semantic relations between nominals), but may not be helpful in other situations, where sequence based recurrent models can produce equal performance. Our results offer insights on the design of neural architectures for representation learning.
from cs.AI updates on arXiv.org http://ift.tt/1B4ICML
via IFTTT
No comments:
Post a Comment