Cs224n tips nlp research ideas final project ideas (lecture 9)
Research ideas:
Learning Temporal Events In text - what came before, what came after
Look at the ‘residual encodings’ in NMT systems - the encoding remaining in the decoder hidden state after <EOS> is produced. These encodings must capture something inherent in the language pair.
Final Project Ideas:
SQuAD question answering
Doing Research in CS
Two starting points:
-
[Nails] Start with a domain problem of interest, and try to find good/better ways to adress it than are currently known/used
-
[Hammer] Start with a technical appraoch of itnerest, and work out good ways to extend or improve it or new ways to apply it.
Find application/task of interest and explore how to approach it effectively.
Analyze behaviour of a model; implment a complex neral architecutre
come up with a new or vairant NN model
If you see a research area wehre a lot of people are going, go somewhere else.
Example of doing your own research
1. Define Task (eg summarization)
2. Define Dataset (online or your own)
3. Split trian val test
4. Define Metrics
5. Establish a baseline. Can use both exiting baselines as well as simple models (ie logistic regression on unigrams and bigrams, or averaging word vectors, or bag of words as baseline). Compute metrics on train AND dev and plot learning curve.
6. Implement NN model
7. Analyze results (get summary stats, data viz, look at errors and hparams), Iterate
8. Try differnet model variants
paperswithcode.com/sota
ARxiv sanity preserver
github.com/niderhoff/nlp-datasets