distributed representations of words
word2vec is a class of neural network models, trained on unlabelled trainign corpus, that produce a vector for each word in the corpus that encode its valuable syntactic and semantic informaton .
Skip-grams (SG) - predict context words given target (position independent)
Continuous Bag of Words (CBOW) - Predict target word from bag-of-words context
Stochastic Gradient Descent
reference
- Linguistic Regularities in Continuous Space Word Representations,
- The amazing power of word vectors
- Word2vec tutorial
- Deep Learning Basics: Neural Networks, Backpropagation and Stochastic Gradient Descent