Softmax alternative for NLP
For large dictionary (>50k words) softmax is often the bottleneck in NLP.

See [Approximating the Softmax](http://ruder.io/word-embeddings-softmax/) or the paper [Breaking the Softmax Bottleneck](https://arxiv.org/abs/1711.03953) and the presentation[Softmax alternatives in Neural Machine Translation](http://www.phontron.com/slides/neubig17mtma.pdf)

Keywords:

- Hierarchical Softmax
- Differentiated Softmax
- CNN-Softmax
- Binary Code Prediction

- Vocabulary Selection

- (Adaptative) Importance Sampling
- Target Sampling
- Noise Contrastive Estimation
- Negative Sampling
- Self-Normalisation
- Infrequent Normalisation

Given the wealth of techniques it's probably best to start with something implemented in a production-ready library like Facebook's [FastText](https://github.com/facebookresearch/fastText), [spaCy](https://github.com/explosion/spaCy) and [Thinc](https://github.com/explosion/thinc), [Rasa NLU](https://rasa.com/docs/nlu/) or [MITIE](https://github.com/mit-nlp/MITIE).