本节介绍最基本的神经网络。

Word embedding, i.e., vector representations of a particular word and also called word vectoring, is important in NLP. It is capable of capturing context of a word in a document, semantic and syntactic similarity, relation with other words, etc.

Word2Vec, as one of the most popular technique to learn word embeddings using shallow neural network, includes:

• 2 algorithms: continuous bag-of-words (CBOW) and skip-gram. CBOW aims to predict a center word from the surrounding context in terms of word vectors. Skip-gram does the opposite, and predicts the distribution (probability) of context words from a center word.
• 2 training methods: negative sampling and hierarchical softmax. Negative sampling defines an objective by sampling negative examples, while hierarchical softmax defines an objective using an efficient tree structure to compute probabilities for all the vocabulary.

The overall goal is to learn the following stuff:

• CS231N-Convolutional Neural Networks for Visual Recognition
• CS224n: Natural Language Processing with Deep Learning
• Linear Algebra ref resources
• Probability and Statistics ref
• Probabilistic Graphical Models Specialization ref cmu
• Statistics for Applications ref
• Convex Optimization website video textbook

Week 1 (28 May - 2 June)

The goal is ::

1. CS231N : Lecture2 - Image Classification
2. CS224n : Introduction and Word Vectors

Week 2 (3 June - 9 June)

1. CS231N : Lecture3 - Loss Functions and Optimization
2. CS224n : Word Vectors 2 and Word Senses

Week 3 (10 June - 16 June)

The goal is ::

1. CS231N : Lecture4 - Introduction to Neural Networks
2. CS224n : Word Window Classification, Neural Networks, and Matrix Calculus

Week 4 (17 June - 23 June)

The goal is ::

1. CS231N : Lecture5 - Convolutional Neural Networks
2. CS224n : Backpropagation and Computation Graphs

Week 5 (24 June - 30 June)

The goal is ::

1. CS231N : Lecture6 - Training Neural Networks, part I
2. CS224n : The probability of a sentence? Recurrent Neural Networks and Language Models

Week 6 (1 July - 7 July)

The goal is ::

1. CS231N : Lecture7 - Training Neural Networks, part II
2. CS224n : Vanishing Gradients and Fancy RNNs

Week 7 (8 July - 14 July)

The goal is ::

1. CS231N : Lecture9 - CNN Architectures
2. CS224n : Machine Translation, Seq2Seq and Attention

Week 8 (15 July - 21 July)

The goal is ::

1. CS231N : Lecture10 - Recurrent Neural Networks
2. CS224n : Question Answering and the Default Final Project

Week 9 (22 July - 28 July)

The goal is ::

1. CS231N : Lecture11 - Detection and Segmentation
2. CS224n : ConvNets for NLP

Week 10 (29 July - 4 Aug)

The goal is ::

1. CS231N : Lecture12 - Visualizing and Understanding
2. CS224n : Information from parts of words: Subword Models

Week 11 (5 Aug - 11 Aug)

The goal is ::

1. CS231N : Lecture13 - Generative Models
2. CS224n : Modeling contexts of use: Contextual Representations and Pretraining

Week 12 (12 Aug - 18 Aug)

The goal is ::

1. ​
2. CS224n: Lecture 14 Transformers and Self-Attention

Week 13 (19 Aug - 15 Aug)

The goal is ::

1. ​
2. CS224n: Lecture 15 Natural Language Generation

Week 14

The goal is ::

1. ​
2. CS224n: Lecture 16 – Coreference Resolution

Week 15

The goal is ::

1. ​
2. CS224n: Lecture 18 – Constituency Parsing, TreeRNNs

Week 16

The goal is ::

1. ​
2. ​

Tutorials of PyTorch and some useful tips.

A high-level plotting library built on top of matplotlab. Seaborn helps resolve the two major problems faced by Matplotlib; the problems are：

• Default Matplotlib parameters
• Working with data frames

It is noted that GAN training is hard and unstable, which results in blury images. In this post, a several techniques are introduced to improve the training stability of GAN.

西班牙语学习！

This post is based on the paper Fully Convolutional Networks for Semantic Segmentation, which aims to perform image segmentation.

Attention in DP and NLP.

Some posts about tensorflow tutorials.