Theory – Page 4

Two sample problem(1): Parzen Windows, Maximum Mean Discrepancy

There is a nice tutorial from Alex. I expanded the math part to show you more details. I used latex then posted screenshots.

NLP 05: From Word2vec to Doc2vec: a simple example with Gensim

Introduction First introduced by Mikolov 1 in 2013, the word2vec is to learn distributed representations (word embeddings) when applying neural network. It is based on the distributed hypothesis that words occur in similar contexts (neighboring words) tend to have similar meanings. Two models here: cbow ( continuous bag of words) where we use aContinue reading “NLP 05: From Word2vec to Doc2vec: a simple example with Gensim”

Deep Learning 12: Energy-Based Learning (2)–Regularization & Loss Functions

First, let’s see what is regularization from a simple example. Then we will have a look at some different types of loss functions. Regularization Reviewed the definition of regularization today from Andrew’s lecture videos.

Deep Learning 11: Energy-Based Learning (1)–What is EBL?

As a part of our goals, it is absolutely important to look back and think about the loss functions we applied, for example, the cross entropy. There are other types, however, targeting on different practical problems and you will need to think about which one is suitable. Besides, the Energy-Based Models (EBMs) provides more. TheseContinue reading “Deep Learning 11: Energy-Based Learning (1)–What is EBL?”

TensorFlow 06: Word Embeddings (1)

(A new learner! Leave comments if you find any mistakes please!) How do we present a word? In TensorFlow, everything, yes everything, flows into the graph, is a tensor. For example, the famous word2vec model is used for learning vector representations of words.

NLP 04: Log-Linear Models for Tagging Task (Python)

We will focus on POS tagging in this blog. Notations While HMM gives us a joint probability on tags and words: . Tags t and words w are one-to-one mapping, so in the series, they share the same length.

TensorFlow 05: Understanding Basic Usage

Until recently, I realized I missed some basics about TF. I went directly to the MNIST when I learned. Also, I asked few people if they have some nice tutorials for TF or for DL. Well, it is not like other modules, where you can easily find good ones like Andrew’s ML. But I didContinue reading “TensorFlow 05: Understanding Basic Usage”

NLP 01: Language Modeling Problems

Lecture notes from Natural Language Processing (by Michael Collins)