Natural Language Processing PyTorch

Deep Learning 17: text classification with BERT using PyTorch

Why BERT If you are a big fun of PyTorch and NLP, you must try to use the PyTorch based BERT implementation! If you have your own dataset and want to try the state-of-the-art model, BERT is a good choice. Please check the code from to get a close look. However, in this post, […]

Algorithm Theory

Understanding Variational Graph Auto-Encoders

Variational Auto-Encoders My post about Auto-encoder. For Variational Auto-Encoders (VAE) (from paper Auto-Encoding Variational Bayes), we actually add latent variables to the existing Autoencoders. The main idea is, we want to restrict the parameters from a known distribution. Why we want this? We wish the generative model to provide more “creative” things. If the model […]

Natural Language Processing Resource

Resources for BioNLP: datasets and tools

Corpora for general medical texts Open Research Corpus Over 39 million published research papers in Computer Science, Neuroscience, and Biomedical. Full dataset 36G, not restricted.

PyTorch Theory

Understanding Graph Convolutional Networks

  Why Graphs? Graph Convolution Networks (GCNs) [0] deal with graphs where the data form with a graph structure. A typical graph is represented as G(V, E), where V is the collection of all the nodes and Eis the collection of all the edges.


LectureBank: a dataset for NLP Education and Prerequisite Chain Learning

Introduction In this blog post, we introduce our AAAI 2019 accepted paper “What Should I Learn First: Introducing LectureBank for NLP Education and Prerequisite Chain Learning.” Our LectureBank dataset contains 1,352 English lecture files collected from university courses in mainly Natural Language Processing (NLP) field. Besides, each file is manually classified according to an existing […]

Deep Learning Python PyTorch

ELMo in Practice

ELMo: Deep contextualized word representations In this blog, I show a demo of how to use pre-trained ELMo embeddings, and how to train your own embeddings.

SlideShare Theory

Slideshare (4): A brief Introduction on Transfer Learning

Please check my notes for Transfer Learning introduction! Transfer Learning


Transfer Learning Materials

Keep Updating

Deep Learning SlideShare Theory

Slideshare (3): Unsupervised Transfer Learning Methods

A brief introduction on unsupervised transfer learning methods. The presentation focused on unsupervised transfer learning methods, introducing feature-based and model-based strategies and few recent papers from ICML, ACL. Unsupervised Transfer Learning Comments are welcomed!

Python Theory

TensorFlow 08: save and restore a subset of variables

TensorFlow provides save and restore functions for us to save and re-use the model parameters. If you have a trained VGG model, for example, it will be helpful for you to restore the first few layers then apply them in your own networks. This may raise a problem, how do we restore a subset of […]