Why BERT If you are a big fun of PyTorch and NLP, you must try to use the PyTorch based BERT implementation! If you have your own dataset and want to try the state-of-the-art model, BERT is a good choice. Please check the code from https://github.com/huggingface/pytorch-pretrained-BERT to get a close look. However, in this post, […]
Variational Auto-Encoders My post about Auto-encoder. For Variational Auto-Encoders (VAE) (from paper Auto-Encoding Variational Bayes), we actually add latent variables to the existing Autoencoders. The main idea is, we want to restrict the parameters from a known distribution. Why we want this? We wish the generative model to provide more “creative” things. If the model […]
Corpora for general medical texts Open Research Corpus Over 39 million published research papers in Computer Science, Neuroscience, and Biomedical. Full dataset 36G, not restricted.
Why Graphs? Graph Convolution Networks (GCNs)  deal with graphs where the data form with a graph structure. A typical graph is represented as G(V, E), where V is the collection of all the nodes and Eis the collection of all the edges.
Introduction In this blog post, we introduce our AAAI 2019 accepted paper “What Should I Learn First: Introducing LectureBank for NLP Education and Prerequisite Chain Learning.” Our LectureBank dataset contains 1,352 English lecture files collected from university courses in mainly Natural Language Processing (NLP) field. Besides, each file is manually classified according to an existing […]
ELMo: Deep contextualized word representations In this blog, I show a demo of how to use pre-trained ELMo embeddings, and how to train your own embeddings.
Please check my notes for Transfer Learning introduction! Transfer Learning
A brief introduction on unsupervised transfer learning methods. The presentation focused on unsupervised transfer learning methods, introducing feature-based and model-based strategies and few recent papers from ICML, ACL. Unsupervised Transfer Learning Comments are welcomed!
TensorFlow provides save and restore functions for us to save and re-use the model parameters. If you have a trained VGG model, for example, it will be helpful for you to restore the first few layers then apply them in your own networks. This may raise a problem, how do we restore a subset of […]