In this short post, I will share a very brief GAN (Generative Adversarial Network) model and in practice, how do we train it using PyTorch. Also, I will include some tips about training as I myself found it is hard to train, especially when working with my own data and model.
Training GAN models
I wrote a blog about how to understand GAN models before, check it out. You can also find PyTorch official tutorial here . We will be focusing on the official tutorial and I will try to provide my understanding and tips of the main steps. Continue reading “Deep Learning 18: GANs with PyTorch”
If you are a big fun of PyTorch and NLP, you must try to use the PyTorch based BERT implementation! If you have your own dataset and want to try the state-of-the-art model, BERT is a good choice.
Please check the code from https://github.com/huggingface/pytorch-pretrained-BERT to get a close look. However, in this post, I will help you to apply pre-trained BERT model on your own data to do classification. Continue reading “Deep Learning 17: text classification with BERT using PyTorch”
My post about Auto-encoder.
For Variational Auto-Encoders (VAE) (from paper Auto-Encoding Variational Bayes), we actually add latent variables to the existing Autoencoders. The main idea is, we want to restrict the parameters from a known distribution. Why we want this? We wish the generative model to provide more “creative” things. If the model only sees the trained samples, it will eventually lose the ability to “create” more! So we add some “noises” to the parameters by forcing the parameters to adapt to a known distribution.
Continue reading “Understanding Variational Graph Auto-Encoders”
Corpora for general medical texts
Over 39 million published research papers in Computer Science, Neuroscience, and Biomedical.
Full dataset 36G, not restricted.
Continue reading “Resources for BioNLP: datasets and tools”
Graph Convolution Networks (GCNs)  deal with graphs where the data form with a graph structure. A typical graph is represented as
G(V, E), where
V is the collection of all the nodes and
Eis the collection of all the edges.
Continue reading “Understanding Graph Convolutional Networks”
In this blog post, we introduce our AAAI 2019 accepted paper “What Should I Learn First: Introducing LectureBank for NLP Education and Prerequisite Chain Learning.”
Our LectureBank dataset contains 1,352 English lecture files collected from university courses in mainly Natural Language Processing (NLP) field. Besides, each file is manually classified according to an existing taxonomy. Together with the dataset, we include 208 manually-labeled prerequisite relation topics. The dataset will be useful for educational purposes such as lecture preparation and organization as well as applications such as reading list generation. Additionally, we experiment with neural graph-based networks and non-neural classifiers to learn these prerequisite relations from our dataset.
Continue reading “LectureBank: a dataset for NLP Education and Prerequisite Chain Learning”
ELMo: Deep contextualized word representations
In this blog, I show a demo of how to use pre-trained ELMo embeddings, and how to train your own embeddings.
Continue reading “ELMo in Practice”