🎉 We are pleased to release the new, 2022 version of the AAN database with over 24,000 resources and over 7,000 lecture notes. Visit aan.how! 2022 TutorialBank This update includes 19,765 manually collected resources with valid URLs, meta-data, organized by topic. We also release an extra batch of 5,001 resources. These resources have valid URLs butContinue reading “AAN.how: All About NLP —— New 2022 Release: over 24,000 resources and over 7,000 lecture notes!”
Category Archives: Theory
Deep Learning 19: Training MLM on any pre-trained BERT models
MLM, masked language modeling, is an important task for trianing a BERT model. In the orignal BERT paper: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding, it is one of the main tasks of how BERT was pre-trained. So if you have your own corpus, it is possible to train MLM on any pre-trainedContinue reading “Deep Learning 19: Training MLM on any pre-trained BERT models”
Slideshare (11): Graph Neural Networks and Applications in NLP
Check out my class talk slides about Graph Neural Networks and their applications in NLP! Covered materials: Semi-Supervised Classification with Graph Convolutional Networks Variational Graph Auto-Encoders Graph Attention Networks Graph Convolutional Networks for Text Classification (AAAI 2019) Heterogeneous Graph Neural Networks for Extractive Document Summarization (ACL 2020) A Graph-based Coarse-to-fine Method for Unsupervised Bilingual LexiconContinue reading “Slideshare (11): Graph Neural Networks and Applications in NLP”
Slideshare (10): Reading notes bout Text Generation, BERT, Knowledge Distillation…
Please check my notes for the Spring Semester 2020: Topics covered: Text Generation BERT understanding Knowledge Distillation NLP Applications EHR+Translation
Paddle Serving: model-as-a-service! Triggered by a single command line, deployment finishes in 10 minutes
To bridge the gap between Paddle Serving and PaddlePaddle framework, we release the new service of PaddleServing: Model As A Service (MAAS) online in Github. With the help of the new service, when a PaddlePaddle model is trained, users now can obtain the corresponding inference model at the same time, making it possible to deployContinue reading “Paddle Serving: model-as-a-service! Triggered by a single command line, deployment finishes in 10 minutes”
Slideshare (9): Transfer learning tutorial 2.0
Please check my notes for an updated version about Transfer Learning with NLP tasks: Irene_TransferLearning_2020Spring A brief outline: Transfer Learning with word embedding Pre-BERT times BERT and its variants Understanding, reducing BERT Transfer Learning in the real world Also, visit my A brief Introduction on Transfer Learning notes from 2019 Spring.
Slideshare (8): Graph Neural Network Paper reading notes
The pdf contains some notes on several papers, some of which only have 1-2 slide pages. Again, red fonts are my thoughts and insights. I wish my notes can help readers to better understand the new concepts and get inspired. Click to view: GNN_notes_Fall2019.pdf
Prepare for the Interviews!
For the past few years after my Master’s, I did many jobs, long term, short term, internship, or full-time. I also had too many interviews, some of them I failed. Together with my friends, we had collected many materials, including basic algorithms, popular questions, basic machine learning knowledge, and deep learning knowledge. Then I organizedContinue reading “Prepare for the Interviews!”
Understanding Variational Graph Auto-Encoders
Variational Auto-Encoders My post about Auto-encoder. For Variational Auto-Encoders (VAE) (from paper Auto-Encoding Variational Bayes), we actually add latent variables to the existing Autoencoders. The main idea is, we want to restrict the parameters from a known distribution. Why we want this? We wish the generative model to provide more “creative” things. If the modelContinue reading “Understanding Variational Graph Auto-Encoders”
Understanding Graph Convolutional Networks
Why Graphs? Graph Convolution Networks (GCNs) [0] deal with graphs where the data form with a graph structure. A typical graph is represented as G(V, E), where V is the collection of all the nodes and Eis the collection of all the edges.
ireneli.eu 