Deep learning references
This section presents a timeline with important scientific papers for deep learning.
The paper date is when it was first published.
| 2018 | - | BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. Google NLP model. [webpage] |
| 2018 | - | Deep contextualized word representations. ELMo, NLP. [webpage] |
| 2017 | - | Attention Is All You Need. Introduces the Transformer network architecture. [webpage] |
| 2016 | - | YOLO9000: Better, Faster, Stronger. YOLOv2, faster (and hotter) CNN for object detection. [webpage] |
| 2016 | - | DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution and Fully Connected CRFs. DeepLab, CNN for semantic segmentation with atrous convolution. |
| 2016 | - | Identity Mappings in Deep Residual Networks. ResNet v2. |
| 2016 | - | Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning. Google classification network. Inception v4 (Inception-ResNet). |
| 2016 | - | Mastering the game of Go with deep neural networks and tree search. The Go board game was one of the greatest challenges in AI. This paper presented a reinforcement learning solution capable of defeating human pro players. |
| 2015 | - | Deep residual learning for image recognition. ResNet, introduced residual connections. |
| 2015 | - | SSD: Single Shot MultiBox Detector Single CNN for object detection. [code] |
| 2015 | - | Rethinking the Inception Architecture for Computer Vision. Google classification network. Inception v2 and v3. |
| 2015 | - | SegNet: A Deep Convolutional Encoder-Decoder Architecture for Image Segmentation. Encoder-decoder image segmentation network. |
| 2015 | - | You Only Look Once: Unified, Real-Time Object Detection. Introduced YOLO, fast (and hot) CNN for object detection. [webpage] |
| 2015 | - | Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Region proposal based solution for object detection in images. |
| 2015 | - | Deep learning. Survey-kind paper from deep learning bosses. |
| 2015 | - | Fast R-CNN. Region proposal based solution for object detection in images. Integrates a region proposal network and a classification network. |
| 2015 | - | Fully Convolutional Networks for Semantic Segmentation. FCN network for image segmentation. Introduced deconvolution(transposed convolution)(or transposed correlation?). |
| 2015 | - | Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift. One of the most important regularization techniques. BN-Inception. |
| 2015 | - | Human-level control through deep reinforcement learning. Deep reinforcement learning to play Atari 2600 games in fancy journal. |
| 2014 | - | Generative Adversarial Nets. Introduced generative adversarial networks. |
| 2014 | - | Sequence to Sequence Learning with Neural Networks. Sequences to sequences map for machine translation. |
| 2014 | - | Going Deeper with Convolutions. Google classification network. Inception v1 (GoogleNet). |
| 2014 | - | Very Deep Convolutional Networks for Large-Scale Image Recognition. VGGNet, introduced factored convolutions. |
| 2014 | - | ImageNet Large Scale Visual Recognition Challenge. ImageNet paper describing the challenge and winner solutions. |
| 2014 | - | Dropout: A Simple Way to Prevent Neural Networks from Overfitting. Regularization technique. |
| 2013 | - | Intriguing properties of neural networks. Introduced adversarial examples and perturbations. |
| 2013 | - | Playing Atari with Deep Reinforcement Learning. Deep reinforcement learning to play Atari 2600 games. Introduced the DQN and experience replay. |
| 2013 | - | Network In Network. NiN, introduced 1x1 convolution. |
| 2013 | - | Rich feature hierarchies for accurate object detection and semantic segmentation. Region proposal based solution for object detection in images. Introduced the R-CNN. |
| 2012 | - | ImageNet Classification with Deep Convolutional neural networks. This paper presents the famous AlexNet network winner of the 2012 Imagenet challenge (ILSVRC2012). |
| 2012 | - | Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. Review on DNNs for acoustic modeling. |
| 2009 | - | Learning deep architectures for AI. Review on deep architecture models. |
| 1998 | - | Gradient-based learning applied to document recognition. Convolutional neural networks for handwriting recognition. |
| 1997 | - | Long Short-Term Memory. Introduced the long short-term memory (LSTM) to store sequential data. |