Papers

Batch Normalization: Accelerating Deep Network Training by Reducing Internal Covariate Shift

• Original Paper introducing Batch Normalization

Lectures

How does Batch Normalization help Optimization?

• Lecture by Andrew Ilyas

Videos

Batch Normalization, 2, 3, 4

• A sequence of videos by Andrew Ng explaining batch normalization in depth

Batch Normalization Walkthrough

• A walkthrough of the Batch Norm paper by Yannic Kilcher

Batch Normalization Explained

• A visual, beginner friendly introduction to Batch Norm with Tensorflow code by Deep Lizard

Batch Norm in Pytorch

• Video of using Batch Norm in a pytorch neural network by Deep Lizard

Posts

Batch Normalization Explained | Papers with Code

• A summary of Batch Norm, along with a link to code and the paper

Code Examples

Batch Norm example on MNIST dataset using numpy another option: here

• Numpy implementation of neural net with batch norm

Batch Norm in Pytorch

• Example of using Batch Norm in a pytorch neural network by Deep Lizard

Textbook Sections

Dive into Deep Learning Chapter 7.5 - Batch Normalization

• Explanation of Batch Norm with mxnet, pytorch, and tensorflow implementations

Deep Learning Chapter 8.7.1 Batch Normalization (Page 313)

• Section on Batch Norm from Ian Goodfellow’s Textbook

APIs

Pytorch:

• Source Code for torch.nn.modules.batchnorm

• BatchNorm1d

• BatchNorm2d

Tensorflow:

• tf.keras.layers.BatchNormalization

• tf.nn.batch_normalization

• tf.compat.v1.layers.batch_normalization