# Unsupervised Deep Learning in Python

#### Description

Autoencoders, Restricted Boltzmann Machines, Deep Neural Networks, t-SNE and PCA

This course is the next logical step in my **deep learning, data science,** and **machine learning** series. Iâ€™ve done a lot of courses about deep learning, and I just released a course about **unsupervised learning**, where I talked about **clustering** and **density estimation**. So what do you get when you put these 2 together? Unsupervised deep learning!

In these course weâ€™ll start with some very basic stuff – **principal components analysis (PCA)**, and a popular nonlinear dimensionality reduction technique known as **t-SNE (t-distributed stochastic neighbor embedding)**.

Next, weâ€™ll look at a special type of unsupervised neural network called the **autoencoder**. After describing how an autoencoder works, Iâ€™ll show you how you can link a bunch of them together to form a deep stack of autoencoders, that leads to better performance of a supervised** deep neural network**. Autoencoders are like a non-linear form of PCA.

Last, weâ€™ll look at **restricted Boltzmann machines (RBMs)**. These are yet another popular unsupervised neural network, that you can use in the same way as autoencoders to **pretrain** your supervised deep neural network. Iâ€™ll show you an interesting way of training restricted Boltzmann machines, known as **Gibbs sampling**, a special case of **Markov Chain Monte Carlo,** and Iâ€™ll demonstrate how even though this method is only a rough approximation, it still ends up reducing other cost functions, such as the one used for autoencoders. This method is also known as **Contrastive Divergence** or **CD-k**. As in physical systems, we define a concept called **free energy** and attempt to minimize this quantity.

Finally, weâ€™ll bring all these concepts together and Iâ€™ll show you visually what happens when you use PCA and t-SNE on the features that the autoencoders and RBMs have learned, and weâ€™ll see that even without labels the results suggest that a pattern has been found.

All the materials used in this course are FREE. Since this course is the 4th in the deep learning series, I will assume you already know calculus, linear algebra, and **Python** coding. You’ll want to install **Numpy,** **Theano, and Tensorflow** for this course. These are essential items in your **data analytics** toolbox.

If you are interested in deep learning and you want to learn about modern deep learning developments beyond just plain **backpropagation**, including using unsupervised neural networks to interpret what features can be automatically and hierarchically learned in a deep learning system, this course is for you.

This course focuses on “**how to build and understand**“, not just “how to use”. Anyone can learn to use an API in 15 minutes after reading some documentation. It’s not about “remembering facts”, it’s about **“seeing for yourself” via experimentation**. It will teach you how to visualize what’s happening in the model internally. If you want **more** than just a superficial look at machine learning models, this course is for you.

“If you can’t implement it, you don’t understand it”

- Or as the great physicist Richard Feynman said: “What I cannot create, I do not understand”.
- My courses are the ONLY courses where you will learn how to implement machine learning algorithms from scratch
- Other courses will teach you how to plug in your data into a library, but do you really need help with 3 lines of code?
- After doing the same thing with 10 datasets, you realize you didn’t learn 10 things. You learned 1 thing, and just repeated the same 3 lines of code 10 times…

Suggested Prerequisites:

- calculus
- linear algebra
- probability
- Python coding: if/else, loops, lists, dicts, sets
- Numpy coding: matrix and vector operations, loading a CSV file
- can write a feedforward neural network in Theano or Tensorflow

WHAT ORDER SHOULD I TAKE YOUR COURSES IN?:

- Check out the lecture “Machine Learning and AI Prerequisite Roadmap” (available in the FAQ of any of my courses, including the free Numpy course)

**Benefits of the course :**

- Understand the theory behind principal components analysis (PCA)
- Know why PCA is useful for dimensionality reduction, visualization, de-correlation, and denoising
- Derive the PCA algorithm by hand
- Write the code for PCA
- Understand the theory behind t-SNE
- Use t-SNE in code
- Understand the limitations of PCA and t-SNE
- Understand the theory behind autoencoders
- Write an autoencoder in Theano and Tensorflow
- Understand how stacked autoencoders are used in deep learning
- Write a stacked denoising autoencoder in Theano and Tensorflow
- Understand the theory behind restricted Boltzmann machines (RBMs)
- Understand why RBMs are hard to train
- Understand the contrastive divergence algorithm to train RBMs
- Write your own RBM and deep belief network (DBN) in Theano and Tensorflow
- Visualize and interpret the features learned by autoencoders and RBMs

**Requirements :**

- Knowledge of calculus and linear algebra
- Python coding skills
- Some experience with Numpy, Theano, and Tensorflow
- Know how gradient descent is used to train machine learning models
- Install Python, Numpy, and Theano
- Some probability and statistics knowledge
- Code a feedforward neural network in Theano or Tensorflow

**Who this course is for :**

- Students and professionals looking to enhance their deep learning repertoire
- Students and professionals who want to improve the training capabilities of deep neural networks
- Students and professionals who want to learn about the more modern developments in deep learning