{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Convolutional Neural Networks on MNIST\n",
    "\n",
    "\n",
    "We will use the same MNIST dataset but each example is reshaped  as a `(28,28,1)` array, in order to be fed to a convolutional layer.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data File found... loading it into memory\n",
      "Data File loaded\n"
     ]
    }
   ],
   "source": [
    "import numpy as np\n",
    "import tensorflow as tf\n",
    "from tensorflow import keras\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "import datetime as dt\n",
    "\n",
    "from pathlib import Path\n",
    "\n",
    "from sklearn.datasets import fetch_openml\n",
    "from sklearn import preprocessing\n",
    "\n",
    "data_home = '/tmp/scikit_learn_data/'\n",
    "datafile = '/tmp/mnist.npz'\n",
    "\n",
    "datapath = Path(datafile)\n",
    "if not(datapath.exists()):\n",
    "    print(\"Data File not found... downloading it\")\n",
    "    Xmnist, ymnist = fetch_openml('mnist_784',\n",
    "                                  version=1,\n",
    "                                  return_X_y=True,\n",
    "                                  data_home=data_home)\n",
    "    np.savez(datapath.as_posix(),\n",
    "             X=np.array(Xmnist, dtype='u8'),\n",
    "             y=np.array(ymnist, dtype='u8'))\n",
    "    print(\"Data File downloaded and saved\")\n",
    "    del Xmnist, ymnist\n",
    "\n",
    "print(\"Data File found... loading it into memory\")\n",
    "data = np.load(datapath.as_posix())\n",
    "Xmnist = data['X']/255.\n",
    "ymnist = keras.utils.to_categorical(data['y'])\n",
    "print(\"Data File loaded\")\n",
    "\n",
    "Xtrain, Ytrain = Xmnist[:50000], ymnist[:50000]\n",
    "Xvalid, Yvalid = Xmnist[50000:60000], ymnist[50000:60000]\n",
    "Xtest, Ytest = Xmnist[-10000:], ymnist[-10000:]\n",
    "\n",
    "Xtrain = Xtrain.reshape((Xtrain.shape[0], 28, 28, 1))\n",
    "Xvalid = Xvalid.reshape((Xvalid.shape[0], 28, 28, 1))\n",
    "Xtest = Xtest.reshape((Xtest.shape[0], 28, 28, 1))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Exercises\n",
    "\n",
    "Let's build a 5 layer MLP with:\n",
    "\n",
    "- 3 convolutional blocks with a number of features of respectively 16 32 64, each block containing:\n",
    "    - Batch Normalizaton at input,\n",
    "    - Conv layer\n",
    "    - Leaky Relu activation \n",
    "    - Max Pooling (2,2 pool size)\n",
    "\n",
    "- 2 dense layers with 28 hidden units.\n",
    "\n",
    "\n",
    "1) What operation should be added in between convolutional and dense layers ?\n",
    "\n",
    "2) Let's try to add dropout and regularization on dense layer.\n",
    "\n",
    "## MNIST Competition\n",
    "\n",
    "Do the best you can on the MNIST dataset !"
   ]
  }
 ],
 "metadata": {
  "celltoolbar": "Tags",
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}