{ "cells": [ { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "## M8 - TP2 \n", "\n", "## Objectifs\n", "\n", "Le but de ce TP est de manipuler, analyser et implémenter les descripteurs mono variables vus en cours comme la moyenne, médiane, modalités, effectifs et fréquence. \n", "\n", "\n", "À la fin du TP, il faudra être capable de prendre en main un jeu de données, et de mettre en oeuvre les descripteurs pertinents au regard de la nature de la variable. \n", "\n", "### Compétences associées :\n", "\n", "| Numéro | Compétence | \n", "|:------------- |:--------------------:|\n", "|PY001 |\tÊtre capable de calculer des statistiques basiques en Python |\n", "|PY002 |\tÊtre capable de calculer des statistiques basiques avec numpy/scipy|\n", "|AS001|\tSavoir identifier les caractéristiques d’une variable (qualitative /quantitative, continue/discrete, |domaine, modalités, etc)|\n", "|AS002|\tÊtre capable de retrouver le nombre d’exemples d’un jeu de données et leur dimensions|\n", "|AS003|\tEtre capable de calculer des statistiques basiques (moyenne, fréquence, médian, etc)|\n", "|AS004|\tEtre capable de choisir la bonne statistique par rapport aux caractéristique des données|\n", "|AS005|\tConnaitre les définitions mathématiques des statistiques de base|\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [ "#importation des bibliothèques\n", "import numpy as np\n", "import matplotlib.pyplot as plt" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "## Des vraies données\n", "1. Récupérez le fichier data1.dat sur moodle (les données du TP1 dans l’onglet TP). Copiez le dans le répertoire courant. Chargez les données du fichier data1.dat et vérifiez que vous l’avez bien fait.\n", " > Utilisez la fonction `loadtxt` de numpy" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "outputs": [], "source": [ "arr = np.loadtxt(\"data1\", dtype=float)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "outputs": [], "source": [ "print(arr.shape)" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "2. Quelles sont les dimensions des données ? Combien d'observations $n$ et chaque observation est décrite par combien de variables $p$ ?" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "source": [ "410 oberservations, 46 variables" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "3. Nous allons travailler sur la treizième variable. Il faut donc extraire la variable indicée par 12 du tableau data1 et affectez la à une variable `x`. Quelle est la nature de cette variable ?" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "outputs": [], "source": [ "x = arr[:,12]\n", "x.dtype # float donc continue \n" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "4. Calculez la moyenne de la variable x de deux manières différentes." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "outputs": [], "source": [ "n = x.shape[0]\n", "np.sum(x) / n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "outputs": [], "source": [ "np.mean(x)" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "5. Calculez, à partir des observations de x, $\\hat{\\mathbb{P}}(x ≤ 50)$ et la fréquence empirique d’observation des la modalité $50 : \\hat{\\mathbb{P}}(x = 50)$" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "outputs": [], "source": [ "np.sum(x<50) / n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "outputs": [], "source": [ "np.sum(x==50) / n" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "6. Recherchez des différentes modalités de la variable x" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "outputs": [], "source": [ "Mx = np.unique(x, axis=0)\n", "Mx\n", "Mi,ni = np.unique(x,return_counts=True) # retourne les effectifs par modalité, alternative à la question suivante" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "7. Calculez des effectifs par modalité" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "outputs": [], "source": [ "ni = np.zeros(Mx.shape)\n", "for i in range(len(Mx)):\n", " ni[i] = np.sum((x==Mx[i]))\n", "ni\n" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "8. Calculez des effectifs cumulés de chaque modalité, les fréquences et les fréquences cumulées." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "outputs": [], "source": [ "Ni = np.cumsum(ni) #Les orienter vers la fonction cumsum\n", "fi = ni/n\n", "Fi = (Ni/n)" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "9. Affichez les résultats de vos calculs dans un tableau présentant les modalités, leurs effectifs, effectifs cumulés, puis les fréquences et les fréquences cumulées." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "outputs": [], "source": [ "for m,n,N,f,F in zip(Mi,ni,Ni,fi,Fi): # N'hésitez pas à insister sur zip : bien utile\n", " print(m,n,N,f,F)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "outputs": [], "source": [ "from tabulate import tabulate # Si tabulate est dispo, sinon tant pis\n", "data = np.vstack([Mx, ni, Ni, fi, Fi ]).T #Ils ont besoin d'aide ici ! Sinon, un simple affichage est ok\n", "table = tabulate(data,headers=['Mx','ni','Ni','fi','Fi'])\n", "\n", "print(table)" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "10. Affichez la fonction de répartition empirique de la variable x." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "outputs": [], "source": [ "#Cas discret étant donné que les modalités sont dénombrables\n", "plt.grid(True)\n", "plt.plot(Mx, Fi, 'o-')" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "11. Calculez la médiane de deux manières différentes : \n", " - En utilisant la fonction NumPy et \n", " - Graphiquement en utilisant la fonction de répartition empirique que vous venez de dessiner.\n", " " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "outputs": [], "source": [ "print(f\"V1 : {np.median(x)}\")\n", "# A la mano\n", "x_sorted = np.sort(x)\n", "N = x.shape[0]\n", "if (N % 2): #C'est impair :\n", " median = x_sorted[(N-1)//2]\n", "else: #C'est pair\n", " median = (x_sorted[N//2] + x_sorted[(N//2)-1])/2\n", "\n", "print(f\"V2 : {median}\")" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "source": [ "**Version 2** : Numpy utilise la definition vu en cours, cf la doc (https://numpy.org/doc/stable/reference/generated/numpy.median.html). Par contre, en lisant dans le graphique, ou le tableau, on peut estimer entre 16,5 et 17 la valeur qui dépasse 0.5. Mais attention, c'est une estimation interpolée sur les données. La définition de la médiane est prise dans l'échantillon." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "12. Visualisez les fréquences observées de trois manières différentes. A l’aide d’un histogramme standard à 10 bins, un histogramme détaillé à 50 bins, et en affichant les effectifs pour chacune des modalités. La quelle de ces trois vue vous parait la plus intéressante ?" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "outputs": [], "source": [ "plt.subplot(3,1,1)\n", "plt.hist(x)\n", "plt.subplot(3,1,2)\n", "plt.hist(x,bins=3)\n", "plt.subplot(3,1,3)\n", "plt.stem(Mx,ni)\n", "# Le premier histogramme me semble meilleur pour avoir une vue globale" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "outputs": [], "source": [ "plt.hist(x,bins=None)" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "## Correction d'examen\n", "Un professeur souhaite étudier statistiquement les notes sur 10 obtenues par ses élèves à un examen. Les résultats sont donnés dans le tableau ci-dessous.\n", "\\begin{array}{c|c}\n", "Nom & Note \\\\\n", " \\hline\n", "mike &6\\\\\n", "bob &5\\\\\n", "evan &9\\\\\n", "ruppert &3\\\\\n", "aaron &3\\\\\n", "steve &1\\\\\n", "zachariy &9\\\\\n", "logan &6\\\\\n", "roger &5\\\\\n", "don &6\\\\\n", "bill &5\\\\\n", "ron &6\\\\\n", "karl &5\\\\\n", "stefen &7\\\\\n", "tom &5\\\\\n", "neil &10\\\\\n", "john &2\\\\\n", "george &1\\\\\n", "dave &1\\\\\n", "tim &1\n", "\\end{array}" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "1. Quel est le domaine de la variable ?\n", "2. Quelles sont les différentes modalités observées ? \n", "3. Quelle la nature de la variable ?" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [ "#Création du tableau\n", "x = np.array([6, 5, 9, 3, 3, 1, 9, 6, 5, 6, 5, 6, 5, 7, 5, 10, 2, 1, 1, 1])\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "outputs": [], "source": [ "# le domaine est { 0 ... 10}\n", "#Calcul des modalités \n", "Mi = np.unique(x) # Modalités\n", "Mi" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "la variable est quantitative discrete" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "2. Le code suivant calcule les effectifs $n_i$ et les effectifs cumulés $N_i$ de chaque modalité. Calculez les effectifs cumulés d'une autre manière et vérifier que vous obtenez les mêmes résultats" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [ "Mi, ni = np.unique(x, return_counts=True)\n", "Ni = np.zeros(ni.shape)\n", "Ni[0] = ni[0]\n", "for i in range(0, len(Ni) - 1): \n", " Ni[i+1] = Ni[i] + ni[i+1]\n", "print(Ni)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "outputs": [], "source": [ "#calcul des effectifs cumulés en 1 ligne\n", "Ni = np.cumsum(ni)\n", "Ni" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "3. Calculez les effectifs par modalité, les fréquences et les fréquences cumulées des notes et affichez-les" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "outputs": [], "source": [ "fi = ni/len(x) # Fréquences\n", "Fi = Ni/len(x) # Fréquences cumulées\n", "\n", "from tabulate import tabulate\n", "data = np.vstack([Mi, ni, Ni, fi, Fi ]).T #besoin d'aide ici ! \n", "table = tabulate(data,headers=['Mi','ni','Ni','fi','Fi'])\n", "\n", "print(table)\n" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "4. Tracez la fonction de répartition empirique des notes." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "outputs": [], "source": [ "#Repartition empirique <=> Fréquences cumulées\n", "plt.grid(True)\n", "plt.plot(Mi, Fi, 'o-')" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "5. Déterminez la médiane et les quartiles des notes" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "outputs": [], "source": [ "quants = np.quantile(x, [0.25,0.5,0.75])" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "6. En déduire le DIQ" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "outputs": [], "source": [ "#DIQ : Q3 - Q1\n", "quants[-1]-quants[0]" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "source": [ "7. Dessinez la boite à moustache associée sur une feuille.\n", "\n", "Selon les calculs effectués, y a-t-il une variable hors épure parmi ces notes ?" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [] }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "outputs": [], "source": [ "dic = plt.boxplot(x)" ] }, { "cell_type": "markdown", "metadata": { "editable": true, "slideshow": { "slide_type": "" }, "tags": [ "correction" ] }, "source": [ "Non" ] } ], "metadata": { "kernelspec": { "display_name": "iml", "language": "python", "name": "iml" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.10.13" } }, "nbformat": 4, "nbformat_minor": 4 }