Redid figs

Report/MLPproject.tex

@@ -105,24 +105,18 @@ When performing the hyperparameter tuning, we started out with a rough grid to g
 \subsection{Caveats and restrictions}
 Although the validation results produced from the script are quite promising there are a couple of important notes to make, not only to better understand the final models but also to avoid pitfalls in potential future projects. Firstly, in our script we decided to not use any standardization as this is a sort of unique case where the models used do not require it. However, it's extremely important to understand that if we were to introduce another model, we would need to standardize the data to ensure that the features contribute equally. Secondly, there are more hyperparameters that one might want to consider as we only used a few of them. The problem with expanding the number of hyperparameters in the grid is that it will exponentially increase the computational load. Therefore we picked a few that we thought were most important. Continuing, the scoring metric used is not always the best choice. We used accuracy, meaning the model tries to correctly label as many datapoints as possible and does not care about keeping a similiar precision for both labels. Our goal of this project is somewhat arbitrary, we mainly want to train and compare models. However if such a model were to be used in a real world application, one might want to change the scoring to better adapt the model to the problem at hand. % Elaborate... Secondly, there are more hyperparameters that one might want to consider... Continuing, the scoring metric used is not always the best choice. In fact, the scoring metric one should use is highly dependent on what one's goal is... 
 
-
-\section{Model Evaluations}
-There are two interesting parts to look at after our analysis. One part is to analyze how well the actual models performed and compare the difference between the two models we have chosen to study. We fine tuned our models using the validation part of the data. After running it on the test data we can see how well it actually performs. A great way to get a quick overview of how well a model classifies is to look at the confusion matrix.
-\subsection{Analyzing the Confusion Matricies}
-
-
-\begin{figure}[!hptb]
+\begin{figure}[!h]
  \centering
  \begin{subfigure}[b]{\columnwidth}
      \centering
-     \includegraphics[width=\textwidth]{CM_dt.png}
+     \includegraphics[width=0.95\textwidth]{CM_dt.png}
      \caption{}
      \label{fig:featureImportanceDT}
  \end{subfigure}
  \hfill
  \begin{subfigure}[b]{\columnwidth}
      \centering
-     \includegraphics[width=\textwidth]{CM_rf.png}
+     \includegraphics[width=0.95\textwidth]{CM_rf.png}
      \caption{}
      \label{fig:featureImportanceRF}
  \end{subfigure}
@@ -130,6 +124,13 @@ There are two interesting parts to look at after our analysis. One part is to an
     \label{fig:}
 \end{figure}
 
+\section{Model Evaluations}
+There are two interesting parts to look at after our analysis. One part is to analyze how well the actual models performed and compare the difference between the two models we have chosen to study. We fine tuned our models using the validation part of the data. After running it on the test data we can see how well it actually performs. A great way to get a quick overview of how well a model classifies is to look at the confusion matrix.
+\subsection{Analyzing the Confusion Matricies}
+
+
+
+
 As we can see in the confusion matricies there is not that big of a difference between the models. Both did an overall good job at identifying the two classes. There is a difference in how well the models did in identifying the two different classes. Overall they performed a lot better at classifying the poor people than the rich. We can see that for the both models are pretty good at classifying the poor class and worse at the rich class. The Random forest model is slightly better than the Decision Tree. This is a very interesting result and maybe not so weird as it first seems. There were a lot more poor people in our training data set than rich people. This would of course train our model to be better at classifying the poor. As well as looking at the classification matricies it is interesting to look at the actual performance metrics that can be calculated from the matricies. 
 \subsection{Analyzing Weighted Performance Metrics}
  We want to analyze to sets of metrics. First we have the validaton Metrics. These metrics can be seen in table(\ref{perfmetric}). Then we have the actual test metrics which is the result from our model. These can be seen in table(\ref{perfmetrictest}). Of note is that all of these metrics are calculated as weighted metrics which means that they account for the class imbalances seen in the confusion matrcies.
@@ -171,20 +172,16 @@ An important thing to touch on is the poor fit on rich people by our model. We s
 \subsection{Overfitting and Underfitting}
 We spent some time tuning the hyperparameters to ensure that we did not overfit. If we compare the validation results with the test results we see that the performance metrics do not change much at all. This is what we want to see as this means that we have avoidede overfitting the model. This means that our model could be used on other similar datasets and hopefully give similar perfomances. We also do not want our model to be underfit. This is a bit harder to validate as we want the errors to be as small as possible for both training and testing and as we stated before I believe that this is a difficult dataaset to get a great fit to. Therefore we believe that we have found a model that has a decent enough balance between bias and variance.
 
-\subsection{Feature Importance}
-Taking a closer look at the feature importance graphs of the two models we notice an interesting difference. The Decision tree which is only one tree focuses has only a few main features where one is the most important. The rest are not used that much or almost not at all. The Random Forest uses a far wider range of features. They also rank the features a bit differently and the best feature for one model is not the best for the other. We considered removing the worst performing features to see if it would make a difference in the performanes. But since they have diffrent worst performing features we reasoned that to keep the comparison as fair as possible it would be more interesting to leave the features as is.
-
-
 \begin{figure}[!hptb]
  \centering
- \begin{subfigure}[b]{0.9\columnwidth}
+ \begin{subfigure}[b]{0.85\columnwidth}
      \centering
      \includegraphics[width=\textwidth]{featureImportanceDT.png}
      \caption{}
      \label{fig:featureImportanceDT}
  \end{subfigure}
  \hfill
- \begin{subfigure}[b]{0.9\columnwidth}
+ \begin{subfigure}[b]{0.85\columnwidth}
      \centering
      \includegraphics[width=\textwidth]{featureImportanceRF.png}
      \caption{}
@@ -194,39 +191,16 @@ Taking a closer look at the feature importance graphs of the two models we notic
     \label{fig:}
 \end{figure}
 
+\subsection{Feature Importance}
+Taking a closer look at the feature importance graphs of the two models we notice an interesting difference. The Decision tree which is only one tree focuses has only a few main features where one is the most important. The rest are not used that much or almost not at all. The Random Forest uses a far wider range of features. They also rank the features a bit differently and the best feature for one model is not the best for the other. We considered removing the worst performing features to see if it would make a difference in the performanes. But since they have diffrent worst performing features we reasoned that to keep the comparison as fair as possible it would be more interesting to leave the features as is.
+
+
 \section{Summary}
 We have succesfully trained two different but similar machine learning models on classifying the monetary status of people based on a bunch of different features. While some trade offs where made in regards to which features where kept and to what we optimized the model for. We still managed to get a respectable result especially regarding the difficult type of data that we had to work with.
 %---------
 %	REFERENCE LIST
 %----------------------------
-\bibliographystyle{model1-num-names}
-\begin{thebibliography}{4}
-\bibitem{Steinhaus:Mathematical}
-    Steinhaus, H.,
-    Mathematical Snapshots,
-    3rd Edition. New York: Dover, pp. 93-94,
-    (1999)
 
-\bibitem{Greivenkamp:FieldGuide}
-    Greivenkamp,
-    J. E., Field Guide to Geometrical Optics,
-    SPIE Press,
-    Bellingham, WA,
-    (2004)
-
-\bibitem{Pedrotti:Introduction}
-    Pedrotti, F.L. and Pedrotti, L.S.,
-    Introduction to Optics,
-    3rd Edition,
-    Addison-Wesley,
-    (2006)
-
-\bibitem{Davis:ChemWiki}
-    UC Davis ChemWiki,
-    Propagation of Error,
-    Available at: \url{https://chem.libretexts.org/Textbook_Maps/Analytical_Chemistry/Supplemental_Modules_(Analytical_Chemistry)/Quantifying_Nature/Significant_Digits/Propagation_of_Error},
-    (Accessed: 10th March 2016).
-\end{thebibliography}