Jany/MLPproject
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Sike

Browse Source
This commit is contained in:
Petrus committed 2025-10-31 16:03:54 +01:00
1 parent 52a78af447
commit abbd45298a
9 files changed
+61 -61
Download Patch File Download Diff File Expand all files Collapse all files

No files matched your search

Decision_tree.ipynb
+49 -49
View File
@@ -6,7 +6,7 @@
"metadata": {},
"source": [
"### Toymodel as an initial test\n",
"This section is mostly the initial feature engineering. We also create a simple toymodel just to see how the decision tree performs"
"This section is mostly the initial feature engineering. We also create a simple toymodel just to see how the decision tree performs and a correlation matrix to find possible redundant features. "
]
},
{
@@ -17,7 +17,7 @@
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA7sAAAOwCAYAAAD/XWM1AAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjcsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvTLEjVAAAAAlwSFlzAAAPYQAAD2EBqD+naQABAABJREFUeJzs3Xd0VPXaxfHvpHdCGqRQQu8dBEE6iChiRVFpooiIICgqIiJgAxVERRRFFKSjCIJ0SOi9914CCamktynvH0CuvKJSEk7K/qzlujA5c2bPXJLJnnN+zzHZbDYbIiIiIiIiIoWIndEBRERERERERHKbyq6IiIiIiIgUOiq7IiIiIiIiUuio7IqIiIiIiEiho7IrIiIiIiIihY7KroiIiIiIiBQ6KrsiIiIiIiJS6KjsioiIiIiISKGjsisiIiIiIiKFjsquiIiIiIiIFDoquyIiIiIiIlLoqOyKiIiIiIhIoaOyKyIiIiIiIoWOyq6IiIiIiIgUOiq7IiIiIiIiUuio7IqIiIiIiEiho7IrIiIiIiIihY7KroiIiIiIiBQ6KrsiIiIiIiJS6KjsioiIiIiISKGjsisiIiIiIiKFjsquiIiIiIiIFDoquyIiIiIiIlLoqOyKiIiIiIhIoaOyKyIiIiIiIoWOyq6IiIiIiIgUOiq7IiIiIiIiUuio7IqIiIiIiEiho7IrIiIiIiIihY7KroiIiIiIiBQ6KrsiIiIiIiJS6KjsioiIiIiISKGjsisiIiIiIiKFjsquiIiIiIiIFDoquyIiIiIiIlLoqOyKiIiIiIhIoaOyKyIiIiIiIoWOyq6IiIiIiIgUOiq7IiIiIiIiUuio7IqIiIiIiEiho7IrIiIiIiIihY7KroiIiIiIiBQ6KrsiIiIiIiJS6KjsioiIiIiISKGjsisiIiIiIiKFjsquiIiIiIiIFDoquyIiIiIiIlLoqOyKiIiIiIhIoaOyKyIiIiIiIoWOyq6IiIiIiIgUOiq7IiIiIiIiUuio7IqIiIiIiEiho7IrIiIiIiIihY7KroiIiIiIiBQ6KrsiIiIiIiJS6KjsioiIiIiISKGjsisiIiIiIiKFjsquiIiIiIiIFDoquyIiIiIiIlLoqOyKiIiIiIhIoaOyKyIiIiIiIoWOg9EBRERE5O7JyMjgwoULWK1Wo6PcMpPJRFBQEG5ubkZHERGRAkBlV0REpIg4e/YsLVs058zZc0ZHuW0lAvwJC19HlSpVjI4iIiL5nMquiIhIEfHTTz8RfSmK2W93wd3Fyeg4tywj28xLX/3BpEmTmDBhgtFxREQkn1PZFRERKSLS0tIo6eNF+/oV/3Gbaat3071N3Zy/7z0VSYC3B+4uTni5Of/n/SLjk/91W6vVhp2d6brbLBYrU1bsxNHejl7t6wPwzk8rea5VbaqVCbhu29CSxUlLS7vp5ywiIkWXyq6IiEgR9tmvG2hSpRSr95wiwNsdNxdHNh06R/j+07SsFcrS7cdpWDkYPy839p+5hJODPQkp6fh6uVG/QhBzwg/w/P31OH4hjgm/b6J+xWCCfDyZvHQ7Lk4O9O/UmPTMbP7cfgw3Z0eqlQ4gMS2D6MupuDg50Kx6GWKSUilX0oeohOScXCF+XsQmqdSKiMjt0zRmERGRIizA2516FYLwcHWiUogfAHHJaVQtHcDpqAT8irlhs4HNBnamK0dkfb3caFkzlIjYJGqUDWDz4fNUDPbFv5g7ZsvfB1+5ODlQLrA4ABar9br/APy93DkZGU+22cLpqASS0jIpE+DNwXPRd/W1EBGRwkVHdkVERIqwa6csD36s6X9ue2+10tf9vXSA9w23Oxd9mQYVg2lZKxSuTlGuWz4Iyt94v/b2drzUseF1tz3YqPLNPgUREZEbUtkVERGRm3bsQiyVgq8cAf6n9bmlA7wp5u7Cj8t34ubiyNMtahF9OYUdxy+QnmWmXMni/LbxEC890JAQ/2IGPRMRESnsVHZFRETkX/20chdWq41ygT6ci7nM9NV78PNyy1mf6+XmzNajESSnZeLt4UKDisEcjYilWY0ybDsaAUCAtwcOdnY42NlRt3wQ249dUNEVEZE8pbIrIiIiN+Xqkl0qBvviYGd33fpca846XBsAlYJ9+W3jIVydHTlw5hLeHq5kW6xkmy1EX06hhLeHUU9DRESKCJVdERER+VfNqpdh5e6TlC/pQ4uaoTfcpknV69fzenu48vz99a+7LcTPK+fPnZtUzaO0IiIiV6jsioiIyL+qEORLhSBfo2OIiIjcEpVdERERuSUbDp4lyMeTcoE+N7X9obPR/LnjGIMfbcrnv23Av5g7VUr5cykhBTcXR05cjMfBzkTXlrX5adUumlQpRf2KwXn+PEREpHBT2RURERG++3M75QKLYzZb2Xz4PL3a1+ObxVvxdHWmZmgJgn29WLz1KD6erlQt7U9kfDIz1+6lfJAvSWmZlAssTru6FUhOz2TrkStDqaqW9ifY14tqZQLYceICAOmZZjKzLZy4GMdzrevwy5o9VAnxY9eJi+w9FQlAtvnv1+oVERG5VXZGBxARERHj1SgTwKaD50jJyKKUvxcX4pKoGVqCsiW9eaBBJc5cSrhue3s7O3yLuVO2hHfOfQFsNq4OqrJiuzKrioiYRA6djeZCXBJebs7YbDYqBPqyaMsRSnh74O7ihJOjA5VDrlzS6FRU/N1/AUREpNAx2WzX3opERESkMHvrrbeY/8uP7Jjw0m3df9rq3XRvUzfXc92KdsN+pm7zDnz//feG5hARkfxPR3ZFRETkphhddEVERG6F1uyKiIjILR+1nbl2L02qlubMpQTOxybSoX5Fdhy/QHqWmcSUDIL9vGhSpRQ/rdqNj6cr91Uvy4ZDZ3G0t6NL85o5+/lh2Q4qBPlSO7QkI35ZzTOtahMRk4i7ixOXLqdgZ2eifb0KTFm+k2FPt8yjZy8iIoWRjuyKiIgUMdNX72HN3lOcj0lk1Iy1zArbB1cL77X//WTuOsb9tpFsswWzxcqq3SdZtfskJyP/t562lH8xWtUuB0CAtwcOdnY42NkR7OdFRpYZs9VKaMniHDhziTIlvLFabXS6p0rO/S8lpODq7AjAmr2nqF8xmNL+xfB0c2bv6Sgqh/hhwsTBs9GU8i92l18lEREp6FR2RUREipiKQb7sOHaB1IwsKoX4cjb6MgC+nm4s23EMACcHe6qU8sPRwR7439Apq/X6UR/7Tkdx6Gw0EbFJZFusZJstlPD2ICYxlaj4FBzt7ahSyh+AjKxsXJ0dWbz1CABHImKwM5k4H5NIcnomRyNisVptZGZbqFOuJB4uTqRkZFG3fOBdfoVERKQw0GnMIiIiRUzjqqVoXLUUQE4R/TcO9nbcX7/idbeVK+nDhdgkaoWWpFZoSQBC/Lxyvl7nakGtViYg57YXH2gIwENXj+62qBl6w8cL+ctR3JpX9+3soF9ZRETk1ujIroiISBE2c+1ezJZbv67tsYuxlCnhnSeZjl2I/dttT7WoecNtRURE/ok+JhURESmCpq/eQ0kfD6w2G0cjYvh982Hqlg/k0LkYmlUvw95TUZQLLE67uhVITs9k65EIAKqW9ifY939HcJfvPE5qRjauTg5cTs1g29EIGlUOwdPVicPnY7i/fkWmrtjFW13uo5ibC5/O30CAtztlArzZfPg8vdrX48uFmykdUIzygb7YmSA+JZ0l247i6+VG/QpBrNp9kqiEFD7u1d7AV0xERAoaHdkVEREpokyYAIhJTKNu+UBORiZQs2wJNhw8S40yAWw6eA4Am+1/a3ZtV5fsZmaZWb7zOPHJ6djZXdnP0YhYWtUuR1pmFhlZZmqFlsTLzZm6FQKJuZxKXHI6Hq5OVArxIyUji1L+XlyIS6JuhUBKeHvQsWEl4lPSAfD1cqNlzVDORSfmZBAREbkVOrIrIiJSBHVrU+cfv3ZtfW7T6mUA8HJz/tua3Wvrb//qgYaVbri/siWK5/x58GNN//b1Zlcfhxtcy7d0gDfuLk7Y20f9y7MRERH5O5VdERGRIsLNzY2o+CRW7DyOu4uT0XFumr2dibrlAlm1+ySnoxJo6uZmdCQRESkAVHZFRESKiGbNmvGZvQNPfzLX6Ci3zcXZmRYtWhgdQ0RECgCTzaZVMCIiIoXZxYsXef/995kyZQqlSpVi0KBBdOjQATu7gjO6w2azsWbNGsaPH8+xY8d49tln+eCDDyhbtqzR0UREJJ9S2RURESmkkpKSGDt2LOPGjcPNzY3hw4fTt29fnJ2djY5228xmMz/++CMjRowgPj6e/v37M2zYMHx8fIyOJiIi+YzKroiISCGTlZXF5MmTGTlyJCkpKQwaNIi33nqLYsWKGR0t16SmpjJu3DjGjh2Lg4MDQ4cO5dVXX8XV1dXoaCIikk+o7IqIiBQSNpuN+fPnM3ToUE6fPk3Pnj0ZOXIkISEhRkfLM9HR0YwaNYrvvvuOwMBARo8ezXPPPYe9vb3R0URExGAFZ7GOiIiI/KPw8HAaN25Mly5dqFKlCnv37mXKlCmFuugCBAQE8PXXX3Po0CHuueceevbsSb169Vi2bBn6PF9EpGhT2RURESnADh48SKdOnWjZsiVWq5W1a9eyePFiatSoYXS0u6pixYrMmzePzZs3U6xYMR544AHatWvHrl27jI4mIiIGUdkVEREpgC5cuMALL7xArVq1OHToELNnz2br1q20bNnS6GiGaty4MeHh4SxcuJCLFy9Sv359nn32WU6fPm10NBERuctUdkVERAqQxMREhg0bRsWKFVm4cCHjx4/n8OHDPPXUUwXqUkJ5yWQy8fDDD7Nv3z4mT57M2rVrqVKlCoMHDyYuLs7oeCIicpdoQJWIiEgBkJWVxbfffsvo0aNJTU1l0KBBvPnmm4VqwnJeSU1NZfz48YwdOxY7OzuGDh3KgAEDNLlZRKSQU9kVERHJx2w2G/PmzWPo0KGcOXOGXr16MXLkSIKDg42OVuBLine truncated
"image/png": "iVBORw0KGgoAAAANSUhEUgAAA7YAAAOwCAYAAAAKo+iFAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjcsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvTLEjVAAAAAlwSFlzAAAPYQAAD2EBqD+naQABAABJREFUeJzs3Xd4VFXixvHvpHdCeughBJLQqxSpKnaavYBlVUBRV9cCgg2kBkURkGJBkCoIUgSkhV5CDyn0UEIoSQhJSJ+Z3x8IKz/LAibcTPJ+nsdnYXLnzjuzJJl3zj3nmKxWqxURERERERERG2VndAARERERERGRf0LFVkRERERERGyaiq2IiIiIiIjYNBVbERERERERsWkqtiIiIiIiImLTVGxFRERERETEpqnYioiIiIiIiE1TsRURERERERGbpmIrIiIiIiIiNk3FVkRERERERGyaiq2IiIiIiIjYNBVbERERERERsWkqtiIiIiIiImLTVGxFRERERETEpqnYioiIiIiIiE1TsRURERERERGbpmIrIiIiIiIiNk3FVkRERERERGyaiq2IiIiIiIjYNBVbERERERERsWkqtiIiIiIiImLTVGxFRERERETEpqnYioiIiIiIiE1TsRURERERERGbpmIrIiIiIiIiNk3FVkRERERERGyaiq2IiIiIiIjYNBVbERERERERsWkqtiIiIiIiImLTVGxFRERERETEpqnYioiIiIiIiE1TsRURERERERGbpmIrIiIiIiIiNk3FVkRERERERGyaiq2IiIiIiIjYNBVbERERERERsWkqtiIiIiIiImLTVGxFRERERETEpqnYioiIiIiIiE1TsRURERERERGbpmIrIiIiIiIiNk3FVkRERERERGyaiq2IiIiIiIjYNBVbERERERERsWkqtiIiIiIiImLTVGxFRERERETEpqnYioiIiIiIiE1TsRURERERERGbpmIrIiIiIiIiNk3FVkRERERERGyaiq2IiIiIiIjYNBVbERERERERsWkqtiIiIiIiImLTVGxFRERERETEpqnYioiIiIiIiE1TsRURERERERGbpmIrIiIiIiIiNk3FVkRERERERGyaiq2IiIiIiIjYNBVbERERERERsWkqtiIiIiIiImLTVGxFRERERETEpqnYioiIiIiIiE1TsRURERERERGbpmIrIiIiIiIiNk3FVkRERERERGyaiq2IiIiIiIjYNBVbERERERERsWkqtiIiIiIiImLTVGxFRERERETEpqnYioiIiIiIiE1TsRURERERERGb5mB0ABEREbl5+fn5nDp1CovFYnSUG2YymahUqRJubm5GRxERERunYisiImKjTp06RYf27Thy9JjRUW6av58fa6OjqVu3rtFRRETEhqnYioiI2Kjp06dz6uRJZr77CJ6uzkbHuWH5RWZeHr+E8ePHM2HCBKPjiIiIDVOxFRERsVE5OTkE+nhxT7Paf3nMtNW76XVH46t/33s0hQBvD9xdnPBy++syfOV+KelZf3usxWLFzs50zW1ms4Vvft2Jo70dz3VuCsB7U1fydMeGRFYPuObYmkE+5OTkXPdzFhER+TMqtiIiImXI6PkbaRVeldV7jhLg7Y6biyOb40+wLvYYHRqEsCzmEM3rVMbPy43YpLM4OdhzITsXXy83mtaqxJx1+3n+7iYcSk7ji4WbaRpWmUo+nkxeFoOLkwP9HmxJbn4hv8QcxM3ZkchqAVzMyeNcxiVcnBy4vW51zmdeomaQD2cuZF3NVcXPi9RMFVgRESkZWhVZRESkDAnwdqdJrUp4uDpRu4ofAGlZOURUC+DYmQv4VXDDagWrFexMl0dafb3c6FA/hFOpmdSrEcCWhJOEVfbFv4I7ReY/Lkrl4uRAzeCKAJgtlmv+A/D3cudISjqFRWaOnblAZk4+1QO8iTtx7pa+FiIiUn5oxFZERKQMuXLZ8Zs92vzPY1tHVrvm79UCvP/0uBPnMmgWVpkODULgt9WMG4dWgtA/P6+9vR2972t+zW33t6hzvU9BRETkhqnYioiIlGMHk1OpXfnyyO5fzaetFuBNBXcXvl2xEzcXRx5v34BzGdnsOJRMbkERNYMq8tOmeHrf25wq/hUMeiYiIlKeqdiKiIiUM1NX7sJisVIz2IcT5zOYvnoPfl5uV+fTerk5s+3AKbJy8vH2cKFZWGUOnErl9nrV2X7gFAAB3h442NnhYGdH49BKxBxMVqkVERHDqNiKiIiUU79NsSWssi8OdnbXzKe1XJ03awWgdmVfftoUj6uzI/uTzuLt4Uqh2UJhkZlzGdkEensY9TRERERUbEVERMqb2+tWZ+XuI4QG+dC+fsifHtMq4tr5t94erjx/d9Nrbqvi53X1z11bRZRQWhERkf9NxVZERKScqVXJl1qVfI2OISIiUmxUbEVERMq5jXHHqeTjSc1gn+s6Pv74OX7ZcZA3u7fh05824l/BnfCq/py9kI2biyOHT6fjYGfiiQ4NmbpqF63Cq9I0rHKJPw8RESm/VGxFRETKoEm/xFAzuCJFRRa2JJzkuc5NmLBkG56uztQPCaSyrxdLth3Ax9OViGr+pKRnMXPtXkIr+ZKZk0/N4Irc1bgWWbn5bEu8vGBURDV/Kvt6EVk9gB2HkwHIzS8iv9DM4dNpPN2pET+s2UN4FT92HT7N3qMpABQW/XEvXBERkeJkZ3QAERERKX71qgewOe4E2XkFVPX3Ijktk/ohgdQI8ubeZrVJOnvhmuPt7ezwreBOjUDvq/cFsFr5bREpC9bL60hx6vxF4o+fIzktEy83Z6xWK7WCfVm0NZFAbw/cXZxwcnSgTpXL2wgdPZN+618AEREpV0xW65VfUyIiImJL3n//faZNmcCecX1v6v7TVu+m1x2Niz3Xjbjvgx+o06IDU6dONTSHiIjYNo3YioiIlFNGl1oREZHiojm2IiIiZdCNjsbOXLuXVhHVSDp7gZOpF7mnaRg7DiWTW1DExew8Kvt50Sq8KlNX7cbH05W2dWuwMf44jvZ2PNqu/tXzfL18B7Uq+dIwJIgPf1jNkx0bcur8RdxdnDibkY2dnYnOTWrxzYqdDHy8Qwk9exERKW80YisiImLjpq/ew5q9Rzl5/iKDZ6xlVvQ++K3cXvnfEXPX89lPmygsMlNktrBq9xFW7T7CkZT/zn+t6l+Bjg1rAhDg7YGDnR0OdnZU9vMir6CIIouFkKCK7E86S/VAbywWKw/eFn71/mcvZOPq7AjAmr1HaRpWmWr+FfB0c2bvsTPUqeKHCRNxx89R1b/CLX6VRESkLFOxFRERsXFhlXzZcTCZS3kF1K7iy/FzGQD4erqxfMdBAJwc7Amv6oejgz3w3wWhLJZrl9rYd+wM8cfPcSo1k0KzhcIiM4HeHpy/eIkz6dk42tsRXtUfgLyCQlydHVmyLRGAxFPnsTOZOHn+Ilm5+Rw4lYrFYiW/0EyjmkF4uDiRnVdA49DgW/wKiYhIWadLkUVERGxcy4iqtIyoCnC1dP4dB3s77m4ads1tNYN8SE7NpEFIEA1CggCo4ud19euNfiujkdUDrt724r3NAXjgt1Hb9vVD/vTxqvxudLb+b+d2dtBbEBERKT4asRURESlDZq7dS5H5xveNPXg6leqB3iWS6WBy6h9ue6x9/T89VkRE5Gbo41IREZEyYPrqPQT5eGCxWjlw6jwLtyTQODSY+BPnub1udfYePUPN4Irc1bgWWbn5bEs8BUBENX8q+/53ZHbFzkNcyivE1cmBjEt5bD9wihZ1quDp6kTCyfPc3TSM737dxbuPtqWCmwtR8zYS4O1O9QBvtiSc5LnOTRj78xaqBVQgNNgXOxOkZ+eydPsBfL3caFqrEqt2H+HMhWyGP9fZwFdMRETKEo3YioiIlBEmTACcv5hD49BgjqRcoH6NQDbGHade9QA2x50AwGr97xzbK7vZ5xcUsWLnIdKzcrGzu3yeA6dS6diwJjn5BeQVFNEgJAgvN2ca1wrmfMYl0rJy8XB1onYVP7LzCqjq70VyWiaNawUT6O3Bfc1rk56dC4Cvlxsd6odw4tzFqxlERESKi0ZsRUREyoCedzT6y69dmU/bpm51ALzcnP8wx/bKfNnfu7d57T89X43Ailf//GaPNn/4+u2/PQ5/sldutQBv3F2csLc/8zfPRkRE5Mao2IqIiNgoNzc3zqZnsnzHQTxdnY2Oc93s7Uw0rhnMmr1HOXomncZubkZHEhERG2eyWnUxkIiIiC06deoUHdq348jRY0ZHuWn+fn6sjY6mbt26RkcREREbpmIrIiJiwy5cuMCAAQP4+uuvqVq1Kh9//DGtWrUyOtZf2r9/P++//z5xcXE8/vjjjB49msqVKxsdS0REbJyKrYiIiI1atmwZr7zyCqdPn+a9997j3Xffxdm59F+SbDabmTBhAgMHDsTV1ZUxY8bwxBNPYDKZjI4mIiI2Sqsii4iI2JjTp0/z6KOPct9991GzZk1iY2P54IMPbKLUAtjb2/Pqq6+SkJBA+/bteeqpp+jcuTOHDh0yOpqIiNgoFVsREREbYTab+fLLLwkPD2fdunXMmDGDlStXEhYWdh33Ln0qV67M3LlzWbp0KYcPH6Z+/foMHjyY/Px8o6OJiIiN0aXIIiIiNmDXrl307t2bHTt20Lt3b4YPH07FihWv4562IScnh08++YSoqChq1qzJxIkT6dixo9GxRETERmjEVkREpBTLzMzk9ddfp3nz5uTn57N582YmTpxYpkotv21dNGzYMPbs2UNAQACdOnWiV69enDt3zuhoIiJiA1RsRURESiGr1cr8+fOJiIjg66+/ZuTIkezcubNUr3hcHOrWrcu6dev4+uuvWbp0KeHh4Xz99ddYLBajo4mISCmmYisiIlLKJCUl8eCDD/Lwww/TtGlT4uPjeeutt3B0dDQ62i1hZ2fHv/71LxITE+nSpQsvvvgi7dq1Y//+/UZHExGRUkrFVkREpJQLine truncated
"text/plain": [
"<Figure size 1200x1200 with 1 Axes>"
]
@@ -35,6 +35,16 @@
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAtkAAAJsCAYAAAAoSr9YAAAAOnRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjEwLjcsIGh0dHBzOi8vbWF0cGxvdGxpYi5vcmcvTLEjVAAAAAlwSFlzAAAPYQAAD2EBqD+naQABAABJREFUeJzs3XdUVMfbwPHvLl26dKR3pNoVe+810Rhj19g1aozGWNCfxhJ7L4m9a+y9YO9dLNgQu6IgLCDS9/0DWFhZqhJN3vmcs+fI3efe++zszDg7O/euRC6XyxEEQRAEQRAE4bORfukEBEEQBEEQBOG/RgyyBUEQBEEQBOEzE4NsQRAEQRAEQfjMxCBbEARBEARBED4zMcgWBEEQBEEQhM9MDLIFQRAEQRAE4TMTg2xBEARBEARB+MzEIFsQBEEQBEEQPjMxyBYEQRAEQRCEz0wMsgVBEAph5cqVSCQSHj9+/NmO+fjxYyQSCStXrvxsxxQEQRC+LDHIFgThiwsNDaV37944OTmhra2NgYEBVatWZc6cOXz48OFLp/fZrF+/ntmzZ3/pNJR07doVPT29XJ+XSCQMGDCgWHNYuHCh+IAhCMJ/jvqXTkAQhP/f9u7dS9u2bdHS0qJz5854e3uTlJTE6dOn+eWXX7h9+zZLly790ml+FuvXr+fWrVsMHjxYabu9vT0fPnxAQ0Pji+X2JS1cuBBTU1O6du36pVMRBEH4bMQgWxCELyYsLIz27dtjb2/P0aNHsbKyUjzXv39/Hj58yN69ez/5PHK5nISEBHR0dHI8l5CQgKamJlLpl/tiTyKRoK2t/cXOLwiCIHx+YrmIIAhfzB9//EFcXBzLli1TGmBncnFx4aefflL8nZKSwoQJE3B2dkZLSwsHBwd+++03EhMTlfZzcHCgWbNmHDx4kPLly6Ojo8OSJUs4fvw4EomEjRs3Mnr0aEqVKkWJEiWIiYkB4MKFCzRq1AhDQ0NKlChBzZo1OXPmTL6vY+fOnTRt2hRra2u0tLRwdnZmwoQJpKamKmJq1arF3r17efLkCRKJBIlEgoODA+SxJvvo0aNUr14dXV1djIyMaNmyJSEhIUox48aNQyKR8PDhQ7p27YqRkRGGhoZ069aN+Pj4Ar8XhZGYmEhgYCAuLi5oaWlha2vL8OHDc7wPK1asoE6dOpibm6OlpUXp0qVZtGiRUoyDgwO3b9/mxIkTinKpVasWZFv/fvr0aQYNGoSZmRlGRkb07t2bpKQkoqOj6dy5M8bGxhgbGzN8+HDkcrnS8adPn05AQAAmJibo6OhQrlw5/v777xyvKXNZzLp163B3d0dbW5ty5cpx8uTJYilDQRD++8RMtiAIX8zu3btxcnIiICCgQPE9e/Zk1apVfPvtt/z8889cuHCByZMnExISwvbt25Vi7927x/fff0/v3r358ccfcXd3Vzw3YcIENDU1GTZsGImJiWhqanL06FEaN25MuXLlCAwMRCqVKgaJp06domLFirnmtXLlSvT09Bg6dCh6enocPXqUsWPHEhMTw7Rp0wAYNWoUMpmM58+fM2vWLIA810IfOXKExo0b4+TkxLhx4/jw4QPz5s2jatWqXL16VTFAz9SuXTscHR2ZPHkyV69e5a+//sLc3JypU6cWqGwjIiIKFJeWlkaLFi04ffo0vXr1wtPTk5s3bzJr1izu37/Pjh07FLGLFi3Cy8uLFi1aoK6uzu7du+nXrx9paWn0798fgNmzZzNw4ED09PQYNWoUABYWFkrnHDhwIJaWlowfP57z58+zdOlSjIyMOHv2LHZ2dkyaNIl9+/Yxbdo0vL296dy5s2LfOXPm0KJFC3744QeSkpLYuHEjbdu2Zc+ePTRt2lTpPCdOnGDTpk0MGjQILS0tFi5cSKNGjbh48SLe3t4FKh9BEAQFuSAIwhcgk8nkgLxly5YFir9+/bockPfs2VNp+7Bhw+SA/OjRo4pt9vb2ckB+4MABpdhjx47JAbmTk5M8Pj5esT0tLU3u6uoqb9iwoTwtLU2xPT4+Xu7o6CivX7++YtuKFSvkgDwsLEwp7mO9e/eWlyhRQp6QkKDY1rRpU7m9vX2O2LCwMDkgX7FihWKbv7+/3NzcXB4ZGanYduPGDblUKpV37txZsS0wMFAOyLt37650zNatW8tNTExUlKSyLl26yIE8H/3791fEr1mzRi6VSuWnTp1SOs7ixYvlgPzMmTN5lkvDhg3lTk5OStu8vLzkNWvWzBGbWdYfvy9VqlSRSyQSeZ8+fRTbUlJS5DY2NjmO83EOSUlJcm9vb3mdOnWUtme+1suXLyu2PXnyRK6trS1v3bp1LqUnCIKQO7FcRBCELyJziYa+vn6B4vft2wfA0KFDlbb//PPPkHEBZXaOjo40bNhQ5bG6dOmitD77+vXrPHjwgA4dOhAZGUlERAQRERG8f/+eunXrcvLkSdLS0nLNLfuxYmNjiYiIoHr16sTHx3P37t0Cvb7sXr16xfXr1+natSslS5ZUbPf19aV+/fqKssiuT58+Sn9Xr16dyMhIRTnnRVtbm8OHD6t8fGzLli14enri4eGhKKeIiAjq1KkDwLFjxxSx2ctFJpMRERFBzZo1efToETKZrMDl0aNHDyQSieLvSpUqIZfL6dGjh2Kbmpoa5cuX59GjR0r7Zs8hKioKmUxG9erVuXr1ao7zVKlShXLlyin+trOzo2XLlhw8eFBp6Y8gCEJBiOUigiB8EQYGBpAxKC2IJ0+eIJVKcXFxUdpuaWmJkZERT548Udru6OiY67E+fu7BgweQMfjOjUwmw9jYWOVzt2/fZvTo0Rw9ejTHoLYwg8lMma8l+xKXTJ6enhw8eJD379+jq6ur2G5nZ6cUl5lrVFSUoqxzo6amRr169QqU24MHDwgJCcHMzEzl82/evFH8+8yZMwQGBnLu3Lkc68NlMhmGhoYFOufHry1zP1tb2xzbo6KilLbt2bOHiRMncv36daU149kH7ZlcXV1zbHNzcyM+Pp63b99iaWlZoHwFQRAQg2xBEL4UAwMDrK2tuXXrVqH2UzU4UkXVnURyey5zlnratGn4+/ur3Ce39dPR0dHUrFkTAwMD/ve//+Hs7Iy2tjZXr15lxIgRec6Af05qamoqt398IeCnSktLw8fHh5kzZ6p8PnPgGxoaSt26dfHw8GDmzJnY2tqiqanJvn37mDVrVqHKJbfXpmp79td76tQpWrRoQY0aNVi4cCFWVlZoaGiwYsUK1q9fX+DzC4IgFIUYZAuC8MU0a9aMpUuXcu7cOapUqZJnrL29PWlpaTx48ABPT0/F9vDwcKKjo7G3ty9yHs7OzpAx8C/ojG6m48ePExkZybZt26hRo4Zie1hYWI7Ygn5AyHwt9+7dy/Hc3bt3MTU1VZrF/ic5Oztz48YN6tatm+fr2b17N4mJiezatUtpJjr7cpJMBS2Xwtq6dSva2tocPHgQLS0txfYVK1aojM/8RiO7+/fvU6JEiVxn7gVBEHIj1mQLgvDFDB8+HF1dXXr27El4eHiO50NDQ5kzZw4ATZo0gYy7UWSXOaP68Z0iCqNcuXI4Ozszffp04uLicjz/9u3bXPfNnE3NPoOalJTEwoULc8Tq6uoWaPmIlZUV/v7+rFq1iujoaMX2W7ducejQIUVZfAnt2rXjxYsX/Pnnnzme+/DhA+/fv4dcykUmk6kc4Orq6iq9zs9FTU0NiUSitJ768ePHSndAye7cuXNKa7WfPXvGzp07adCgQa6z6YIgCLkRM9mCIHwxzs7OrF+/nu+++w5PT0+lX3w8e/YsW7ZsUfwKoJ+fH126dGHp0qWKJRoXL15k1apVtGrVitq1axc5D6lUyl9//UXjxo3x8vKiW7dulCpVihcvXnDs2DEMDAzYvXu3yn0DAgIwNjamS5cuDBo0CIlEwpo1a1Qu0yhXrhybNm1i6NChVKhQAT09PZo3b67yuNOmTaNx48ZUqVKFHj16KG7hZ2hoyLhx44r8Wj9Vp06d2Lx5M3369OHYsWNUrVqV1NRU7t69y+bNmxX3Jm/QoAGampo0b96c3r17ExcXx59//om5uTmvXr1SOma5cuVYtGgREydOxMXFBXNzc8WFlJ+iadOmzJw5k0aNGtGhQwfevHnDggULcHFxITg4OEe8t7c3DRs2VLqFH8D48eM/ORdBEP4f+tK3NxEEQbh//778xx9/lDs4OMg1NTXl+vr68qpVq8rnzZundAu85ORk+fjx4+WOjo5yDQ0Nua2trXzkyJFKMfKMW/g1bdo0x3kyb+G3ZcsWlXlcu3ZN3qZNG7mJiYlcS0tLbm9vL2/Xrp08KChIEaPqFn5nzpyRV65cWa6joyO3traWDx8+XH7w4EE5ID927JgiLi4uTt6hQwe5kZGRHFDczk/VLfzkcrn8yJEj8qpVq8p1dHTkBgYG8ubNm8vv3LmjFJN5C7+3b98qbVeVpypdunSR6+rq5vr8x7fwk2fcBm/q1KlyLy8vuZaWltzY2Fherlw5+fjx4+UymUwRt2vXLrmvr69cW1tb7uDgIJ86dap8+fLlOfJ6/fq1vGnTpnJ9fX05oLgNX+ZruHTpUoFes6rXsmzZMrmrq6tcS0tL7uHhIV+xYoVif1Wvc+3atYr4MmXKKL1/giAIhSGRf+6rYgRBEAThX0YikdC/f3/mz5//pVMRBOE/QqzJFgRBEARBEITPTAyyBUEQBEEQBOEzE4NsQRAEQRAEQfjMxCBbEARB+H9PLpeL9diC8C9x8uRJmjdvjrW1NRKJJNfbcmZ3/PhxypYti5aWFi4uLqxcubLY8xSDbEEQBEEQBOFf4/379/j5+bFgwYICxYeFhdG0aVNq167N9evXGTx4MD179uTgwYPFmqe4u4ggCIIgCILwrySRSNi+fTutWrXKNWbEiBHs3buXW7duKba1b9+e6OhoDhw4UGy5iZlsQRAEQRAE4YtKTEwkJiZG6ZGYmPhZjn3u3Dnq1auntK1Line truncated
"text/plain": [
"<Figure size 800x600 with 2 Axes>"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"name": "stdout",
"output_type": "stream",
@@ -48,11 +58,7 @@
" accuracy 0.81 6033\n",
" macro avg 0.74 0.75 0.74 6033\n",
" weighted avg 0.81 0.81 0.81 6033\n",
"\n",
"Index(['age', 'workclass', 'education.num', 'marital.status', 'occupation',\n",
" 'relationship', 'race', 'sex', 'capital.gain', 'capital.loss',\n",
" 'hours.per.week', 'native.country'],\n",
" dtype='object')\n"
"\n"
]
}
],
@@ -65,6 +71,7 @@
"from sklearn.tree import DecisionTreeClassifier, plot_tree\n",
"from sklearn.preprocessing import LabelEncoder\n",
"from sklearn.metrics import classification_report, confusion_matrix, ConfusionMatrixDisplay\n",
"import seaborn as sns\n",
"\n",
"# Load dataset\n",
"df = pd.read_csv('./Datasets/adult.csv', comment = '#')\n",
@@ -84,7 +91,7 @@
" le = LabelEncoder()\n",
" df_encoded[column] = le.fit_transform(df_encoded[column])\n",
"\n",
"# Now properly separate features and target\n",
"# Separate features and target\n",
"X = df_encoded.drop(columns=['income'])\n",
"y = df_encoded['income']\n",
"\n",
@@ -124,10 +131,15 @@
"plt.title(\"Confusion Matrix\")\n",
"plt.show() \n",
"\n",
"print(\"Classification Report:\")\n",
"print(classification_report(y_val, y_pred, target_names=[\"Lower-earning\", \"Higher-earning\"]))\n",
"# Create a correlation matrix\n",
"matrix = X_train.corr()\n",
"plt.figure(figsize=(8,6))\n",
"sns.heatmap(matrix, annot=True, cmap=\"coolwarm\", fmt=\".2f\", linewidths=0.5)\n",
"plt.title(\"Correlation Heatmap\")\n",
"plt.show()\n",
"\n",
"print(X_train_val.columns)"
"print(\"Classification Report:\")\n",
"print(classification_report(y_val, y_pred, target_names=[\"Lower-earning\", \"Higher-earning\"]))"
]
},
{
@@ -143,7 +155,7 @@
},
{
"cell_type": "code",
"execution_count": 12,
"execution_count": 24,
"id": "e68b1ea9",
"metadata": {},
"outputs": [],
@@ -286,7 +298,7 @@
},
{
"cell_type": "code",
"execution_count": 8,
"execution_count": 25,
"id": "d6fd1fee",
"metadata": {},
"outputs": [
@@ -306,15 +318,15 @@
"\n",
"------ Results Sorted by Accuracy ------\n",
" Model Accuracy Precision Recall F1 Score Best Params Training Time (s) Prediction Time (s) Total Time (s)\n",
" XGBoost 0.868059 0.863590 0.868059 0.863957 {'learning_rate': 0.2, 'max_depth': 5, 'n_estimators': 100} 28.797149 0.005661 28.802809\n",
"Random Forest 0.858942 0.853479 0.858942 0.853443 {'max_depth': 20, 'min_samples_split': 10, 'n_estimators': 100} 166.736673 0.098411 166.835085\n",
"Decision Tree 0.848334 0.844900 0.848334 0.846228 {'max_depth': 10, 'min_samples_split': 5} 9.839182 0.004644 9.843826\n",
" XGBoost 0.868059 0.863590 0.868059 0.863957 {'learning_rate': 0.2, 'max_depth': 5, 'n_estimators': 100} 54.522120 0.022445 54.544564\n",
"Random Forest 0.858942 0.853479 0.858942 0.853443 {'max_depth': 20, 'min_samples_split': 10, 'n_estimators': 100} 152.761754 0.094559 152.856313\n",
"Decision Tree 0.848334 0.844900 0.848334 0.846228 {'max_depth': 10, 'min_samples_split': 5} 7.268746 0.003260 7.272006\n",
"\n",
"------ Results Sorted by Total Time ------\n",
" Model Accuracy Precision Recall F1 Score Best Params Training Time (s) Prediction Time (s) Total Time (s)\n",
"Decision Tree 0.848334 0.844900 0.848334 0.846228 {'max_depth': 10, 'min_samples_split': 5} 9.839182 0.004644 9.843826\n",
" XGBoost 0.868059 0.863590 0.868059 0.863957 {'learning_rate': 0.2, 'max_depth': 5, 'n_estimators': 100} 28.797149 0.005661 28.802809\n",
"Random Forest 0.858942 0.853479 0.858942 0.853443 {'max_depth': 20, 'min_samples_split': 10, 'n_estimators': 100} 166.736673 0.098411 166.835085\n"
"Decision Tree 0.848334 0.844900 0.848334 0.846228 {'max_depth': 10, 'min_samples_split': 5} 7.268746 0.003260 7.272006\n",
" XGBoost 0.868059 0.863590 0.868059 0.863957 {'learning_rate': 0.2, 'max_depth': 5, 'n_estimators': 100} 54.522120 0.022445 54.544564\n",
"Random Forest 0.858942 0.853479 0.858942 0.853443 {'max_depth': 20, 'min_samples_split': 10, 'n_estimators': 100} 152.761754 0.094559 152.856313\n"
]
}
],
@@ -336,7 +348,7 @@
},
{
"cell_type": "code",
"execution_count": 9,
"execution_count": 26,
"id": "4185a428",
"metadata": {},
"outputs": [
@@ -350,15 +362,15 @@
"\n",
"------ Results Sorted by Accuracy ------\n",
" Model Accuracy Precision Recall F1 Score Best Params Training Time (s) Prediction Time (s) Total Time (s)\n",
" XGBoost 0.868556 0.864049 0.868556 0.864181 {'learning_rate': 0.2, 'max_depth': 5, 'n_estimators': 75} 24.887917 0.011055 24.898972\n",
"Random Forest 0.858942 0.853479 0.858942 0.853443 {'max_depth': 20, 'min_samples_split': 10, 'n_estimators': 100} 91.218867 0.037435 91.256302\n",
"Decision Tree 0.849163 0.843062 0.849163 0.839388 {'max_depth': 8, 'min_samples_split': 6} 0.872687 0.001455 0.874143\n",
" XGBoost 0.868556 0.864049 0.868556 0.864181 {'learning_rate': 0.2, 'max_depth': 5, 'n_estimators': 75} 27.877137 0.011091 27.888228\n",
"Random Forest 0.858942 0.853479 0.858942 0.853443 {'max_depth': 20, 'min_samples_split': 10, 'n_estimators': 100} 136.946045 0.074518 137.020563\n",
"Decision Tree 0.849163 0.843062 0.849163 0.839388 {'max_depth': 8, 'min_samples_split': 6} 2.567322 0.006399 2.573721\n",
"\n",
"------ Results Sorted by Total Time ------\n",
" Model Accuracy Precision Recall F1 Score Best Params Training Time (s) Prediction Time (s) Total Time (s)\n",
"Decision Tree 0.849163 0.843062 0.849163 0.839388 {'max_depth': 8, 'min_samples_split': 6} 0.872687 0.001455 0.874143\n",
" XGBoost 0.868556 0.864049 0.868556 0.864181 {'learning_rate': 0.2, 'max_depth': 5, 'n_estimators': 75} 24.887917 0.011055 24.898972\n",
"Random Forest 0.858942 0.853479 0.858942 0.853443 {'max_depth': 20, 'min_samples_split': 10, 'n_estimators': 100} 91.218867 0.037435 91.256302\n"
"Decision Tree 0.849163 0.843062 0.849163 0.839388 {'max_depth': 8, 'min_samples_split': 6} 2.567322 0.006399 2.573721\n",
" XGBoost 0.868556 0.864049 0.868556 0.864181 {'learning_rate': 0.2, 'max_depth': 5, 'n_estimators': 75} 27.877137 0.011091 27.888228\n",
"Random Forest 0.858942 0.853479 0.858942 0.853443 {'max_depth': 20, 'min_samples_split': 10, 'n_estimators': 100} 136.946045 0.074518 137.020563\n"
]
},
{
@@ -433,7 +445,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 27,
"id": "776cdb17",
"metadata": {},
"outputs": [
@@ -441,27 +453,15 @@
"name": "stdout",
"output_type": "stream",
"text": [
"Tuning Decision Tree ...\n"
]
},
{
"ename": "KeyboardInterrupt",
"evalue": "",
"output_type": "error",
"traceback": [
"\u001b[31m---------------------------------------------------------------------------\u001b[39m",
"\u001b[31mKeyboardInterrupt\u001b[39m Traceback (most recent call last)",
"\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[13]\u001b[39m\u001b[32m, line 12\u001b[39m\n\u001b[32m 10\u001b[39m \u001b[38;5;66;03m# Train decision tree without redundant features\u001b[39;00m\n\u001b[32m 11\u001b[39m cmp_dt = ClassifierComparisonOpt(X_dt, y_train_val)\n\u001b[32m---> \u001b[39m\u001b[32m12\u001b[39m \u001b[43mcmp_dt\u001b[49m\u001b[43m.\u001b[49m\u001b[43mfit_models\u001b[49m\u001b[43m(\u001b[49m\u001b[43m{\u001b[49m\u001b[33;43m'\u001b[39;49m\u001b[33;43mDecision Tree\u001b[39;49m\u001b[33;43m'\u001b[39;49m\u001b[43m:\u001b[49m\u001b[43m \u001b[49m\u001b[43mfine_grid\u001b[49m\u001b[43m[\u001b[49m\u001b[33;43m'\u001b[39;49m\u001b[33;43mDecision Tree\u001b[39;49m\u001b[33;43m'\u001b[39;49m\u001b[43m]\u001b[49m\u001b[43m}\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 14\u001b[39m \u001b[38;5;66;03m# Train XGBoost without redundant features\u001b[39;00m\n\u001b[32m 15\u001b[39m cmp_xgb = ClassifierComparisonOpt(X_xgb, y_train_val)\n",
"\u001b[36mCell\u001b[39m\u001b[36m \u001b[39m\u001b[32mIn[12]\u001b[39m\u001b[32m, line 63\u001b[39m, in \u001b[36mClassifierComparisonOpt.fit_models\u001b[39m\u001b[34m(self, models_with_params)\u001b[39m\n\u001b[32m 60\u001b[39m grid_search = GridSearchCV(model, param_grid, scoring=\u001b[33m'\u001b[39m\u001b[33maccuracy\u001b[39m\u001b[33m'\u001b[39m, cv=cv, n_jobs=-\u001b[32m1\u001b[39m)\n\u001b[32m 62\u001b[39m start_train = time.time()\n\u001b[32m---> \u001b[39m\u001b[32m63\u001b[39m \u001b[43mgrid_search\u001b[49m\u001b[43m.\u001b[49m\u001b[43mfit\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mX_train\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43my_train\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 64\u001b[39m end_train = time.time()\n\u001b[32m 66\u001b[39m best_model = grid_search.best_estimator_\n",
"\u001b[36mFile \u001b[39m\u001b[32m~/Documents/MLP/Projects/MLPproject/.venv/lib64/python3.13/site-packages/sklearn/base.py:1365\u001b[39m, in \u001b[36m_fit_context.<locals>.decorator.<locals>.wrapper\u001b[39m\u001b[34m(estimator, *args, **kwargs)\u001b[39m\n\u001b[32m 1358\u001b[39m estimator._validate_params()\n\u001b[32m 1360\u001b[39m \u001b[38;5;28;01mwith\u001b[39;00m config_context(\n\u001b[32m 1361\u001b[39m skip_parameter_validation=(\n\u001b[32m 1362\u001b[39m prefer_skip_nested_validation \u001b[38;5;129;01mor\u001b[39;00m global_skip_validation\n\u001b[32m 1363\u001b[39m )\n\u001b[32m 1364\u001b[39m ):\n\u001b[32m-> \u001b[39m\u001b[32m1365\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[43mfit_method\u001b[49m\u001b[43m(\u001b[49m\u001b[43mestimator\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43margs\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mkwargs\u001b[49m\u001b[43m)\u001b[49m\n",
"\u001b[36mFile \u001b[39m\u001b[32m~/Documents/MLP/Projects/MLPproject/.venv/lib64/python3.13/site-packages/sklearn/model_selection/_search.py:1051\u001b[39m, in \u001b[36mBaseSearchCV.fit\u001b[39m\u001b[34m(self, X, y, **params)\u001b[39m\n\u001b[32m 1045\u001b[39m results = \u001b[38;5;28mself\u001b[39m._format_results(\n\u001b[32m 1046\u001b[39m all_candidate_params, n_splits, all_out, all_more_results\n\u001b[32m 1047\u001b[39m )\n\u001b[32m 1049\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m results\n\u001b[32m-> \u001b[39m\u001b[32m1051\u001b[39m \u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43m_run_search\u001b[49m\u001b[43m(\u001b[49m\u001b[43mevaluate_candidates\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 1053\u001b[39m \u001b[38;5;66;03m# multimetric is determined here because in the case of a callable\u001b[39;00m\n\u001b[32m 1054\u001b[39m \u001b[38;5;66;03m# self.scoring the return type is only known after calling\u001b[39;00m\n\u001b[32m 1055\u001b[39m first_test_score = all_out[\u001b[32m0\u001b[39m][\u001b[33m\"\u001b[39m\u001b[33mtest_scores\u001b[39m\u001b[33m\"\u001b[39m]\n",
"\u001b[36mFile \u001b[39m\u001b[32m~/Documents/MLP/Projects/MLPproject/.venv/lib64/python3.13/site-packages/sklearn/model_selection/_search.py:1605\u001b[39m, in \u001b[36mGridSearchCV._run_search\u001b[39m\u001b[34m(self, evaluate_candidates)\u001b[39m\n\u001b[32m 1603\u001b[39m \u001b[38;5;28;01mdef\u001b[39;00m\u001b[38;5;250m \u001b[39m\u001b[34m_run_search\u001b[39m(\u001b[38;5;28mself\u001b[39m, evaluate_candidates):\n\u001b[32m 1604\u001b[39m \u001b[38;5;250m \u001b[39m\u001b[33;03m\"\"\"Search all candidates in param_grid\"\"\"\u001b[39;00m\n\u001b[32m-> \u001b[39m\u001b[32m1605\u001b[39m \u001b[43mevaluate_candidates\u001b[49m\u001b[43m(\u001b[49m\u001b[43mParameterGrid\u001b[49m\u001b[43m(\u001b[49m\u001b[38;5;28;43mself\u001b[39;49m\u001b[43m.\u001b[49m\u001b[43mparam_grid\u001b[49m\u001b[43m)\u001b[49m\u001b[43m)\u001b[49m\n",
"\u001b[36mFile \u001b[39m\u001b[32m~/Documents/MLP/Projects/MLPproject/.venv/lib64/python3.13/site-packages/sklearn/model_selection/_search.py:997\u001b[39m, in \u001b[36mBaseSearchCV.fit.<locals>.evaluate_candidates\u001b[39m\u001b[34m(candidate_params, cv, more_results)\u001b[39m\n\u001b[32m 989\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m.verbose > \u001b[32m0\u001b[39m:\n\u001b[32m 990\u001b[39m \u001b[38;5;28mprint\u001b[39m(\n\u001b[32m 991\u001b[39m \u001b[33m\"\u001b[39m\u001b[33mFitting \u001b[39m\u001b[38;5;132;01m{0}\u001b[39;00m\u001b[33m folds for each of \u001b[39m\u001b[38;5;132;01m{1}\u001b[39;00m\u001b[33m candidates,\u001b[39m\u001b[33m\"\u001b[39m\n\u001b[32m 992\u001b[39m \u001b[33m\"\u001b[39m\u001b[33m totalling \u001b[39m\u001b[38;5;132;01m{2}\u001b[39;00m\u001b[33m fits\u001b[39m\u001b[33m\"\u001b[39m.format(\n\u001b[32m 993\u001b[39m n_splits, n_candidates, n_candidates * n_splits\n\u001b[32m 994\u001b[39m )\n\u001b[32m 995\u001b[39m )\n\u001b[32m--> \u001b[39m\u001b[32m997\u001b[39m out = \u001b[43mparallel\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 998\u001b[39m \u001b[43m \u001b[49m\u001b[43mdelayed\u001b[49m\u001b[43m(\u001b[49m\u001b[43m_fit_and_score\u001b[49m\u001b[43m)\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 999\u001b[39m \u001b[43m \u001b[49m\u001b[43mclone\u001b[49m\u001b[43m(\u001b[49m\u001b[43mbase_estimator\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1000\u001b[39m \u001b[43m \u001b[49m\u001b[43mX\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1001\u001b[39m \u001b[43m \u001b[49m\u001b[43my\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1002\u001b[39m \u001b[43m \u001b[49m\u001b[43mtrain\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtrain\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1003\u001b[39m \u001b[43m \u001b[49m\u001b[43mtest\u001b[49m\u001b[43m=\u001b[49m\u001b[43mtest\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1004\u001b[39m \u001b[43m \u001b[49m\u001b[43mparameters\u001b[49m\u001b[43m=\u001b[49m\u001b[43mparameters\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1005\u001b[39m \u001b[43m \u001b[49m\u001b[43msplit_progress\u001b[49m\u001b[43m=\u001b[49m\u001b[43m(\u001b[49m\u001b[43msplit_idx\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mn_splits\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1006\u001b[39m \u001b[43m \u001b[49m\u001b[43mcandidate_progress\u001b[49m\u001b[43m=\u001b[49m\u001b[43m(\u001b[49m\u001b[43mcand_idx\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mn_candidates\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1007\u001b[39m \u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mfit_and_score_kwargs\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1008\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 1009\u001b[39m \u001b[43m \u001b[49m\u001b[38;5;28;43;01mfor\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43m(\u001b[49m\u001b[43mcand_idx\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mparameters\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m(\u001b[49m\u001b[43msplit_idx\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m(\u001b[49m\u001b[43mtrain\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43mtest\u001b[49m\u001b[43m)\u001b[49m\u001b[43m)\u001b[49m\u001b[43m \u001b[49m\u001b[38;5;129;43;01min\u001b[39;49;00m\u001b[43m \u001b[49m\u001b[43mproduct\u001b[49m\u001b[43m(\u001b[49m\n\u001b[32m 1010\u001b[39m \u001b[43m \u001b[49m\u001b[38;5;28;43menumerate\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43mcandidate_params\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1011\u001b[39m \u001b[43m \u001b[49m\u001b[38;5;28;43menumerate\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43mcv\u001b[49m\u001b[43m.\u001b[49m\u001b[43msplit\u001b[49m\u001b[43m(\u001b[49m\u001b[43mX\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43my\u001b[49m\u001b[43m,\u001b[49m\u001b[43m \u001b[49m\u001b[43m*\u001b[49m\u001b[43m*\u001b[49m\u001b[43mrouted_params\u001b[49m\u001b[43m.\u001b[49m\u001b[43msplitter\u001b[49m\u001b[43m.\u001b[49m\u001b[43msplit\u001b[49m\u001b[43m)\u001b[49m\u001b[43m)\u001b[49m\u001b[43m,\u001b[49m\n\u001b[32m 1012\u001b[39m \u001b[43m \u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 1013\u001b[39m \u001b[43m\u001b[49m\u001b[43m)\u001b[49m\n\u001b[32m 1015\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mlen\u001b[39m(out) < \u001b[32m1\u001b[39m:\n\u001b[32m 1016\u001b[39m \u001b[38;5;28;01mraise\u001b[39;00m \u001b[38;5;167;01mValueError\u001b[39;00m(\n\u001b[32m 1017\u001b[39m \u001b[33m\"\u001b[39m\u001b[33mNo fits were performed. \u001b[39m\u001b[33m\"\u001b[39m\n\u001b[32m 1018\u001b[39m \u001b[33m\"\u001b[39m\u0Line truncated
"\u001b[36mFile \u001b[39m\u001b[32m~/Documents/MLP/Projects/MLPproject/.venv/lib64/python3.13/site-packages/sklearn/utils/parallel.py:82\u001b[39m, in \u001b[36mParallel.__call__\u001b[39m\u001b[34m(self, iterable)\u001b[39m\n\u001b[32m 73\u001b[39m warning_filters = warnings.filters\n\u001b[32m 74\u001b[39m iterable_with_config_and_warning_filters = (\n\u001b[32m 75\u001b[39m (\n\u001b[32m 76\u001b[39m _with_config_and_warning_filters(delayed_func, config, warning_filters),\n\u001b[32m (...)\u001b[39m\u001b[32m 80\u001b[39m \u001b[38;5;28;01mfor\u001b[39;00m delayed_func, args, kwargs \u001b[38;5;129;01min\u001b[39;00m iterable\n\u001b[32m 81\u001b[39m )\n\u001b[32m---> \u001b[39m\u001b[32m82\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m \u001b[38;5;28;43msuper\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43m)\u001b[49m\u001b[43m.\u001b[49m\u001b[34;43m__call__\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43miterable_with_config_and_warning_filters\u001b[49m\u001b[43m)\u001b[49m\n",
"\u001b[36mFile \u001b[39m\u001b[32m~/Documents/MLP/Projects/MLPproject/.venv/lib64/python3.13/site-packages/joblib/parallel.py:2072\u001b[39m, in \u001b[36mParallel.__call__\u001b[39m\u001b[34m(self, iterable)\u001b[39m\n\u001b[32m 2066\u001b[39m \u001b[38;5;66;03m# The first item from the output is blank, but it makes the interpreter\u001b[39;00m\n\u001b[32m 2067\u001b[39m \u001b[38;5;66;03m# progress until it enters the Try/Except block of the generator and\u001b[39;00m\n\u001b[32m 2068\u001b[39m \u001b[38;5;66;03m# reaches the first `yield` statement. This starts the asynchronous\u001b[39;00m\n\u001b[32m 2069\u001b[39m \u001b[38;5;66;03m# dispatch of the tasks to the workers.\u001b[39;00m\n\u001b[32m 2070\u001b[39m \u001b[38;5;28mnext\u001b[39m(output)\n\u001b[32m-> \u001b[39m\u001b[32m2072\u001b[39m \u001b[38;5;28;01mreturn\u001b[39;00m output \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m.return_generator \u001b[38;5;28;01melse\u001b[39;00m \u001b[38;5;28;43mlist\u001b[39;49m\u001b[43m(\u001b[49m\u001b[43moutput\u001b[49m\u001b[43m)\u001b[49m\n",
"\u001b[36mFile \u001b[39m\u001b[32m~/Documents/MLP/Projects/MLPproject/.venv/lib64/python3.13/site-packages/joblib/parallel.py:1682\u001b[39m, in \u001b[36mParallel._get_outputs\u001b[39m\u001b[34m(self, iterator, pre_dispatch)\u001b[39m\n\u001b[32m 1679\u001b[39m \u001b[38;5;28;01myield\u001b[39;00m\n\u001b[32m 1681\u001b[39m \u001b[38;5;28;01mwith\u001b[39;00m \u001b[38;5;28mself\u001b[39m._backend.retrieval_context():\n\u001b[32m-> \u001b[39m\u001b[32m1682\u001b[39m \u001b[38;5;28;01myield from\u001b[39;00m \u001b[38;5;28mself\u001b[39m._retrieve()\n\u001b[32m 1684\u001b[39m \u001b[38;5;28;01mexcept\u001b[39;00m \u001b[38;5;167;01mGeneratorExit\u001b[39;00m:\n\u001b[32m 1685\u001b[39m \u001b[38;5;66;03m# The generator has been garbage collected before being fully\u001b[39;00m\n\u001b[32m 1686\u001b[39m \u001b[38;5;66;03m# consumed. This aborts the remaining tasks if possible and warn\u001b[39;00m\n\u001b[32m 1687\u001b[39m \u001b[38;5;66;03m# the user if necessary.\u001b[39;00m\n\u001b[32m 1688\u001b[39m \u001b[38;5;28mself\u001b[39m._exception = \u001b[38;5;28;01mTrue\u001b[39;00m\n",
"\u001b[36mFile \u001b[39m\u001b[32m~/Documents/MLP/Projects/MLPproject/.venv/lib64/python3.13/site-packages/joblib/parallel.py:1800\u001b[39m, in \u001b[36mParallel._retrieve\u001b[39m\u001b[34m(self)\u001b[39m\n\u001b[32m 1789\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m \u001b[38;5;28mself\u001b[39m.return_ordered:\n\u001b[32m 1790\u001b[39m \u001b[38;5;66;03m# Case ordered: wait for completion (or error) of the next job\u001b[39;00m\n\u001b[32m 1791\u001b[39m \u001b[38;5;66;03m# that have been dispatched and not retrieved yet. If no job\u001b[39;00m\n\u001b[32m (...)\u001b[39m\u001b[32m 1795\u001b[39m \u001b[38;5;66;03m# control only have to be done on the amount of time the next\u001b[39;00m\n\u001b[32m 1796\u001b[39m \u001b[38;5;66;03m# dispatched job is pending.\u001b[39;00m\n\u001b[32m 1797\u001b[39m \u001b[38;5;28;01mif\u001b[39;00m (nb_jobs == \u001b[32m0\u001b[39m) \u001b[38;5;129;01mor\u001b[39;00m (\n\u001b[32m 1798\u001b[39m \u001b[38;5;28mself\u001b[39m._jobs[\u001b[32m0\u001b[39m].get_status(timeout=\u001b[38;5;28mself\u001b[39m.timeout) == TASK_PENDING\n\u001b[32m 1799\u001b[39m ):\n\u001b[32m-> \u001b[39m\u001b[32m1800\u001b[39m \u001b[43mtime\u001b[49m\u001b[43m.\u001b[49m\u001b[43msleep\u001b[49m\u001b[43m(\u001b[49m\u001b[32;43m0.01\u001b[39;49m\u001b[43m)\u001b[49m\n\u001b[32m 1801\u001b[39m \u001b[38;5;28;01mcontinue\u001b[39;00m\n\u001b[32m 1803\u001b[39m \u001b[38;5;28;01melif\u001b[39;00m nb_jobs == \u001b[32m0\u001b[39m:\n\u001b[32m 1804\u001b[39m \u001b[38;5;66;03m# Case unordered: jobs are added to the list of jobs to\u001b[39;00m\n\u001b[32m 1805\u001b[39m \u001b[38;5;66;03m# retrieve `self._jobs` only once completed or in error, which\u001b[39;00m\n\u001b[32m (...)\u001b[39m\u001b[32m 1811\u001b[39m \u001b[38;5;66;03m# timeouts before any other dispatched job has completed and\u001b[39;00m\n\u001b[32m 1812\u001b[39m \u001b[38;5;66;03m# been added to `self._jobs` to be retrieved.\u001b[39;00m\n",
"\u001b[31mKeyboardInterrupt\u001b[39m: "
"Tuning Decision Tree ...\n",
"Tuning XGBoost ...\n",
"Tuning Random Forest ...\n",
"\n",
"------ Results Sorted by Accuracy ------\n",
" Model Accuracy Precision Recall F1 Score Best Params Training Time (s) Prediction Time (s) Total Time (s)\n",
" XGBoost 0.867893 0.863335 0.867893 0.863496 {'learning_rate': 0.15, 'max_depth': 5, 'n_estimators': 100} 27.545653 0.011063 27.556716\n",
"Random Forest 0.859606 0.854161 0.859606 0.853520 {'max_depth': 15, 'min_samples_split': 8, 'n_estimators': 150} 117.926460 0.047736 117.974195\n",
"Decision Tree 0.848666 0.843533 0.848666 0.837078 {'max_depth': 8, 'min_samples_split': 4} 1.503573 0.004421 1.507995\n"
]
}
],
@@ -504,7 +504,7 @@
},
{
"cell_type": "code",
"execution_count": null,
"execution_count": 28,
"id": "a470fd20",
"metadata": {},
"outputs": [
Report/MLPproject.aux
+1 -1
View File
@@ -57,7 +57,6 @@
\newlabel{rf_metrics@cref}{{[table][4][]4}{[1][4][]4}}
\@writefile{toc}{\contentsline {subsection}{\numberline {5.4}Overfitting and Underfitting}{4}{subsection.5.4}\protected@file@percent }
\@writefile{toc}{\contentsline {subsection}{\numberline {5.5}Feature Importance}{4}{subsection.5.5}\protected@file@percent }
\@writefile{toc}{\contentsline {section}{\numberline {6}Summary}{4}{section.6}\protected@file@percent }
\newlabel{fig:featureImportanceDT}{{2(a)}{5}{\relax }{figure.caption.6}{}}
\newlabel{fig:featureImportanceDT@cref}{{[subfigure][1][2]2(a)}{[1][4][]5}}
\newlabel{sub@fig:featureImportanceDT}{{(a)}{5}{\relax }{figure.caption.6}{}}
@@ -69,6 +68,7 @@
\@writefile{lof}{\contentsline {figure}{\numberline {2}{\ignorespaces The feature importance graphs for the Decision Tree model and the Random Forest model based on the validation data.\relax }}{5}{figure.caption.6}\protected@file@percent }
\newlabel{fig:}{{2}{5}{The feature importance graphs for the Decision Tree model and the Random Forest model based on the validation data.\relax }{figure.caption.6}{}}
\newlabel{fig:@cref}{{[figure][2][]2}{[1][4][]5}}
\@writefile{toc}{\contentsline {section}{\numberline {6}Summary}{5}{section.6}\protected@file@percent }
\ttl@finishall
\newlabel{LastPage}{{}{5}{}{page.5}{}}
\xdef\lastpage@lastpage{5}
Report/MLPproject.fdb_latexmk
+6 -6
View File
@@ -1,6 +1,6 @@
# Fdb version 4
["pdflatex"] 1761920867.71702 "/home/petrus/Documents/MLP/Projects/MLPproject/Report/MLPproject.tex" "MLPproject.pdf" "MLPproject" 1761920868.83079 0
"/home/petrus/Documents/MLP/Projects/MLPproject/Report/MLPproject.tex" 1761920867.50658 24533 58032bad0234d994ba6556d7acc5212e ""
["pdflatex"] 1761921952.93435 "/home/petrus/Documents/MLP/Projects/MLPproject/Report/MLPproject.tex" "MLPproject.pdf" "MLPproject" 1761921953.99247 0
"/home/petrus/Documents/MLP/Projects/MLPproject/Report/MLPproject.tex" 1761921952.70534 24812 36992a9467feb6ff9f9f97a89afe5aee ""
"/usr/share/texlive/texmf-dist/fonts/enc/dvips/base/8r.enc" 1737590400 4850 80dc9bab7f31fb78a000ccfed0e27cab ""
"/usr/share/texlive/texmf-dist/fonts/map/fontname/texfonts.map" 1577235249 3524 cb3e574dea2d1052e39280babc910dc8 ""
"/usr/share/texlive/texmf-dist/fonts/tfm/adobe/helvetic/phvb7t.tfm" 1136768653 2240 eb56c13537f4d8a0bd3fafc25572b1bd ""
@@ -131,10 +131,10 @@
"/var/lib/texmf/web2c/pdftex/pdflatex.fmt" 1761127067 7753793 c9f4d2c19ab997188c605d7179b0cdc0 ""
"CM_dt.png" 1761920482.34887 97023 ce9f07bdb4551ffd7f80782b99a54328 ""
"CM_rf.png" 1761920484.96582 98726 a24b8d53317f0e7e65e41ed83ef8fae5 ""
"MLPproject.aux" 1761920868.72356 6698 e699ab45a2056e84f281588212bdf2ec "pdflatex"
"MLPproject.out" 1761920868.72456 3113 d57c5f2b0e6699323b0a2645b9706cce "pdflatex"
"MLPproject.tex" 1761920867.50658 24533 58032bad0234d994ba6556d7acc5212e ""
"MLPproject.toc" 1761920868.72456 1587 d275c5e85ba45c005c3baf7931c510a7 "pdflatex"
"MLPproject.aux" 1761921953.8967 6698 d2e044226fe88697053e22eec695a818 "pdflatex"
"MLPproject.out" 1761921953.8997 3113 d57c5f2b0e6699323b0a2645b9706cce "pdflatex"
"MLPproject.tex" 1761921952.70534 24812 36992a9467feb6ff9f9f97a89afe5aee ""
"MLPproject.toc" 1761921953.8997 1587 6a7d8c5cbfca28921bcd78f124e2ec7a "pdflatex"
"SelfArx.cls" 1761125830.98333 7316 506603b27aab6da8087bc0f1ee693041 ""
"featureImportanceDT.png" 1761403205.10917 60078 4a2e56e2a45ae2ae5e41b9830c1bbcea ""
"featureImportanceRF.png" 1761403205.11075 61794 6b3eefc625dd3da8a3dbf302174c614c ""
Report/MLPproject.log
+3 -3
View File
@@ -1,4 +1,4 @@
This is pdfTeX, Version 3.141592653-2.6-1.40.25 (TeX Live 2023) (preloaded format=pdflatex 2025.10.22) 31 OCT 2025 15:27
This is pdfTeX, Version 3.141592653-2.6-1.40.25 (TeX Live 2023) (preloaded format=pdflatex 2025.10.22) 31 OCT 2025 15:45
entering extended mode
restricted \write18 enabled.
file:line:error style messages enabled.
@@ -726,9 +726,9 @@ Here is how much of TeX's memory you used:
38909 multiletter control sequences out of 15000+600000
569401 words of font info for 297 fonts, out of 8000000 for 9000
1137 hyphenation exceptions out of 8191
75i,12n,77p,1656b,605s stack positions out of 10000i,1000n,20000p,200000b,200000s
75i,12n,77p,1812b,605s stack positions out of 10000i,1000n,20000p,200000b,200000s
</usr/share/texlive/texmf-dist/fonts/type1/urw/helvetic/uhvb8a.pfb></usr/share/texlive/texmf-dist/fonts/type1/urw/helvetic/uhvr8a.pfb></usr/share/texlive/texmf-dist/fonts/type1/urw/helvetic/uhvro8a.pfb></usr/share/texlive/texmf-dist/fonts/type1/urw/times/utmr8a.pfb>
Output written on MLPproject.pdf (5 pages, 305630 bytes).
Output written on MLPproject.pdf (5 pages, 305913 bytes).
PDF statistics:
191 PDF objects out of 1000 (max. 8388607)
148 compressed objects within 2 object streams
Report/MLPproject.pdf
BIN
View File
Binary file not shown.
Report/MLPproject.synctex.gz
BIN
View File
Binary file not shown.
Report/MLPproject.tex
+1 -1
View File
@@ -197,7 +197,7 @@ Table (\ref{dt_metrics}) and (\ref{rf_metrics}) shows the class-wise metrics of
At a first glance at both the confusion matricies and the performance metrics the models do not look to be that good. But what has to be considered is the data that we are analyzing. We are looking at what possible indicators there are for a person to earn more than a certain amount of money. This is real world data and in the real world there is a lot of unique ways of earning money. While there certainly are some indicators that will clearly tell that somebody is earning a lot of money, there are other factors that are not as telling. This means that some features are less important than others. This can be seen in our models in the feature importance graphs in figure(\ref{fig:featureImportanceDT}) and (\ref{fig:featureImportanceRF}). This also means that there will be plenty of outliers in the data. No matter how good the model is, it cannot possibly catch all of these outliers. If it did it would be overfitted. We simply cannot expect a model to have very good accuracy on this type of data set.
An important thing to touch on is the poor fit on higher-earning people by our model. We see that both models produce a precision of 77\% on the lower-earning individuals, which is quite bad compared to the precision of 87\% and 89\% on the higher-earning individuals. This means that out of all individuals predicted as higher-earning, only 77\% are correctly predicted. Even more notably, there is a very big discrepancy on the recall between the two classes. Recalls of 56\% and 63\% for the higher-earning class compared to 95\% and 94\% shows that out of all the higher-earning individuals, the models are not good at correctly detecting them as higher-earning. As we talked about above there may be many reasons for this poor fit. Of note is that we have optimized this model to find the best accuracy on all data point. We therefore stride to classify as many total data points correctly as possible and not on getting the best average for the classes separetly. Since there are more lower-earning people in our dataset it is very reasonable for the model to have optimised for that as well since it gives the best weighted accuracy. As previosly stated, the scoring metrics used for training the models should be adapted based on the problem at hand. If the problem requires similiar metrics across the classes, one should instead consider using scoring metrics such as balanced accuracy score, which are adapted to produce such results.
An important thing to touch on is the poor fit on higher-earning people by our model. We see that both models produce a precision of 77\% on the lower-earning individuals, which is quite bad compared to the precision of 87\% and 89\% on the higher-earning individuals. This means that out of all individuals predicted as higher-earning, only 77\% are correctly predicted. Even more notably, there is a very big discrepancy on the recall between the two classes. Recalls of 56\% and 63\% for the higher-earning class compared to 95\% and 94\% shows that out of all the higher-earning individuals, the models are not good at correctly detecting them as higher-earning. Additionally, the F1-score of both classes demonstrates the discrepancy of the overall performances across the classes. It shows that the harmonic average of precision and recall is significantly lower for the lower-earning individuals than for the higher earning individuals. As we talked about above there may be many reasons for this poor fit. Of note is that we have optimized this model to find the best accuracy on all data point. We therefore stride to classify as many total data points correctly as possible and not on getting the best average for the classes separetly. Since there are more lower-earning people in our dataset it is very reasonable for the model to have optimised for that as well since it gives the best weighted accuracy. As previosly stated, the scoring metrics used for training the models should be adapted based on the problem at hand. If the problem requires similiar metrics across the classes, one should instead consider using scoring metrics such as balanced accuracy score, which are adapted to produce such results.
\subsection{Overfitting and Underfitting}
We spent some time tuning the hyperparameters to ensure that we did not overfit. If we compare the validation results with the test results we see that the performance metrics do not change much at all. This is what we want to see as this means that we have avoidede overfitting the model. This means that our model could be used on other similar datasets and hopefully give similar perfomances. We also do not want our model to be underfit. This is a bit harder to validate as we want the errors to be as small as possible for both training and testing and as we stated before I believe that this is a difficult dataaset to get a great fit to. Therefore we believe that we have found a model that has a decent enough balance between bias and variance.
Report/MLPproject.toc
+1 -1
View File
@@ -16,5 +16,5 @@
\contentsline {subsection}{\numberline {5.3}Analyzing the Performance}{4}{subsection.5.3}%
\contentsline {subsection}{\numberline {5.4}Overfitting and Underfitting}{4}{subsection.5.4}%
\contentsline {subsection}{\numberline {5.5}Feature Importance}{4}{subsection.5.5}%
\contentsline {section}{\numberline {6}Summary}{4}{section.6}%
\contentsline {section}{\numberline {6}Summary}{5}{section.6}%
\contentsfinish
decision_tree.pdf
BIN
View File
Binary file not shown.