add find the best model in tuning and fix typos
"3. [Tune Model Hyperparameters](#3-Tune-Model-Hyperparameters)\n",
" * 3.1 [Start a Hyperparameter Sweep](#3.1-Start-a-Hyperparameter-Sweep)\n",
" * 3.2 [Monitor HyperDrive runs](#3.2-Monitor-HyperDrive-runs)\n",
+ " * 3.3 [Find the Best Model](#3.3-Find-the-Best-Model)\n",
"- [References](#References)"
]
},
"```"
]
},
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "The training process follows the steps:\n",
+ "1. Create or load the dataset vocabulary\n",
+ "2. Train on the training dataset for each batch epoch (batch size = 48 updates)\n",
+ "3. Evaluate on the validation dataset for every 10 epochs\n",
+ "4. Find the local minimum point on validation loss\n",
+ "5. Save the best model and stop the training process"
+ ]
+ },
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Interpret the Training Results**\n",
"\n",
- "The training process follows the steps:\n",
- "1. Create or load the dataset vocabulary\n",
- "2. Train on the training dataset for each batch epoch (batch size = 48 updates)\n",
- "3. Evaluate on the validation dataset for every 10 epochs\n",
- "4. Find the local minimum point on validation loss\n",
- "5. Save the best model and stop the training process\n",
- "\n",
- "The following chart is the results with different nodes with AmlCompute:\n",
+ "The following chart shows the model validation loss (the less loss, the better performance) with different nodes with AmlCompute:\n",
"\n",
"| Standard_NC6 | AML_1node | AML_2nodes | AML_4nodes | AML_8nodes |\n",
"| --- | --- | --- | --- | --- |\n",
- "| Best_val_loss | 4.81 | 4.78 | 4.77 | 4.58 |\n"
+ "| Best_val_loss | 4.81 | 4.78 | 4.77 | 4.58 |\n",
+ "\n",
+ "From the chart, we can tell training with more nodes, the performance is getting better with lower loss."
]
},
{
" ```"
]
},
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## 3.3 Find the Best Model"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Once all the runs complete, we can find the run that produced the model with the lowest loss."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "best_run = hyperdrive_run.get_best_run_by_primary_metric()\n",
+ "best_run_metrics = best_run.get_metrics()\n",
+ "print(best_run)"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "print('Best Run is:\\n Validation loss: {0:.5f} \\n Learning rate: {1:.5f} \\n'.format(\n",
+ " best_run_metrics['best_val_loss'][-1],\n",
+ " best_run_metrics['lr'])\n",
+ " )"
+ ]
+ },
{
"cell_type": "markdown",
"metadata": {},
