Fix comments in notebook
"\\leftarrow\n",
"\\begin{bmatrix} \\mathbf m \\\\ \\mathbf S \\end{bmatrix}\n",
"-\n",
- "\\alpha \\mathcal{\\mathbf F}^{-1} \\begin{bmatrix} \\nabla \\mathbf m \\\\ \\nabla \\mathbf S \\end{bmatrix}\n",
+ "\\alpha \\mathcal{\\mathbf F}^{-1} \\nabla \\begin{bmatrix} \\mathbf m \\\\ \\mathbf S \\end{bmatrix}\n",
"\\end{align}\n",
"\n",
"where $\\alpha$ is a step size and $\\mathbf F$ is the **Fisher information matrix** corresponding to this distribution. This is known as **natural gradient descent**, or **NGD**.\n",
" output = model(x_batch)\n",
" loss = -mll(output, y_batch)\n",
" minibatch_iter.set_postfix(loss=loss.item())\n",
- "\n",
- " # Multiply the loss by the amount of data in order to take the appropriate step size!\n",
" loss.backward()\n",
" variational_ngd_optimizer.step()\n",
" \n",
