fixed lists in the markdown --> HTML conversion
"These choices reproduce the structure of model M2 and its corresponding inference network in reference [1].\n",
"\n",
"We translate this model and guide pair into Pyro code below. Note that:\n",
- "1. The labels `ys`, which are represented with a one-hot encoding, are only partially observed (`None` denotes unobserved values).\n",
- "2. `model()` handles both the observed and unobserved case.\n",
- "3. The code assumes that `xs` and `ys` are mini-batches of images and labels, respectively, with the size of each batch denoted by `batch_size`. "
+ "\n",
+ "- The labels `ys`, which are represented with a one-hot encoding, are only partially observed (`None` denotes unobserved values).\n",
+ "\n",
+ "- `model()` handles both the observed and unobserved case.\n",
+ "\n",
+ "- The code assumes that `xs` and `ys` are mini-batches of images and labels, respectively, with the size of each batch denoted by `batch_size`. "
]
},
{
"cell_type": "code",
- "execution_count": 1,
+ "execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
},
{
"cell_type": "code",
- "execution_count": 2,
+ "execution_count": 3,
"metadata": {},
"outputs": [],
"source": [
"where $\\tilde{p_l}({\\bf x,y})$ is the empirical distribution over the labeled (or supervised) data and $\\alpha' \\equiv \\frac{\\alpha}{|\\mathcal{D}_{supervised}|}$. Note that we've introduced an arbitrary hyperparameter $\\alpha$ that modulates the importance of the new term.\n",
"\n",
"To learn using this modified objective in Pyro we do the following: \n",
- "1. We use a new model and guide pair (see the code snippet below) that corresponds to scoring the observed label ${\\bf y}$ for a given image ${\\bf x}$ against the predictive distribution $q_{\\phi}({\\bf y}~|~ {\\bf x})$ \n",
- "2. We specify the scaling factor $\\alpha'$ (`aux_loss_multiplier` in the code) in the `pyro.observe` call by making use of the `log_pdf_mask` argument. Note that `log_pdf_mask` was used to similar effect in the [Deep Markov Model](http://pyro.ai/examples/dmm.html) to implement KL annealing.\n",
- "3. We create a new `SVI` object and use it to take gradient steps on the new objective term"
+ "\n",
+ "- We use a new model and guide pair (see the code snippet below) that corresponds to scoring the observed label ${\\bf y}$ for a given image ${\\bf x}$ against the predictive distribution $q_{\\phi}({\\bf y}~|~ {\\bf x})$ \n",
+ "\n",
+ "- We specify the scaling factor $\\alpha'$ (`aux_loss_multiplier` in the code) in the `pyro.observe` call by making use of the `log_pdf_mask` argument. Note that `log_pdf_mask` was used to similar effect in the [Deep Markov Model](http://pyro.ai/examples/dmm.html) to implement KL annealing.\n",
+ "\n",
+ "- We create a new `SVI` object and use it to take gradient steps on the new objective term"
]
},
{
"cell_type": "code",
- "execution_count": 3,
+ "execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
