Minor edits to svi_part_ii
"So far so good. Pyro can now leverage the conditional indendency of the observations given the latent random variable. But how this does actually work? Basically `pyro.irange` is implemented using a context manager. At every execution of the body of the `for` loop we enter a new (conditional) independence context which is then exited at the end of the `for` loop body. Let's be very explicit about this: \n",
"\n",
"- because each `pyro.observe` statement occurs within a different execution of the body of the `for` loop, Pyro marks each observation as independent\n",
- "- that this independence is properly a _conditional_ independence _given_ `latent_fairness` follows because `latent_fairness` is sampled _outside_ of the context of `data_loop`.\n",
+ "- this independence is properly a _conditional_ independence _given_ `latent_fairness`, because `latent_fairness` is sampled _outside_ of the context of `data_loop`.\n",
"\n",
"Before moving on, let's mention some gotchas to be avoided when using `irange`. Consider the following variant of the above code snippet:"
]
},
"outputs": [],
"source": [
+ "# WARNING do not do this!\n",
"my_reified_list = list(pyro.irange(\"data_loop\", len(data)))\n",
"for i in my_reified_list: \n",
" pyro.observe(\"obs_{}\".format(i), dist.bernoulli, data[i], f)"
"cell_type": "markdown",
"metadata": {},
"source": [
- "This will _not_ achieve the desired behavior, since `list()` will enter and exit the `data_loop` context completely before a single `pyro.observe` statement is called. Similarly, the user needs to take care not to leak mutable computations across the boundary of the context manager, as this may lead to subtle bugs.\n",
+ "This will _not_ achieve the desired behavior, since `list()` will enter and exit the `data_loop` context completely before a single `pyro.observe` statement is called. Similarly, the user needs to take care not to leak mutable computations across the boundary of the context manager, as this may lead to subtle bugs. For example, `pyro.irange` is not appropriate for temporal models where each iteration of a loop depdends on a previous iteration; in this case a `range` should be used instead.\n",
"\n",
"## `iarange`\n",
"\n",
- "Conceptually `iarange` is the same as `irange` except that it is a vectorized operation. As such it potentially enables large speed-ups compared to the explicit `for` loop that appears with `irange`. Let's see how this looks for our running example. First we need `data` to be in the form of a tensor:"
+ "Conceptually `iarange` is the same as `irange` except that it is a vectorized operation (as `torch.arange` is to `range`). As such it potentially enables large speed-ups compared to the explicit `for` loop that appears with `irange`. Let's see how this looks for our running example. First we need `data` to be in the form of a tensor:"
]
},
{
"source": [
"Importantly, `iarange` now returns a tensor of indices `ind`, which, in this case will be of length 5. Note that in addition to the argument `subsample_size` we also pass the argument `size` so that `iarange` is aware of the full size of the tensor `data` so that it can compute the correct scaling factor. Just like for `irange`, the user is responsible for selecting the correct datapoints using the indices provided by `iarange`. \n",
"\n",
- "Finally, note that the user must pass the argument `use_cuda=True` to `irange` or `iarange` if `data` is on the GPU. [**IS THIS REMARK ON THE MARK?**]\n",
+ "Finally, note that the user must pass the argument `use_cuda=True` to `irange` or `iarange` if `data` is on the GPU.\n",
"\n",
"### Custom subsampling strategies with `irange` and `iarange`\n",
"\n",
