finished multitask comments
"cell_type": "markdown",
"metadata": {},
"source": [
- "This notebook demonstrates how to perform multitask regression with kernels.IndexKernel. Multitask regression, first introduced in [this paper](https://papers.nips.cc/paper/3189-multi-task-gaussian-process-prediction.pdf) learns similarities in the outputs simultaneously. It's useful when you are performing regression on multiple functions that share similarities (such as being sinusoidial). "
+ "This notebook demonstrates how to perform multitask regression with kernels.IndexKernel. Multitask regression, first introduced in [this paper](https://papers.nips.cc/paper/3189-multi-task-gaussian-process-prediction.pdf) learns similarities in the outputs simultaneously. It's useful when you are performing regression on multiple functions that share the same inputs, especially if they have similarities (such as being sinusodial). \n",
+ "\n",
+ "Multitask regression on `n` tasks essentially learns `n^2` tasks (here $2\\cdot2=4$). The extra tasks are learning the cross correlations between outputs. This allows for more accurate predictions on both tasks."
]
},
{
"outputs": [],
"source": [
"from torch.autograd import Variable\n",
- "# Training points are every 0.1 in [0,1]\n",
+ "# Training points are every 0.1 in [0,1] (note that they're the same for both tasks)\n",
"train_x = Variable(torch.linspace(0, 1, 11))\n",
"# y1s are indexed 0, y2s are indexed 1\n",
"y1_inds = Variable(torch.zeros(11).long())\n",
" self.mean_module = ConstantMean()\n",
" # We use the very common RBF kernel\n",
" self.covar_module = RBFKernel(log_lengthscale_bounds=(-3, 3))\n",
- " ## Need explanation from Geoff on this\n",
+ " # We learn an IndexKernel for 2 tasks\n",
+ " # (so we'll actually learn 2x2=4 tasks with correlations)\n",
" self.task_covar_module = IndexKernel(n_tasks=2, rank=1)\n",
"\n",
" def forward(self,x,i):\n",
" # Get predictive mean\n",
" mean_x = self.mean_module(x)\n",
- " ## Get base covariance (?)\n",
+ " # Get all covariances, we'll look up the task-speicific ones\n",
" covar_x = self.covar_module(x)\n",
- " ## Make indexed operation on it to get different tasks (?)\n",
+ " # # Get the covariance for task i\n",
" covar_i = self.task_covar_module(i)\n",
" covar_xi = covar_x.mul(covar_i)\n",
" return GaussianRandomVariable(mean_x, covar_xi)\n",
