add timings note
" print(ct_uid)"
]
},
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "## Run timings to compare the numpy generator pulling from cache to the tfrecord dataset\n",
+ "\n",
+ "While the numpy generator is super flexible, able to be generated a fresh from a set of DICOM\n",
+ "files and it will fall back to creating the mask if a cache doesn't exist it is about 50%\n",
+ "slower than the tfrecord dataset.\n",
+ "\n",
+ "Nevertheless, both the tfrecord dataset and the number generator dataset take on the order of\n",
+ "10 ms / item for this example with 5 masks.\n",
+ "\n",
+ "At that time cost, there doesn't seem to be a massive advantage for using the tfrecord.\n",
+ "The trade off in loss of flexibility doesn't appear to be worth it.\n",
+ "\n",
+ "Nevertheless, there may be a significant time difference should the numpy generator need to\n",
+ "copy the data onto the GPU. Potentially the tfrecord dataset may just load the data directly\n",
+ "onto the GPU saving a copy step. Will need to determine experimentally which dataset provides\n",
+ "the best development flexibility / training time trade off."
+ ]
+ },
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
- "source": []
+ "source": [
+ "%%timeit\n",
+ "\n",
+ "# Numpy generator dataset\n",
+ "for ct_uid, x_grid, y_grid, input_array, output_array in dataset.take(15):\n",
+ " pass"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "%%timeit\n",
+ "\n",
+ "# tfrecord dataset\n",
+ "for ct_uid, x_grid, y_grid, input_array, output_array in loaded_dataset.take(15):\n",
+ " pass"
+ ]
}
],
"metadata": {
