Add instruction for installing s2s-ft
"metadata": {},
"source": [
"## Before you start\n",
- "Set `QUICK_RUN = True` to run the notebook on a small subset of data and a smaller number of steps"
+ "Set `QUICK_RUN = True` to run the notebook on a small subset of data and a smaller number of steps. If `QUICK_RUN = False`, the notebook takes about 10 hours to run on a VM with 4 16GB NVIDIA V100 GPUs. "
]
},
{
"metadata": {},
"outputs": [],
"source": [
- "QUICK_RUN = False"
+ "QUICK_RUN = True"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Manually install the s2s-ft package from the [UniLM repo](https://github.com/microsoft/unilm) by running the following commands in your bash shell. \n",
+ "(We are working on making the s2s-ft package available on PyPI and this step will not be necessary in the future. )\n",
+ "\n",
+ "```\n",
+ "conda activate nlp_gpu\n",
+ "git clone https://github.com/microsoft/unilm.git\n",
+ "cd unilm/s2s-ft\n",
+ "pip install -e .\n",
+ "```"
]
},
{
"BEAM_SIZE = 5\n",
"FORBID_IGNORE_WORD = \".\"\n",
"\n",
- "# mixed precision setting\n",
+ "# mixed precision setting. To enable mixed precision training, follow instructions in SETUP.md. \n",
"FP16 = False"
]
},
"## Distributed training with DistributedDataParallel (DDP)\n",
"This notebook uses DataParallel for multi-GPU training by default. In general, DistributedDataParallel(DDP) is recommended because of its better performance. See details [here](https://pytorch.org/tutorials/intermediate/ddp_tutorial.html). \n",
"Since DDP requires multiprocess and can not be run from the notebook, we provide a python script [abstractive_summarization_unilm_cnndm.py](./abstractive_summarization_unilm_cnndm.py) to demonstrate how to use DDP. \n",
- "**Note that the python script set `FP16=True` by default. If you don't have apex installed as instructed above, set `FP16=False` instead.**"
+ "**Note that the python script set `fp16=False` by default. If you have enabled mixed precision training following the instructions in SETUP.md, you can call the script with an additional argument \"--fp16 true\".** "
]
}
],
"metadata": {
"kernelspec": {
- "display_name": "nlp_gpu",
+ "display_name": "nlp_gpu_test_new",
"language": "python",
- "name": "nlp_gpu"
+ "name": "nlp_gpu_test_new"
},
"language_info": {
"codemirror_mode": {
