add uploading files in default ds in section 1.4
"## 1.4 Upload to Azure Blob Storage\n",
"We make the data accessible remotely by uploading that data from your local machine into Azure. Then it can be accessed for remote training. The datastore is a convenient construct associated with your workspace for you to upload or download data. You can also interact with it from your remote compute targets. It's backed by an Azure Blob storage account.\n",
"\n",
- "**Note: User needs to upload all the files under `data_folder` MANUALLY to Azure Blob storage account for now, because the uploading function has bugs on Azure Blob.**"
+ "**Note: If you already has all the files under `clean/snli_1.0/` in your default datastorage, you DO NOT need to redo this section.**"
]
},
{
"metadata": {},
"outputs": [],
"source": [
- "data_folder = os.path.join(BASE_DATA_PATH, \"clean/snli_1.0/\")"
+ "data_folder = os.path.join(BASE_DATA_PATH, \"clean/snli_1.0/\")\n",
+ "ds = ws.get_default_datastore()\n",
+ "print(ds.datastore_type, ds.account_name, ds.container_name, data_folder)\n",
+ "\n",
+ "ds.upload(src_dir=data_folder, target_path='data/preprocessed', overwrite=True, show_progress=True)"
]
},
{
