Tutorial 2 - Uploading files to the Blackfynn platform

This tutorial describes how to upload files to the Blackfynn platform using the Blackfynn Agent. It assumes that the user has a Blackfynn account and has setup his/her password. It also assumes the user has configured the agent using the method described in tutorial 1.

Creating a dataset

All files on the Blackfynn platform belong to a dataset. Therefore, we first need to create a dataset before we can upload files to the platform. You create a dataset using the init command.

$ blackfynn_agent init newDataset 'This is a test dataset for tutorial 2.'

Created dataset newDataset (N:dataset:xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx)

The dataset is now created. To verify, you can open the web application and see the dataset listed in your account. you can also list all datasets using the Agent with the datasets command.

$ blackfynn_agent datasets

+-----------------------------+----------------------+
| DATASET                     | NAME                 |
+-----------------------------+----------------------+
| N:dataset:xxxxxxxx-xxxx-xx  | newDataset           |

Uploading data

Now, let’s upload some files to the dataset we just created. The upload command has multiple options to specify exactly which files should be uploaded. Some examples are provided below, but be sure to check out the documentation on this command for a complete view of its capabilities.

Examples

  • Upload a single image file from the ~/Desktop/ folder to the “newDataset” dataset.
$ blackfynn_agent upload --dataset newDataset ~/Desktop/image.jpg
  • Upload all files in the ~/Desktop/ folder to the “newDataset” dataset. This command will not recurse into subdirectories, it will only upload files that live in ~/Desktop/.
$ blackfynn_agent upload --dataset newDataset ~/Desktop/
  • Upload all files in the ~/Desktop/ folder and all files in all subfolders of ~/Desktop/ using the --recursive flag. This flag will also preserve the folder structure in the Blackfynn platform.
$ blackfynn_agent upload --dataset newDataset --recursive ~/Desktop/
  • Apply filters to the files that are uploaded when uploading from a directory using the --include and --exclude flags. These flags support globbing syntax. The following command will only upload DICOM images (.dcm files) from the Desktop folder.
$ blackfynn_agent upload --dataset newDataset --include='*.dcm' ~/Desktop/
  • Pass in a list of files to be uploaded. These files will be uploaded to the root collection in the “newDataset” dataset on the Blackfynn platform. When passing in more than one argument to the upload command, all arguments must be files (directories are not supported using this syntax).
$ blackfynn_agent upload --dataset newDataset ~/Desktop/image.jpg ~/Desktop/subfolder/image2.jpg