-
Notifications
You must be signed in to change notification settings - Fork 25
Uploading Files Using Gsutil Tool
Our wizard is useful if you want to upload a couple, small files into the portal. Often primary data (eg. Fastq files) are many files or big files. To upload fastq files, it is easier to use more appropriate tools like the gsutil tool (a part of the Google Cloud SDK). This tool allows you to upload files directly into your study in the cloud. This tool does require some use of command line, but we are happy to help you get acclimated to the process. Let's get started!
-
Download Google Cloud SDK (which contains the Gsutil Tool). The following are the official install instructions for several operating systems. If you do not have administrative rights to your system, we found the following document's instructions for installing from a tar archive very useful. If you use these instructions, step 3 and 4 are optional.
-
Have your data (eg.fastq.gz files) in a directory you can access with a command line console (terminal program on a Mac).
-
Get your bucket id from the portal team. Email us at single_cell_portal@broadinstitute.org and we will set up a bucket for you!
- Move to your directory.
cd /the/path/to/your/directory/holding/your/data
- Authenticate to get access to the portal project.
gcloud init
-
Type 'Y' to login.
-
If you are on your machine a page will be opened in your browser. If you are in a terminal on a server a URL will be given to open in a browser (paste this URL in a browser).
-
Log in with the google account you use with the portal.
-
Copy the code and paste it in your terminal.
-
Select the project 'single-cell-portal'
-
Copy a file to the bucket
gsutil cp file.fastq.gz gs://bucket_id
- Or copy all compressed fastq files to the bucket.
gsutil cp *.fastq.gz gs://bucket_id
- Done! Let us know you are complete and we will attach your data to the portal study.
- If you get a syntax error for a *.py file on the gcloud init command, set up your system to point to a version of python 2.7.X. Use the following command with the path to your python install and try again.
export CLOUDSDK_PYTHON=/path/to/your/python_2.7.1-sqlite3-rtrees/bin/python