Globus is the preferred data transfer tool for use when doing large scale data transfers with external partners/research organisations. Both Research Data Store and CaStLeS storage are accessible using Globus and any researcher registered on a BEAR project with storage should automatically have access to Globus for data transfers.

Whilst Globus can be used to transfer data to other Data Transfer Nodes, this guide is focussed on getting started using the BEAR Globus endpoint.

In addition to using the Globus web application, the command line tools are also available on BlueBEAR login nodes.

Getting started using Globus

Globus is a data transfer tool which allows large amounts of data to be scheduled for transfer and is typically used for moving data between research organisations. As part of the BEAR services, we now offer a Globus endpoint/data transfer node (DTN) which enables BEAR storage to be accessed via Globus services. When the data transfer occurs, unlike a drag and drop in e.g. Windows, the transfer takes place directly between two Globus DTN.  Parallel streams are used to make the data transfer significantly faster and ensures the transfer is completed successfully in the background.

In order to access the DTN, you will need to be registered on a BEAR project. Globus works by creating data transfer control channels via the web services, which directly connect the DTN systems to perform the data transfer reliably and in the background. If you are doing a large transfer, it may take some time to complete, however closing the browser etc. will not stop the transfer as this happens directly between the DTN systems.

Logging In

The login process uses the University of Birmingham’s “shibboleth” single sign-on service and is connected to Globus via a broker called CILogon. When you access the service, you should only enter your University username and password at the University single sign on portal, all authentication beyond this is done using secure tokens passed by your web-browser. When you first use Globus, you will be prompted to authorise passing of these credentials, which will create and link a Globus account to your University account.

To access Globus, you should browse to https://app.globus.org

First time login

The steps below show you how to login to Globus for the first time and how to perform this account linking.  When visiting the Globus site, you will be asked to choose how to login:

Search for “University of Birmingham” in the “Look-up your organisation” box:

Please ensure that you select “University of Birmingham” and NOT “University of Birmingham Development IDP”. Globus uses federated authentication via a broker called CILogon, when you click “Continue” you will be redirected via CILogon to the University’s SSO web page:

Use your normal University username and password (ADF), you will then be redirected back to the Globus website:

If you have previously used Globus at another organisation then you can “Link to an existing account”, otherwise just click the “Continue” button. In order for you to use Globus, you need to confirm you are happy for your email address etc. to be passed to Globus to create an account:

You can opt in to receive Globus email updates if you wish. Once you click continue you will see:

You need to click Allow in order to give Globus permission to transfer data files. Once you have logged in for the first time, you will not need to link your account again and usually the web browser will remember your preferred organisation to sign in via.

Accessing the BEAR endpoint

To transfer data to/from Research Data Store/CaStLeS storage, you need to connect to the “University of Birmingham BEAR” endpoint. Once logged into Globus, select the “File Manager” icon:

In the “Collection” box start search for “bear”:

In this example, you can see “University of Birmingham BEAR” as the second item found, when you click on this endpoint, you should automatically be connected to the BEAR endpoint. By default this shows your BlueBEAR home directory (this may be empty if you are only a user of RDS storage):

To access other parts of the storage, you can type the path to your RDS folder in the “Path” box, e.g. “/rds/projects/2015/XXX”, note that this is the “BEAR Linux path” rather than the path you might use from Windows or a Mac. You can check what access to projects you have and the find the “BEAR Linux path” by logging into https://bearadmin.bham.ac.uk/ and then clicking on the relevant project code.

Transferring data using Globus

When you transfer data using the Globus web application, this is a scheduled data transfer that happens in the background. You can transfer data both from and to Birmingham’s DTN. To start a data transfer, select the File Manager in the Globus web application and connect to the BEAR DTN. To the right of the window, you should see a “Transfer or Sync to…” link. When you click this, the window should split:

You can then click the “Transfer or sync to…” box at the on the right which will allow you to select another Globus endpoint. Search for the Globus DTN you are wanting to transfer data from/to in the “Collection” box. In the following example we are using the ESnet test endpoint at CERN – this is a public endpoint which can be used for testing downloads from another endpoint but you cannot transfer to it. You may wish to try using the "Shared EMBL-EBI public endpoint" to do test transfers as the ESnet test endpoint currently fails:

Select the file(s) that you wish to transfer either from Birmingham on the left or to Birmingham on the right (hold the CTRL key to select multiple files) note: the "Shared EMBL-EBI public endpoint" is read-only meaning you can only transfer data from them to Birmingham. You should then click the “Start” button under the right hand column to indicate you want to transfer from "Shared EMBL-EBI public endpoint" to BEAR. This will then schedule the data transfer. You can check on the progress of the transfer by selecting “Activity” on the left hand tool bar, depending on your account settings, you may also receive an email once the data transfer has completed:

In this example you can see one completed transfer and one transfer which has just started. 

Note that when accessing Globus endpoints, you may need to use local credentials to access the endpoint, for example if you are transferring data from Diamond Light Source, you may be asked to enter your Diamond Light Source username and password:

Need help/report a problem?

If you need assistance with Globus or to report a problem, please raise a Service Desk ticket - select "Make a Request", "Research", "Log a Fault with BEAR/CaStLeS".