Data Transfer

If you have a need to transfer large amounts of research data with either external people or organisations, there are a number of BEAR services available to support this. In all cases, when transferring data, you should take appropriate measures to ensure that sensitive data is protected or encrypted prior to the transfer being undertaken.

Globus Connect Server

Globus Connect Server is a tool used for transferring multi-TB data sets. A data transfer node (DTN) is required at each end for the transfer to work. The data transfer is usually controlled using the Globus web service which automatically manages background data transfers to be completed. The BEAR Globus endpoint is directly connected to both the Research Data Store and CaStLeS storage facilities. Globus is used in a number of large international scientific organisations, for example ORNL, CERN, Sanger, Crick, Archer (RDF) and so is well suited for large inter-institutional data transfers.

In addition to this, it is possible to use the Globus Connect Personal on other devices (such as laptops) to transfer data quickly to and from the Research Data Store or CaStLeS. Note that we recommend you are not connected to the University's Remote Access Service (VPN) when you try to transfer data using Globus Connect Personal.

Access to Globus is normally via the Globus web portal. Any researcher registered on a BEAR project with Research Data Store or CaStLeS storage can use the Globus end point to transfer data without prior registration.

Globus Connect Server is the preferred solution for large scale data transfers as the transfer can occur in the background with integrity checks built in to ensure successful transfer of the data.

The Birmingham endpoint currently only supports encrypted transfers, if the remote endpoint does not support encrypted transfers, then your transfer may fail.

A quick start guide on using Globus is available online.
For the external collacorator the following is guidance on how to setup sharing.

Other command line tools (rsync/sftp/ftp)

Other command line tools may be necessary to transfer data from some organisations. Where this is required, it is preferrable that the data transfer is done from a system which directly accesses the Research Data Store. For example on BlueBEAR login nodes. Note that it is not possible to undertake such transfers from BlueBEAR compute nodes. Whilst computation work is nor permitted on BlueBEAR login nodes, there is a specific exemptions for the use of these kinds of data transfer tools.


Professional Services