Along with the Swift and S3 APIs, Horizon, and DuraCloud, there are additional tools that can be used to manage your content in the OLRC.
RClone is a command-line tool that can be used to transfer data to the OLRC. One of the major benefits of rclone is its ability to synchronize data (similar to a tool like rsync), meaning that only data that has changed will be uploaded. RClone is our recommended tool for uploading/downloading large volumes of data, as it has proven to be fast and reliable.
To use rclone with the OLRC:
- Download and install rclone
- In your terminal, set up your environment with:
- Follow the prompts to set up your configuration, setting your storage type as “OpenStack Swift”. Additional setup values can be found in the OpenRC file available in Horizon. See the rclone swift documentation for additional setup help and commands.
Cyberduck is a free software GUI client that supports OpenStack Swift operations, including bulk uploads and downloads, and runs on Windows and macOS.
These instructions have been written using macOS, but the steps should be similar in a Windows environment.
When uploading large volumes of data with Cyberduck, we strongly recommend verifying the data was uploaded via another method (e.g. with Horizon or the command-line) as some users have had issues with failed uploads when uploading large amounts of data using Cyberduck.
To use the Cyberduck GUI:
- Download Cyberduck and follow the installation instructions.
- After you have installed and opened Cyberduck, open a new connection and enter your credentials:
- Server: olrc2auth.scholarsportal.info
- Port: 443
- Project/Domain/Username: Enter the name of your project, domain (e.g. your institution domain), and your account name, formatted as “project:domain:username”. For example, “general:scholarsportal:kaitlin”. You can find these details in the Horizon dashboard as well.
- Enter your password (the same password you use to login to Horizon or use the Swift CLI).
- Click “Connect”.
- Once you have connected, you will see a list of the containers in the project. You can now create or delete containers and upload or download data from your local machine.