Episode

Storage for the output files

The processing workflow writes the output files to a storage from which they can be downloaded afterwards.

For this tutorial, we use Infomaniak Object Storage.

Move to your local working environment and get access to the storage by sourcing the OpenStack RC file that you have downloaded previously. If you use a virtual environment, activate it.

cd <your_path>/infomaniak-cloud-test
source venv/bin/activate # if you created a virtual environment with name venv
source <path_to_rc_file_if_not_local/>PCU-XXXXXX-openrc.txt

Storage

Create

Create the storage - if not already done - with

openstack container create mystorage

You can list the existing storage containers with

openstack container list

Test

Before continuing, go through some basic operations for the storage container.

Write

Create a test file and write it into the storage

echo "Storage test" > test.txt
openstack object create mystorage test.txt

Inspect

You can list the contents with

$ openstack object list mystorage
+----------+
| Name     |
+----------+
| test.txt |
+----------+

or

$ swift list mystorage
test.txt

You can also login to the Infomaniak Horizon dashboard, reachable from a top-right button of the Cloud Computing dashboard or from the cloud computing project view.

You will find the storage container by choosing “Object Store” from the left bar.

Download

The file is now stored on the Infomaniak object storage. You can access it anywhere provided you have sourced the OpenStack RC file and have the Python tools (openstackand swift installed).

You can download it with (to a different file with the -o option, omit it to keep the same name):

$ swift download mystorage test.txt -o downloaded.txt
test.txt [auth 1.021s, headers 1.428s, total 1.430s, 0.000 MB/s]

or write to stdout with

$ swift download mystorage test.txt -o -
Storage test

A download option is also available from the Horizon dashboard

Delete

You can delete the test file with

swift delete mystorage  test.txt

Costs

Storage

The cost for the storage is currently 0.000013 €/GB/hour, i.e. 0.11 €/GB/year, 114 €/TB/year. In the context of this tutorial, the cost is very small, but keep it in mind if you plan to store large output files for a long time.

Data access

Uploading to the storage is free.

Downloading is free for the 10 first TB per month and then 0.00811 €/GB i.e 8.11 €/TB.