CMS open data
processing on Infomaniak cloud resources

Learn to run a typical CMS open data processing job using public cloud resources from Infomaniak.

Prepare your computing environment to interact with computing and storage resources on cloud.

Learn more about CMS open data by working through the tutorials in one of our workshops.

This tutorial gives an estimate of the cost and time needed for a typical processing task

Learning Objectives

Episode	Learning Objectives
Introduction	Understand the motivation for using a Kubernetes cluster from public cloud providers. Learn the tools used to set up the Kubernetes cluster and run the processing workflow.
Storage for the output files	Learn to create a persistent storage for the job output. Learn to list the contents of the storage. Understand the persistent storage costs.
Kubernetes cluster	Learn to create a Kubernetes cluster. Access the cluster and inspect it from the command line.
Set up a workflow	Deploy Argo Workflows services to the cluster. Submit a test job. Find the output in the storage.
Scaling up	Optimize the cluster setup for a full dataset processing. Learn about job configuration. Get an idea of cost and time need for full-scale processing.
Discussion	Learn about the practical user experience with CMS open data processing on Infomaniak resources.

Time	Episode	Questions
	Setup	Before you start
00:00	Introduction 20 min	What are public cloud providers? Why would you use them? What is Kubernetes? What else do you need?
00:20	Storage for the output files 25 min	How to create an object storage container on Infomaniak resources? What are the basic operations? What are the costs of storage and download?
00:45	Kubernetes cluster 65 min	How to create a Kubernetes cluster on Infomaniak resources? How to access the cluster from the command line?
01:50	Set up a workflow 45 min	How to set up Argo Workflow engine? How to submit a test job? Where to find the output?
02:35	Scaling up 15 min	What is an optimal cluster setup? What is an optimal job configuration? How much does it cost?
02:50	Discussion 15 min	What was the user experience? What worked well? What difficulties were observed?

Contributor	GitHub
Kati Lassila-Perini	@katilp