Introduction
Overview
Teaching: 7 min
Exercises: 0 minQuestions
What is CMSSW?
How is CMSSW structured?
Objectives
Understand what CMSSW is and how it is organized
Overview
The CMS Software (CMSSW) is a collection of software libraries that the CMS experiment uses in order to acquire, produce, process and even analyze its data. The program is written in C++ but its configuration is manipulated using the Python language.
CMSSW is built around a Framework, an Event Data Model (EDM), and Services needed by the simulation, calibration and alignment, and reconstruction modules that process event data so that physicists can perform analysis. The primary goal of the Framework and EDM is to facilitate the development and deployment of reconstruction and analysis software.
The CMSSW repository is on Github. You can browse this huge amount of code, search through it using the CMSSW Software Cross Reference or explore the documentation here.
CMSSW is a continuously-evolving project. Historically, there has been many releases, which are handled on Github using branches. Be aware that in this workshop we will use release CMSSW_7_6_7 (the official release for 2015 open data), which can be found in the CMSSW_7_6_X branch of the repository. This branch may differ a little or a lot compared to the bleeding-edge one in the master branch, so make sure you are always referencing to the historical one.
Structure and architecture
As it was mentioned above, the CMSSW software is used for almost all computing activities in CMS. From data acquisition to data analysis, using different pieces of CMSSW is very intuitive. Different modules (or plugins) have different functionalities. Some, for instance, are in charge of setting up certain services like the magnetic field configuration (we call this type of code Setup), while others help you create some object that was not there before (EDProducers) or analyze final data (EDAnalyzers). You can find details here. In this workshop, we are only going to look at EDAnalyzers. In fact, while setting up your environment using Docker, you already created a simple instance of such an analyzer.
As an example of modularity, take a look at the package used for reconstructing tracks. It has many sub-packages that put in evidence the many bits involved in making a track from detector sensor information. One of those sub-packages is the TrackProducer, which is in charge of putting (recording) the track information in the event. Note the structure of this sub-package:
It has the usual look of a C++ repository. Commonly, you can find a src directory with the bulk of the C++ programming (*.cc files), an interface directory, with mostly the header files (*.h files) matching the code in the src, and a python directory, where configuration files, written in Python, are stored. Some other accesories are in other directories. Of course, this is not a standard rule, and many times the structure follows a different logic. There is also a Buildfile, which controls the package dependencies at compile time.
All these packages are, in a sense, plugins to the main Framework, which is also a package by itself.
The event data architecture is modular, just as the framework is. Different data layers (using different data formats) can be configured, and a given application can use any layer or layers. The following diagram illustrates this concept if one thinks about how the information from tracks of charged particles is organized:
All the information regarding the physics of a collision is stored in the Event. Computationally, one can think of the Event as an object from which you can pull all the information you need from the collision.
CMS uses different data formats, which are arranged in tiers. Currently CMS open data comes only in the AOD (Run 1 data from 2010-2012) and miniAOD (Run 2 data from 2015) format. In this workshop we will use the latter format, i.e., miniAOD as we will learn to analyze 2015 data.
Key Points
The CMS SoftWare (CMSSW) is the software used by the CMS experiment for acquiring, producing, processing and analyzing its data.
CMSSW is built in a modular fashion around a main Framework.
CMSSW is needed to decode data formats like AOD and miniAOD
Installation and Execution
Overview
Teaching: 0 min
Exercises: 20 minQuestions
How do I install CMSSW?
How do I compile and execute CMSSW?
Objectives
Review the steps necessary to setup a CMSSW area.
Learn how to compile and execute CMSSW jobs.
Setting up your CMSSW area
If you completed the lessons on Docker containers you should already have a working CMSSW area.
If you have closed the container, re-start it with:
docker start -i <theNameOfyourContainer>
Make sure you are in the CMSSW_7_6_7/src area of the container. This is the default directory when you start the container, but if you have changed to another directory, come back to it with:
cd /code/CMSSW_7_6_7/src
Note that we are not really “installing” CMSSW but setting up an environment for it. CMSSW was already installed in the container. Every time you start the container a script runs by default to set up a few environmental variables that are needed.
For instance, you can check where your CMSSW_RELEASE_BASE variable points to:
echo $CMSSW_RELEASE_BASE
The variable points to a local CMSSW install in the Docker container:
/cvmfs/cms.cern.ch/slc6_amd64_gcc493/cms/cmssw/CMSSW_7_6_7
Try a different variable
Can you check where the variable CMSSW_BASE points to?
cmsRun, the CMSSW executable
All the packages that comprise the CMSSW release in use have been already compiled and linked to one single executable, which is called cmsRun. So, unless you want to create your own plugin (addition) for the software, you won’t even have to re-compile. You can actually try to execute this command by itself, but it will give you a configuration error:
cmsRun
cmsRun: No configuration file given.
For usage and an options list, please do 'cmsRun --help'.
So, inevitably, the cmsRun executable needs a configuration file. This configuration file must be written in Python. Do not worry, we will expand on this later. For now, let’s try a simple example.
Run with a configuration
Now try to run, but this time with a configuration file.
Solution
We could simply repeat what we already did while setting up our Docker container: run with the
Demo/DemoAnalyzer/demoanalyzer_cfg.pypython configuration file.
To run in a more efficient way, although it is not strictly necessary, we use the bash redirector>, the redirection ofstderrtostdout(2>&1), and the trailing run-in-the-background control operator&. This allows us to send the output to adummy.logfile and run the proces in the background so we can still interact with the terminal.cmsRun Demo/DemoAnalyzer/python/ConfFile_cfg.py > dummy.log 2>&1 &You can check the development of your job with
tail -f dummy.logWhen finished, if you dumped the content of
dummy.log, and docat dummy.logyou’ll get an output similar to:
14-Jul-2022 14:42:10 CEST Initiating request to open file root://eospublic.cern.ch//eos/opendata/cms/Run2015D/SingleElectron/MINIAOD/08Jun2016-v1/10000/001A703B-B52E-E611-BA13-0025905A60B6.root %MSG-w XrdAdaptor: file_open 14-Jul-2022 14:42:11 CEST pre-events Data is served from cern.ch instead of original site eospublic %MSG 14-Jul-2022 14:42:13 CEST Successfully opened file root://eospublic.cern.ch//eos/opendata/cms/Run2015D/SingleElectron/MINIAOD/08Jun2016-v1/10000/001A703B-B52E-E611-BA13-0025905A60B6.root (/code/CMSSW_7_6_7/src) cat dummy.log 14-Jul-2022 14:42:10 CEST Initiating request to open file root://eospublic.cern.ch//eos/opendata/cms/Run2015D/SingleElectron/MINIAOD/08Jun2016-v1/10000/001A703B-B52E-E611-BA13-0025905A60B6.root %MSG-w XrdAdaptor: file_open 14-Jul-2022 14:42:11 CEST pre-events Data is served from cern.ch instead of original site eospublic %MSG 14-Jul-2022 14:42:13 CEST Successfully opened file root://eospublic.cern.ch//eos/opendata/cms/Run2015D/SingleElectron/MINIAOD/08Jun2016-v1/10000/001A703B-B52E-E611-BA13-0025905A60B6.root Begin processing the 1st record. Run 257645, Event 1184198851, LumiSection 776 at 14-Jul-2022 14:42:23.233 CEST Begin processing the 2nd record. Run 257645, Event 1184202760, LumiSection 776 at 14-Jul-2022 14:42:23.234 CEST Begin processing the 3rd record. Run 257645, Event 1183968519, LumiSection 776 at 14-Jul-2022 14:42:23.234 CEST Begin processing the 4th record. Run 257645, Event 1183964627, LumiSection 776 at 14-Jul-2022 14:42:23.235 CEST Begin processing the 5th record. Run 257645, Event 1184761030, LumiSection 776 at 14-Jul-2022 14:42:23.235 CEST Begin processing the 6th record. Run 257645, Event 1184269130, LumiSection 776 at 14-Jul-2022 14:42:23.235 CEST Begin processing the 7th record. Run 257645, Event 1184358918, LumiSection 776 at 14-Jul-2022 14:42:23.236 CEST Begin processing the 8th record. Run 257645, Event 1183874827, LumiSection 776 at 14-Jul-2022 14:42:23.236 CEST Begin processing the 9th record. Run 257645, Event 1184415529, LumiSection 776 at 14-Jul-2022 14:42:23.237 CEST Begin processing the 10th record. Run 257645, Event 1184425291, LumiSection 776 at 14-Jul-2022 14:42:23.237 CEST 14-Jul-2022 14:42:23 CEST Closed file root://eospublic.cern.ch//eos/opendata/cms/Run2015D/SingleElectron/MINIAOD/08Jun2016-v1/10000/001A703B-B52E-E611-BA13-0025905A60B6.root ============================================= MessageLogger Summary type category sev module subroutine count total ---- -------------------- -- ---------------- ---------------- ----- ----- 1 XrdAdaptor -w file_open 1 1 2 fileAction -s file_close 1 1 3 fileAction -s file_open 2 2 type category Examples: run/evt run/evt run/evt ---- -------------------- ---------------- ---------------- ---------------- 1 XrdAdaptor pre-events 2 fileAction PostEndRun 3 fileAction pre-events pre-events Severity # Occurrences Total Occurrences -------- ------------- ----------------- Warning 1 1 System 3 3
Compilation
We use scram, the release management tool used for CMSSW, to compile (build) the code:
scram b
Reading cached build data
>> Local Products Rules ..... started
>> Local Products Rules ..... done
>> Building CMSSW version CMSSW_7_6_7 ----
>> Entering Package Demo/DemoAnalyzer
>> Creating project symlinks
src/Demo/DemoAnalyzer/python -> python/Demo/DemoAnalyzer
>> Leaving Package Demo/DemoAnalyzer
>> Package Demo/DemoAnalyzer built
>> Subsystem Demo built
>> Subsystem BigProducts built
>> Local Products Rules ..... started
>> Local Products Rules ..... done
gmake[1]: Entering directory `/code/CMSSW_7_6_7'
>> Creating project symlinks
src/Demo/DemoAnalyzer/python -> python/Demo/DemoAnalyzer
>> Done python_symlink
>> Compiling python modules cfipython/slc6_amd64_gcc493
>> Compiling python modules python
>> Compiling python modules src/Demo/DemoAnalyzer/python
>> All python modules compiled
>> Pluging of all type refreshed.
>> Done generating edm plugin poisoned information
gmake[1]: Leaving directory `/code/CMSSW_7_6_7'
Note that scram only goes into the Demo/DemoAnalyzer package that we created locally to validate our setup in a previous lesson. The rest of the packages, that live on the installation area (remember the area where CMSSW_RELEASE_BASE points to), were already compiled. Since there is nothing new to compile, it finishes very quickly. In a later episode we will modify this DemoAnalyzer and will need to compile again.
Point to be made: if you compile at main
srclevel, all the packages in there will be compiled. However, if you go inside a specific package or sub-package, like ourDemo/DemoAnalyzer, only the code in that subpackage will be compiled.
Additional goodies
Your CMSSW environment comes with other executable scripts/tools that can be very useful. An example of those is the mkedanlzr script that we used already to create the DemoAnalyzer package. This script creates skeletons for EDAnalyzers that can be later modified or expanded. Notice that this package, DemoAnalyzer, has a similar structure as any of the CMSSW packages we mentioned before.
One can find out about other scripts like mkedanlzr by typing mked and hitting the Tab key (maybe twice):
mked + Tab + Tab
mkedanlzr mkedfltr mkedlpr mkedprod
In this workshop, however, we will not be using those other ones.
There are also additional scripts, like the Event Data Model(EDM) utilities, the hltGetConfiguration trigger dumper, or the cmsDriver, which can be very useful. Now, let’s check an example.
Finding the EventSize of a ROOT EDM file
Now, as a simple exercise, use one of the EDM utilities mentioned above to find out about the number of events in the ROOT file that is in the
Demo/DemoAnalyzer/python/ConfFile_cfg.pyconfig file of your analyzer package. Try to figure it out from the documentation above before looking at the solution.Solution
After checking the documentation above, the following one-liner will work.
edmEventSize -v root://eospublic.cern.ch//eos/opendata/cms/Run2015D/SingleElectron/MINIAOD/08Jun2016-v1/10000/001A703B-B52E-E611-BA13-0025905A60B6.root | grep EventsFile root://eospublic.cern.ch//eos/opendata/cms/Run2015D/SingleElectron/MINIAOD/08Jun2016-v1/10000/001A703B-B52E-E611-BA13-0025905A60B6.root Events 49871So, the ROOT file has 49871 events.
Key Points
A CMSSW area is not really installed but set up.
cmsRunis the CMSSW executable. There are also utilitarian scripts.You can compile CMSSW with
scram b
EDAnalyzers
Overview
Teaching: 7 min
Exercises: 1 minQuestions
What is an EDAnalyzer and what does it contain?
What files are relevant in an EDAnalyzer?
Objectives
Learn what an EDAnalyzer is and how it is structured
Learn what c++, python and xml files are relevant.
Structure
First, make sure you start up your container as discussed in the previous episode.
EDAnalyzers are modules that allow read-only access to the Event. They are useful to produce histograms, reports, statistics, etc. Take a look at the DemoAnalyzer package that we created while validating our CMSSW working environment; it is an example of an EDAnalyzer.
Go to your CMSSW_7_6_7/src area, for example:
cd /code/CMSSW_7_6_7/src
Let’s explore the DemoAnalyzer package:
ls Demo/DemoAnalyzer/
doc plugins python test
ls Demo/DemoAnalyzer/plugins/
BuildFile.xml DemoAnalyzer.cc
ls Demo/DemoAnalyzer/python/
CfiFile_cfi.py CfiFile_cfi.pyc ConfFile_cfg.py ConfFile_cfg.pyc __init__.py __init__.pyc
Note that it has a similar structure as any of the CMSSW packages we mentioned before. In this sense, our DemoAnalyzer is just one more CMSSW package that we created privately. However, the headers and implementation of our simple DemoAnalyzer are coded in one single file under the plugins directory. The file was automatically named DemoAnalyzer.cc when we created the package.
CMSSW could be very picky about the structure of its packages. Most of the time, scripts or other tools expect to have a
Package/Sub-Packagestructure, just like ourDemo/DemoAnalyzerexample.
We also notice we have a python configuration file called ConfFile_cfg inside the python directory. This is the default configurator for the DemoAnalyzer.cc code.
Finally, there is a BuildFile.xml, inside the plugins directory, where we can include any dependencies if needed so our code can compile without problems.
All EDAnalyzers are created equal; of course, if made with the same mkedanlzr, they will look identical. The DemoAnalyzer.cc is a skeleton, written in C++, that contains all the basic ingredients to use CMSSW libraries. So, in order to perform a physics analysis, and extract information from our CMS open data, we just need to understand what to add to this code and how to configure it. This is, one way or another, the first and unskippable step when using CMS open data.
Key Points
An EDAnalyzer is a an edm class that generates a template for any analysis code using CMSSW.
There are essentially three important files in an EDAnalyzer package, the source code in c++, the python config file and a Buildfile for tracking dependencies.
The Source
Overview
Teaching: 10 min
Exercises: 30 minQuestions
What are the elements of the source code of an EDAnalyzer?
How do I modify the source to get additional information?
Objectives
Learn the basic structure of the C++ implementation of an EDAnalyzer.
Learn the basics on how to modify the source in order to extract physics information good for analysis.
Playing with the DemoAnalyzer.cc file
The DemoAnalyzer.cc file in the Demo/DemoAnalyzer/plugins/ directory is the main file of our EDAnalyzer. As it was mentioned, the default structure is always the same. Have a look at what is inside it using your favourite editor of your local computer. Remember that you will find the Demo area under the cms_open_data_work/CMSSW_7_6_7/src directory on your local computer. As the working directory has been mounted into the container, all changes take effect there as well.
The first thing that you will see is a set of includes:
// system include files
#include <memory>
// user include files
#include "FWCore/Framework/interface/Frameworkfwd.h"
#include "FWCore/Framework/interface/EDAnalyzer.h"
#include "FWCore/Framework/interface/Event.h"
#include "FWCore/Framework/interface/MakerMacros.h"
#include "FWCore/ParameterSet/interface/ParameterSet.h"
These are the most basic Framework classes that are needed to mobilize the CMSSW machinery. In particular, notice the Event.h class. This class contains essentially all the accessors that are needed to extract information from the Event, i.e., from the particle collision. Another important class is the ParameterSet.h. This one will allow us to extract configuration parameters, which can be manipulated using the Demo/DemoAnalyzer/python/ConfFile_cfg.py python file.
Something important to take into account is that you can learn a lot about the kind of information you have access to by exploring the code in the CMSSW repository on Github. For instance, you can look at the Event.h header and check all the available methods. You will notice, for instance, the presence of the getByToken accessors; we will be using one these to access physics objects.
When exploring CMSSW code on Github, remember to choose the CMSSW_7_6_X branch to match the release we are using for this workshop.
If you need to look at CMS Run 1 open data, please look at previous tutorials. The essential ideas are similar, but due to the different dataformats (miniAOD in Run 2 as opposed to AOD in Run 1), accessing the information is a bit different. This workshop focuses on Run 2 (miniAOD format) data.
Including muon headers
Let’s pretend that we are interested in extracting the energy of all the muons in the event. We would need to add the appropriate classes for this. After quickly reviewing this chapter (make sure you pick the
Run 2data tabs) of the CMS Open Data Guide (which is still under construction), we conclude that we need to add this header line to our analyzer://classes to extract Muon information #include "DataFormats/PatCandidates/interface/Muon.h"Let’s add it at the end of the header section together with the standard vector C++ library:
#include<vector>So the header section of the
Demo/DemoAnalyzer/plugins/DemoAnalyzer.ccsource file becomes:// system include files #include <memory> // user include files #include "FWCore/Framework/interface/Frameworkfwd.h" #include "FWCore/Framework/interface/one/EDAnalyzer.h" #include "FWCore/Framework/interface/Event.h" #include "FWCore/Framework/interface/MakerMacros.h" #include "FWCore/ParameterSet/interface/ParameterSet.h" //class to extract muon info #include "DataFormats/PatCandidates/interface/Muon.h" #include<vector>
Next, you will see the class declaration:
//
// class declaration
//
// If the analyzer does not use TFileService, please remove
// the template argument to the base class so the class inherits
// from edm::one::EDAnalyzer<> and also remove the line from
// constructor "usesResource("TFileService");"
// This will improve performance in multithreaded jobs.
class DemoAnalyzer : public edm::one::EDAnalyzer<edm::one::SharedResources> {
public:
explicit DemoAnalyzer(const edm::ParameterSet&);
~DemoAnalyzer();
static void fillDescriptions(edm::ConfigurationDescriptions& descriptions);
private:
virtual void beginJob() override;
virtual void analyze(const edm::Event&, const edm::EventSetup&) override;
virtual void endJob() override;
// ----------member data ---------------------------
};
The first thing one notices is that our class inherits from the edm::EDAnalyzer class. It follows the same structure as any class in C++. The declaration of the methods reflect the functionality needed for particle physics analysis. Their implementation is further below in the same file.
Declaring info containers
Let’s add the declaration of a vector for our energy values:
std::vector<float> muon_e;and a token, which is needed for registering for data access.
This section becomes:
// // class declaration // // If the analyzer does not use TFileService, please remove // the template argument to the base class so the class inherits // from edm::one::EDAnalyzer<> and also remove the line from // constructor "usesResource("TFileService");" // This will improve performance in multithreaded jobs. class DemoAnalyzer : public edm::one::EDAnalyzer<edm::one::SharedResources> { public: explicit DemoAnalyzer(const edm::ParameterSet&); ~DemoAnalyzer(); static void fillDescriptions(edm::ConfigurationDescriptions& descriptions); private: virtual void beginJob() override; virtual void analyze(const edm::Event&, const edm::EventSetup&) override; virtual void endJob() override; // ----------member data --------------------------- std::vector<float> muon_e; //energy values for muons in the event edm::EDGetTokenT<pat::MuonCollection> muonToken_; };
Next, we can see the constructor and destructor of our DemoAnalyzer class:
//
// constructors and destructor
//
DemoAnalyzer::DemoAnalyzer(const edm::ParameterSet& iConfig)
{
//now do what ever initialization is needed
usesResource("TFileService");
}
DemoAnalyzer::~DemoAnalyzer()
{
// do anything here that needs to be done at desctruction time
// (e.g. close files, deallocate resources etc.)
}
Note that a ParameterSet object is passed to the constructor. This is then the place where we will read any configuration we might end up implementing through our Demo/DemoAnalyzer/python/ConfFile_cfg.py python configuration file.
Registering for data access
Before a module, or a class helper delegated by a module, can access data it must have registered with the framework ahead of the time that it will be making a data access request. This registration must happen in the module’s constructor. Registration is accomplished by changing the constructor to
// DemoAnalyzer::DemoAnalyzer(const edm::ParameterSet& iConfig) { //now do what ever initialization is needed usesResource("TFileService"); muonToken_ = consumes<pat::MuonCollection>(edm::InputTag("slimmedMuons")); }
Do not mind the rather complicated way in which the token objects are handled. The good thing is that the same structure repeats for almost
any object you need from the event. The slimmedMuons tag refers to the collection stored inside the miniAOD ROOT files. There could be more than one collection for a given physical object as it can be seen in this table. This will be important to remember when we make this piece configurable.
The heart of the source file is the analyze method:
// ------------ method called for each event ------------
void
DemoAnalyzer::analyze(const edm::Event& iEvent, const edm::EventSetup& iSetup)
{
using namespace edm;
#ifdef THIS_IS_AN_EVENT_EXAMPLE
Handle<ExampleData> pIn;
iEvent.getByLabel("example",pIn);
#endif
#ifdef THIS_IS_AN_EVENTSETUP_EXAMPLE
ESHandle<SetupData> pSetup;
iSetup.get<SetupRecord>().get(pSetup);
#endif
}
Anything that goes inside this routine will loop over all available events. The CMSSW Framework will take care of that, so you do not really have to write a for loop to go over all events. Note that an edm::Event object and a edm::EventSetup object are passed by default. While from the Event we can extract information like physics objects, from the EventSetup we can get information like trigger prescales.
Get the muons energy
Now let’s add a few lines in the analyzer so we can retrieve the energy of all the muons in each event. We will print out this information as an example. Again, after checking out this guide, the analyze method becomes:
// ------------ method called for each event ------------ void DemoAnalyzer::analyze(const edm::Event& iEvent, const edm::EventSetup& iSetup) { using namespace edm; //clean the container muon_e.clear(); //define the handler, a token and get the information by token Handle<pat::MuonCollection> mymuons; iEvent.getByToken(muonToken_, mymuons); //if collection is valid, loop over muons in event if(mymuons.isValid()){ for (const pat::Muon &itmuon : *mymuons){ muon_e.push_back(itmuon.energy()); } } //print the vector of muons for each event for(unsigned int i=0; i < muon_e.size(); i++){ std::cout <<"Muon # "<<i<<" with E = "<<muon_e.at(i)<<" GeV."<<std::endl; } #ifdef THIS_IS_AN_EVENT_EXAMPLE Handle<ExampleData> pIn; iEvent.getByLabel("example",pIn); #endif #ifdef THIS_IS_AN_EVENTSETUP_EXAMPLE ESHandle<SetupData> pSetup; iSetup.get<SetupRecord>().get(pSetup); #endif }
The other methods are designed to execute instructions according to their own name description.
// ------------ method called once each job just before starting event loop ------------
void
DemoAnalyzer::beginJob()
{
}
// ------------ method called once each job just after ending the event loop ------------
void
DemoAnalyzer::endJob()
{
}
For instance, any instructions placed inside the beginJob or endJob routine will be executed every time the Framework starts or ends a
computing job, respectively. One may also use the beginRun and endRun routines to, for example, execute code that needs to be run at the
beginning of each CMS Run, like the trigger routines.
Now, it is time to compile. Remember that while editing is done on your local computer, compiling and running must be done in the container. So let’s move to the container shell and compile (heads-up: it will fail; see below)
scram b
Work assignment
The compilation will invariably fail. This is because the Muon class we added introduced some dependencies that need to be taken care of in the
BuildFile.xml. We will deal with this in a moment. Now, however, it is a good time to submit your assignment for this lesson. One of the assignments is to copy the compilation error message you got and paste it to the corresponding section in our assignment form; remember you must sign in and click on the submit button in order to save your work. You can go back to edit the form at any time.
So let’s modify the Demo/DemoAnalyzer/BuildFile.xml to include DataFormats/PatCandidates dependencies. It should look like:
<use name="FWCore/Framework"/>
<use name="FWCore/PluginManager"/>
<use name="FWCore/ParameterSet"/>
<use name="DataFormats/PatCandidates"/>
<flags EDM_PLUGIN="1"/>
Now, if you compile again, it should work. Then, we can run with the cmsRun executable:
cmsRun Demo/DemoAnalyzer/python/ConfFile_cfg.py > mylog.log 2>&1 &
Let’s check the log file:
cat mylog.log
14-Jul-2022 17:22:35 CEST Successfully opened file root://eospublic.cern.ch//eos/opendata/cms/Run2015D/SingleElectron/MINIAOD/08Jun2016-v1/10000/001A703B-B52E-E611-BA13-0025905A60B6.root
(/code/CMSSW_7_6_7/src) cat mylog.log
14-Jul-2022 17:22:32 CEST Initiating request to open file root://eospublic.cern.ch//eos/opendata/cms/Run2015D/SingleElectron/MINIAOD/08Jun2016-v1/10000/001A703B-B52E-E611-BA13-0025905A60B6.root
%MSG-w XrdAdaptor: file_open 14-Jul-2022 17:22:33 CEST pre-events
Data is served from cern.ch instead of original site eospublic
%MSG
14-Jul-2022 17:22:35 CEST Successfully opened file root://eospublic.cern.ch//eos/opendata/cms/Run2015D/SingleElectron/MINIAOD/08Jun2016-v1/10000/001A703B-B52E-E611-BA13-0025905A60B6.root
Begin processing the 1st record. Run 257645, Event 1184198851, LumiSection 776 at 14-Jul-2022 17:22:37.137 CEST
Begin processing the 2nd record. Run 257645, Event 1184202760, LumiSection 776 at 14-Jul-2022 17:22:39.003 CEST
Begin processing the 3rd record. Run 257645, Event 1183968519, LumiSection 776 at 14-Jul-2022 17:22:39.003 CEST
Muon # 0 with E = 15.2976 GeV.
Begin processing the 4th record. Run 257645, Event 1183964627, LumiSection 776 at 14-Jul-2022 17:22:39.014 CEST
Muon # 0 with E = 6.0168 GeV.
Begin processing the 5th record. Run 257645, Event 1184761030, LumiSection 776 at 14-Jul-2022 17:22:39.014 CEST
Begin processing the 6th record. Run 257645, Event 1184269130, LumiSection 776 at 14-Jul-2022 17:22:39.014 CEST
Muon # 0 with E = 8.94318 GeV.
Begin processing the 7th record. Run 257645, Event 1184358918, LumiSection 776 at 14-Jul-2022 17:22:39.014 CEST
Muon # 0 with E = 4.05406 GeV.
Begin processing the 8th record. Run 257645, Event 1183874827, LumiSection 776 at 14-Jul-2022 17:22:39.015 CEST
Muon # 0 with E = 4.2305 GeV.
Begin processing the 9th record. Run 257645, Event 1184415529, LumiSection 776 at 14-Jul-2022 17:22:39.015 CEST
Muon # 0 with E = 8.99778 GeV.
Begin processing the 10th record. Run 257645, Event 1184425291, LumiSection 776 at 14-Jul-2022 17:22:39.015 CEST
Muon # 0 with E = 8.0897 GeV.
Muon # 1 with E = 4.36828 GeV.
14-Jul-2022 17:22:39 CEST Closed file root://eospublic.cern.ch//eos/opendata/cms/Run2015D/SingleElectron/MINIAOD/08Jun2016-v1/10000/001A703B-B52E-E611-BA13-0025905A60B6.root
=============================================
MessageLogger Summary
type category sev module subroutine count total
---- -------------------- -- ---------------- ---------------- ----- -----
1 XrdAdaptor -w file_open 1 1
2 fileAction -s file_close 1 1
3 fileAction -s file_open 2 2
type category Examples: run/evt run/evt run/evt
---- -------------------- ---------------- ---------------- ----------------
1 XrdAdaptor pre-events
2 fileAction PostEndRun
3 fileAction pre-events pre-events
Severity # Occurrences Total Occurrences
-------- ------------- -----------------
Warning 1 1
System 3 3
Interestingly, as one can see, there are muons in a SingleElectron dataset. This is, of course, expected.
Key Points
The C++ source code file of an EDAnalyzer is taylored for particle physics analysis under the CMSSW Framework.
This source file needs to be modified according to the analyzer needs
The Configuration
Overview
Teaching: 20 min
Exercises: 30 minQuestions
What are the key elements of a CMSSW configuration file?
What kind of elements can I control at the configuration level?
Objectives
Learn the basic structure of the Python implementation of a CMSSW configuration file.
Learn how to modify a config file in order to change parameters and/or run additional code.
CMSSW configuration Framework
The CMS software framework uses a “software bus” model. A single executable, cmsRun, is used, and the modules are loaded at runtime. A configuration file, fully written in Python, defines which modules are loaded, in which order they are run, and with which configurable parameters they are run. Note that this is not an interactive system. The entire configuration is defined once, at the beginning of the job, and cannot be changed during running. This is the file that you “feed” cmRun when it is executed.
Playing with the ConfFile_cfg.py file
In the case of our DemoAnalyzer we have been working with, its configuration is the ConfFile_cfg.py file, which resides in the Demo/DemoAnalyzer/python directory of your CMSSW Demo package.
If we explore what is in the Demo/DemoAnalyzer/python directory:
ls Demo/DemoAnalyzer/python
we will get:
CfiFile_cfi.py CfiFile_cfi.pyc ConfFile_cfg.py ConfFile_cfg.pyc __init__.py __init__.pyc
You will note that there is also a CfiFile_cfi.py in there. We will not pay attention to this file now, but it instructive to point out that the _cfg and _cfi descriptors are meaningful. While the former one defines a top level configuration, the latter works more like a module initialization file. There are also _cff files which bear pieces of configuration and so they are dubbed config fragments. You can read a bit more about it in this subsection of the Workbook.
Ok, so the file we will play around with is just the top level Demo/DemoAnalyzer/python/ConfFile_cfg.py. Open it in the editor of your local computer from your working directory in cms_open_data_work/CMSSW_7_6_7/src.
The first instructions that you will find in all the top level CMSSW config files are the lines
import FWCore.ParameterSet.Config as cms
process = cms.Process("Demo")
The first line imports our CMS-specific Python classes and functions, and the second one creates a process object. This refers to a CMSSW process (the one we will be configuring, of course). Essentially, the main idea is that we will be feeding to our process all the tasks that we need executed by the CMSSW software or its plugins. The process needs always a name. It could be any short word, but it is usually chosen so it is meaningful. For instance, if the main task would be to process the high level trigger information, an adequate name will be “HLT”; if the process is actually the full reconstruction of the data, then it is most likely assigned the name “RECO”. For our demo, we will leave our creativity aside and just call it “Demo” (you can, of course, change it to your linking).
Then, you will notice a line that loads something:
process.load("FWCore.MessageService.MessageLogger_cfi")
Actually, because of the _cfi tag, we know it is presumably a piece of Python code that initializes some module. Indeed, it is the MessageLogger service. As the name describes, it controls how the message logging is handled during the job execution. The string "FWCore.MessageService.MessageLogger_cfi" tells you exactly where to look for it on Github if you ever need it. Note the structure matches the repository’s, except that the python directory name is always omitted when loading modules this way (this is why it is often important to put config files in the python directory).
There is a whole Workbook section regarding this module, but let’s just look at a simple example.
Changing the logging frequency in our CMSSW job
Suppose you want the Framework to report every 5 events instead of each event. Then one can simply add this line
process.MessageLogger.cerr.FwkReport.reportEvery = 5right below the load line (actually the place where you copy it hardly matters, but it helps to keep things organized):
process.load("FWCore.MessageService.MessageLogger_cfi") process.MessageLogger.cerr.FwkReport.reportEvery = 5
Note that all we are doing is loading the MessageLogger module and changing just one parameter, the one in this line, instead of going with the default value, which is one.
For the next line
process.maxEvents = cms.untracked.PSet( input = cms.untracked.int32(10) )
it is easy to guess that it controls the number of events that are going to be processed in our CMSSW job. It is worth noting that maxEvents is an untracked variable within the Framework. In general, the system keeps track of what parameters are used to create each data item in the Event and saves this information in the output files. This can be used later to help understand how the data was made. However, sometimes a parameter will have no effect on the final objects created. Such parameters are declared untracked. More information can be found in the Workbook.
Let’s change the number of events to 100:
process.maxEvents = cms.untracked.PSet( input = cms.untracked.int32(100) )
Next, there is the first module (also an object by itself) we are attaching to our process object:
process.source = cms.Source("PoolSource",
# replace 'myfile.root' with the source file you want to use
fileNames = cms.untracked.vstring(
#'file:myfile.root'
'root://eospublic.cern.ch//eos/opendata/cms/Run2015D/SingleElectron/MINIAOD/08Jun2016-v1/10000/001A703B-B52E-E611-BA13-0025905A60B6.root'
)
)
Inside the process object there must be exactly one object assigned that has Python type Source and is used for data input. There may be zero or more objects for each of many other Python types. In the official production configurations there can be hundreds or even thousands of objects attached to the process. Your job is configured by your choice of objects to construct and attach to the process, and by the configuration of each object. (This may be done via import statements or calls to the load function, instead of or in addition to object construction.) Some of the Python types that may be used to create these objects are listed in the Workbook.
Explore the PoolSource C++
Generally, all modules in our python configuration are associated with its corresponding C++ code. By searching the CMSSW Github repository, would you be able to point exactly to the line of C++ code where the
fileNamesparameter is read?solution
You will find that the label first appearing in a Python CMSSW module is actually the name of the C++ code. So, we would expect that there be a class
PoolSource.h(and perhaps its implementation asPoolSource.C) associated with the “PoolSource” label. Let’s then go to the Github CMSSW repository and simply search forPoolSource.Cusing the search field of that page. Immediately, in the search results, we notice there is aIOPool/Input/src/PoolSource.ccfile that we can browse. After looking for the variablefileNames, we find that this parameter is read in this line. Note that it is in the constructor of the object where theParameterSetobjects are read.
Note also that the fileNames variable is a vstring, i.e., a vector of strings in the C++ sense. In Python, it is a list, so you can very well input a comma separated list of files. There is a drawback, though. In general, our open datasets will contain more than 255 files, which is the limit for the number of arguments a Python function can take, so very long vstrings cannot be created in one step. There are various alternatives to circumvent this problem. To run over massive amounts of ROOT files, one will usually use the FileUtils module to load index files instead of individual ROOT files.
Configure our DemoAnalyzer
The second to last line in our configuration
process.demo = cms.EDAnalyzer('DemoAnalyzer'
)
has to do with our recently created DemoAnalyzer. This module is now just a declaration of existance because it is empty. Let’s put it to work:
Making our EDAnalyzer configurable
Recall that, previously, we registered data access for slimmedMuons. However as, we saw, there could be some objects which have more than one collection, like it can be seen in this table. Therefore we would like to make the selection of this tag (slimmedMuons) configurable. Would you be able to implement that?
Solution
Since we are going to make our DemoAnalyzer configurable, the first thing we need to do is to modify the C++ source of our analyzer in order to accommodate configurability. Let’s modify then the
Demo/DemoAnalyzer/plugins/DemoAnalyzer.ccfile. Again, following the logic in the Physics Ojects guide and using an editor, we should read the token from the ParameterSet.We will have to read this InputTag from the configuration. As it was noted above, this is done in the constructor. One option is to read and assign the value to the relevant variable directly. The constructor will need to be modified like:
DemoAnalyzer::DemoAnalyzer(const edm::ParameterSet& iConfig): muonToken_(consumes<pat::MuonCollection>(iConfig.getParameter<edm::InputTag>("muons"))) { //now do what ever initialization is needed usesResource("TFileService"); }Here we will be reading the
Collectionvariable from configuration (which is of typeedm::InputTag, which is essentially a string) and will store it in themuonToken_variable. Note the parameter read is called muonsIn this way, the string tag slimmedMuons is not harcoded any more; it can be changed if needed at configuration time.
Finally, let’s change the
Demo/DemoAnalyzer/python/ConfFile_cfg.pyby replacing our empty module statement:
process.demo = cms.EDAnalyzer('DemoAnalyzer')with
process.demo = cms.EDAnalyzer('DemoAnalyzer', muons = cms.InputTag("slimmedMuons") )In this way, we are now able to enter “slimmedMuons” or the token of any other collection, depending on our needs.
Now, before re-compiling our code, let’s check that our python configuration is ok. Go to the container shell. We can validate the syntax of your configuration using python:
python Demo/DemoAnalyzer/python/ConfFile_cfg.py
If there are no errors, you are good to go.
Since we modified the source code, let’s compile the code again, with scram in the container:
scram b
If everything goes well, you should see something like:
>> Local Products Rules ..... started
>> Local Products Rules ..... done
>> Building CMSSW version CMSSW_7_6_7 ----
>> Entering Package Demo/DemoAnalyzer
>> Creating project symlinks
src/Demo/DemoAnalyzer/python -> python/Demo/DemoAnalyzer
>> Compiling edm plugin /code/CMSSW_7_6_7/src/Demo/DemoAnalyzer/plugins/DemoAnalyzer.cc
>> Building edm plugin tmp/slc6_amd64_gcc493/src/Demo/DemoAnalyzer/plugins/DemoDemoAnalyzerAuto/libDemoDemoAnalyzerAuto.so
Leaving library rule at src/Demo/DemoAnalyzer/plugins
@@@@ Running edmWriteConfigs for DemoDemoAnalyzerAuto
--- Registered EDM Plugin: DemoDemoAnalyzerAuto
>> Leaving Package Demo/DemoAnalyzer
>> Package Demo/DemoAnalyzer built
>> Subsystem Demo built
>> Subsystem BigProducts built
>> Local Products Rules ..... started
>> Local Products Rules ..... done
gmake[1]: Entering directory `/code/CMSSW_7_6_7'
>> Creating project symlinks
src/Demo/DemoAnalyzer/python -> python/Demo/DemoAnalyzer
>> Done python_symlink
>> Compiling python modules cfipython/slc6_amd64_gcc493
>> Compiling python modules python
>> Compiling python modules src/Demo/DemoAnalyzer/python
>> All python modules compiled
@@@@ Refreshing Plugins:edmPluginRefresh
>> Pluging of all type refreshed.
>> Done generating edm plugin poisoned information
gmake[1]: Leaving directory `/code/CMSSW_7_6_7'
Finally, let’s run the CMSSW job:
cmsRun Demo/DemoAnalyzer/python/ConfFile_cfg.py > mylog.log 2>&1 &
If you check the development of the job with
tail -f mylog.log
Eventually, as the job progresses, you will see something like:
14-Jul-2022 18:37:29 CEST Successfully opened file root://eospublic.cern.ch//eos/opendata/cms/Run2015D/SingleElectron/MINIAOD/08Jun2016-v1/10000/001A703B-B52E-E611-BA13-0025905A60B6.root
Begin processing the 1st record. Run 257645, Event 1184198851, LumiSection 776 at 14-Jul-2022 18:37:31.600 CEST
Muon # 0 with E = 15.2976 GeV.
Muon # 0 with E = 6.0168 GeV.
Begin processing the 6th record. Run 257645, Event 1184269130, LumiSection 776 at 14-Jul-2022 18:37:33.710 CEST
Muon # 0 with E = 8.94318 GeV.
Muon # 0 with E = 4.05406 GeV.
Muon # 0 with E = 4.2305 GeV.
Muon # 0 with E = 8.99778 GeV.
Muon # 0 with E = 8.0897 GeV.
Muon # 1 with E = 4.36828 GeV.
14-Jul-2022 18:37:33 CEST Closed file root://eospublic.cern.ch//eos/opendata/cms/Run2015D/SingleElectron/MINIAOD/08Jun2016-v1/10000/001A703B-B52E-E611-BA13-0025905A60B6.root
=============================================
MessageLogger Summary
type category sev module subroutine count total
---- -------------------- -- ---------------- ---------------- ----- -----
1 XrdAdaptor -w file_open 1 1
2 fileAction -s file_close 1 1
3 fileAction -s file_open 2 2
type category Examples: run/evt run/evt run/evt
---- -------------------- ---------------- ---------------- ----------------
1 XrdAdaptor pre-events
2 fileAction PostEndRun
3 fileAction pre-events pre-events
Severity # Occurrences Total Occurrences
-------- ------------- -----------------
Warning 1 1
System 3 3
Change the InputTag
Now, change the name of the
InputColletionfrom “slimmedMuons” to “slimmedElectrons” in your configuration and run again without re-compiling the code. Do you see any difference?Solution
After changing the tag to “slimmedElectrons” this is the output we get:
14-Jul-2022 18:31:47 CEST Initiating request to open file root://eospublic.cern.ch//eos/opendata/cms/Run2015D/SingleElectron/MINIAOD/08Jun2016-v1/10000/001A703B-B52E-E611-BA13-0025905A60B6.root 220714 18:31:47 5623 secgsi_InitProxy: cannot access private key file: /home/cmsusr/.globus/userkey.pem %MSG-w XrdAdaptor: file_open 14-Jul-2022 18:31:47 CEST pre-events Data is served from cern.ch instead of original site eospublic %MSG 14-Jul-2022 18:31:49 CEST Successfully opened file root://eospublic.cern.ch//eos/opendata/cms/Run2015D/SingleElectron/MINIAOD/08Jun2016-v1/10000/001A703B-B52E-E611-BA13-0025905A60B6.root Begin processing the 1st record. Run 257645, Event 1184198851, LumiSection 776 at 14-Jul-2022 18:31:51.574 CEST Begin processing the 6th record. Run 257645, Event 1184269130, LumiSection 776 at 14-Jul-2022 18:31:51.576 CEST 14-Jul-2022 18:31:51 CEST Closed file root://eospublic.cern.ch//eos/opendata/cms/Run2015D/SingleElectron/MINIAOD/08Jun2016-v1/10000/001A703B-B52E-E611-BA13-0025905A60B6.root ============================================= MessageLogger Summary type category sev module subroutine count total ---- -------------------- -- ---------------- ---------------- ----- ----- 1 XrdAdaptor -w file_open 1 1 2 fileAction -s file_close 1 1 3 fileAction -s file_open 2 2 type category Examples: run/evt run/evt run/evt ---- -------------------- ---------------- ---------------- ---------------- 1 XrdAdaptor pre-events 2 fileAction PostEndRun 3 fileAction pre-events pre-events Severity # Occurrences Total Occurrences -------- ------------- ----------------- Warning 1 1 System 3 3Of course, we are not going to see anything because there is no slimmedElectrons tag for muon objects.
Running some already-available CMSSW code
The last line in our Demo/DemoAnalyzer/python/ConfFile_cfg.py config file is
process.p = cms.Path(process.demo)
The “software bus” model that was mentioned in the introduction of this episode can be made evident in this line. CMSSW executes its code using Paths (which in turn could be arranged in Schedules). Each Path can execute a series of modules (or Sequences of modules). In our example we have just one Path named p that executes the demo process, which corresponds to our DemoAnalyzer.
In general, however, we could add more modules. For instance, the Path line could look like
process.mypath = cms.Path (process.m1+process.m2+process.s1+process.m3)
, where m1, m2, m3 could be CMSSW modules (individual EDAnalyzers, EDFilters, EDProducers, etc.) and s1 could even be a modules Sequence.
Adding a Trigger Filter
In CMSSW, there are other types of code one can execute. Some of these are known as EDFilters. As the name implies, they can be used to filter events. For instance, one could use the HLTHighLevel filter class to only run over events that have passed a certain kind of trigger.
We can get a hint of its usage by exploring the constructor of that code (remember that the configuration parameters are passed there). After some careful inspection we conclude that what we need to add to our configuration is the following module:
process.hltHighLevel = cms.EDFilter("HLTHighLevel", TriggerResultsTag = cms.InputTag("TriggerResults","","HLT"), HLTPaths = cms.vstring('HLT_Ele27_WPLoose_Gsf_v*'), # provide list of HLT paths (or patterns) you want eventSetupPathsKey = cms.string(''), # not empty => use read paths from AlCaRecoTriggerBitsRcd via this key andOr = cms.bool(True), # how to deal with multiple triggers: True (OR) accept if ANY is true, False (AND) accept if ALL are true throw = cms.bool(True) # throw exception on unknown path names )Of course, we need to add this module to the CMSSW path for execution. When running first, it will filter events that do not have electrons firing a version of the
HLT_Ele27_WPLoose_Gsftrigger. You will learn more about triggers during the workshop.The full config file
Do not forget to switch back to “slimmedMuons” for the InputTag. The full config file will then look like:
import FWCore.ParameterSet.Config as cms process = cms.Process("Demo") process.load("FWCore.MessageService.MessageLogger_cfi") process.MessageLogger.cerr.FwkReport.reportEvery = 5 process.maxEvents = cms.untracked.PSet( input = cms.untracked.int32(10) ) process.source = cms.Source("PoolSource", # replace 'myfile.root' with the source file you want to use fileNames = cms.untracked.vstring( #'file:myfile.root' 'root://eospublic.cern.ch//eos/opendata/cms/Run2015D/SingleElectron/MINIAOD/08Jun2016-v1/10000/001A703B-B52E-E611-BA13-0025905A60B6.root' ) ) process.demo = cms.EDAnalyzer('DemoAnalyzer', muons = cms.InputTag("slimmedMuons") ) process.hltHighLevel = cms.EDFilter("HLTHighLevel", TriggerResultsTag = cms.InputTag("TriggerResults","","HLT"), HLTPaths = cms.vstring('HLT_Ele27_WPLoose_Gsf_v*'), # provide list of HLT paths (or patterns) you want eventSetupPathsKey = cms.string(''), # not empty => use read paths from AlCaRecoTriggerBitsRcd via this key andOr = cms.bool(True), # how to deal with multiple triggers: True (OR) accept if ANY is true, False (AND) accept if ALL are true throw = cms.bool(True) # throw exception on unknown path names ) process.p = cms.Path(process.hltHighLevel+process.demo)Without even having to compile again, the execution of the trigger path will stop if the
hltHighLevelfilter module throws aFalseresult. The output becomesBegin processing the 1st record. Run 257645, Event 1184198851, LumiSection 776 at 14-Jul-2022 21:02:00.208 CEST Begin processing the 6th record. Run 257645, Event 1184269130, LumiSection 776 at 14-Jul-2022 21:02:02.132 CEST Muon # 0 with E = 8.94318 GeV. Muon # 0 with E = 4.05406 GeV. Muon # 0 with E = 4.2305 GeV. Muon # 0 with E = 8.99778 GeV. 14-Jul-2022 21:02:02 CEST Closed file root://eospublic.cern.ch//eos/opendata/cms/Run2015D/SingleElectron/MINIAOD/08Jun2016-v1/10000/> 001A703B-B52E-E611-BA13-0025905A60B6.root ============================================= MessageLogger Summary type category sev module subroutine count total ---- -------------------- -- ---------------- ---------------- ----- ----- 1 XrdAdaptor -w file_open 1 1 2 fileAction -s file_close 1 1 3 fileAction -s file_open 2 2 type category Examples: run/evt run/evt run/evt ---- -------------------- ---------------- ---------------- ---------------- 1 XrdAdaptor pre-events 2 fileAction PostEndRun 3 fileAction pre-events pre-events Severity # Occurrences Total Occurrences -------- ------------- ----------------- Warning 1 1 System 3 3It is obvious that the first event (at least) was filtered out by by the trigger filter we added.
Congratulations!!, you have made it to the end the lesson.
Key Points
The Python implementation of the configuration of CMSSW is fundamentally modular.
The CMSSW configuration allows for the usage of already available code and/or make changes to yours without having to recompile.