Author: iglez

A PROOF Analysis Framework
I. González Caballero1,3, A.Y. Rodríguez Marrero2,
E. Fernández del Castillo2, A. Cuesta Noriega1
1Universidad de Oviedo - Spain, 2Instituto de Física de Cantabria (UC-CSIC) - Spain
3Isidro.Gonzalez.Caballero@cern.ch

Evolution of data formats in a typical LHC experiment

PROOF, computer farms and batch systems

Smaller size per event
Less events per dataset

As data evolves higher level physics objects are added to the structure. Other operations reduce the amount of data that a given analysis needs to process in the end.

The Parallel ROOT Facility provides an efficient and transparent way to process ROOT trees distributedly. Clusters of computers like Tier-2 and Tier-3 centres can be used to improve processing performance. Batch and Cloud systems can be used to build PROOF clusters dynamically by using some specific tools: PROOF Cluster, PoD, Cloud Cluster,...
Modern many-core desktops or laptops can also benefit from PROOF using the PROOF-Lite mode.
Being able to run in sequential mode is important for debugging.
For more details see poster: Integrating PROOF Analysis in Cloud and Batch Clusters

The data analysis development cycle

PROOF Analysis Framework design goals

Implement an analysis algorithm

Run over data and wait for results

Look at results and think on improvements

Some sort of interactivity is desirable so the developer time is spent thinking rather than waiting for results. To achieve this a large amount of CPU is needed during short periods of time.

Physicists should concentrate on building the analysis rather than on the computing technicalities
Migrating traditional ROOT based analyis code should be easy
Code should be valid across the different PROOF modes supported
The need for special configurations on the computing infrastructures used should be kept minimal
PAF achieves these objectives through an automatically generated TSelector subclass that contains all the tree information. Another class managing all the interaction with PROOF exposes only the configuration settings.

PAF — Analysis code

PAF — PROOF Configuration

The analysis code is implemented in a subclass of the automatically generated PAF selector. Four virtual methods can be specialised to:
● Read input parameters
● Initialise analysis result objects
● Do the appropiate calculation at each event and update analysis results
● Perform final operations on results
Analysis result data is automatically saved to a ROOT file for later inspection.

A macro file starts the PROOF session by setting:
● PROOF Mode, number of workers to use,...
● Input data files
● Output file where results are to be stored
● Dynamic histograms : Histograms that will be updated as they get filled
● Additional analysis packages hosting modularised code (ex. official electron selection)
● Input parameters for the analysis
● Selector containing the analysis code

PAF Example session

A typical PAF session runs from inside ROOT invoking the configuration macro from a normal terminal .
PAF starts by setting (or reusing if possible) a PROOF session according to the chosen mode.
It then transparently generates, compiles, packages and uploads to the workers all the code that needs it.
At this point a window , showing the progress of the event processing and providing access to peformance information and logs, is launched.