r1 - 26 Jul 2018 - 06:55:59 - LaurentDuflotYou are here: TWiki >  Atlas Web  >  Informatique > GuideAnalyse > GuideAnalyse26Jul2018

User guide for physics analysis

This page is dedicated to the use of the available resources at the Lyon Computing Centre to perform a physics analysis. One should have read before the New user's guide, and the Grid user's guide.

Physics Analysis tools

We detail here a few link to help to start to perform physics analyses. Most of these informations concern the tools available through Athena. They are not specific to the use of the Lyon Computing Centre and thus only links to the usual CERN we pages are provided.

CERN wiki pages

The CERN Physics Analysis Work Book (select the release 21 version of the workbook for xAOD analysis) aims to give information to ATLAS members on how to start physics analyses.

Non-grid analysis

Non grid analysis are done through the Grid Engine batch scheduler. In particular the use of command qsub is described here and look at the Grid Engine Tutorial from 1012 (this was Oracle Grid Engine, minor difference can show with the current version of GE in use at CCIN2P3). One will take care in particular to declare usage of some ressources using the qsub options -l sps=1 -l dcache=1 -l xrootd=1 (chose the one(s) you use)

Distributed analysis

Currently, there are two official ways of submitting jobs to analyze large samples of data, GANGA and pathena/prun. More information are available at PhysicsAnalysisWorkBookBatch2.

Panda tools

The panda-based tools for submiting grid jobs are described here. The PhysicsAnalysisWorkBookBatch2 has example and a few tips, a general introduction and a few additional tips are given here. Panda provides two commands for job submision, prun and pathena, and one for monitoring / handling, pbook. As explained in the panda wiki, these tools can be installed locally so that you can submit jobs from a machine in your lab (or from your laptop), you would also need to have the basic grid environment installed and setup.

  • pathena: Is almost equivalent to athena myjoboption.py. It actually configures athena at submission time to figure out your job configuration, in particular which output to expect. I needs at least a joboption, an input dataset (container) and the name for the output dataset container.

  • prun: Allows to run custom executables so it is more flexible but then you need to handle yourself how to use the input files and specify which output files you produce. In particular, prun can be used to run root jobs on the grid.

  • pbook: A command line or gui to query the status of your jobs, cancel or retry them. Actually needs to connect to the panda DB to get updated status for the jobs and that is often quite slow.

Panda should give you the best site to run on your input dataset, ingeneral it is best not interfere with it but you can exclude sites with --excludedSites or force a a particular site with --site. You can have a look at the test jobs results from the ganga robot, e.g. ANALY_IN2P3-CC-T2, to find out if a site is known to have problems. Panda tries to figure out how to split the input dataset in subjobs but sometime fails, you would then have to force the number of subjobs by --nFilesPerJob=N or fix the total size per job via --nGBPerJob=S (where S can be set to MAX to use all disk space on the worker node, however sometimes jedi gets confused and cannot fit correctly file on scratch with this option, from experience a value of 10 gives good results). Given how job priorities are calculated, it is best to keep the number of subjobs as low as possible. A couple additional remarks on job submission

  • You can use a dataset container as input to your panda job event if the datasets are not localised on a single grid site, panda will split the job accordingly and put the corresponding output datasets in your output container.
  • Instead of a input dataset, you can specify a GRL (and possibly the data tier, e.g. ESD or AOD, you want to run on), and panda will figure out the datasets.
  • The output datasets of your jobs are stored in the SCRATCHDISK zone of the site your job ran on. However, these areas a regularly cleaned up (after ~7 days except for US sites, unless the disk becomes full, see here). You can directly subscribe your output datasets to a more permanent storage, e.g. your favorite LOCALGROUPDISK, via the --destSE option.

Monitoring jobs and data

You can monitor your jobs via your panda web page, bigpanda-new.cern.ch/user/yourname/?display_limit=100&day=1. When a subjob fails, you have access (most of the time...) to the logfile so that you can understand why it failed. If this looks like a glitch on the site, you can recover your failed jobs once all subjobs are finished:
  • either by resubmiting the exact same pathena / prun command, possibly vetoing the site if looks really sick
  • or by using pbook and then retry(jobID)

A more generic monitoring of the grid (jobs, data transfers, databases) is provided by ADC monitoring. Of particular interest to end users are the

Old versions of this page

-- LaurentDuflot - 13 Nov 2013 Cleanup links and delete obsolete info

Edit | WYSIWYG | Attach | Printable | Raw View | Backlinks: Web, All Webs | History: r1 | More topic actions
Powered by TWiki
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback