Exporting system libraries for

ATLAS on special facilities

Alessandro De Salvo
14-02-2013

A. De Salvo 14 febbraio 2013

The problem: external libraries availability

We currently use a meta-rpm (HEP_OSlibs) to install the

experiments dependencies in sites

Common files for all the experiments, not present in the software releases of
any experiment

Compatibility libraries

Special system libraries

Working quite well in standard WLCG sites

May create problems for non-WLCG sites, running a stock SL6/RHEL6

Clusters where ATLAS is not officially supported, like University research computing
centers

Not so many sites having the problem, but may be large clusters (~100k cores)

May be important for HPC sites too

Those category of sites may need to export the dedicated libs in a different way

Possible solutions [1]

Installing libraries in the ATLAS CVMFS production space

Installing the needed libraries in CVMFS is possible, but there are a number
of possible drawbacks

Hard to maintain: need to keep them up-to-date and react very quickly in case of
emergencies, like security treats, to avoid compromising all the sites

Need to isolate the full list of libs required by ATLAS and install automatically any needed
dependency, the list of files to install may become very huge

Just works for ATLAS

Installing libraries in a dedicated CVMFS common space, like SFT

Same problems as the previous option (mainainability, security, ...)

Works for every experiment
May not work correctly with CVMFS access via parrot, due to the limitations
on the concurrent access of multiples repositories and cross-referencing
issues
We could export CVMFS via nfs to overcome the parrot limitations, but the
computing facilities should support NFS, which could not be true in all the
cases
3

Possible solutions [2]

Using CernVM contents in a chrooted/exported environment

CernVM images have all the needed software, up-to-date, to run every LHC
experiments code
The root fs of CernVM3 is available in CVMFS

/cvmfs/cernvm-prod.cern.ch/cvm3

Good solution to have the correct libs in place, from a different repo
Not using CernVM itself, just using its contents
Still need to understand how to handle efficiently multiple repos when using
parrot

CVMFS Fuse vs CVMFS Parrot

CVMFS Fuse

CVMFS Parrot

Jakob Blomer

Current approach (from Dave Lesny)

ACE cache

Use parrot/CVMFS to provide access to all Atlas repositories

Setup a parrot cache and cvmfs "alien caches" to be shared between all jobs
on the node (or cluster)
Use fakeroot/fakechroot to provide access to an image which contains
HEP_OSlibs and all other needed products
Use parrot to map /usr/lib, /usr/lib64, /usr/bin, etc into this image (overriding
the local system libraries, binaries, etc)
Setup a Certificate Authority
Setup a OSG WN Client installation on the node
Setup an $OSG_APP, etc so Pilots can run properly

Possible improvements

Use CernVM3 instead of a custom image, either via a standard image or

exporting the CVMFS files