The DIRAC framework and DIRAC4EGI service

DIRAC is a software framework for distributed computing that acts as an intermediate layer between users and resources: as such, it is considered as an interware. DIRAC allows for interoperability by providing simplified interfaces to all kind of computing resources (illustration 2): grid worker nodes, cloud virtual machines or direct access to a batch system, this both in HTC and HPC contexts.

 

Description of the service

DIRAC is a software framework for distributed computing that acts as an intermediate layer between users and resources: as such, it's considered as an interware. DIRAC allows for interoperability by providing simplified interfaces to all kind of computing resources (illustration 2): grid worker nodes, cloud virtual machines or direct access to a batch system, this both in HTC and HPC contexts.

Illustration 1: DIRAC, the interware

The main components of the DIRAC framework are a workload management system based on pilot jobs to broke computing tasks on all kind of available resources, a Transformation system that acts a basic data driven workflow manager, and a Data Management system that provides access to storage resources and full data catalog implementation with advanced meta-data handling.

The DIRAC4EGI service is provided to the EGI community as a workload management service used to distribute the users' computing tasks among the available resources both HTC and cloud. The service  supports established Virtual Organization as well as the long tail of users.

The dirac.egi.eu service suite comprises a back-end for distributed computing, a web portal for general scientific computing needs, a set of software tools and user community support. The service relies on a platform hosted at the CYFRONET computing center, with user and technical support provided by the Università Autonoma de Barcelona and the DIRAC@IN2P3 team from CPPM, LUPM and CC-IN2P3.

Web links:

 

Dirac usage

Dirac has originally been developed to support the production activities of the LHCb experiment at CERN (approximately 10 years ago). Today, it has become a general purpose software, targeting several large scientific communities. The DIRAC consortium was founded to drive the development, maintenance and documentation of the software as an open source project developed by user communities for users.

LHCb remains was a leader in the development and usage of DIRAC, but more than 10 communities linked to large and small scale experiments now use either a proper instance of DIRAC with dedicated plugins around the world, or the DIRAC4EGI multi-community instance, as shown on figure 2.

Illustration 2: DIRAC user communities

 

Advantages of the service

DIRAC, both as a framework and through the DIRAC4EGI service, provides a unique access to distributed computing, making it no more something exotic and easily usable in a daily work by users in various scientific domains.

Indeed, the path for newcomers to use DIRAC for their specific project is first to try it out and get familiar with it via testing on through the DIRAC4EGI service. Then if DIRAC meets their requirements and if a heavy workload is planned, the community needs to organize to setup a proper instance of the DIRAC platform and eventually develop its own plugins on top of it to better handle its specific use cases.

The strength of DIRAC really lies in its open source user driven development scheme that makes it easy for a new community to adapt the tool to their need, and even give feedback to the main development tree. New users can rely on the long run on the seriousness of the DIRAC consortium and its associated community that aims to provide support to all users, as is obvious from the liveliness of the DIRAC user forum.

Contact person and website

  • Johan Bregeon
  • Andrei Tsaregorodtsev, DIRAC consortium representative and DIRAC4EGI technical advisor
Website: 
http://dirac.egi.eu
Author: 
DIRAC