CTA - Workflow Management System API

Introduction

The INAF CTA science gateway aims at providing a web instrument for high energy astrophysics. It  leverages open source technologies giving web access to a set of tools and software widely used by the CTA community. An extended (though not exhaustive) list of tools provided by this technology embrace XANADU software package, GammaLib & ctools, Fermi Science Tools, Aladin, IRAF. The gateway is based on the Liferay platform. It provides a Workflow Management System (WMS) with a customizable graphical web user interface and a web-desktop environment.

The integrated WMS is based on WS-PGRADE/gUSE that seamlessly enables the execution of astronomical and physics workflows (and jobs) on major platforms such as DIRAC (Distributed Infrastructure with Remote Agent Control) INTERWARE, ARC, Globus, gLite, UNICORE(Uniform Interface to Computing Resources), PBS as well as web services or clouds.

A plethora of mature workflow systems has evolved that support diverse concepts and languages with different strengths and emphases on different modes of processing. Few workflow systems deliver the power of diverse digital resources and most of the web-based creation and editing tools either require local software installations with inherent security problems or offer incomplete functionalities.

Therefore WS-PGRADE/gUSE has been selected mainly because of:

1.     its usability via web-based user interfaces;
2.     its availability, with respect to licensing terms and cost;
3.     its anticipated long-term support, e.g. via an active open-source community; and
4.     its ability to deal efficiently with the scales of data, computation and concurrent use required.

gUSE enables users convenient and easy access to distributed computing infrastructures (e.g. clusters and clouds). The integration with DIRAC INTERWARE is of particular relevance since new generation astrophysical/cosmological experiments, with very large processing and storage needs, are currently investigating the usability of DIRAC in this context.

WMS Interface Description

Interface with the WMS is implemented through web services  which exposes the whole life-cycle of a workflow management through SOAP and REST APIs.

The interface provides functionalities for workflow submission, obtaining status during
execution in short or in detailed format, for stopping, suspending, or rescuing the workflow,
and for downloading outputs.

Endpoint: https://cta-sg.oact.inaf.it/wspgrade/RemoteServlet

Useful operations include:

•     submit: Submits the workflow to the science gateway WMS
•     info: access the status of the workflow (workflow statuses include: submitted/running/finished/error/suspended/invalid)
•     stop: aborts and deletes the workflow
•     download: downloads the produced results of the workflow

Source Code and Documentation

API Source Code: https://sourceforge.net/projects/guse/  
API Samples: https://sourceforge.net/projects/guse/files/3.7.4/Documentation/RemoteAPI_Usage_Examples.zip
API Configuration:https://sourceforge.net/projects/guse/files/3.7.4/Documentation/RemoteAPI_Configuration.pdf

Summary of existing software to be used

Taverna, Kepler, Triana, Pegasus, WS-pGrade/gUSE, YaBi, DAGman

Contact

• Alessandro Costa    INAF-OACT

• Eva Sciacca        INAF-OACT

• Fabio Vitello        INAF-OACT