The LOFAR Default Pre-Processing Pipeline (DPPP) software reads and writes radio-interferometric data in the form of Measurement Sets. It goes through visibilities in time order and contains standard operations like averaging, phase-shifting and flagging bad stations. Between the steps in a pipeline, the data is not written to disk, making this tool suitable for operations where I/O dominates.
As part of the ASTERICS/OBELICS project, gain calibration was implemented as a part of DPPP, using the StefCal algorithm. Furthermore, a move towards open standards for calibration tables was implemented, by adopting the HDF5 file format.
Specification of the data reduction is done by parameter sets (‘parsets’) which are used throughout the LOFAR system. An example data reduction could be to concatenate several subbands in frequency, flag the data (using default parameters), average the data, calibrate it, and average some more. Intermediate data products are stored in this example, if this is not necessary the steps can be skipped by leaving them out of the ‘steps’ parameter.
msin = [band1.MS, band2.MS] # Bands will be concatenated in frequency
steps = [aoflag, avg1, out1, cal, out2, avg2]
aoflag.type = aoflagger
avg1.type = average
avg1.timestep = 4 # Average every 4 timeslots
out1.type = out
out1.name = allbands_avg_t4.MS # Save the averaged file
cal.type = calibrate
cal.skymodel = 3C196.skymodel
cal.applysolution = true
cal.parmdb = instrument.parmdb
out2.type = out
out2.name = allbands_avg_t4.MS # Add calibrated data to allbands_avg4_t4
out2.datacolumn = CORRECTED_DATA # Save calibrated data in another column
avg2.type = average
avg2.timestep = 2 # average down some more
avg2.freqstep = 16
msout = allbands_avg_t8_f16.MS # Save final averaged data
The DPPP software is distributed as part of the LOFAR software suite, under the GPL v3 license. The source code can be obtained from https://svn.astron.nl/LOFAR/
Documentation for DPPP is found at https://www.astron.nl/lofarwiki/doku.php?id=public:user_software:documentation:ndppp