Become familiar with EVN data reduction in AIPS or CASA. Here you can find the basic commands to calibrate your EVN data using any of the two packages (AIPS instructions on the left column, CASA instructions on the right column). Hover your mouse over the different parameters to know their meaning and the explanation about the chosen values. Note that VLBI data reduction under CASA is still under heavy development, and thus several options available in AIPS may still be missing in CASA.
You can also access the EVN Data Reduction Tutorials to follow a hands on calibration.
- First steps.
- Inspecting the data.
- Split the data.
- Post-imaging steps:
- Specific observing modes:
- Frequently Asked Questions (FAQ).
Obtaining the data
At this point you should have several FITS IDI files containing the visibility data from your observation and a
tasav.FITS file containing the calibration tables generated by the EVN Pipeline.
Start AIPS and load the data
AIPS can be started by typing (you may need to initialize the environment before hand):
aips tv=local:0. Introduce the AIPS user number that you wish to use for the session.
The EVN data need to be loaded as:
default fitld datain 'PWD:experiment_1_1.IDI digicor -1 doconcat 1 ncount n outname 'exp' go fitld recat
And the same for the
tasav FITS file:
tget fitld datain 'PWD:experiment_tasav.FITS digicor -1 doconcat -1 ncount 1 outname 'exp' go fitld
The useful calibration tables from the EVN Pipeline must be transfered from the tasav file to the data. In the case of EVN data, this means the CL table no. 2 (calibration table containing the parallactic angle and a-priori gain corrections), the a-priori flagging (FG) table, and, optionally, the bandpass (BP) calibration.
default tacop getn 2 geton 1 inext 'CL' invers 2 ncount 1 go tacop
Optionally the bandpass calibration from the EVN Pipeline can be copied. Otherwise a manual bandpass calibration may be preferred.
tget tacop inext 'FG' invers 1 ncount 1 go tacop
tget tacop inext 'BP' invers 1 ncount 1 go tacop
Start CASA and load the data
Before starting CASA it is required to download the external CASA VLBI Python module located under this link, which will append calibration meta-data (system temperature, gain, flags) to the FITS-IDI file.
Note that appending system temperature and gain curve information is only required if you are analyzing observations older than 1 January 2022. Otherwise this information is already available in the provided FITS-IDI files. Running the module scripts will notify you of this and not alter any existing data. You will still need the module to convert the UVFLG information.
To import the module in CASA, set your
$PYTHONPATH variable to point to the directory where you have saved the downloaded module.
You need another two files from the EVN Data Archive in addition to the previously mentioned FITS-IDI files. Under the "Pipeline" tab, download both the Associated EVN calibration (
ANTAB file) and the UVFLG flagged data (
Now CASA can be started by typing
casa. For proper functionality of the CASA VLBI module, make sure you run the most recent CASA distribution with the highest Python version.
To import the module at the CASA prompt type
from casavlbitools import fitsidi
Generate a sorted list of all your FITS-IDI files
from natsort import natsort_keygen import glob myidifiles = sorted(glob.glob('experiment*IDI*'),key=natsort_keygen())
The natsort module may need to be separately installed, you need this to get natural sorting if you have more than 9 FITS-IDI files.
First the system temperature information is added. This requires a list of all the FITS-IDI files in the variable idifiles.
The module will give an error if there is already system temperature information present in the FITS-IDI files. This is normal behaviour. Depending on the size of your FITS-IDI files and your machine CPU/memory, this can take anywhere from a few minutes to ~half an hour. Be patient.
Second, this is repeated with the gain curve information.
Note the subtle difference: appending the gain curve requires only one of the FITS-IDI files. As above, this module gives an error if the gain information is already present, which is also benign. This step is fast.
Finally, the UVFLG flags from the archive need to be converted into a CASA readable format. This module writes the flags to a file in your working directory.
fitsidi.convert_flags('experiment.uvflg', myidifiles, outfile=experiment.flag)
Now the IDI FITS files may be converted to a Measurement Set (MS) file, the native format for CASA:
importfitsidi(vis='experiment.ms', fitsidifile=myidifiles, constobsid=True, scanreindexgap_s=15.0)
The flag table may be applied to the data:
And an a-priori gain calibration may be generated with:
flagdata(vis='experiment.ms', mode='list', inpfile='experiment.flag', reason='any', action='apply', flagbackup=True, savepars=False)
Note that CASA handles differently the parallactic angle corrections. They are applied on-the-fly for each calibration task instead of being corrected at the beginning like in AIPS.
gencal(vis='experiment.ms', caltable='cal.tsys', caltype='tsys', uniform=False) gencal(vis='experiment.ms', caltable='cal.gcal', caltype='gc')
Inspect the data
First, information about the number of scans, times, sources, and antennas can be retrieved by
default listr getn 1 optype 'SCAN' go listr
Data should be checked carefully before calibration to identify possible issues like radio frequency interference (RFI), dropouts of stations and some particular times, or bad data in general. We recommend to do a manual flagging of the data at this point and indentify the edge channels where the signal already dropped (and should thus be flaged). AIPS has a large number of possible tasks to perform these steps, among others
default prtan getn 1 go prtan
TVFLG. Pick your preferred one.
And you can visualize the data interactively with
Among other options, you can set the parameters
coloraxis(either when calling
plotmsor inside the interactive window) to display different information like amplitudes, phases, times, uv distances, or u versus v. This tool also allows you to flag the bad data.
Depending on the data to be analyzed, some atmospheric corrections may be recommended. If the observation was conducted at frequencies of around 5 GHz or below, the data may suffer from ionospheric dispersive delays (different at each antenna and proportional to the Total Electron Content, TEC, along the line of sight and inversely proportional to the square of the frequency) that can be compensated.
vlbatecr downloads the required ionospheric data at the time of the observation and produces a calibration.
A new CL table should have been created (number 3 if no additional steps have been done up to now).
run vlbautil default vlbatecr getn 1 go vlbatecr
tec_maps downloads the required data:
from casatasks.private import tec_maps tec_image, tec_rms_image, plotname = tec_maps.create(vis='experiment.ms', doplot=True) gencal(vis='experiment.ms', caltable='cal.tecim', caltype='tecim', infile='tec_image') viewer('experiment.ms.IGS_TEC.im') viewer('experiment.ms.IGS_RMS_TEC.im')
If the observation was conducted at frequencies above 20 GHz, you may want to correct for tropospheric effects.
Removing instrumental delay
Phase jumps between subbands (or IFs) are related to instrumental delays at each telescope and thus can be corrected by using the data from one of the strong calibrators and apply the solutions for the whole observation. Note that if the data only have one subband (or if you conduct a subband-independent global fringe calibration), this step is not necessary.
default fring getn 1 timerang d1 h1 m1 s1 d2 h2 m2 s2 docal 1 gainuse n doband 1 bpver 1 weightit 1 refant a solint 0 dparm(9) 1 snver 1 go fring
fringefit(vis="experiment.ms", caltable="cal.sbd", timerange='h1:m1:s1~h2:m2:s2', solint='inf', zerorates=True, refant='a' minsnr=10, gaintable=['cal.tsys', 'cal.gcal', ...] parang=True)
In AIPS look at the solutions with
default snplt getn 1 inext 'SN' invers 1 opty 'dela' dotv 1 nplots 8 tvini go snplt
In CASA look at the solutions with:
Note that they can also be plotted by using the
plotms(vis='cal.sbd', xaxis='frequency', yaxis='phase', antenna='a' correlation='rr,ll', timerange='h1:m1:s1~h2:m2:s2', averagedata=True, avgtime='120')
In AIPS apply the solutions to the data and create a new calibration (CL) table.
default clcal getn 1 gainver n gainuse n+1 snver 1 interpol '' opcode 'cali' refant a go clcal
In CASA apply the calibration to the data.
applycal(vis='experiment.ms', gaintable=['cal.tsys', 'cal.gcal', 'cal.sbd', ...], parang=True)
NOTE: In some cases it is not possible to obtain a time range at which all stations were observing. In those cases multiple
FRINGinstances must be run. In AIPS, run the first
FRINGexactly in the same way but use
optype 'CALP'when running
CLCALwith above settings (so it keeps data from antennas with no solutions in the previous SN table, i.e. the stations that did not observe in the considered scan).
FRINGagain for a different time range where the remaining station(s) observed. In this case you need to run the task with the following parameters:
default fring getn 1 timerang d1 h1 m1 s1 d2 h2 m2 s2 docal 1 gainuse n doband 1 bpver 1 weightit 1 refant a1 dofit a2, a3,... antenna a1, a2, a3,... solint 0 dparm(9) 1 snver s go fringRepeat these steps (
default clcal getn 1 gainver n gainuse n+1 snver s interpol '' opcode 'calp' refant a1 antenna a2, a3,... go clcal
CLCAL) as many times as required to get solutions for all antennas.
In AIPS the bandpass table can be imported (highly recommended) from the EVN Pipeline results with
TACOP (previous steps). In some cases, a manual bandpass calibration must be performed to recover a flat response of the instruments across all the band:
default bpass getn 1 calsour 'bandpass-calibrator','' docal 1 gainuse n refant a solint -1 weightit 1 soltype 'l1r' bpassprm(1) 0 bpassprm(2) 1 bpassprm(10) 1 go bpass
NOTE: You should pick the same scan used for the instrumental delay calibration. Otherwise, the bandpass calibration should be done after the global fringe-fitting (following step). In any case, note that the bandpass correction may introduce again some offsets between subbands/IFs. Therefore, it is recommended to run again the instrumental delay correction (and/or global fringe-fit) afterwards.
In CASA the manual bandpass calibration must be performed by running:
bandpass(vis='experiment.ms', caltable='cal.bpass' field='bandpass-calibrator', gaintable=['cal.tsys', 'cal.gcal', ...], solnorm=True, solint='inf', refant='a', bandtype='B', parang=True)
Global fringe (or frequency and time-dependent phase calibration)
Phases must be corrected for the whole observation. For VLBI data that means that delays and rates must be corrected by using fringe fit. If the target source is bright and compact enough, it can directly be used in the fringe-fit. However, in most of the cases a nearby calibrator (the so-called phase calibrator) should be used for that and then solutions must be extrapolated to the target source position.
default fring getn 1 calsour 'calsource1', 'calsource2', '' docal 1 gainuse n doband 1 bpver 1 timerang 0 weightit 1 refant a solint t aparm(5) 1 aparm(6) 3 aparm(9) 1 dparm 1 200 50 0 search a2, a3, a4 snver 2 go fring
fringefit(vis='experiment.ms', caltable='cal.mbd', solint='t', combine='spw', field='calsource1, calsource2,...', refant='a', minsnr=7, gaintable=['cal.tsys', 'cal.gcal', 'cal.bpass', 'cal.sbd', ...], parang=True)
solint) should be long enough to produce solutions with a good signal-to-noise ratio, but it must be smaller than the time interval at which you see a significant phase evolution (which would de-correlate the signal). Typically it is of the order of a few minutes for standard arrays and EVN frequencies. Also note that it is not possible to combine IFs (
aparm(5) 1in AIPS or
combine='spw'in CASA) if the instrumental delay correction was not performed beforehand.
In AIPS apply the solutions to the data and create a new calibration (CL) table. It is recommended to apply the solutions to one source at the time:
Repeate to all sources used in fring. Then if solutions must be transfer to the target (like in phase-referencing experiments) then:
default clcal getn 1 calsour 'sourceN' '' source calsour gainver n gainuse n+1 snver 2 interpol 'self' opcode 'cali' refant a go clcal
tget clcal calsour 'phaseCalibrator' '' source 'target' interpol 'ambg' opcode 'cali' refant a go clcal
In CASA apply the calibration to the data:
applycal(vis='experiment.ms', gaintable=['cal.tsys', 'cal.gcal', 'cal.bpass', 'cal.sbd', 'cal.mbd', ...], spwmap=[, , , N*, ...], parang=True)
NOTE: Fringe fit assumes that your source(s) are point-like (unresolved for all baselines) and located at the phase center position. However, in some cases this may not be true and a source model can be provided to for a more accurate fit. In
FRING(in AIPS), add the parameters
get2n i(where i refers to the image file of the source) and
cmethod 'dft'(for a better model interpolation).
In the case of spectral line observations,
CVELmay be executed to provide the conversion between frequencies and velocities.
Split and store the data
The final calibration may be applied to the data and different datasets can be produced for the different sources.
To store the UVDATA outside AIPS:
default split getn 1 source '' docal 1 gainuse n doband 1 bpver 1 flagv 0 aparm 2 0 go split
default fittp getn i dataout 'PWD:filename.uvfits' go fittp
To store the data as UVFITS:
split(vis='experiment.ms', field='source_name', outputvis='source_name.ms', datacolumn='corrected')
exportuvfits(vis='filename.ms', field='sources', fitsfile='filename.uvfits', datacolumn='corrected')
Now images for each source can be produced. Depending on the type of observations, the required parameters may differ significantly. In the case of continuum data, an image can be obtained with:
In the case of spectral line data, a cube may be wanted. Select multiple channels with the desired averaging to obtain it.
default imagr getn n imsize 512 cellsize c niter 10000 gain 0.05 cmethod 'dft' robust r dotv 1 go imagr
tclean(vis='filename.ms', field='source', imagename='source.image', specmode='mfs', nterms=1, deconvolver='hogbom', gridder='standard', imsize=1024, cell='XXmas', weighting='natural', niter=10000, interactive=True, savemodel='modelcolumn')
Image fitting and statistics
Different measurements can be performed in the obtained images. The most basic ones are:
tvall: show the image.
imstat: provides the statistics of the image (like rms, maximum and minumum brightnesses).
tvstat: similar to imstat but a region in the image can be specified.
jmfit: fit a Gaussian to the image.
default fittp getn i dataout 'PWD:filename.fits' go fittp
viewer: show the image and allows interactive measurements.
imstat: provides the statistics of the image (like rms, maximum and minumum brightnesses).
imfit: fit a Gaussian to the image.
Typically this calibration can still be improved by self-calibrating the target data (or the phase calibrator data and then transfer the solutions to the target if this is faint). If a good source model has already been created, then this one can be used to improve the calibration:
It will create a
default calib getn n get2n m nmaps 1 refant a weightit 1 soltype 'l1r' cmethod 'dft' solint t aparm 3 0 solmode 'p' go calib
CALIBfile with the calibrated data which can be used to image again the source (now with a better calibration). Repeat this process iteractively until reaching the best possible calibration. After some rounds of phase-only selfcalibration, and only if the model is good enough, a phase-and-amplitude selfcalibration may be performed (setting
If the selfcalibration has been performed in a calibrator source, then the solutions may be transfered to the original UVDATA file and applied to the target source, to be later on imaged again.
First the SN table created in the SPLIT file used for the selfcalibration must be copied to the original UVDATA:
default tacop getn n geton 1 inext 'SN' invers s ncount 1 outvers 0 go tacop
Now a new CL (calibration) table may be created with
CLCALin the same way as before by using this SN table. Split the target source data again applying this CL table and the fully calibrated dataset will be ready to be imaged.
In CASA: (NOTE: THIS SECTION IS UNDER TESTING)
Apply the calibration:
gaincal(vis='filename.ms', caltable='cal.pcal' field='calibrator', gaintype='T', solint='inf', refant='a', calmode='p', combine='spw', minsnr=10, minblperant=4)
Repeat this process iteractively until reaching the best possible calibration. After some rounds of phase-only selfcalibration, and only if the model is good enough, a phase-and-amplitude selfcalibration may be performed (setting
applycal(vis='filename.ms', caltable='cal.pcal' field='sources', gainfield='calibrator', gaintable=['cal.pcal'], calwt=False, flagbackup=False, interp='linearperobs')
Multi phase-center observations
The EVN supports multi phase-center correlations, where different correlations at different positions (within the primary beam) can be performed to observe multiple source with only one pointing. In these cases, the calibration may be performed in the same way for only one phase center. Once it is done, you can apply the same calibration to the other phase centers.
Primary beam corrections
When you image a source that is significantly offset from the phase center (the position in the sky where all antennas were pointing to; like in the case of multi phase-cetner observations), you may need to correct your data for the primary beam of each individual antenna (their response pattern outside the pointing position). For observations correlated after January 2021, JIVE directly provides these corrections within the calibration tables produced by the EVN Pipeline. You only need to copy the SN #2 (table containing the amplitude corrections to apply for each antenna and source) and apply them to your data during the data reduction.
For older observations, please contact the EVN Support Scientists at JIVE (usersupport (at) jive.eu) to get information about how to apply this correction in your case.
Frequency Asked Questions (FAQ)
- Uncertainties in total flux densities in VLBI data
- Amplitudes in VLBI data are based on the measurement of antenna sensitivities and system temperatures during the observation. Given that no reference source can be used to set the true amplitude level (like it is done in connected interferometers with the known amplitude calibrators), the final amplitudes fully rely on these measurements and estimations. There is thus an uncertainty that propagates to the final absolute flux density scale in your data. Although it cannot be easily measured, in VLBI observations this method is typically accurate up to 10-15%. This should thus be the uncertainty quoted in the absolute flux density measurements that you perform on your data.
- How to measure flux densities from the data
- There are typically two different approaches to measure the flux density of a source in your VLBI data: doing it in the image plane or in the uv plane.
- DO NOT USE CLCOR (AIPS) on EVN data
- The use of CLCOR in AIPS to move the phase center of your data to a different position should be avoided.
- Gain calibration issues
- In some data it may happen that one (or some) antennas exhibit a significant offset in the gain calibration, producing amplitudes that are significantly off from what it was expected (e.g. when compared to the other baselines). This issue can be corrected via self-calibration (producing an accurate model with the other baselines), or with tasks like
gscalein packages like
- When do self-calibration?
- Self-calibration can only be reliably used when certain conditions are met. You can read the guide from Sabrian Stierwalt (NRAO) about when, why, and how to do self-calibration on your data.