huibintemaspampipeline
Differences
This shows you the differences between two versions of the page.
Both sides previous revisionPrevious revisionNext revision | Previous revisionNext revisionBoth sides next revision | ||
huibintemaspampipeline [2017/06/26 22:22] – [Limited bandwidth observations] huibintema | huibintemaspampipeline [2019/02/18 11:37] – [Old hardware-correlator observations] huibintema | ||
---|---|---|---|
Line 1: | Line 1: | ||
===== Running the SPAM pipeline ===== | ===== Running the SPAM pipeline ===== | ||
- | This section | + | This section |
- | Let's assume you have requested and downloaded from the GMRT archive in LTA format. You will end up with 3 files: | + | ----- |
+ | |||
+ | ==== Basic pipeline run ==== | ||
+ | |||
+ | Let's assume you have requested and downloaded | ||
* <project name> | * <project name> | ||
* <project name> | * <project name> | ||
Line 13: | Line 17: | ||
<code python> | <code python> | ||
lta_file_name = " | lta_file_name = " | ||
- | uvfits_file_name = " | + | convert_lta_to_uvfits( lta_file_name ) |
- | convert_lta_to_uvfits( lta_file_name, uvfits_file_name | + | |
</ | </ | ||
+ | This creates a UVFITS file in the fits subdirectory, | ||
Then we derive calibration and flagging information from the primary calibrator(s), | Then we derive calibration and flagging information from the primary calibrator(s), | ||
<code python> | <code python> | ||
+ | uvfits_file_name = " | ||
pre_calibrate_targets( uvfits_file_name, | pre_calibrate_targets( uvfits_file_name, | ||
</ | </ | ||
Line 36: | Line 41: | ||
./ | ./ | ||
- | There' | + | The pipeline run can be summarized by typing: |
+ | <code python> | ||
+ | summarize_spam_log( " | ||
+ | </ | ||
+ | |||
+ | |||
+ | There' | ||
----- | ----- | ||
+ | ==== Combining multiple LTA files of one observation ==== | ||
+ | |||
+ | Sometimes a single observation has been broken up into multiple LTA files because the data recording was interrupted (e.g., because of a power failure or system reset). Note that this is different from having a single project spread out over multiple observations. The multiple LTA files can be individually converted to UVFITS before combining: | ||
+ | <code python> | ||
+ | lta_file_name_1 = " | ||
+ | convert_lta_to_uvfits( lta_file_name_1 ) | ||
+ | lta_file_name_2 = " | ||
+ | convert_lta_to_uvfits( lta_file_name_2 ) | ||
+ | </ | ||
+ | Then the resulting UVFITS files can be combined as follows: | ||
+ | <code python> | ||
+ | uvfits_file_name_1 = " | ||
+ | uvfits_file_name_2 = " | ||
+ | uvfits_file_name = " | ||
+ | combine_uv( uvfits_file_name_1, | ||
+ | </ | ||
+ | The output UVFITS can then be passed to // | ||
+ | |||
+ | ----- | ||
+ | |||
+ | ==== Combining observations of the same target ==== | ||
+ | |||
+ | Combining observations on the same target from multiple observations is only possible when the frequency setup was exactly the same during the multiple observations. This is (almost?) always the case when a target was observed during a single project spread over multiple nights. | ||
+ | |||
+ | First, convert and pre-calibrate the observations per night. Then make sure that the multiple pre-calibrated UVFITS files of the target are all located in the same (fits) directory, and have a common naming scheme. E.g., < | ||
+ | <code python> | ||
+ | uvfits_file_name = " | ||
+ | process_target( ' | ||
+ | </ | ||
+ | This will read in all UVFITS files that match the wildcard query, concatenate them, and process them as one. | ||
+ | |||
+ | ----- | ||
+ | |||
+ | ==== Changing image weights ==== | ||
+ | |||
+ | The user can change some of the imaging parameters (AIPS IMAGR style), including the image weights. By default, the main pipeline uses no UV range cuts, and sets the Briggs robust parameter to -1 to compensate for the broad PSF wings due to the centrally condensed UV coverage. This can be changed as follows: | ||
+ | <code python> | ||
+ | imagr_params = { ' | ||
+ | process_target( target_uvfits_file_name, | ||
+ | </ | ||
+ | |||
+ | ----- | ||
+ | |||
+ | ==== Keeping main pipeline intermediate files ==== | ||
+ | |||
+ | The main pipeline takes the data through various repetitions of calibration, | ||
+ | <code python> | ||
+ | process_target( target_uvfits_file_name, | ||
+ | </ | ||
+ | |||
+ | ----- | ||
==== Dual-frequency observations ==== | ==== Dual-frequency observations ==== | ||
In case of dual-frequency observations, | In case of dual-frequency observations, | ||
<code python> | <code python> | ||
- | convert_lta_to_uvfits( lta_file_name, | + | convert_lta_to_uvfits( lta_file_name, |
- | convert_lta_to_uvfits( lta_file_name, | + | convert_lta_to_uvfits( lta_file_name, |
</ | </ | ||
The two output UVFITS files can then be processed further as usual. Please note that you may have to manually limit the frequency channel range for the 235 MHz observations, | The two output UVFITS files can then be processed further as usual. Please note that you may have to manually limit the frequency channel range for the 235 MHz observations, | ||
Line 61: | Line 123: | ||
gv ./ | gv ./ | ||
</ | </ | ||
- | Select a channel range over which the bandpass phases are well-behaved (linear) for most antennas, | + | Select a channel range over which the bandpass phases are well-behaved (approximately |
+ | {{ : | ||
+ | |||
+ | Note down the lower- and upper-limit of the channel range. Then re-run | ||
<code python> | <code python> | ||
- | channel_range = [ 10, 225 ] | + | channel_range = [ 150, 425 ] # example channel range to keep |
pre_calibrate_targets( uvfits_file_name, | pre_calibrate_targets( uvfits_file_name, | ||
</ | </ | ||
----- | ----- | ||
+ | |||
+ | ==== Old hardware-correlator observations ==== | ||
+ | |||
+ | If you downloaded data from cycle 17 or earlier, it is likely correlated using the GMRT Hardware Backend (GHB; a.k.a. the hardware correlator). For frequencies of 325 MHz and higher, the 32 MHz bandwidth is typically split over an upper-side band (USB) and lower-side band (LSB), both captured in separate LTA files (typical extensions are .lta and .ltb). In SPAM, both sideband LTA files need to be pre-processed separately: | ||
+ | <code python> | ||
+ | convert_lta_to_uvfits( lta_file_name ) | ||
+ | convert_lta_to_uvfits( ltb_file_name ) | ||
+ | </ | ||
+ | The next step is run using the // | ||
+ | <code python> | ||
+ | pre_calibrate_targets( uvfits_file_name_lta, | ||
+ | pre_calibrate_targets( uvfits_file_name_ltb, | ||
+ | </ | ||
+ | The resulting UVFITS files for USB and LSB per target can be combined: | ||
+ | <code python> | ||
+ | uvfits_file_name_usb = " | ||
+ | uvfits_file_name_lsb = " | ||
+ | uvfits_file_name = " | ||
+ | combine_usb_lsb( uvfits_file_name_usb, | ||
+ | </ | ||
+ | The output UVFITS file can be processed further in the main pipeline. | ||
+ | |||
+ | Regarding the main pipeline, there are two options that may be relevant to get to better results. The first option related to the situation explained above, where two sidebands (USB and LSB) are joined together to cover 32 MHz of bandwidth. In that case, it may help to turn on an image-based flagging option that treats the joined USB and LSB separately. Reason for this is that the USB and LSB have separate signal chains, and thus there can be system problems that relate only to one of the two sidebands. | ||
+ | <code python> | ||
+ | process_target( target_uvfits_file_name, | ||
+ | </ | ||
+ | |||
+ | The second option is to turn on baseline-based calibration, | ||
+ | <code python> | ||
+ | process_target( target_uvfits_file_name, | ||
+ | </ | ||
+ | |||
+ | ----- | ||
+ | |||
+ | ==== Using different calibration models ==== | ||
+ | |||
+ | By default, a point source model based on the NVSS, WENSS, VLSSr, SUMSS, and MGPS-2 catalogs is used to bootstrap the phase calibration at the start of the main pipeline (// | ||
+ | <code python> | ||
+ | process_target( target_uvfits_file_name, | ||
+ | </ | ||
+ | |||
+ | In case of observations of the same field in multiple GMRT bands, it is often useful to start with processing the lowest frequency data (which means the widest field-of-view, | ||
+ | |||
+ | <code python> | ||
+ | catalog_name = "< | ||
+ | catalog = read_pybdsm_ascii_catalog( catalog_name ) | ||
+ | source_list = create_source_list_from_catalog( catalog ) | ||
+ | resolution = 15. # representative resolution of model image in arcsec | ||
+ | process_target( target_uvfits_file_name, | ||
+ | </ | ||
+ | |||
+ | ----- | ||
+ | |||
+ | Feedback: [[huib.intema@curtin.edu.au|Click here]] | ||
huibintemaspampipeline.txt · Last modified: 2020/10/05 17:46 by huibintema