%
% this document belongs to the gildas CVS and in the src gildas 
% distribution is to be compiled with 'make pdbi-cookbook.pdf', so 
% that all the sty files are properly found  
%
\documentclass[11pt]{article}
\include{gildas-def} % GILDAS specific definitions

\textheight 560pt
\textwidth 420pt

\makeindex{}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{document}

\newcommand{\IPB}{{\tt IPB}} 
\newcommand{\hpb}{{\tt hpb}} 


\begin{center}
  \huge{}
  \textbf{IRAM NOEMA} \\
  \textbf{Data Reduction CookBook} \\[2\bigskipamount]
  \Large{}
  September 2010\\[\bigskipamount]
  Version 5.0\\[2\bigskipamount]
  \large{}
\end{center}

This document describes how to reduce NOEMA 
observations and gives some ideas to perform first analysis and
imaging. Sect.\ \ref{getstarted} explains how to get started, from
planning your trip to Grenoble to having a project account. The
standard procedure to calibrate NOEMA data with the \clic\ software
package is described in Sect.\ \ref{dataclic}. A few instructions to
start the data analysis with the \mapping\ software package are given
in Sect.\ \ref{datamap}. A theoretical description of the calibration
as well as a description of the extended pipeline (or AoD) {\tt First
  Look} report are annexed in Apps.\ \ref{calibprinciples} and
\ref{pipeshowsec} respectively.


\bigskip
\bigskip


%\hspace{6cm}
\begin{description}
\item[Documentation] \mbox{}\\
 \credit{A.~Castro-Carrizo$^1$, R.~Neri$^1$}{}{S.~Guilloteau, 
R.~Lucas, A.~Dutrey, and S.~Radford}
\end{description}
1. IRAM

\bigskip
\bigskip

\begin{rawhtml}
  Note: this is the on-line version of the IRAM NOEMA Data reduction CookBook.
  A <A HREF="../../pdf/pdbi-cookbook.pdf"> PDF version</A>
  is also available.
\end{rawhtml}

\bigskip{} %

Related information is available in:
\begin{latexonly}
  \begin{itemize}
  \item{IRAM NOEMA: Introduction}
  \item{IRAM NOEMA: OBS Users Guide}
  \item{IRAM NOEMA: Atmospheric Calibration}
  \item{CLIC: Continuum and Line Interferometric Calibration}
  \item{MAPPING: Imaging and Deconvolution of Aperture Synthesis Data}
  \item{MIS:  Millimeter Interferometry Simulation Tools}
  \item{GREG: Graphical Possibilities}
  \item{SIC:  Command Line Interpretor}
  \end{itemize}
\end{latexonly}
%
\begin{rawhtml}
  <UL>
  <LI> IRAM NOEMA: <A HREF="../noema-intro-html/noema-intro.html">
  Introduction</A>
  <LI> IRAM NOEMA: <A HREF="../obs-html/obs.html"> 
  OBS Users Guide</A>
  <LI> <A HREF="../clic-html/clic.html"> CLIC:</A>  
  Continuum and Line Interferometric Calibration 
  <LI> <A HREF="../map-html/map.html"> MAP:</A>  
  Imaging and Deconvolution of Aperture Synthesis Data
  <LI> <A HREF="../mis-html/mis.html"> MIS:</A>
  Tools for Millimeter Interferometry Simulation
  <LI> <A HREF="../greg-html/greg.html"> GREG: Graphical Possibilities</A>
  <LI> <A HREF="../sic-html/sic.html">   SIC:  Command Line Interpretor</A>
  </UL>
\end{rawhtml}

\newpage
\tableofcontents{}


%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\newpage

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{\label{getstarted}Getting started}

\subsection{Preparing your travel}

The day by day execution of Plateau de Bure projects can be followed
at {\tt http://www.iram.fr/IRAMFR/PDB/project.html}.  The status of
the observations of each project can be checked at {\tt
http://www.iram.fr/IRAMFR/PDB/} {\tt ongoing.html}. The {\it
scientific secretary} informs the project principal investigator on
the completion of his/her program and on the steps to follow to
organize the travel to IRAM Grenoble, where the data reduction is
normally carried out. By following these instructions, you should get
in contact with your {\it local contact}\footnote{A local contact is
assigned to each A and B-rated project which does not have in-house
collaborator. He/She assists you in the preparation of observing
procedures and in the data reduction.}  or with your collaborator at
IRAM (who is the project local contact) to find some convenient dates
for both of you.  Remote data reduction is possible in exceptional
cases, after agreement with the local contact and thereby with the
{\it scientific coordinator}.

For most projects a week at Grenoble is enough to finish the data
calibration and perform first analysis. An experimented astronomer may
need less time. We recommend to spend one day on data analysis after
calibration, so that we can help if there are difficulties and return
back to data calibration if needed. You are invited to consult with
your local contact before deciding on the duration of your stay. To
avoid overbooking the computer facilities, no more than two groups are
accepted simultaneously. You should consult the visitor list before
discussing dates with the local contact. You will tell the scientific
secretary on the agreed dates as soon as possible, so that she can
book a computer, and may help in hotel booking and in other questions
related to your visit. Visits must be announced at least two weeks in
advance.  You can consult the conditions for financial support for
visiting astronomers at {\tt
http://www.iram.fr/IRAMFR/PDB/bure$\_$mission.html}. Finally, note
that IRAM is closed from Friday 6pm to Monday 8am.


\subsection{\label{IRAMcomp}Working on the IRAM computers}

As soon as you arrive at IRAM Grenoble, an account is created for your
project. Project accounts are in general allocated for a period of two
weeks and are automatically removed at the end of that period (with no
archiving or warning). You should get in contact with your local
contact in case you need more time. 

\bigskip

At your arrival a computer at the IRAM terminal room is booked for the
reduction of your project. Your local contact will provide some
information on the computer directories, download the raw data and
have a first look with you at the calibration procedure.

Project accounts contain a README file, the {\tt reports}, {\tt calib}
and {\tt maps} directories and the {\tt tmp $-$> /scratch/xxxx/}
link. In README you will find a description of the account directories
and some other helpful information. The {\tt reports} directory
contains the calibration results obtained by the {\it
pipeline}\footnote{Pipeline normally refers to the automatic
calibration taking place at Bure, which is afterwards verified,
sometimes also improved, by the astronomer on duty.}, sometimes also
those from an improved calibration by the {\it astronomer on duty
(AoD)}\footnote{Astronomer on Duty taking care of the NOEMA observations
and of the first data calibration}, and AoD's notes to help with the
calibration (Sect.\ \ref{pipeline}). The data calibration is, by
default, performed in the {\tt calib} directory, further data analysis
in {\tt maps}.

In order to download the project raw data, the command {\tt getproj}
must be executed in the shell prompt. The data are stored in the
scratch directory linked by tmp, and are visible from all \clic\
sessions launched on the computer where {\tt getproj} was executed.

By default the \gildas\ daily version is used. You can execute "{\tt
gagiram} mmmyy" (mmm = month, yy = year) to switch to a previous
\gildas\ release.

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%\newpage
%\section{\label{backup}Backup}
\bigskip

At the end of the calibration you may want to save a copy of the
project account. Reduced data can either be transferred via {\tt scp}
to your home institute or archived on DVDs. From the account home the
command ``{\rm buildDVD .}''  allows you to burn in a DVD all the things
there contained. Note that the {\tt IPB} data files are in the scratch
of the assigned computer and so by keeping the default distribution
they are not saved in the DVD. Finally, it is strongly recommended to
check the integrity of the recorded data before the account expires.

\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%


\section{\label{dataclic}Data Calibration with CLIC}

\clic\ is the software to calibrate NOEMA data. The calibration of most
projects is carried out with a few interactive procedures and widgets
provided within \clic, so most astronomers do not need to know much
about \clic\ commands. We recommend however to have a look in the
\clic\ manual\footnote{available on-line at {\tt http://www.iram.fr/}
  {\tt IRAMFR/GILDAS/}} at the following commands: {\tt help}, {\tt
  file}, {\tt find}, {\tt list}, {\tt plot}, {\tt set x} and {\tt set
  y}. During the calibration you will become familiar with others,
e.g.\ {\tt solve}. In general we recommend to read the \clic\ manual
to have a deeper understanding of the calibration possibilities.

In some of the following sections, paragraphs written with {\it
italic} fonts are aimed to solve problems that may appear in the
calibration. You can skip these paragraphs in a first reading.

\subsection{\label{pipeline}NOEMA pipeline and AoD notes}

In the project account, the {\tt reports} directory contains the first
calibration results produced or revised by the AoD
(Sect.\ \ref{IRAMcomp}).  The gzipped postscript and ascii files
report on the calibration. Particularly those whose names include {\tt
  -pipe} result from the automatic data reduction or pipeline
(Sect.\ \ref{IRAMcomp}).  You will get familiar with the calibration
reports in Sects.\ \ref{createhpbsec}, \ref{firstlooksec} and
\ref{standardcalibsec}. The AoD performing the observations evaluates
the data quality and calibration, and may write notes in the ascii
file of extension {\tt note}. It is recommended to read the notes
relative to each track before starting its calibration. The files of
extension {\tt hpb}, so-called {\it header} files, store the
parameters derived from the calibration.

Having a look at the automatic reports before starting the calibration
may be useful, mainly for an experimented astronomer. They can be
used, for instance, to decide on the first track to calibrate, likely
the one with the best flux calibration. In addition, note that the AoD
at Bure performs a comprehensive analysis of the observations, and
his/her notes (in the {\tt note} file) may be useful to identify
problems, which perhaps are visible in some of the plots of the {\tt
  First Look} report (see Sect.\ \ref{firstlooksec} and
App.\ \ref{pipeshowsec}). Recommendations concerning the calibration might
also be added to the AoD notes.

Nowadays the {\tt pipeline} is distributed with every release of {\tt
  GILDAS}, although it can only be used with data obtained later than
January 2007.  You just need to type ``{\tt @pipeline `project\_name'
  `date'} '' and the pipeline will be launched, producing outputs very
similar to those obtained at the observatory. It has become usual
among NOEMA users to start the data calibration by launching the {\tt
  pipeline}. If this first calibration is not satisfactory, the
analysis of the {\tt pipeline} report, with the help of the AoD notes
and the local contact, allow improving easily the data
calibration. Anyway, we recommend to have a critical look at the {\tt
  pipeline} output protocols before making use of the {\tt hpb}
files. Particularly, it is known that the {\tt Data Quality
  Assessment} procedure launched at the end of the calibration often
flags out more data than actually needed. If you plan to recalibrate
all or parts of the calibration from the {\tt pipeline}-produced {\tt
  hpb} file, we recommend first to remove all the {\tt Data Quality
  Assessment} flags by using the following commands:

\hspace{3cm} \begin{minipage}{17cm}
{\tt 
\noindent
file both `{\it file\_name.hpb}' \\
find \\
store flag redu /ant all /reset \\
store flag redu /base all /reset \\
}
\end{minipage}

At the end of the new calibration you can use the {\tt Data Quality
  Assessment} procedure (Sect.\ \ref{visibassess}) to flag the
undesirable data according to the needs of your project.



\subsection{\label{scansec}How the data look like}

Let us assume that we are already looking at the data, in \clic.
After typing in \clic\ {\tt find} and later {\tt list} we will see,
e.g.:
\begin{verbatim}
60 7684 QA12 3C454.3      P CORR CO10     6Cq-N17      12-NOV-2007 21:58  2.9
61 7685 QA12 3C454.3      P CORR CO10     6Cq-N17      12-NOV-2007 21:59  2.9
62 7686 QA12 3C454.3      P CORR CO10     6Cq-N17      12-NOV-2007 22:00  2.9
\end{verbatim}
where each line corresponds to an observation {\it subscan}\footnote{A
  {\it subscan} (which is a scan partition) is the shortest
  acquisition for which spectral information is
  stored. Frequency-averaged acquisitions are stored for each second,
  and are known as {\it records}}. Each subscan contains the following
information:
\begin{verbatim}
 - First column: Observation number
 - Second column: Scan number
 - Third column: Project name       
 - Fourth column: Source name
 - Fifth column: Type of source (O=object, P=phase calibrator)
 - Sixth column: Type of scan procedure
 - Seventh column: Line name
 - Eighth column: Array configuration, i.e. antenna positions
 - Nineth column: Date
 - Tenth column: UT time
 - Eleventh column: Hour angle
\end{verbatim}
The usual observational procedures (column sixth) and characteristics
are:
\begin{verbatim}
 CORR: cross-correlation; 1 subscan (ss) is obtained per scan 
 GAIN: cross-correlation to measure the sideband rejection; 1 ss
 FOCU: focus measurements in all the antennas; 5 ss
 POIN: interferometric pointing in all the antennas; 2 ss
 FLUX: cross-correlation to measure the flux; 1 ss
 IFPB: IF passband calibration, by observing a noise diode; 2 ss
 AUTO: autocorrelation; 1 ss
 CALI: atmospheric calibration, autocorr. on SKY, HOT [COLD] load; 2 [3] ss
 SKYD: sequence of autocorrelations to calibrate the 22 GHz receivers; 12 ss
\end{verbatim}
By using the mentioned procedures, a typical sequence of observations
is:
\begin{verbatim}
 - Receiver tuning followed by a sideband gain measurement
 - Radio frequency passband calibration on a bright source
 - Acquisitions on at least a flux calibrator, MWC 349 if possible
 - Check the pointing and focus on the predefined phase calibrator(s)
 - Possibly a flux measurement on this calibrator(s)
 - Cross-correlations on the phase calibrator(s)
 - Cross-correlations on source
\end{verbatim} 
About 3 minutes are spent on each phase calibrator, some more time if
pointing and focus are performed, every $\sim$ 23 minutes on
source. Pointing and focus are normally repeated every two transitions
of $\sim$ 23 minutes on source. {\tt IFPB}, {\tt AUTO} and {\tt CALI}
scans are obtained before correlations to calibrate the observations
in real time (see Sect.\ \ref{onlinecal}).


\subsection{Getting started with CLIC}

Just type \clic\ in the shell prompt, and click on the \clic\ widget
shown in Figure \ref{clicmenu} to have the menu. 

\begin{figure}[bh]
\begin{center}

\hspace{-5cm}

%\includegraphics[height=6cm]{clic.eps} 
%\includegraphics[height=1.5cm]{pdbi-cookbook-01} 
\includegraphics[height=6cm]{pdbi-cookbook-01} 
\caption{\clic\ menu}
\label{clicmenu}
\end{center}
\end{figure}

In the following sections we describe the steps to follow in order to
calibrate a standard project, following the options listed from top to
bottom in the \clic\ menu. 


\subsection{\label{createhpbsec}Open raw data file}

The data obtained at Plateau de Bure are stored in files of extension
{\tt IPB}. The parameters derived from the calibration are stored in
files of extension {\tt hpb} (or {\it header} files). A naked version
of these files can be created with the option {\tt Open raw data file}
(Figure \ref{createhpb}) in the \clic\ menu (Figure \ref{clicmenu}).

First of all, the location of the raw data ({\tt IPB} files) must be
defined, either by clicking in {\tt RAW DATA DIRECTORIES} (see Figure
\ref{createhpb}) or within the {\tt .gag.dico} gildas file. In the
IRAM project accounts (see Sect.\ \ref{IRAMcomp}) the location of the
raw data is already predefined. A raw data file must be selected among
the {\tt Project$\_$Date} choices. {\tt OPEN and FIND} opens this file
and proposes a standard name for the output file. {\tt CREATE HEADER
FILE} creates a {\tt hpb} file ready to calibrate the data in
antenna-based mode. By default {\tt Mode} (in the widget) is NEW, and
so a new {\tt hpb} file is created each time. By selecting OLD,
visibilities can be added to an existing {\tt hpb} file, which is very
convenient if observations were contiguous in time.

\begin{figure}[h]
\begin{center}

\hspace{-3cm}

%\includegraphics[height=4cm]{createhpb.eps} 
\includegraphics[height=4.5cm]{pdbi-cookbook-02} 
\caption{{\tt Open raw data file}, in the \clic\ menu.}
\label{createhpb}
\end{center}
\end{figure}

In this and the following widgets there are at the top the options
{\tt GO} and {\tt ABORT}.  {\tt GO} is aimed to run all the options
proposed as clickable buttons, from left to right, with no stop. In
general, its use is only recommended for the {\tt First Look} (see
Sect.\ \ref{firstlooksec}). {\tt ABORT} closes the widget and returns
to the parent widget.

As mentioned in Sect.\ \ref{pipeline}, the {\tt hpb} files resulting
from the NOEMA pipeline calibration are given in the directory {\tt
reports}. We recommend not to use them to create the final {\it
uv-}tables since, for the time being, they were not created with this
purpose and so may include severe flags or artifacts due to a further
AoD analysis.


\subsection{\label{firstlooksec}First Look}

To have a look at the conditions in which the observations were
obtained click on the {\tt First Look} in the \clic\ menu (Figure
\ref{clicmenu}). An extended version of the {\tt First Look} is used
by the AoD and by the pipeline procedure (see
App.\ \ref{pipeshowsec}).

An {\tt hpb} file name is needed to start with the {\tt First Look}
widget (see Figure \ref{firstlook}). The scan range can also be
specified. By default all the scans are taken into account (i.e.\ from
{\tt 0} to {\tt 10000}). The receiver with which the source(s) was
observed is determined by clicking on {\tt Select}.

{\tt Select} is also necessary to assess the presence of data, and to
define variables needed in the following calibration steps. The
options from {\tt Meteo} to {\tt Water} can be clicked with no defined
order. Each of them creates one or two plots and then pauses waiting
for a ``{\tt continue}'', in the \clic\ prompt, in order to save the
displayed plots in postscript files. {\tt Print} combines all these
postscript files in a file called show-'date'-'projectname'.ps,
including also a description of the observational setup and a detailed
scan list. 

This package is actually not interactive since it does not admit an
input other than ``{\tt continue}''. With {\tt GO} we can therefore
obtain the same result as by clicking through the buttons, from {\tt
Select} to {\tt Print}. It is recommended to check the {\tt First
Look} report to identify instrumental features, errors, verify weather
conditions, etc. This information becomes particularly relevant to
take adequate decisions in case of calibration difficulties.
%After the calibration, the ``Quality Assessment''
%procedure will verify if pointing, focus and tracking errors are
%important enough with respect to the beam size to propose flagging on
%source data affected.
 

Note that all the plots created with the {\tt First Look} are
presented for {\it physical antennas}\footnote{Each NOEMA antenna has a
reference number that identifies it, from 1 to 6 corresponding to the
order in which they were built. When we refer to the antennas by using
this reference number, we call them physical antennas.}, while the
calibration procedures refer to {\it logical antennas}\footnote{\clic\
uses a numbering for the antennas proper to each observation, going
from 1 to the number of used antennas.  We refer to logical antennas
when we use their reference number in \clic. Note that the logical
antennas differ from the physical ones if the array has less than 6
antennas and the missing antennas are not the last ones. }. Sometimes
the AoD refers in his/her notes to physical antennas, but note that
all the \clic\ commands refer to the logical ones. In the section 1.2
of the calibration report (see Sect.\ \ref{pipeline} and
\ref{standardcalibsec}), efficiencies are presented for logical
antennas, the equivalent physical numbers are shown into parenthesis.
%Logical and physical antennas may differ when
%observations were performed with less than six antennas.

\begin{figure}
\begin{center}

\hspace{-5cm}

\includegraphics[height=5.5cm]{pdbi-cookbook-03}
\caption{{\tt First Look}, in the \clic\ menu}
\label{firstlook}
\end{center}
\end{figure}

\bigskip

\noindent
{\bf Select}, in addition to selecting the used receiver(s), adopts
default settings for data presentation, reports on the amount of
correlations obtained, defines scan ranges between tunings by looking
at the {\tt gain} scans, reports on the used baselines and antennas,
on the array configuration, on the used calibrators and their fluxes,
and the tuned band. {\tt Select} also introduces in the widget the
option to select telescope configuration if that changed during the
observations. Information is given in the line of commands, which is
specially relevant for calibration (as shown in Figure \ref{prompt1}).

\begin{figure}
\begin{center}

\hspace{-5cm}

\includegraphics[height=19cm]{pdbi-cookbook-04}
\caption{Information given in the line of commands after {\tt Select}, in the 
{\tt First Look} and the {\tt Standard Calibration} sections.}
\label{prompt1}
\end{center}
\end{figure}

\bigskip

\noindent 
{\bf Meteo} provides information on the atmospheric conditions at the
time of the observations: Ambient temperature (in K), relative
humidity (\%), pressure (mbar) and the mean and maximum wind speed
(m/s).

\bigskip

\noindent 
{\bf Elev-Azi} plots the elevation and azimuth of each scan, so that
we can see the relative position of a source with respect to the
calibrators.

\bigskip

\noindent 
{\bf Poin-Foc} presents the pointing corrections (in arc seconds) in
azimuth and elevation, and the focus corrections (in mm) per
antenna. Note that pointing and focus measurements are performed
typically every $\sim$ 50 min (i.e.\ every two transitions on source)

\bigskip

\noindent 
{\bf Tracking} shows the rms of the tracking variations (in arc
seconds) in azimuth and elevation.

\bigskip

\noindent {\bf Tot. Pow.} displays the total power (in K) obtained per
{\it correlator input}\footnote{There are two correlators at the
  observatory: 1) The Narrow-band correlator accepts two 1 GHz input
  signals, from the 4 GHz band obtained per polarization. Narrow
  inputs refer to the two 1GHz-bandwidth signals entering into the
  narrow-band correlator, which correspond to quarters of the receiver
  bands. The astronomer decides on the configuration of the eight
  spectral units of the narrow-band correlator within the two selected
  Narrow inputs.
% 
2) The Widex correlator is composed of four units, each one accepting
2 GHz bandwidth. Units 1 and 3 and units 2 and 4 are placed
consecutively to cover a bandwidth of 3.8 GHz respectively in the IF
of H and V polarization receivers. Widex inputs correspond hence to
the four Widex units)}.

%\bigskip

%\noindent 
%{\bf Cable} to be removed

\bigskip                                                           
                            
\noindent 
{\bf Tsys} plots the system temperature (in K) along the track, for
the tuned and the rejected band. Note that after January 2007 all
receivers installed at Plateau de Bure are operated in single
side-band mode. A plot is created per correlator
input\footnotemark[8].

\bigskip

\noindent 
{\bf Water} shows the values derived for the water vapor pressure (in
mm) by modeling the atmospheric behavior from the data obtained with
the astronomical receivers. The water vapor pressure ({\tt WVR
  H$_2$O}) obtained from the 22GHz receivers is also shown. Note that
the 22GHz receivers are only sensitive to water vapor, while the
astronomical ones permit to obtain a rough estimate of the water in
the atmosphere, including liquid water. A plot is created per
correlator input\footnotemark[8].

\bigskip

\noindent 
{\bf Print} combines all the plots in a file of name
show-`date'-`projectname'.ps.

\medskip

\noindent 
{\bf AoD or pipeline First Look:} The astronomer on duty carrying out
the observations performs a deeper analysis of the data. The file
created by the AoD {\tt First Look} includes more technical plots,
related to receiver calibration, bandpass stability, correlator
tweaks, 22\,GHz receivers, cables transporting the synthesizer
signals, etc (briefly described in App.\ \ref{pipeshowsec}). The AoD
may include remarks in the project {\tt note} file (see
Sect.\ \ref{pipeline}) if the inspection of these plots reveals
problems to be considered in the calibration.

\subsection{\label{standardcalibsec}Standard Calibration}

This tool is designed to calibrate a standard NOEMA project. The name of
the {\tt hpb} ({\it header}) file must be specified in the {\tt
Standard Calibration} widget (Fig.\ \ref{standardcalib}) to start. A
limited scan range can be selected to perform a complete or partial
calibration. By default all the scans are considered. The receiver
band used in the observations is determined by clicking on {\tt
Select}.

\begin{figure}
\begin{center}

\hspace{-5cm}

\includegraphics[height=6.5cm]{pdbi-cookbook-05}
\caption{{\tt Standard Calibration}, in the \clic\ menu}
\label{standardcalib}
\end{center}
\end{figure}


The calibration must be done by clicking on the proposed steps from
left to right. {\tt Select} is so needed at first, and also before
repeating any part of the calibration.
%; in those cases, the following steps must be also repeated.  
The last option {\tt Print} combines all the postscript files created
of each calibration step in a single file called
`date'-`projectname'.ps. This package is an interactive tool which
pauses for an input from the astronomer. It is not recommended
therefore to click on the {\tt GO} (see Sect.\ \ref{createhpbsec}).

The option {\tt Use previous settings} (in the widget; Fig.\
\ref{standardcalib}) permits to adopt settings coming from a previous
calibration of the selected {\tt hpb} file, with respect to the
atmospheric phase correction, scan ranges, selected bandpass
calibrator, etc. 

Finally, note that all the plots created within the {\tt Standard
  Calibration} are presented for logical antennas, which are sometimes
different from the physical ones shown in the {\tt First Look} outputs
(see Sect.\ \ref{firstlooksec} and App.\ \ref{pipeshowsec}).


\subsubsection{\label{select}Select}

It selects the used receiver band(s), establishes defaults for data
presentation, reports on the amount of correlations obtained, scan
ranges between tunings, array configuration(s), lists the physical
baselines and antennas, builds a first flux list and reports on the
tuned band. In addition, it defines internal variables which are later
used by the calibration procedures.

The procedure also selects the RF calibrator, at first the brightest
observed source, otherwise that selected in a previous RF
calibration. The selection is reported in the line of commands, as
shown in Figure \ref{prompt1}. Sometimes, because of technical
problems or to improve sensitivity, another calibrator must be
considered. In those cases ``let band$\_$source {\it
`a$\_$new$\_$calibrator'} '' must be typed in \clic\ to select {\it
`a$\_$new$\_$calibrator'}.

{\tt Select} deduces which are the phase calibrators observed
alternatively with the source, hence suitable to calibrate phase and
amplitude evolution in time. The names of the selected calibrators are
stored in the variable {\tt phcal}. You can decide about the content
of {\tt phcal} as follows: ``let phcal `{\it calibrator1}' `{\it
  calibrator2}' ... ''. Note that ``let phcal `*' '' should be
selected if the (frequency averaged) phases from the H and V
polarization receivers are not identical, which for instance can be
seen in the {\tt RF phases} plot in the {\tt First Look} (see more
details in Sect.\ \ref{phcal}).

By default, the atmospheric phase corrections derived from the 22GHz
receivers are used according to the {\tt PhCor} evaluation (see
Sect.\ \ref{phcor}). However, sometimes it is recommended (by the {\tt
  Autoflag} procedure or by the AoD) not to use the phase corrections,
for instance due to problems with the 22GHz receivers.  ``let
do$\_$atm no'' allows ignoring the atmospheric phase corrections.

{\bf Several gain ranges:} From time to time, the {\tt Select}
procedure pauses to inform on the presence of different gain ranges,
i.e. scan intervals with different tuning characteristics. The RF
calibration is then performed independently for each gain range ({\it
  i}), and a calibrator is so assigned to the variable
``band$\_$source[{\it i}]'' for each range. When the procedure pauses,
{\tt continue} should be typed in the line of commands to continue.

\subsubsection{Autoflag}

Different verifications are performed in all the obtained scans, which
are flagged if anomalies are identified. For instance, {\tt Autoflag}
checks that the timing of the acquisitions is correct. It also
verifies that no source data are surrounded only by flagged calibrator
acquisitions, which for instance may happen if an antenna is
shadowed. A message is given for information if problems are found,
reporting about introduced flags.

{\bf Warnings, periods of observational peculiarities:} If the
observations were obtained in periods with special conditions or
difficulties, a message is shown to inform about it and about the
possible consequences in the calibration.

\subsubsection{\label{phcor}PhCor}

{\tt PhCor} compares the amplitude of each scan with and without
atmospheric phase correction. The phase correction is only adopted
when it reduces the phase decorrelation within each scan.  Note that
the spectral units, L01 to L12, do not store information on time
scales smaller than a scan, while the continuum units, C01 to C12, do
so; two streams of data are so stored at Bure for each L0i and C0i
unit, respectively with and without atmospheric phase correction.

A green line on the bottom of the plot produced during the flux
calibration (see Figure \ref{solveflux}) shows, when being above zero,
the scans for which the program {\tt PhCor} activates the atmospheric
phase correction.

In addition, this procedure checks for interferences in the WVR
channels, if it was not done before (for instance by the pipeline).

Note that the data are not corrected for atmospheric decorrelation if
``let do$\_$atm no'' was selected (see Sect.\ \ref{select}).

At the end, a procedure is launched to verify if the phase calibrators
are polarized, setting the variable {\tt do$\_$avpol} accordingly (see
Sect.\ \ref{ampcal}) to define the amplitude calibration mode.



\subsubsection{\label{rfcal}RF calibration}

The goal is to measure the receiver bandpass (RF) to calibrate the
source data. For this, a bright calibrator is observed for some
minutes. By default, in a first calibration the procedure {\tt Select}
selects the brightest observed source for the RF calibration. The
selected RF calibrator can be changed by modifying the variable
``band$\_$source'' as explained in Sect.\ \ref{select}. If several
gain ranges are found, an independent RF calibration is performed for
each range.

The procedure pauses to propose fits for amplitudes and phases for
each correlator input (as shown in Figure \ref{rfcalib}); by default
the solutions for the Narrow correlator inputs are obtained with
``solve rf 12 20 /plot'', where ``12'' is the polynomial degree to
fit amplitudes and ``20'' for phases (higher degrees are used for the 
Widex correlator units). 
\begin{figure}
\begin{center}

\hspace{-5cm}

\includegraphics[height=10cm]{pdbi-cookbook-06}
\caption{Example of the RF calibration of data obtained for the
second Narrow correlator input}
\label{rfcalib}
\end{center}
\end{figure}
Changing the polynomial degrees with respect to the ones proposed by
the automatic procedure rarely improves the RF calibration. In the
pause, ``continue'' (or ``c'') should be typed to accept the solution
and store the calibration parameters in the {\tt hpb} file.



\bigskip

{\it \small 

Sometimes the fit of the bandpass looks poor with deviations of a few
degrees (in phase) or a few percents (in amplitude) between the
measured data and fits are observed.  For the time being there are no
simple means to reduce these small differences.  In general, they have
no influence on the results.

Sometimes the procedure crashes (when fitting) because the data of an
antenna is flagged. Two solutions exist: (1) Mask (see point 2 in
Sect.\ \ref{faq}) shadow, saturati[on] or redu[ction] flags to use
these data for the RF calibration if the signal to noise ratio is good
enough. (2) Select another calibrator by typing ``let band$\_$source
{\it `a$\_$new$\_$calibrator'} '' as indicated in Sect.\ \ref{select}.

If delays are observed (constant slope in phases vs.\ frequency for
one or more antennas) we may correct for it to decrease the phase
decorrelation within scans. 
%This is more relevant when a larger frequency interval separates 
%both narrow inputs, often when both inputs correspond to the same 
%polarization. 
By using the commands ``solve delay /plot'' and ``modify delay'' we
can correct for this (see Sect.\ \ref{diff}).

Differences in the LO1ref frequencies result in frequency offsets in
the RF amplitude profiles. If the AoD (through the project notes)
informs you on the presence of a big LO1ref jump (which is often
linked to cable phase jumps) we recommend to calibrate the RF with the
strongest phase calibrator observed cyclically with the source, for
which the source LO1ref frequency was properly considered.  }

\subsubsection{\label{phcal}Phase calibration}

\begin{figure}
\begin{center}

\hspace{-5cm}

\includegraphics[height=10cm]{pdbi-cookbook-07}
\includegraphics[height=10cm]{pdbi-cookbook-08}
\caption{Example of the phase calibration of data obtained for the
horizontal polarization. The command ``solve phase /plot'' was used in
the first plot, ``solve phase /plot /break 1 14.7 3 15.2'' was used in
the second one. The introduction of sharp breaks in the phase
calibration should always result from an analysis of the reasons for
such phase jumps. }
\label{solvephase}
\end{center}
\end{figure}

In most projects we observe, alternatively to the source(s), one or
two phase calibrators, for which `ideal' phases (without instrumental
contribution) are known to be zero for all the baselines. The track of
these phases allows us to measure and remove possible instrumental
contributions. The phase fittings for the calibrators observed
cyclically with the source (set by {\tt Select}, Sect.\ \ref{select})
are obtained with the command ``solve phase /plot''. Breaks and jumps
can be introduced in the fit with the option ``solve phase /break {\it
  `break$\_$degree' `break$\_$time'} ''. {\it Break$\_$degree} is the
order of the polynomial derivative that is discontinuous at the break
point, the sharpest one being 0. {\it Break$\_$time} is the UT time
(h) at which the break should happen. (See two examples of phase
calibration in Figure \ref{solvephase}.) The option ``/weight'' can be
used to take into account the weight ($\sim$ SNR) of the different
calibrators, and ``/polynomial'' to impose polynomials of a given
degree (the last option is incompatible with the use of breaks).  More
information can be found in the \clic\ manual, or by typing ``help
solve phase'' in \clic. The procedure pauses after proposing a fit. A
previous solution can be removed from the plot with ``clear seg'' to
try a new one. A solution is accepted by entering ``c'' in the line of
commands. The phases are calibrated independently for each
polarization, and stored in the {\tt hpb} file at the end of the
procedure.


The choice of phase calibrators used to calibrate the source phases
and amplitudes is performed by {\tt Select} (see
Sect.\ \ref{select}). If you disagree with the choice, it can be
modified with ``let phcal `{\it list$\_$of$\_$calibrators}' ''. The
phases and amplitudes of the calibrators not included in this `{\it
  list$\_$of$\_$calibrators}' are self-calibrated. Note that if the
frequency-averaged phases from H and V polarization receivers are not
equal (which for instance can be seen in the {\tt RF phases} plot, see
Sect.\ \ref{pipeshowsec}) self-calibration should be avoided, and
hence ``let phcal `{*}' '' should be entered after {\tt Select}
(Sect.\ \ref{select}).

%By default the phase calibration is performed by using the continuum
%units, and the atmospheric phase correction as concluded after {\tt
%PhCor}, for each polarization.

\bigskip

{\it \small 

If due to some poor data obtained in a limited time interval, the fit
fails in other interval in which the phases seem well constrained, we
can divide the phase calibration in two parts, by specifying the scan
intervals in the main calibration widget. We should just pay attention
that no source data remains uncalibrated between the two selected
intervals.

From time to time the procedure crashes because too many calibrator
data are flagged for an antenna or baseline. To find a proper solution
for the other baselines we can just mask (see point 2 in
Sect.\ \ref{faq}) whichever the flags, solve the phases for all
baselines, and reset at the end these masks. Verify at the end that
the source data that were calibrated with flagged calibrator data are
not used.

If the data obtained for a calibrator are too bad, and are not needed
in the calibration, we can flag them as indicated in Sect.\
\ref{faq}.  }


\subsubsection{\label{fluxcalibsec}Flux calibration}

When clicking on {\tt FLUX} (in the CLIC menu; Fig.\ \ref{clicmenu}) a
widget similar to the one shown in Fig.\ \ref{fluxcalib} is
opened. The flux calibration is an iterative process in which the
known flux of one or more calibrators is fixed to determine the
efficiencies (Jy/K) of the antennas, which are then used to estimate
the flux of the other calibrators. When a flux is fixed, {\tt SOLVE}
derives efficiencies and fluxes. {\tt GET RESULT}, {\tt STORE}, and
{\tt PLOT} store the solved flux densities and plot the amplitudes
scaled by the derived fluxes (in K/Jy). These scaled amplitudes
correspond to the inverse of the antenna efficiencies, that, in an
ideal project, should remain constant and equal to their nominal
values.

The amplitudes obtained for a calibrator often vary along the track
due to effects of a changing atmosphere or instrumental
problems. Antenna efficiencies should be estimated by considering the
best data ranges. Observational glitches, data obtained with bad
pointings or focus measurements, or limited intervals of bad data
should not be considered. If for an observed polarization amplitude
oscillations on a 24 hour scale are observed, this is often an
indication of that the emission is polarized. Having H and V
polarizations is needed to confirm the presence of polarized
emission. (The degree of polarization of the NOEMA phase calibrators
is evaluated and archived at the observatory, and so can be checked by
your local contact if needed.) Anyway, there is nothing to do in the
flux calibration with respect to this.  The {\tt Scan List} option
permits to select the scan ranges to be considered in the calculation
of fluxes and efficiencies.  After {\tt PLOT}, scan numbers can be
determined by using the command ``cursor'' and clicking on the display
(an example is shown in Figure \ref{solveflux}).

All the calibrators observed during the track are shown in this
widget. Currently the main flux calibrator is MWC\,349, which is
observed in most of the projects; when included, a flux is proposed to
be fixed, as we can see in Fig.\ \ref{fluxcalib}. Its use must be
however considered by checking the quality of the observed
correlations: i.e.\ correlations on MWC\,349 showing a big scatter in
amplitude should not be considered unless they are representative of
the track observing conditions.  The other calibrators, the one used
to calibrate the RF and also the phase calibrators, can be used in
this process. Their flux may be known from other tracks observed close
in time. Note also that we monitor the flux of the brightest
calibrators. Your local contact can provide you this information, and
also an estimate of the right efficiencies with a reasonable
accuracy. For example, for a track observed in good conditions the
expected efficiencies for the different antennas should range from 20
to 25 Jy/K, from 25 to 32 Jy/K, and from 32 to 45 Jy/K for receiver
bands 1, 2, and 3 respectively. Values much larger than those expected
should be well explained by the observational conditions. Efficiencies
significantly smaller are not possible.


\begin{figure}[]
\begin{center}

\hspace{-8cm}

\includegraphics[height=10cm]{pdbi-cookbook-09}
\caption{Flux Calibration widget, showing on top the solved
antenna efficiencies. Note that these are characteristics of the
antennas, and the different values obtained between antennas should be
well understood by the AoD at the time of the observations and likely
also by your local contact. For instance, the low efficiency of A6 at
the moment of these observations was due to a polluted injected LO
signal; a new LO box was installed in A6 just after these
observations.  In the second row we define the scan intervals to be
considered in the flux calibration. }
\label{fluxcalib}
\end{center}
\end{figure}
By properly using this method, the absolute calibration obtained is
typically precise to less than 10$\%$ at 3mm and $\sim$ 20$\%$ at
1mm. Special attention should be payed to the relative flux
calibration between different tracks; normally the different tracks
provide complementary information in the {\it uv}-plane, and so a
wrong relative flux calibration can be interpreted as source
structure.

\begin{figure}[]
\begin{center}

\hspace{-8cm}

\includegraphics[height=10cm]{pdbi-cookbook-10}
\caption{Flux Calibration plot, showing the amplitudes scaled by the
solved fluxes. Note that the amplitudes from both Narrow correlator
inputs are averaged in this plot. }
\label{solveflux}
\end{center}
\end{figure}

As mentioned in Sect.\ \ref{phcor} the green lines at the bottom of
the plots show, when being above zero, the regions in which the
atmospheric phase correction is applied as resulted from {\tt
PhCor}.

Note finally that the option {\tt CHECK}, at the top of the FLUX
calibration widget (Fig.\ \ref{fluxcalib}), permits to obtain a
solution by fixing a reference calibrator and ignoring scans of
quality below a certain threshold. This may be used as a first try, a
second iteration is often needed (after storing the first one). A
solution is stored with {\tt GET RESULT} and {\tt STORE}, and a new
{\tt PLOT} should be created accordingly.
\bigskip

{\it \small 

The flux calibration is performed by averaging the amplitudes from all
the spectral units, i.e.\ from all the correlator
inputs\footnotemark[8], H and V polarizations. Delays (see
Sect.\ \ref{rfcal} and \ref{faq}) result in flux losses due to that
phases are frequency averaged. To correct for remaining delays you
should follow the instructions given in Sect. \ref{diff}.  Also, as
mentioned in Sect.\ \ref{phcal}, differences in the (frequency
averaged) phases from H and V polarization receivers introduce
amplitude losses in the flux calibration for the RF and flux
calibrators. See Sect.\ \ref{phcal} to correct for this effect. Note
anyway that either the presence of delays or polarization differences
are rare, since the standard NOEMA observing procedures correct for
them at the very beginning of each track.

If flagged data are masked, such masks should be reset before the flux
calibration. Since fluxes are solved by averaging the data of all the
spectral units, we may not be able to identify problems coming from
some flagged data from, for example, one of the narrows. }



\begin{figure*}[h!t]
\centering
\includegraphics[height=8cm]{pdbi-cookbook-11}
\includegraphics[height=8cm]{pdbi-cookbook-12}
\caption{Example of the standard amplitude calibration, performed
independently for the V and H polarization receivers. In this example
the calibrator emission is found to be polarized. The option ``let
do$\_$avpol yes'' will allow repeating the amplitude calibration by
averaging the calibrator emission for both polarizations, as shown in 
Figure \ref{avpol}.}
\label{amppol}
\end{figure*}

\subsubsection{\label{ampcal}Amplitude calibration}

The amplitude calibration is similar to the phase calibration in its
concept: fitting and removing the amplitude changes observed in the
calibrators along the track. The command to be used is ``solve
amplitude'' with the same options as for the phase calibration, in
Sect. \ref{phcal}.  Scaled amplitudes are plotted, so we can verify
that the flux calibration, in relative terms, was correct.

Sometimes we find complementary (between polarization V and H)
variations of the amplitudes in time, indicating that the calibrator
emission is polarized (see Figure \ref{amppol}). The amplitudes of
both polarizations should be then averaged to cancel such variations
(see Figure \ref{avpol}). By typing in the line of commands ``let
do$\_$avpol yes'' the amplitude calibration is performed in the
average mode. At the end of the {\tt PhCor} procedure an assessment on
the polarization of the phase calibrators is performed (if not done
before, for instance by the pipeline), and the variable {\tt
  do$\_$avpol} is set accordintly.

\begin{figure}[h!t]
\begin{center}
\includegraphics[height=10cm]{pdbi-cookbook-13}
\caption{Example of the amplitude calibration in the average mode,
i.e.\ by averaging the emission of V and H polarization receivers. }
\label{avpol}
\end{center}
\end{figure}


\bigskip

{\it
\small

The amplitude calibration procedure presents similar characteristics
and problems to those described for the phase calibration, in the
paragraphs with italic-fonts in Sect.\ \ref{phcal}.

In addition, pointing and focus problems do often result in amplitude
losses, mainly at higher frequencies, when the primary beam is
smaller. A proper fit of the amplitudes affected by these problems may
allow to correct for it; note that this is only valid if the source
emission is expected to be centered and compact. We should also
consider the proximity of the source to the calibrator(s).  }

\subsubsection{Print}

{\tt Print} combines all the postscript files created at each step of
the calibration to produce a file named `date'-`projectname'.ps. In
its first pages the calibration quality is summarized, including
resulting efficiencies and flux densities, and the rms of the RF,
phase and amplitude calibrations.


\subsubsection{\label{diff}Usual difficulties}

\noindent 
{\bf Baselines}

\begin{figure}
\begin{center}

\hspace{-5cm}

\includegraphics[height=10cm]{pdbi-cookbook-14}
\caption{Example of phases affected by a `baseline' problem. The
separation of the phases obtained for the two phase calibrators as
well as the large phase drifts are characteristics of a wrong baseline
model. In this case, the position of antennas 3 and 5 were wrong in
$\sim$ 5mm. }
\label{baseline}
\end{center}
\end{figure}

Sometimes, during the observations of a project, a good baseline model
(a precise determination of the antenna positions) is not
available. This introduces typical phase drifts that are often well
identified by an experienced astronomer (see Figure
\ref{baseline}). Normally the AoD reports on this in the project {\tt
notes} and often proposes a baseline solution obtained hours or days
later. Baseline solutions obtained close to each track are given in
the directory {\tt baselines}.  Otherwise, they can be provided by the
local contact. To apply a baseline model, ``set /def'' and ``find''
should be entered at first in \clic, followed by the command ``modify
antenna 1...2...3...4...5...6.../offset 99'' as proposed by the best
baseline model, either found in the in the AoD notes or in the {\tt
baselines} directory. Note that this command refers to logical
antennas, which should be well identified within the solution if the
number of antennas is smaller than six. Verify with your local contact
if any doubt.

\bigskip


\noindent 
{\bf Delays}


\begin{figure}
\begin{center}

\hspace{-5cm}

\includegraphics[height=10cm]{pdbi-cookbook-15}
\caption{Example of a RF calibration where we can appreciate a remaining  
delay in the baselines including A2. Delays can be solved with the 
commands ``solve delay'' and ``modify delay'' (see Sect.\ \ref{diff}). }
\label{delay}
\end{center}
\end{figure}

As mentioned in Sect.\ \ref{rfcal}, sometimes data must be corrected
for delays (see Figure \ref{delay}). To determine the delays we first
select a polarization (with ``set polar ..'') and then plot phases
(with ``set y phase'') vs.\ the if1 frequency (or band frequency, with
``set x if1''), preferably after selecting antenna-mode (with ``set
antenna all'').  Delays are determined with ``solve delay /plot
/print'' for each polarization. To enter a solution, all concerned
scans must be selected with ``find'' and then the command ``modify
delay ...''  must be typed, as proposed by the solution. After
correcting delays the RF calibration must be repeated.

Note that, for the time being, the continuum units are not corrected
offline for delays. If delays are modified, we advise using spectral
units (instead of continuum ones) at each step of the calibration
process.

\bigskip

\noindent 
{\bf Atmospheric Phase Correction}

By default the atmospheric phase correction derived from the 22GHz
receivers is applied according to the results obtained with the {\tt
PhCor} procedure. We can however remove it by typing ``let do$\_$atm
no'' in the line of commands before performing any calibration.  If a
previous calibration was already carried out applying the atmospheric
phase correction, the option {\tt no} must be also selected in {\tt
Use previous settings}. The whole calibration must be then repeated.
On the upper-left corner of each data plot we can see {\it Ph:
Abs./Rel(A). Atm}, where {\it Atm} reports on the presence of the
atmospheric phase correction (according to {\tt PhCor}).

See additional information in Sect.\ \ref{phcor}.

\bigskip

\noindent 
{\bf Several telescope configurations}

If a track includes observations with different telescope
configurations, for example if an antenna was removed during the
observations, after clicking on {\tt Select} the different
configurations will appear in the main calibration widget.  The
calibration must be perform independently for each configuration. {\tt
Use previous settings} is then set off.


\subsubsection{\label{faq}FAQ}

\bigskip
\noindent
$\bullet$ {\it How to proceed to repeat a calibration step?}  If you want
to repeat a part of the calibration, you do not need to repeat all the
calibration steps happening before, but those taking place later
according to the scheme here described. Note that clicking on {\tt
Select} (within the {\tt Standard Calibration} widget) is always
recommended before repeating any calibration step, in order to have
proper settings.

\bigskip
\noindent
$\bullet$ {\it How can I mask flagged data?} Sometimes procedures
crash because a lot of data of an antenna or baseline are flagged. We
can verify the kind of flags with the command ``list /flag''.  We can
mask them with the command ``mask {\it flag$\_$label} /ant {\it i}''
(or ``/baseline {\it ij}''). For example, ``mask shadow /ant 3'' would
mask the flags of antenna 3 (logical) due to shadowing.  Masking flags
without a clear understanding of what it means is not
recommended. However, masking certain calibrator flags may be
convenient in some cases (see Sect.\ \ref{rfcal}, \ref{phcal} and
\ref{ampcal}). If flagged data are masked in some part of the
calibration, the masking should be reset with ``mask /reset'' (e.g.\
``mask shadow /ant 3 /reset'') at the end of the calibration step.

\bigskip
\noindent
$\bullet$ {\it Is there any command in \clic\ that refers to physical
antennas?} No. The physical antenna numbers are just shown in the
plots of the {\tt First Look} procedures, because the command ``plot
/physical'' is the default there. Otherwise, all \clic\ commands and
procedures use the logical numbers for antennas. This must be taken
into account if AoD notes refer to physical antennas, or when applying
a new baseline model in observations with an incomplete array.


\bigskip
\noindent
$\bullet$ {\it How can I flag data?} After selecting the data to be
flagged with ``find'', you can use the command ``store flag {\it
flag$\_$label} /ant {\it i}'' (or ``/baseline {\it ij}'').  We can
also delete them by changing their quality with the command ``store
quality 9 /ant {\it i}'' (or ``/baseline {\it ij}''). Please, ask your
local contact if any doubt.

For example, the following commands would allow flagging correlations
obtained with all antennas on a source of name starting by 1823,
between the scans 234 and 250, with the label ``redu'':\\
\hspace{1cm} {\tt CLIC> find /proc corr flux /sou 1823* /scan 234 250 \\ 
\hspace{1cm}  CLIC> store flag redu /ant all \\}


\subsubsection{Other calibration procedures}

\noindent 
{\bf Baseline-based calibration}

\bigskip

Sometimes we have difficulties to have antenna-based calibration
solutions. In almost all the cases this can be solved by finding the
origin of the problem, often due to the presence of corrupted data.
If the difficulties are found to be due to baseline-dependent
problems, for instance related to the correlator, to the atmosphere,
etc, it may be better to calibrate in baseline-based mode. In that
case the {\tt hpb} file must be created by using the \clic\ command
``copy header base'' instead of ``copy header antenna'' (the default
in Sect.\ \ref{createhpbsec}).  The baseline-based solution can be
then obtained by defining ``set rf/ phase/ amplitude baseline'' before
solving and storing.  Antenna-based solutions can be also stored in
these {\tt hpb} files by using ``set rf/ phase/ amplitude antenna'',
followed by solve and store. Note that the last solution stored is the
one considered by default. Get in contact with your local contact for
a more detailed explanation if needed.


\bigskip

\noindent 
{\bf Self-calibration}

\bigskip

Self-calibration can be performed by selecting ``Self-cal on point
source'' in the \clic\ menu (Fig.\ \ref{clicmenu}), with a procedure
similar to that here described for a ``Standard Calibration'' that
includes also the source correlations.

\subsection{\label{visibassess}Data quality assessment}

\begin{figure}[h]
\begin{center}

\hspace{-5cm}

\includegraphics[height=7cm]{pdbi-cookbook-16}
\caption{{\tt Data Quality Assessment}, in the \clic\ menu}
\label{assess}
\end{center}
\end{figure}

An assessment of the quality of the calibrated data can be performed
by selecting in the \clic\ menu (Fig.\ \ref{clicmenu}) the option {\tt
Data quality assessment} (see Fig.\ \ref{assess}).  This procedure
creates a {\tt clic} procedure to flag data not satisfying predefined
quality criteria.  By default two different criteria are adopted to
assess the data quality, depending if the project aim is detecting or
mapping source emission. In the first case, the criterion is just the
phase stability. In the second, data whose phase stability results in
a {\tt seeing} larger than the maximum defined in the widget are not
considered. The adopted thresholds to assess the quality of the
amplitude and phase calibrations change with the receiver band. The
considered limits to evaluate the quality of pointing, focus and
tracking are defined in relation with the primary beam and the
wavelength. Plots are created to identify with shadowed areas the data
of poor quality according to the limits defined in the widget (Fig.\
\ref{assess}). Histograms presenting a statistical analysis of the
data quality are created at the end.

By clicking on {\tt Check Now} a {\tt clic} procedure is created to
flag the data shown in shadowed areas of the plots. It is to the user
to {\tt Apply} or not this procedure. The option {\tt Edit} permits to
modify the procedure before applying it.



\subsection{\label{writetablesec}{\it UV-}Table Creation}

\begin{figure}[h]
\begin{center}

\hspace{-5cm}

\includegraphics[height=10cm]{pdbi-cookbook-17} 
\caption{{\tt Write a UV Table}, in the \clic\ menu. }
\label{table}
\end{center}
\end{figure}

A {\it uv-}table with calibrated source visibilities can be created by
selecting ``Write a UV Table'' in the \clic\ menu (Fig.\
\ref{clicmenu}). A widget similar to the one shown in Fig.\ \ref{table} is
opened. The {\tt hpb} file name, table name (with no extension),
source name, receiver band, tuned side band and table mode (with
spectral -{\tt LINE}- or without spectral -{\tt CONT}- information)
are to be specified. In the widget, the atmospheric phase correction
must be disabled if it was not taken into account in the
calibration. The rest frequency can be redefined, as well as the table
resampling for the {\tt LINE} mode. A {\it uv-}table is newly created
if the option {\tt New Table} is set {\tt on}; visibilities are added
to an existing {\it uv-}table if the option {\tt New Table} is set
{\tt off}.

This procedure creates (or includes) the selected calibrated source
visibilities into the specified {\it uv-}table, but also creates (or
updates) a {\tt clic} procedure, which can be executed by entering
``@mytable-table.clic'' in \clic\ to produce the same table (of name
``mytable'' in this example).  Editing a script to create tables (the
file ``mytable-table.clic'' in our example, see Fig.\ \ref{mosaic}) is
easy and likely faster than filling in all the options available in
the widget to create tables. This {\tt clic} procedure defines the
{\tt hpb} file, calibration and table settings, selects the
correlations on source, and finally uses the \clic\ command ``table'',
possibly with the options ``/resample'' and ``/frequency''.  In the
\clic\ menu there exist other more complex procedures to create
tables, but likely none of them is as fast and flexible as editing
table scripts is.

Particularly for mosaics a table must be created for each observed
offset, all of them with a generic common name followed by an offset
number (an example is shown in Fig.\ \ref{mosaic}). The imaging
\mapping\ procedures (presented in Sect.\ \ref{datamap}) will so
recognize the mosaic and properly proceed with the imaging.



\subsection{Calibrating and/or merging with data previous to 2007}
 
\subsubsection{\clic\ backwards compatible}

The current versions of \clic\ are backwards compatible and hence can
be also used to calibrate data obtained before 2007, when a new
generation of receivers was installed at Plateau de Bure (see also
Sect. \ref{veryold}).
% However, not enough tests have been made yet to discard the presence of 
% bugs. 
% Let us know (at sog@iram.fr) if you find problems or have doubts.
% For the time being, we recommend to work with the version of January 
% 2007 to ensure a proper calibration and table creation of data previous 
% to 2007. {\tt CLIC07} is for the time being distributed with all the 
% GILDAS releases, and allows using the \clic\ version of January 2007.

\subsubsection{uv-tables obtained before 04/2004 }

The format of the {\it uv-}tables changed in March 30, 2004. The
\mapping\ command ``uvt$\_$convert'' updates the {\it uv-}tables'
format to be treated with a recent {\tt GILDAS} version and
particularly to mix the old tables with recent data.

\begin{figure}[t]
\begin{center}

\hspace{-5cm}

\includegraphics[height=11cm]{pdbi-cookbook-18}
\caption{ Script to create {\it uv-}tables for a mosaic of two points,
as edited by emacs. Note that for the data of the second day the same
settings adopted but for the atmospheric phase correction and the
considered scan range.}
\label{mosaic}
\end{center}
\end{figure}

\subsubsection{Merging old and new data}

In order to merge old and new data the best is likely to create the
{\it uv-}tables from the old and the new hpb files, with the same CLIC
version dated later than April 2004, as described in Sect.\
\ref{writetablesec}:
%
1) Using the widget to create a table or just to add visibilities in a
previous table by setting on or off the option {\tt New table} (see
Fig.\ \ref{table}).
%
2) Editing a table script and using the command {\tt table} with the
options {\tt new} and/or {\tt old} as shown in Fig.\ \ref{mosaic},
depending if the table is to be created or if just visibilities from a
second data set are added.

Otherwise, {\it uv-}tables can be merged with the procedure ``run
uv\_merge'' either in \clic\ or in \mapping, provided they have the
same spectral setup. Contact your local contact if your situation does
not correspond to any of the mentioned cases.



\subsubsection{Continuum projects between 11/2005 and 01/2007}

In November 25, 2005, prototypes of the new generation of receivers
were installed on antenna 6, while the other antennas continued
observing with the old receivers. This situation remained until
January 2007, when all antennas were equipped with new receivers. As a
consequence, for those projects observed with DSB tuning the creation
of continuum tables from all baselines except for those with antenna 6
should be made by using both bands, but the baselines with antenna 6
should just include the tuned band (pure SSB). Note that this affects
all 1mm continuum projects, where the tuning was DSB by default with
the old receivers.

To create continuum tables we recommend: First, storing in the {\it
uv-}table the data coming from all the antennas for the band tuned in
antenna 6. Second, storing the visibilities obtained for the other
band, ignoring antenna 6 data. For example, for a project LSB tuned in
antenna 6, but DSB in the others, we would proceed as follows:\\

\hspace{3cm} \begin{minipage}{17cm}
{\tt 
\noindent
set band lsb \\
table {\it table\_name} \\
set band usb \\
mark data /ant 6 \\
table {\it table\_name} /nocheck scan 
}
\end{minipage}



\subsubsection{\label{veryold}Very Old Projects}

The calibration of very old projects (older $\sim$ 1995) is likely not
possible with widgets because of the many big changes over the
years. Consult the SOG (at sog@iram.fr) if needed.

\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{\label{datamap}First Instructions for Data Analysis with MAPPING}

\mapping\ is the GILDAS software package to image and analyze NOEMA
data. The tools described here are complex procedures built over
simple \mapping\ commands. We recommend to look at the \mapping\
manual to have a deeper understanding of imaging and deconvolution
concepts and of the command functionalities.

\begin{figure}[h]
\begin{center}

\hspace{-5cm}

\includegraphics[height=3.5cm]{pdbi-cookbook-19}
\caption{\mapping\ menu.}
\label{mapping}
\end{center}
\end{figure}


The menu shown in Fig.\ \ref{mapping} leads to two main sections, {\tt
Operations on {\it UV-}tables} and {\tt Imaging and deconvolution}. In
the widgets resulting from the selection of these options (Figs.\
\ref{uvactions} and \ref{imaging}) the left column lists the
procedures to be run, their control parameters can be edited by
clicking on the related button in the central column, help is
displayed by pushing on the right buttons.  At the bottom of all the
``Parameters'' widgets (see e.g.\ Fig.\ \ref{uvshow} and \ref{uvfit})
there are the options {\tt GO} and {\tt DISMISS}. {\tt GO} executes
the procedure with the specified parameters, {\tt DISMISS} closes the
widget.

In the below sections, the paragraphs starting with {\it At the
prompt:} describe how to run the same procedures through the command
line instead of through the widgets. While this possibility exists for
all the widget functionalities, information is given for some of
them. In general, typing``{\tt go} {\it procedure$\_$name}'' is needed
to call \mapping\ procedures. To modify the control parameters ``{\tt
let} {\it variable$\_$name value}'' is used. Their content can be
checked with ``{\tt input} {\it procedure$\_$name}'' or by typing
``{\tt exam} {\it variable$\_$name}''. The name and a short
description of the control variables can be found by clicking where
the values for parameters are requested, in the widgets.

In addition, a few variables are used by (almost) all the
procedures. You can also define their values directly at the prompt.
For instance ``{\tt let name} ...'' is used to define the generic name
(without extension) of your table or image, ``{\tt let first} ...'' to
define the first channel to be considered, ``{\tt let last} ...'' to
define the last one.


\subsection{Operations on {\it UV} table}


By clicking in the \mapping\ menu (Fig.\ \ref{mapping}) on {\tt
Operations on {\it UV} table} the widget presented in Fig.\
\ref{uvactions} is opened.  There, by clicking on the buttons in the
column on the left, we run the corresponding procedures, some of them
described below. We strongly recommend to look at the adopted
parameters (in the central column) before executing them.

The main procedures related to the analysis of the data in the {\it
uv-}plane are described in the following sections:

\bigskip
\bigskip

\begin{figure}[hb]
\begin{center}

\hspace{0cm}

\includegraphics[height=5.7cm]{pdbi-cookbook-20}
\caption{Operations on {\it UV} table, from the \mapping\ menu.}
\label{uvactions}
\end{center}
\end{figure}


%\newpage

\subsubsection{\label{uvshowsec}UVSHOW}

UVSHOW plots the {\it uv-}data. By default amplitudes vs. {\it
uv-}radii are presented, for all visibilities and spectral
channels. The {\tt UVSHOW parameters} widget (see Fig.\ \ref{uvshow})
permit to select the visibility's characteristic (amplitude, real
part, weight, etc.) to plot. By default the different tracks are
plotted with different colors. Plotting a model fitted by UVFIT is
possible by clicking in the corresponding option.

\bigskip

\begin{table}[h]
{\it \small 

\vspace{0cm}

\noindent
At the prompt:\\
Procedure: {\tt go uvshow} \\

\vspace{-1cm}

\hspace{5cm} \begin{minipage}{17cm}

\noindent
Some variables:\\
{\tt let ytype amp} \\ 
{\tt let ytype weight} \\ 
{\tt let xtype radius} \\ 
{\tt let xtype time} \\ 
{\tt let uvshow$\%$fit no/yes} \\ 
{\tt let uvshow$\%$zero yes/no} \\ 
{\tt let uvshow$\%$track yes/no} \\

\end{minipage}
}
\end{table}


\begin{figure}[h]
\begin{center}

\hspace{-5cm}

\includegraphics[height=7.5cm]{pdbi-cookbook-21}
\caption{UVSHOW parameters, to plot the {\it UV-}data.}
\label{uvshow}
\end{center}
\end{figure}


\subsubsection{UVSHIFT}

Procedure to shift the phase center of a {\it UV-}table. Absolute or
relative coordinates can be introduced. The position angle of the
reference axes (which is then considered by the plotting procedures)
can also be changed.

%\bigskip

\begin{table}[h]
{\it \small 
\noindent
At the prompt:\\
Procedure: {\tt go uv$\_$shift} \\
}
\end{table}

\subsubsection{UVFIT}

Procedure to fit the obtained {\it uv-}data with the Fourier Transform
of the following functions (see Fig.\ \ref{uvfit}): a point, a
circular Gaussian, an elliptical Gaussian, a circular disk, an
elliptical disk, a circular or elliptical ring, an exponential
brightness distribution, a distribution proportional to radius$^{-2}$,
and a distribution proportional to radius$^{-3}$. Some parameters can
be fixed by defining them in the option {\tt Parameters}.  They are
the offset R.A., offset Dec and flux for a point function, offset
R.A., offset Dec, flux and diameter for c$\_$gauss, offset R.A.,
offset Dec, flux, major and minor diameter and position angle for
e$\_$gauss, offset R.A., offset Dec, flux and diameter for c$\_$disk,
offset R.A., offset Dec, flux, major and minor diameter and position
angle for e$\_$disk, offset R.A., offset Dec, flux, inner diameter and
outer diameter for a ring, offset R.A., offset Dec, flux and diameter
for the functions called expo, power-2 and power-3.

The fitted function can be subtracted in a residual table (of
extension uvfit) that later can be plotted with UVSHOW (Sect.\
\ref{uvshowsec}).
\begin{figure}[h]
\begin{center}

\hspace{-5cm}

\includegraphics[height=6.8cm]{pdbi-cookbook-22}
\includegraphics[height=6.8cm]{pdbi-cookbook-23}
\caption{{\it Left:} UV$\_$FIT parameters, to fit the {\it UV-}data
with the Fourier Transform of simple functions. {\it Right:} PLOTFIT
parameters, to plot the fitting results obtained with UVFIT.}
\label{uvfit}
\end{center}
\end{figure}


\subsubsection{PLOTFIT}

PLOTFIT is aimed to plot the results of UVFIT. The number of
parameters to plot (see Fig.\ \ref{uvfit}) is to be defined if larger
than 3. By default it presents the position of the center of the
fitted function, R.A. and Dec, and the flux at each velocity. Several
options are available for each fitting. Error bars are plotted by
default.

\newpage

\subsection{Imaging and deconvolution}


\begin{figure}
\begin{center}

\hspace{-5cm}

\includegraphics[height=5.5cm]{pdbi-cookbook-24}
\caption{Imaging and deconvolution, from the \mapping\ menu.}
\label{imaging}
\end{center}
\end{figure}

By clicking in the \mapping\ menu (Fig.\ \ref{mapping}) on {\tt
Imaging and deconvolution} in the widget presented in Fig.\
\ref{imaging} is opened.  The menus at the top of Fig.\ \ref{imaging}
permit to run some of the most used procedures.  The list of options
presented below in the widget (Fig.\ \ref{imaging}) allow us to
execute the same procedures (on the left) and modify the corresponding
parameters (at the center).


\subsubsection{SETUP}

{\tt SETUP} performs a first check of the data related to a common
{\it generic name}\footnote{Generic name refers to the table name
without extension, and, for mosaics, without the offset specification
(see Sect.\ \ref{writetablesec}).}, and reports on possible
consistency problems if mosaics or short-spacing data.  Short-spacing
tables to be merged with interferometric data are supposed to have the
same generic name with the extension {\tt tab}.  For mosaics, all the
fields must keep a common generic name followed by {\it -i} for each
field {\it i}; just the generic name should be then specified in the
imaging procedures. From this initial inspection of the data, map and
pixel sizes are recommended and later used by default in the imaging
process.

\subsubsection{UVSHORT}

Four steps are proposed in UVSHORT (setup, table, merge and clean) to
merge interferometric and single-dish data.  The compatibility of both
data sets is checked, a {\it uv-}table is created from the single-dish
data, both data sets are merged and imaged in order to verify the
consequences on the beam.  A {\tt uvt} table is created with the
resulting merged data. The default UVSHORT parameters are well
adapted for merging NOEMA and 30m-telescope data.

We recommend to use the procedure {\tt UVSHOW} (see Sect.\
\ref{uvshowsec}) to verify the merged {\it uv} data, and particularly
to verify that the flux calibration is not incompatible. For example,
by plotting amplitude versus {\it uv} radius we can check the flux of
both data sets at their interface, i.e.\ at a {\it uv} radius of 15m
(for the NOEMA array).

\subsubsection{\label{uvmapsec}UVMAP}

Procedure to compute a dirty map (with extension {\tt lmv}) and beam
(with extension {\tt beam}) from {\it uv-}tables. For mosaics a unique
dirty image is created, as well as individual ones for each
field. UVMAP estimates which are the best values for pixel and map
sizes. We recommend to use them. By default images are created by
using natural weights for the visibilities. Uniform weights can be
defined in the widget (see Fig.\ \ref{uvmap}), but indeed correspond
to robust weighting.

\begin{figure}[h]
\begin{center}

\hspace{-5cm}

\includegraphics[height=9cm]{pdbi-cookbook-25}
\caption{UVMAP parameters, to create a dirty ({\tt .lmv}) image.}
\label{uvmap}
\end{center}
\end{figure}

\noindent
{\bf UVSTAT weight/taper} The advantages and inconveniences of
modifying the weights from natural to robust are quantified, and
numbers are shown by UVSTAT in mode {\tt weight}. The advantages and
inconveniences of tapering can be verified with UVSTAT in mode {\tt
taper}.

\bigskip

\noindent
{\bf DIRTY/BEAM/PRIMARY/WEIGHTS} at the top of the imaging widget
(Fig.\ \ref{imaging}) are aimed to plot the results from UVMAP, that
is ``mytable''{\tt.lmv}, ``mytable''{\tt.beam}, and, if mosaicing, also
``mytable''{\tt.lobe} and ``mytable''{\tt.weight}.

\bigskip

\begin{table}[h]
{\it \small 

\noindent
At the prompt:\\
Procedure: {\tt go uv$\_$map} \\
}

{\it \small 

\noindent
{\rm To plot}:\\
Procedure: {\tt go bit} \\

\vspace{-1cm}

\hspace{5cm} 
\begin{minipage}{17cm}

\noindent
Some variables:\\
{\tt let type lmv} \\
{\tt let type beam} \\ 
{\tt let first 7} \\  
\end{minipage}
}
\end{table}



\subsubsection{\label{supportsec}SUPPORT}

Procedure to define polygons inside which the CLEANing procedure will
look for CLEAN components. A common polygon can be defined for all the
channels, or one polygon per channel. Polygons can be defined with the
cursor or by introducing the needed parameters to define elliptical or
rectangular supports. All these possibilities can be easily identified
in the SUPPORT widget.


\bigskip

\begin{table}[h]
{\it \small 

\noindent
At the prompt:\\
Procedure: {\tt go support} \\

\vspace{-1cm}

\hspace{5cm} \begin{minipage}{17cm}

\noindent
Some variables:\\
{\tt let support$\%$oneperplane yes/no} \\ 
{\tt let support$\%$kind cursor/ellipse/rect} \\ 
\end{minipage}
}
\end{table}



\subsubsection{\label{cleansec}CLEAN}

It identifies in the dirty map CLEAN components and convolve them with
a deduced synthetic beam. Different CLEANing methods can be
used. HOGBOM and CLARK are recommended for being the most robust (see
{\tt MAPPING} manual for details). By using {\tt SUPPORT}s the search
for CLEAN components can be restricted to the areas where the source
is known to be (when the variable {\tt myclean\%support}=yes, see
Sect.\ \ref{supportsec}). The criterion to stop the search for CLEAN
components within the residual map can be modified by changing the
{\tt Stopping criteria} options in the CLEAN parameters widget (Fig.\
\ref{clean}).  By default no supports are used in a first CLEANing of
a dirty map, but can be defined for later CLEANing. The option to
CLEAN with supports can also be set on or off in the CLEAN parameters
widget. By default the flux accumulated in the identified CLEAN
components is shown (if {\tt myclean\%show}=yes), and stored in a file
of name including {\tt -cct}.

%\newpage

\begin{figure}[h]
\begin{center}

\includegraphics[height=11cm]{pdbi-cookbook-26}
\caption{Clean parameters, to identify CLEAN components and convolve
them with the deduced synthetic beam.}
\label{clean}
\end{center}
\end{figure}

\noindent
{\bf HOGBOM/CLARK/others} at the top of the imaging widget permits
CLEANing by using the defined parameters.

\bigskip

\noindent
{\bf RESIDUALS/CLEAN/CCT} at the top of the imaging widget plots the
resulting residuals (stored in a file of extension {\tt .lmv-res}),
the CLEANed image (in a file of extension {\tt .lmv-clean}) or the
found CLEAN components (in a file of extension {\tt .lmv-cct}).


\begin{table}[h]
{\it \small 

\noindent
At the prompt:\\
Procedure: {\tt go clean} \\

\vspace{-1cm}

\hspace{5cm} \begin{minipage}{17cm}

\noindent
Some variables:\\
{\tt let method hogbom/clark} \\
{\tt let myclean$\%$show yes/no}\\ 
{\tt let myclean$\%$support yes/no}\\ 
{\tt let niter 1500} \\
{\tt let ares 1e-3}\\
\end{minipage}
}
\end{table}

\bigskip

\begin{table}[h]
{\it \small 

\noindent
{\rm To plot}:\\
Procedure: {\tt go bit} \\

\vspace{-1cm}

\hspace{5cm} 
\begin{minipage}{17cm}

\noindent
Some variables:\\
{\tt let type lmv-clean} \\
{\tt let first 23 } \\ 
{\tt let last 45} \\  
{\tt let type lmv-res} \\
\end{minipage}
}
\end{table}

\newpage

\subsubsection{VIEW}

Procedure to plot resulting images and cubes. VIEW plots the averaged
image, a selected channel, the line profile obtained from a certain
position, and the integrated line profile. Ranges and channels to
plot, and areas and pixels for which profiles should be shown, are
selected by clicking on each plot with the mouse. Typing {\tt h}
anywhere in the plotting window displays help information on the
terminal. BIT presents the images obtained for all the spectral
channels (within the specified first and last channels)

\bigskip

\noindent
{\bf VIEW/BIT} at the top of the imaging widget permits selecting the
procedure to plot.

%\begin{figure}
%\begin{center}

%\hspace{-5cm}

%\includegraphics[height=6cm]{view.eps}
%\caption{View parameters.}
%\label{view}
%\end{center}
%\end{figure}

\bigskip

\begin{table}[h]
{\it \small

\noindent 
At the prompt:\\
Procedure: {\tt go view}\\
}

\bigskip

{\it \small 

\noindent
Procedure: {\tt go bit} \\

\vspace{-.5cm}

\hspace{5cm} 
\begin{minipage}{17cm}

\noindent
Some variables:\\
{\tt let type lmv-clean} \\
{\tt let first 23 } \\ 
{\tt let last 45} \\  
{\tt let size 50} \\
{\tt let spacing 3e-3} \\
\end{minipage}
}
\end{table}


\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\appendix{}

\section{\label{calibprinciples}Appendix: Calibration Principles}

An ``ideal'' interferometer samples the Fourier transform of the sky
brightness multiplied by the antenna beam pattern; these samples are
called {\bf visibilities}. In practice, because of noise,
imperfections, sampling, and other effects, several terms perturb the
visibilities to produce the true output of a ``real''
interferometer. The goal of the calibration is to identify and
estimate all the important instrumental and atmospheric effects and
apply appropriate corrections to recover the true visibilities.

\subsection{Standard Decomposition of Visibilities}

For any given complex number $Z$, let us call $PZ$ its phase, $AZ$ its
amplitude, and $Z^*$ its complex conjugate.  The observed visibility
$V_{ijk}(t)$ is a complex number representing the amplitude and phase of
the signal detected on baseline $ij$, from spectral channel $k$, at time
$t$,
\begin{equation}
V_{ijk}(t) = AV_{ijk}(t) . \exp(-i.PV_{ijk}(t))
\end{equation}

This visibility is the Fourier transform of the product of the primary beam
patterns of the antennas and the brightness distribution of the observed
source, sampled at the point $(u(t),v(t))_{ij}$ corresponding to baseline
$ij$ at time $t$,
\begin{equation}
V_{ijk}(t) = FT ( B_i(x,y,x_0+x_i,y_0+y_i) B^*_j(x,y,x_0+x_j,y_0+y_j) 
             I(x,y,k) ) (u,v)_{ij}
\end{equation}
where
\begin{itemize}
\item $FT$ is the Fourier Transform operation, $x,y$ are integration
  parameters,
\item $I(x,y,k)$ is the brightness distribution of the source at frequency
  $k$,
\item $B_i(x,y,x',y')$ is the voltage pattern of antenna $i$ pointed in
  direction $(x',y')$,
\item $(x_0,y_0)$ is the pointing direction of the antennas,
\item $(x_i,y_i)$ is the pointing error of antenna $i$, and
\item $(u,v)_{ij}$ are the projected coordinates of baseline $ij$, in
  wavelength units.
\end{itemize}
If we further assume that all antennas are equal, their pointing errors are
negligible, their beam shape does not depend on the pointing direction, and
the fractional bandwidth is small ($\delta\nu/\nu << 1$), this equation
reduces to
\begin{equation}
V_{ijk}(t) = FT ( P(x,y) I(x,y,k) ) (u,v)_{ij}
\end{equation}
where $P(x,y)$ is the power beam pattern of the antennas.

The antennas, receivers, cables, and correlators all introduce additional
modifications to this visibility. These perturbations can be formally
decomposed into
\begin{equation}
V_{ijk} = A_i A^*_j S_{ik} S^*_{jk} C_{ijk} R_{ijk} + O_{ijk} + N_{ijk}
\end{equation}
where
\begin{itemize}
\item $A_i$ is the complex gain of antenna $i$ (amplitude and phase),
\item $S_{ik}$ is the complex gain of channel $k$ for antenna $i$,
\item $C_{ijk}$ is the complex gain of channel $k$ for correlator entry
  $ij$,
\item $R_{ijk}$ is the theoretical visibility of the source,
\item $O_{ijk}$ is a non random error on channel $k$ for correlator entry
  $ij$.  These errors have various origins, such as finite bandpass,
  bandpass mismatch between antennas $i$ and $j$, etc...), and
\item $N_{ijk}$ is a random error due to detection noise.
\end{itemize}

Provided the design of the interferometer system is adequate, the terms
appearing in this decomposition have the following properties:
\begin{itemize}
\item $N_{ijk}(t)$ is normally distributed with known variance.
\item $O_{ijk}$ is negligible with respect to $N_{ijk}$. We will assume
  $O_{ijk} = 0$ in the following discussion.
\item $C_{ijk}(t)$ is only weakly time dependent. This factor is introduced
  essentially by analog filters in the correlator and residual (constant)
  delay offsets between subbands; it may depend strongly on $k$.
\item $S_{ik}(t)$ is only weakly time dependent. This factor is introduced
  by receivers and IF cables. The frequency ($k$) dependence is weak.
\item $A_i(t)$ depends on antenna pointing (amplitude only), focus
  (amplitude and phase), and on atmosphere (amplitude and phase).
\end{itemize}

Let $ W = V - O - N = V - N $ to first order.  Here, $PN_{ijk}(t)$ is a
random phase and $AN_{ijk}(t)$ has known variance, {\tt <}$AN${\tt >}. Then
\begin{equation}
PW_{ijk}(t) = PA_i(t)+PS_{ik}(t)-PA_j(t)-PS_{jk}(t)+PC_{ijk}(t)+PR_{ijk}(t)
\end{equation}
\begin{equation}
PW_{ijk}(t) = PV_{ijk}(t) + PD_{ijk}(t)
\end{equation}
where
\begin{itemize}
\item $PA_i$ is the instrumental phase on antenna $i$
\item $PS_{ik}$ is the relative phase of channel $k$ for antenna $i$
\item $PC_{ijk}$ is the relative phase of channel $k$ for correlator entry
  $ij$, independent of the $PS_{ik}$
\item $PR_{ijk}$ is the source phase on baseline $ij$ for channel $k$.
  $R_{ijk}(t)$ only depends on $ij$ through the coordinates of antennas $i$
  and $j$, via $(u(t),v(t))_{ij}$.
\item $PD_{ijk}$ is a phase noise introduced by measurement noise $N_{ijk}$
\end{itemize}

Then, to first order,

\begin{itemize}
\item $PC_{ijk}$ can be measured independently, and is only weakly time
  dependent,
\item $PS_{ik}$ is constant, providing the receiver is not retuned,
\item $PA_i$ is time variable, on many different timescales, and
\item $PD_{ijk}$ has known variance, depending on $W_{ijk}$/{\tt <}$AN${\tt
    >}.
\end{itemize}

Similar relations can be expressed for the intensities:

\begin{equation}
AW_{ijk}(t) = AA_i(t).AS_{ik}(t).AA_j(t).AS_{jk}(t)).AC_{ijk}(t).AR_{ijk}(t)
\end{equation}
and
\begin{equation}
AW_{ijk}(t) = AV_{ijk}(t) + AD_{ijk}(t)
\end{equation}
where
\begin{itemize}
\item $AA_i$ is the gain of antenna $i$ (including effects due to
  atmospheric absorption, focus, receiver gain, pointing, etc...),
\item $AS_{ik}$ is the relative gain of channel $k$ for antenna $i$,
\item $AC_{ijk}$ is the relative gain of channel $k$ for correlator entry
  $i,j$, measured independently from $AS_{ik}$,
\item $AR_{ijk}$ is the Source intensity on baseline $ij$. $AR_{ijk}(t)$
  depends on $ij$ only through antenna coordinates.
\item $AD_{ijk}$ has known variance {\tt <}$AN${\tt >}.
\end{itemize}

The purpose of calibration is to determine as best as possible these
various functions, taking advantage of the time independence of some
parameters, and of the weak chromaticity of the atmosphere.

\subsection{Baseline versus Antenna based calibration}

Although many of these calibration parameters are antenna based,
\clic\ is able to keep simultaneously two values for these parameters:
an antenna-based value and a baseline-based value.  Whenever an
antenna-based calibration function is computed (i.e. a part of the
$A_i$, or of the $S_{ij}$), \clic\ offers the possibility of solving
for an antenna-based or a baseline-based function. In general, solving
for antenna based parameters should be preferred, since this mode
offers less free parameters (so that they are better constrained) and
uses a priori knowledge of the instrument (e.g. the phase closure
relations). However, in some cases, well-identified baseline-related
problems make it difficult to calibrate in antenna-based mode, so it
may be needed to look for baseline-based solutions, mainly for
amplitude or phase calibration.

\subsection{\label{onlinecal}Plateau de Bure Online Calibration}

Some of the mentioned terms are partially estimated on real time so
that the stored visibilities are already corrected from their
contribution. Specific OBS commands, a dedicated {\tt GILDAS} software
package ({\tt RDI}) and some \clic\ {\tt procedures} are used to
produce and reduce the following scans:
\begin{itemize}
\item {\tt CALI}: Each scan consists of two or three subscans,
  obtained by the OBS command {\tt CALIBRATE}. They are used to
  measure the atmospheric transmission and receiver temperature,
  converting backend counts in antenna temperature outside atmosphere
  (which is a factor in $AA_i$, excluding pointing and focus terms).
\item {\tt IFPB}: Two subscans are obtained by the OBS command
  BANDPASS, which switches the correlator entries to a common noise
  source. Thus all the $AC_{ijk}$ and $PC_{ijk}$ are measured, except
  for a residual delay (visible as a constant phase slope with respect
  to frequency) due to path differences between the noise source and
  the antenna connections to the correlator.
\item {\tt GAIN}: This procedure allows measuring the sideband gain
  ratio of the receivers, by looking at a strong cosmic source (in
  practice, only $AI_i$ is computed, although the phase term $PI_i$
  could also be derived). Since this ratio only depends on the
  receiver tuning, in particular on the backshort tuning, a {\tt
  CALIBRATE} scan is made just after tuning modifications.
\end{itemize}

Results from {\tt CALI} and {\tt IFPB} observations are applied to the
data by the automatic data compression job. Corrections from tuning
modifications are introduced in OBS by the operator just after {\tt
GAIN}, at the beginning of the observations. Therefore, in principle,
amplitude errors should only be due to phase noise, pointing and focus
errors.

\subsection{Offline Calibration; the rules of the game}

The purpose of the offline calibration (carried out when the
acquisition process is ended at the observatory) is to estimate as
precisely as possible all the various terms of Equation 4.
%The calibration receipes described in this document are based entirely 
%on their relative importance and behaviour (time and frequency dependence).
The basic principle is to measure the visibilities of sources with
known (spatial and frequency) structure, in practice continuum
point-like sources (called calibrators), which flux (so $b_{ijk}$) is
also known or can be deduced.  By simple division, the product
$A_i.A^*_j.S_{ij}.S^*_{ij}$ is determined at the time intervals in
which these calibrators are observed, and interpolated for the
intermediate times in which the project source is.  The signal to
noise ratio of the calibrator measurements is the main limiting factor
in this process.
%especially for the $S_{ik}$ terms since the bandwidths of the spectral 
%channels are narrow. 
%It is then necessary to take full advantage of the low time or low
%frequency dependence of all factors to optimize the signal to noise
%ratio on all determinations.

Another important limitation concerns the validity of the
interpolations used to predict the values of the calibration factors
on the studied sources. It is desirable to estimate the quality of
these predictions, and to ignore data which for any reason may deviate
substantially from the predictions (for example due to instrumental
problems, strong atmospheric variations, etc).


\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\section{\label{pipeshowsec}Appendix: Pipeline or AoD First Look}

The {\tt First Look} report created by the {\tt pipeline} is much
longer than what is available under the \clic\ {\tt First Look} widget
(Sect.\ \ref{firstlooksec}). This report provides detailed and
critical information of system performance to the AoD. It shows hence
various technical aspects which may be difficult to interpret by
non-trained astronomers. Here we briefly describe the content of these
plots.

\begin{figure}[h]
\begin{center}

\hspace{-5cm}

\includegraphics[height=3.6cm]{pdbi-cookbook-27}
\caption{Pipeline or AoD {\tt First Look} widget}
\label{pipefirstlook}
\end{center}
\end{figure}
\bigskip

\noindent
{\bf Summary}: The first page of the {\tt First Look} report shows the
correlator setup, for the Narrow-band and Widex correlator
units\footnotemark[8], and the number of correlations obtained on each
source.  Warnings are often included, and are mainly addressed to the
AoDs for technical assessment.

\bigskip

\noindent 
{\bf Meteorogical data}: See details in Sect.\ \ref{firstlooksec}. The
verification of the wind speed is particularly important for AoDs if
tracking problems are found in the ``Elevation and Azimuth'' plot.

\bigskip

\noindent 
{\bf Elevation and Azimuth}: See description in
Sect.\ \ref{firstlooksec}.  Colored lines are included to mark flagged
scans or records (seconds). Deep blue lines refer to flags due to
antenna shadowing, red to correlator unit flags, solid pale blue lines
to {\tt DATA} (correlator related) flags, dashed pale blue lines to
tracking problems, green to phase lock losses, yellow to doppler or time
issues, and pink to outliers in the system temperature, pointing
problems or user or pipeline flagged data. More details about the
flags can be found in ``Flag data List'' paragraph presented below.

\bigskip

\noindent 
{\bf Pointing and Focus corrections}: See description in
Sect.\ \ref{firstlooksec}. Colored lines are included if pointing or
focus corrections change by more than 30\% the size of the primary
beam or wavelength, respectively.

\bigskip

\noindent {\bf Antenna Tracking Errors}: See description in Sect.\
\ref{firstlooksec}. Problems are often related to strong wind. It may
happen, though it remains very rare, that large tracking errors
indicate a technical problem. This monitoring is particularly
important to foresee interventions by the technical staff in NOEMA
antennas.

\bigskip

\noindent 
{\bf 22GHz monitors} present the results obtained in time from the
22GHz receiver in each antenna. The first three plots show the counts
obtained in each of the three channels of the 22GHz receivers. They
are combined to produce the `triple' values, which are used to model
the atmosphere, disentangle between water vapor and droplets, and
predict the atmospheric phase fluctuations by which the
interferometric data are affected. This is then used to reduce of
phase decorrelation within each scan. Derived water vapor amounts are
displayed in the `Water Monitoring' plots. The given ambient
temperatures are obtained by sensors placed close to the 22GHz
receivers. Peltier Temperatures are directly related to the receiver
performance, and should remain within the plot limits.

\bigskip

\noindent 
{\bf CALI scans vs time} plots show the time evolution of the {\tt
  CALI} autocorrelation scans, which can consist of two (on the hot
load and sky respectively) or three (on the cold load, hot load and
sky) subscans. Measurements on the cold load are typically performed
every $\sim$ 50 min, and are used to derive and monitor receiver
temperatures. Differences between antennas are normally linked to
receiver attenuations, and differences between units are often linked
to correlator tweaks.  In correct weather conditions, the sky
autocorrelations show a constant value because the signal variations
due to airmass change is compensated by correlator tweaking. This
tweaking effect is however visible in the autocorrelations on the
loads. (Note that Widex tweak levels change in large steps.) Strong
variations in the input signal often result in changes in the tweak
values, and accordingly in the {\tt CALI} autocorrelations.

\bigskip

\noindent 
{\bf IFPB scans vs time} plots present the time evolution of the
amplitudes of the {\tt IFPB} correlations obtained on the noise source
(see Sect.\ \ref{scansec}). Absolute amplitude values change from unit
to unit, depending on the IF phase delay. As the noise diode provides
a constant input, {\tt IFPB} amplitudes should remain constant along a
track.  Sharp variations in the tweak levels (for instance due to bad
weather) can however result in changing amplitudes of the IFPB scans,
due to tweaking adjustment effects.

\bigskip

\noindent 
{\bf Tweak levels vs time} plots show the tweak levels for each of the
Narrow correlator units, per correlator sub-band. Values remaining
constant (in correct weather conditions) in time correspond to the
{\tt IFPB} scans, tweak values change with the airmass for the other
acquisitions.

\bigskip

\noindent 
{\bf Monitoring the time, UTC, NTP and PPS}. The monitoring of the
time synchronization signals is important as it is essential to stop
fringes, ensure a good pointing, etc. This plot helps us to monitor
differences in time between the UTC (from the maser-synchronized GPS)
and NTP ([GPS-]synchronized NTP) times, as well as time offsets
between hardware clocks (PPS used by the correlator from
maser-synchronized GPS time) and software synchronized events (NTP).

\bigskip

\noindent 
{\bf Receiver Temperatures in the IF} plots present the receiver
temperature computed along the IF bandwidth from one of the first {\tt
  CALI} scans including a cold load acquisition (often called
`cal-cold' scan). Different colors are used to identify the different
correlator inputs. These plots shows better than any other the
presence of parasites and tuning features. Particularly, marks are
included to identify the known system parasites (from the IF processor
at 6300, 4500 MHz, and at 3 and 4 times the LO1ref local oscillator
frequencies). Dotted lines represent the values stored in the data
header, computed by the online {\tt RDI} software.

\bigskip

\noindent 
{\bf Receiver Temperatures vs Time} plots display the evolution of the
mean receiver temperature values averaged over the
correlator-input\footnotemark[8] bands. They should not change by more
than a few K in projects with stable tunings. Tunings affected by
strong parasites may show changing receiver temperatures. An
intervention from the frontend group may happen as a consequence of
the information extracted from these plots.

\bigskip

\noindent 
{\bf Dewar Temperature} plots show the temperatures measured at
various stages in the cryostat.

\bigskip

\noindent 
{\bf Observing List} summarizes the sequence of obtained scans. {\tt
CALI} and {\tt IFPB} acquisitions are ignored in this list.

\bigskip

\noindent 
{\bf Flagged data List} shows all the flagged records and scans. It
should be consistent with the color marks in the 'Elevation-Azimuth'
plots.

\bigskip

\noindent 
{\bf Total Power vs time}: See description in
Sect.\ \ref{firstlooksec}. Values are presented per each calibration
unit (which correspond to correlator inputs\footnotemark[8]): Two
Narrow Quarters and four Widex units. Moderate differences (of a few
K) in the values from the different units are due to the different
covered frequency ranges.

\bigskip

\noindent 
{\bf Cable Phase} plots present the phase delays (in degrees) produced
at the Master Frequency level by the cables in their movements, mostly
due to antenna tracking, for the used and unused bands.  Colored lines
mark changes that could result in phase variations larger than
30$^{\rm o}$. Significant changes in the LO1ref local oscillator
frequencies are also displayed in grey lines.

\bigskip                                                           
                            
\noindent 
{\bf System Temperatures vs time}: See description in
Sect.\ \ref{firstlooksec}. Plots are shown per calibration unit.

\bigskip

\noindent 
{\bf Water Vapor vs time}: See first description in
Sect.\ \ref{firstlooksec}. A plot is created per calibration unit. A
differential plot is created to compare the results obtained from all
the antennas. Pale blue lines in the WVR H$_2$O plots mark the update
of the 22GHz receiver calibration by the online software.

\bigskip

\noindent 
{\bf RF phases} plots compare the phases obtained per correlator input
in the (IF1) frequency band. Remaining phase delays can be identified
in these plots. One plot is created for the first correlation scan at
the project start, another for the last correlation scan at the
project end. Note that if the (frequency averaged) phases from H and V
polarization receivers are not equal, ``let phcal `{*}' '' should be
entered after {\tt Select} (see Sects.\ \ref{select} and \ref{phcal}).


\bigskip

\noindent 
{\bf Amplitudes for Narrow and Widex correlator inputs}: Comparison of
the signal level obtained from all the correlator
inputs\footnotemark[8] (or calibration units), which -in principle-
should be almost identical. Uncorrected delays result in amplitude
differences among the correlator units, which become particularly
important between the Widex and the Narrow-band correlator
units. Delays can be identified in the {\tt RF phases} and the {\tt RF
  calibration} plots, and the data can be corrected for them by
following the instructions in Sect.\ \ref{diff}.


\newpage
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%




\end{document}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

