\documentclass[11pt]{book}
\usepackage{fancyhdr}
\include{gildas-def}

\makeindex{}

\newcommand{\emm}[1]{\ensuremath{#1}}
\newcommand{\emr}[1]{\emm{\mathrm{#1}}}
\newcommand{\unit}[1]{\emm{\,{#1}}}

\newcommand{\Hz}[1]{\unit{Hz}}

\newcommand{\paren}[1]  {\emm{\left(  #1 \right) }} % Parenthesis. 
\newcommand{\cbrace}[1] {\emm{\left\{ #1 \right\}}} % Curly Braces.
\newcommand{\hbrace}[1] {\emm{\left\{ #1 \right.}}  % Left Curly Brace.
\newcommand{\bracket}[1]{\emm{\left[  #1 \right] }} % Brackets.

\newcommand{\mjd}[1]{\emm{\emr{mjd}_{#1}}}
\newcommand{\lamoff}[1]{\emm{l_{#1}}}

\newcommand{\fortran}[1]{\texttt{#1}}
\newcommand{\imbfitskey}[1]{\texttt{#1}}
\newcommand{\command}[1]{\texttt{#1}}
\newcommand{\brand}[1]{\textbf{#1}}

\newcommand{\mrtcal}  {\brand{MRTCAL}}
\newcommand{\mira}    {\brand{MIRA}}
\newcommand{\classic} {\brand{CLASSIC}}
\newcommand{\cdc}     {\brand{CLASSIC} Data Container}
\newcommand{\clib}    {\brand{CLASSIC} Library}
\newcommand{\classdf} {\brand{CLASS} Data Format}
\newcommand{\pako}    {\brand{PAKO}}
\newcommand{\cfitsio} {\brand{CFITSIO}}
\newcommand{\imbfits} {\brand{IMBFITS}}

\newcommand{\nint}{\emr{nint}}
\newcommand{\nused}{\emr{USED}}
\newcommand{\spacing}{\emr{SPACING}}
\newcommand{\bandwidth}{\emm{W}}
\newcommand{\nslices}{\emm{N}}
\newcommand{\quotient}{\emm{q}}
\newcommand{\remainder}{\emm{r}}

% Makes fancy headings and footers.
\pagestyle{fancy}

\fancyhf{} % Clears all fields.

\rhead{\scshape \nouppercase{\rightmark}}   %
\fancyfoot[C]{\bfseries \thepage{}}

\renewcommand{\headrulewidth}{0pt}    %
\renewcommand{\footrulewidth}{0pt}    %

\renewcommand{\sectionmark}[1]{%
  \markright{\thesection. \MakeLowercase{#1}}}
\renewcommand{\sectionmark}[1]{}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{document}

\frontmatter{} %

\thispagestyle{empty}

\begin{center}
  \includegraphics[width=5cm]{gildas-logo}\\[2\bigskipamount]
  \huge{}
  \mrtcal\ Programmer Manual\\[3\bigskipamount]
  \large{}
  Questions? Comments? Bug reports? Mail to: \texttt{gildas@iram.fr}\\[2\bigskipamount]
  The \gildas{} team welcomes an acknowledgment in publications\\
  using \gildas{} software to reduce and/or analyze data.\\
  Please use the following reference in your publications:\\
  \texttt{http://www.iram.fr/IRAMFR/GILDAS}\\[2\bigskipamount]
\end{center}

\begin{description}
\item[Documentation] \mbox{}\\
  \credit{S.~Bardeau$^{1}$}{J.~Pety$^{1,2}$, A.~Sievers$^3$}{}
\item[Software] \mbox{}\\
  \credit{S.~Bardeau$^{1}$, A.~Sievers$^3$}{J.~Pety$^{1,2}$}{}
\end{description}
1. IRAM (Grenoble)\\
2. Observatoire de Paris\\
3. IRAM (Granada)

\begin{rawhtml}
  Note: this is the on-line version of the MRTCAL Programmer Manual.
  A <A HREF="../../pdf/mrtcal-pro-manual.pdf"> PDF version</A>
  is also available.
\end{rawhtml}

\bigskip{} %

Related information is available in:
\begin{latexonly}
  \begin{itemize}
  \item{MRTCAL User Manual}
  \item{CLASS Manual}
  \item{GREG: Graphical Possibilities}
  \item{SIC:  Command Line Interpretor}
  \end{itemize}
\end{latexonly}
%
\begin{rawhtml}
  <UL>
  <LI> <A HREF="../mrtcal-user-manual-html/mrtcal-user-manual.html"> MRTCAL User Manual</A>
  <LI> <A HREF="../class-html/class.html"> CLASS Manual</A>
  <LI> <A HREF="../greg-html/greg.html"> GREG: Graphical Possibilities</A>
  <LI> <A HREF="../sic-html/sic.html">   SIC:  Command Line Interpretor</A>
  </UL>
\end{rawhtml}

\tableofcontents{}

\mainmatter{}%

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\chapter{Chunks and chunksets}

For the processing of the \imbfits{} files, \mrtcal{} introduces the
concept of \emph{chunks} (see Fig~\ref{fig:chunks}). One dump of the
\imbfits{} \imbfitskey{DATA} column is a collection of such
chunks. \mrtcal{} can make no assumption on the way they are ordered (this
comes from ???). The \imbfitskey{PART} column indicates to which spectrum
each chunk belongs (spectra are identified with integer numbers). Each
``set of chunks belonging to the same final spectrum'' is called a
\emph{chunkset} in the \mrtcal{} nomenclature. From one chunkset, \mrtcal{}
will produce one spectrum\footnote{For debugging purpose, \mrtcal{} can
  also produce one spectrum per chunk.}.

The chunks in each \imbfitskey{DATA} row can belong to several spectra
(\eg{}, 8 for FTS), and by definition to as many chunksets. \mrtcal{} maps
those chunksets internally as a \fortran{chunkset\_1D} Fortran structure,
\ie{}, an array of chunksets.

The \imbfitskey{DATA} column has as many rows as the number of pixels of
the receiver times the number of time dumps. This introduces 2 new
dimensions which are used to map all\footnote{\mrtcal{} can also map a
  fraction of the \imbfitskey{DATA} column, process it, and then iterate on
  the next fraction. This can be useful on memory-limited machines.}, the
\imbfitskey{DATA} column into a \fortran{chunkset\_3D} Fortran
structure. The 2 new dimensions here are \emph{pixels} and \emph{time} (see
Fig~\ref{fig:chunksets}).

For its internal needs, \mrtcal{} has to refer to a single time dump in the
3rd dimension. It can also time-average this 3rd dimension. In those 2
use-cases, the final product is then available through a {\tt chunkset\_2D}
Fortran structure.

\begin{figure}[p]
  \centering
  \includegraphics[width=\textwidth]{imbfits-chunks}
  \caption{How one dump in the \imbfits{} \imbfitskey{DATA} column is
    divided into chunks, gathered by family (chunksets), and reordered to
    produce the spectra.}
  \label{fig:chunks}
\end{figure}

\begin{figure}[p]
  \centering
  \includegraphics[width=\textwidth]{imbfits-chunksets}
  \caption{How all the dumps in the \imbfits{} \imbfitskey{DATA} column is
    divided into 3D chunksets. After this, they can be time-averaged, or
    can be accessed one by one.}
  \label{fig:chunksets}
\end{figure}

\section{Chunks slicing}
\label{sec:slicing}

In order to be able to make calibration at a bandwidth smaller than the
hardware chunks, \mrtcal{} is able to map the raw data block read with
\cfitsio{} into smaller \emph{memory} chunks. The main advantage is that
this approach has very well defined boundaries: When the \imbfits{}
\imbfitskey{BACKEND} table is read, it can be replaced on-the-fly with its
sliced version. All the rest of the code does not have to know this
happened. The downside is that the more memory chunks are defined, the more
overheads they carry (e.g. description of their spectroscopic axis, calls
to chunk-by-chunk subroutines, stitching, etc). Experience shows that
slicing the 1400 MHz of FTS chunks into 14 chunks of 100 MHz introduces a
penalty of 10\% in the whole computation time (reading, slicing,
calibrating, writing).

In practice, user can request the bandwidth he wishes. In order to slice
the \nused{} channels into comparable pieces, \mrtcal{} will choose the
nearest bandwidth which gives an integer number of slices
\begin{equation}
  \nslices = \nint \paren{\frac{\nused \times |\spacing|}{\bandwidth}}
\end{equation}
where \bandwidth{} is the requested bandwidth. \nslices{} must also be
corrected so that there is at least 1 and at most \nused{} channels per
subchunk
\begin{equation}
  \nused \ge \nslices \ge 1.
\end{equation}

While the \imbfitskey{CHANS} value (total number of channels) is often a
multiple of 2, \nused{} channels have no special value. In particular, it
may not be a multiple of \nslices{}. Their relationship can be written as
\begin{equation}
  \nused = \quotient \times \nslices + \remainder,
\end{equation}
where \quotient{} and \remainder{} are respectively the integer quotient
and the remainder of the division $\nused/\nslices$. Since $0 \le
\remainder \le \nslices-1$, \nused{} can be divided in \nslices{}
subchunks, \remainder{} of them having one extra channel
\begin{equation}
  \nused = \bracket{\quotient \times (\nslices-\remainder)} + \bracket{(\quotient+1) \times \remainder}.
\end{equation}
This gives the best channel division and a negligible difference of
subchunks bandwidth.

\begin{figure}[p]
  \centering
  \includegraphics[width=\textwidth]{imbfits-subchunks}
  \caption{Hardware chunks can be virtually divided into smaller pieces
    just by redefining the \imbfitskey{BACKEND} table. In this example, the
    table is sliced from 1 chunk to 3 subchunks (3 times more lines in the
    table). The same raw data block read with \cfitsio{} will be mapped by
    Mrtcal with a larger number of \emph{memory} chunks.}
  \label{fig:subchunks}
\end{figure}

ZZZ Say a word about the ``famous'' special channel with its half
weight. Slicing does not have any special problem with it.

\section{Polarimetry}

The presence of polarimetric measurements in the \imbfits{} file is
described in the \imbfitskey{POLAR} column of the \imbfitskey{BACKEND}
table. For a non-polarimetric scan, all the chunks in the
\imbfitskey{POLAR} column have the \imbfitskey{NONE} value. For a
polarimetric scan, values can be \imbfitskey{NONE}, \imbfitskey{AXIS},
\imbfitskey{REAL} or \imbfitskey{IMAG} (note that non-polarimetric chunks
-\imbfitskey{NONE}- can be mixed with polarimetric ones).

In addition, calibration scans for polarimetric measurements must
provide a {\tt calGrid} subscan. It absent, \mrtcal{} will mark this
scan failed as it can not be used to calibrate polarimetric science
scans.

\chapter{Reading the {\tt DATA} column}

\section{Discarding bad time dumps}
\label{sec:data}

The BackendDATA table provides the data dumps collected during the
subscan integration. In particular, it shows the dump {\tt MJD}, its
integration time {\tt INTEGTIM}, its phase number through cycles ({\tt
  ISWITCH}) and the data itself (channels) in the {\tt DATA} column.

It may happen that a dump is flagged out by the control system. In
this case, its {\tt ISWITCH} value is 0. Such a dump should be
discarded during the calibration process. This is achieved with the
command {\tt MSET CALIBRATION BAD YES|NO} (default {\tt NO},
i.e. discard bad time dumps).

In practice, this is not just a matter of skipping undesired rows when
dealing with the {\tt DATA} column. The first issue is that the values
in the associated columns are unreliable. This is particularly true
with the {\tt MJD} value which is not set to the actual MJD of the
dump. This introduces inconsistent values in this column, resulting in
an unsorted column, breaking the dichotomic search engine. In order to
solve this issue, \mrtcal\ \emph{compresses} at read time the {\tt
  MJD}, {\tt INTEGTIM}, and {\tt ISWITCH} columns, discarding the rows
where {\tt ISWITCH} is null. The size of those columns is decreased
from $N_{tot}$ to $N_{good}$ (with $N_{tot} = N_{good} + N_{bad}$).

\mrtcal\ keeps also track of the discarded dumps thanks to two new
columns added. One column named {\tt FOREPOIN} (size $N_{tot}$)
provides a forward pointer, i.e. {\tt FOREPOIN(i)} is the position of
the $i^{th}$ dump (as found in the original columns) in the compressed
columns. If the dump is bad, the corresponding position is set to 0
(i.e. dump not available in the compressed columns). It also adds a
column named {\tt BACKPOIN} (size $N_{good}$) which provides a
backward pointer: {\tt BACKPOIN(j)} is the position of the $j^{th}$
dump (as found in the compressed columns) in the original columns.

At this stage, \mrtcal\ has patched the BackendDATA columns so that
the bad dumps are removed, but this is not fully transparent for later
use.

\section{Dump cycle identification}

From a general point of view, the dumps have to be calibrated by
cycles, each cycle repeating the same sequence of several phases. In
order to do this, \mrtcal\ identifies those sequences in the {\tt
  ISWITCH} column. As this column may have been compressed, the
identification also takes into account the {\tt BACKPOIN} column: a
cycle is valid if the expected sequence is found in the compressed
column AND if the associated back pointers are contiguous in the
original columns.

\section{Reading the {\tt DATA} column by pieces}

\mrtcal\ is designed to be able to process the subscans within an
arbitrary limited buffer size ({\tt MSET BOOKKEEPING SPACE Value},
default 512 MBytes). If the buffer size is large enough, the whole
{\tt DATA} column can be loaded at once in memory and its dumps can be
treated in a row. If the buffer is not large enough, the {\tt DATA}
column has to be load by pieces, and each piece has to be processed
separately from the other.

In practice, the calibration routine asks the reading routine to load
the piece of {\tt DATA} which contains \emph{at least} a desired range
of dumps (usually a cycle of a few dumps). \emph{At least} means that
the reading routine ensures that the range will be present entirely in
the buffer, but more dumps will also be loaded in order to fill at
best the buffer. On subsequent requests, if the desired range is
already present in the buffer, nothing will be reloaded, in order to
save calls to {\tt CFITSIO} and I/O accesses.

At this stage, one should consider that {\tt CFITSIO} is asked to read
a block of rows from the {\tt DATA} column. {\tt CFITSIO} is not able
to skip some bad rows here and there, and it would be unefficient to
ask for reading the desired rows one by one. So {\tt CFITSIO} reads a
piece of the {\tt DATA} column, including good and bad dumps. But
right after, \mrtcal\ sets a 3D chunkset ($N_{set} \times N_{pix}
\times N_{time}$) pointing to the good dumps only. Then the calling
routine has just to work with the 3D chunkset, with no need to care
for bad dumps.

\chapter{Positions}

\section{Reference position}
\label{sec:reference}

For each spectrum, the Class Data Format expects a reference (usually
source) position. In \mrtcal, this reference is taken from the
\imbfitskey{LONGOBJ} and \imbfitskey{LATOBJ} (\emph{Source longitude
  and latitude in basis frame}) in the Scan HDU. The coordinate system
of these values are taken from header cards \imbfitskey{CTYPE1} and
\imbfitskey{CTYPE2} ({\tt RA} and {\tt DEC} for equatorial system,
{\tt GLON} and {\tt GLAT} for galactic system). Note that the trailing
characters describing a projection system (\eg\ {\tt -SFL}) are
useless: they do not apply to any data which can be found in the Scan
HDU.

\section{Scan offsets}
\label{sec:scanoffsets}

From the absolute reference defined at \ref{sec:reference}, the
\imbfits{} format can define an intermediate position, through the
\imbfitskey{SYSOFF}, \imbfitskey{XOFFSET} and \imbfitskey{YOFFSET}
columns in the \imbfitskey{SCAN} table
\begin{itemize}
\item if only the \emph{Nasmyth} row is present, an implicit
  \emph{projection} row is added at read time with (0,0) offsets,
\item if the \emph{projection} row is available, the associated offsets are
  added to the antenna offsets (see next section) and used into the
  \classdf,
\item if something else than \emph{projection} is available, \eg\
  \emph{horizontalTrue}, a specific support is not yet available in
  \mrtcal{} and an error is raised. However, if the associated offsets are
  (0,0)\footnote{This can happen \eg{}, if the user has commanded {\tt
      OFFSETS 0.0000000E+00 0.0000000E+00 /SYSTEM "trueHorizon"} in \pako},
  Mrtcal will tolerate this system of offsets. This is a temporary
  solution, waiting for a full support of all systems.
\end{itemize}

\section{Antenna offsets}

The antenna offsets are found in the \imbfitskey{LONGOFF} and
\imbfitskey{LATOFF} columns in the Antenna Slow HDU. These antenna
offsets are to be added to the scan offsets defined at
\ref{sec:scanoffsets}. Note that these offsets describe the
\emph{primary dish} pointing position. In case of wobbler switching,
the beam does not point to this position (see section~\ref{sec:wsw}
for details).

\subsection{Projection system}

The antenna offsets are expressed in a system described by the card
\imbfitskey{SYSTEMOF} in the Antenna HDU. If its value is:
\begin{itemize}
\item {\tt projection}, this means the offsets are radio-projected in
  the current coordinate system, \ie\ the system found at
  \ref{sec:reference}; the offsets can be used ``as is'' in
  \class{},
\item something else, \eg\ {\tt horizontalTrue} for {\tt calSky}, the
  offsets are not expressed in the correct frame and should be
  recomputed to the desired sky coordinate system (\class{} does not
  offer the possibility to express the reference in one system and the
  offsets in another). ZZZ this is not yet implemented, warning for
  now.
\end{itemize}

\subsection{Interpolation}
\label{sec:offset}

The antenna offsets are derived from the Antenna Slow table together
with other position values. Namely the 5 following elements have to be
computed:
\begin{itemize}
\item lambda offset from reference position,
\item beta offset,
\item azimuth,
\item elevation,
\item local sidereal time.
\end{itemize}
All these elements are available in the \imbfitskey{ANTSLOW} table, at a
typical sampling rate of 1 Hz. Since the spectra dumps can be produced at
different time sampling (no assumption is made on the rate or its
variations), each spectrum positions are interpolated from the
\imbfitskey{ANT} table thanks to their respective \emph{Modified Julian
  Day} values
\begin{equation}
  f = \frac{\mjd{S} - \mjd{A}(j)}{\mjd{A}(j+1) - \mjd{A}(j)}
\end{equation}
where \mjd{S} is the spectrum MJD value, and $\mjd{A}(j)$ is the MJD value
of the $j^{th}$ trace in the \imbfitskey{ANTSLOW} table. $j$ is computed
thanks to a dichotomic search in the table such as
\begin{equation}
  \mjd{A}(j) \le \mjd{S} < \mjd{A}(j+1).
\end{equation}
$f$ being the interpolation fraction between the $j^{th}$ and $j+1^{th}$
trace, the positions are interpolated by
\begin{equation}
  \lamoff{S} = \lamoff{A}(j) + f \times \bracket{\lamoff{A}(j+1)-\lamoff{A}(j)},
\end{equation}
where \lamoff{S} and \lamoff{A} are the spectrum and antenna lambda offsets
respectively. Same formula applies for the beta, azimuth, and elevation
values.

If \mjd{S} is found beyond the \imbfitskey{ANTSLOW} table limits, the
boundary values are applied without extrapolation. However, this is not
expected to happen since such spectra should be rejected since they are out
of the on-track range.

\subsection{Wobbler switching}
\label{sec:wsw}

In case of wobbler switching, the \imbfitskey{LONGOFF} and
\imbfitskey{LATOFF} columns describe the primary dish direction on
sky. Because the secondary mirror switches with additional offsets
between the ON and OFF positions, \mrtcal\ assumes that the ON phases
always have (0,0) antenna offsets. For simplicity, the OFF phases
offsets are also set to (0,0) (instead of $\pm \Delta/2$) but the
\class\ switching section will describe properly the throw when saving
the ON-OFF result, or when saving the ON and OFF separately. Remember
that the wobbler throw is usually along azimuth with a given
projection, while the \class\ offsets are described in equatorial or
galactic system with radio projection: their addition is not
straightforward.

\begin{figure}[p]
  \centering
  \includegraphics[width=\textwidth]{wsw-asymmetrical}
  \caption{Asymmetrical wobbler switching illustration. For each
    subscan, the antenna (primary dish) points at $\pm \Delta/2$ away
    from the source, usually in the azimuthal direction (blue
    line). This is called \emph{antenna noding}. During the subscan,
    the secondary mirror switches (red line) from the ON position (at
    offset 0) to one OFF position (at offset $\pm \Delta$). $\Delta$
    is the \emph{wobbler throw}. This pattern, which alternates
    between OFF1 and OFF2 at each subscan, is called
    \emph{asymmetrical wobbler switching}, as opposed to the
    \emph{symmetrical wobbler switching} (see details in
    Fig~\ref{fig:wsw-sym}).}
  \label{fig:wsw-asym}
\end{figure}

\begin{figure}[p]
  \centering
  \includegraphics[width=\textwidth]{wsw-symmetrical}
  \caption{Symmetrical wobbler switching illustration. Same as
    Fig~\ref{fig:wsw-asym}, except that the subscans are observed with
    symmetry around a given time. Thanks to this, and assuming that
    the atmospheric properties drift linearly through time, the
    atmospheric variations between the 2 OFF1 subscans and between the
    2 OFF2 subscans average at the same level.}
  \label{fig:wsw-sym}
\end{figure}

\subsection{Slow and Fast traces}

The \imbfits{} files provide 2 antenna streams and associated tables:
The slow (1 Hz) and the fast (128 Hz) traces. For each of those
traces, some antenna position parameters are available, plus the
associated time as MJD values.

As of today (\imbfits{} version 2), the 2 traces are in the same FITS
extension. Both have the same number of rows, but the fast trace has 128
values per column and per row, while the slow trace has 1 value per column
and per row. However, even if they have the same number of rows, they are
INDEPENDENT: For example, their MJDs are misaligned by a $\sim$2 seconds
shift (2 rows). They should not be correlated! In the \imbfits{} version 3,
the 2 traces will be clearly separated in 2 FITS extensions, which will
avoid any misinterpretation.

Note that the caveat exposed above explains why \mrtcal{} selects more
dumps than \mira{} in on-the-fly maps: \mira{} looks at the antenna slow
column \imbfitskey{TRACEFLAG} to flag out MJD values in the antenna
fast. Because of the $\sim$2 seconds shift between the 2 traces, more time
dumps are discarded.

\section{Offsets of multi-pixel receiver}

In the \imbfits{} files, the backend table describes how the chunks
cover the bandwidth and from which receiver they come from (\eg\ H or
V polarization). In this table, the column {\tt PIXEL} also describes
from which pixel the signal comes. This is useful in the context of
multi-pixel receivers. In such a case, there are several pixels which
do not look at the same position on sky. In the frontend table header,
the derotator mode (\emph{sky}, \emph{frame}, or \emph{horizontal})
and angle describe the commanded inputs from the user. The derotator
table, which is present in the \imbfits\ files only if relevant,
describes the commanded and actual derotator position through time, at
typical intervals of 5 seconds. We describe below how \mrtcal\ makes
use of these informations.

\subsection{Handling the derotator}

If the subscan has no derotator table, the current pixel (obviously
the unique pixel) is left centered on the current position (reference
+ position offset) computed in the section~\ref{sec:offset}. On the
other hand, if a derotator table is present, \mrtcal\ will apply a
per-pixel offset.

First of all, \mrtcal\ tries to guess the \emph{actual} derotator
angle \emph{on sky}. The strategy is the following:
\begin{itemize}
\item angle on sky will be chosen from the column {\tt sAct} (``sky
  Actual''), as opposed to {\tt fAct} (``frame Actual'') and {\tt
    hAct} (``horizontal Actual''). This assumes {\tt sAct} values are
  reliable even if the current derotator mode is not \emph{sky}.
\item if the column {\tt sAct} is empty (this can happen around
  midnight), a warning is raised and the returned value is the
  commanded value as found in the frontend table header. In this case,
  the derotator mode must be \emph{sky}.
\item if the column {\tt sAct} provides no value within the subscan
  range, a warning is raised.
\item if the column {\tt sAct} has a single row, or if the chunk MJD
  is beyond the table limits, the unique/nearest value is used.
\item if the chunk MJD lies within the column {\tt sAct} limits, the
  derotator angle is interpolated according the associated {\tt MJD}
  column and to the chunk MJD computed earlier.
\item if the chunk MJD is further than 5 seconds\footnote{5 seconds is
  the derotator sampling rate under normal conditions.} from the
  nearest point in the column {\tt sAct}, an additional warning is
  raised.
\item finally, if the actual angle is more than 0.5 degrees away from
  the commanded value (\eg\ derotator has reached its rotating limit),
  a warning is raised.
\end{itemize}

\subsection{Computing the pixel coordinates}

Given the actual position of the derotator, \mrtcal\ can compute the
pixel position on sky from the current telescope position. This
position has been pre-computed using the radio projection ({\tt
  LONGOFF} and {\tt LATOFF} columns in the IMB-FITS), only the
per-pixel offsets are to be added. \mrtcal\ checks first if it knows
the offsets for the current receiver (as found in the column {\tt
  RECNAME} in the frontend table). The pixels offsets are not
self-described in the \imbfits: they must be hardcoded in the program,
with a correct correspondance between the pixel number found in the
backend table and the hardcoded offsets\footnote{See the Fig.4 in the
  \emph{HERA user manual} for this receiver.}. Once the offset of the
current pixel is found, the strategy is the following.
\begin{itemize}
\item The current position (reference + position offset) is
  unprojected to absolute spherical coordinates.
\item The pixel position angle on sky is computed, given the pixel
  position in the array and the derotator angle.
\item The spherical distance from the array center is added using an
  inverse Haversine formula, i.e., the pixel position in absolute spherical
  coordinates is found by adding an arc of great-circle towards the correct
  direction.
\item The resulting position is reprojected again as a reference +
  position offset in the current projection system.
\end{itemize}
The deprojection, offset in spherical coordinates, and reprojection
are done by the Gildas internal engines.

\subsection{Description of the radio projection}

The unprojection-reprojection approach to add the pixel offsets is more
accurate than simple offset addition to projected position, in particular
far away from the equator of the coordinate system. If we look at the radio
projection (its standard FITS name is Global Sinusoidal, abbreviated into
GLS) of the full sky shown in Fig.~\ref{fig:radio}, we can see that one can
expect strong distortions near the pole. Note that the distortions depend
on the declination of the object and NOT on the declination of the
projection center\footnote{... while most of the other projections offer
  minimal distortions near the projection center!}. In other words, the
plane of projection is not tangent to the projection center, but to the
equator instead. This is directly linked to the mathematical formula of the
radio projection
\begin{eqnarray}
  x & = & (A-A_0) \times \mathrm{cos}(d), \\
  y & = & d-d_0,
\end{eqnarray}
where ($A_0,d_0$) is the projection center, ($A,d$) the object
absolute coordinates, and ($x,y$) its offsets in the projected
map. The key point in this projection is that $x$ is a function of $d$
and not $d-d_0$. Hence, the deformation for wide fields of view will depend
both on the distance to the projection center and the source declination!
That's why the radio projection is deprecated and IAU recommends to replace
it by the Sanson-Flamsteed projection (abbreviated in SFL).
\begin{figure}[t]
  \centering
  \includegraphics[width=\textwidth]{sky-radio}
  \caption{Radio projection of the full sky (right ascension from -12
    to 12 hours, declination from -90 to 90 degrees), for four
    projection centers (marked with a blue cross) at 0 right ascension
    and 0, 30, 60, 90 degrees declination. The parallels (resp. the
    meridians) are spaced by 30 degrees (resp. 2 hours).}
  \label{fig:radio}
\end{figure}

The distortions near the pole may thus have a non-negligible impact on
observations. Figures~\ref{fig:map-0-0} to~\ref{fig:map--1-0} show the
effect in different conditions. The geometry of the multi-beam array shows
no visible distortions, even one degree away from the source position, when
the source position is located on the equator. However, the distorsions of
the array geometry increase with the distance to the projection center when
the source is located at high declination. Moreover, for the same source
coodinates, the deformation at high source declination depends on the right
ascension of the projection center (cf.\ Fig.~\ref{fig:map--1-0}), but they are independent of the
projection center declination (cf.\ Fig.~\ref{fig:map-0--15})!

\begin{figure}[htbp]
  \centering
  \includegraphics[width=\textwidth]{sky-radio-0-0}
  \caption{Effects of the radio projection on observations with HERA
    for source centers at 4h, 12h, 20h Right Ascension and 0, 44, 88
    degrees declination (blue crosses). In this case, {\bf the
      projection center is aligned on the source center.} The green
    polygon shows the shape of a $1 \time 1$ square degree field in
    sky spherical coordinates. In addition, the red parallelograms
    show the resulting shape of a $480''\times480''$ square multi-beam
    array as a function of the distance from the source center.
    Finally, the parallels (resp. meridians), shown as black dashed
    lines, are separated by 0.5 degree (resp. 1 degree).}
  \label{fig:map-0-0}
\end{figure}

\begin{figure}[htbp]
  \centering
  \includegraphics[width=\textwidth]{sky-radio-0--15}
  \caption{Same as Fig.~\ref{fig:map-0-0}, except that {\bf the projection
      center is located 15 degrees south the source center.} The
    deformations are independent of the declination of the projection
    center!}
  \label{fig:map-0--15}
\end{figure}

\begin{figure}[htbp]
  \centering
  \includegraphics[width=\textwidth]{sky-radio--1-0}
  \caption{Same as Fig.~\ref{fig:map-0-0}, except that {\bf the projection
      center is located 1 hour angle west from the source center.} The
    deformations depend on the right ascension of the projection center.}
  \label{fig:map--1-0}
\end{figure}

\clearpage
\subsection{In practice: Effect on Polaris observations}

For reference, we first remind the observing strategy advised in the HERA
manual. Two main points will help us explain the behavior on Polaris.
\begin{itemize}
\item The array is rotated by a given angle that ensures Nyquist sampling
  and optimal sky coverage in a single observation (i.e., each portion of
  the sky is observed only by one of the nine pixels during a scan). This
  requires two contiguous subscans.
\item However, the observing strategy usually set up at Pico Veleta samples
  regularly the \emph{projected} sky and \emph{not} the sky itself as naive
  users tries to obtained rectangular projected maps. Moreover the HERA
  receiver array is a perfect square in focal plane and on the sky, but
  \emph{not} in the projected map!
\end{itemize}
Under observer-friendly conditions, namely \emph{relatively} small maps
near the projection center and maps \emph{near} the equator, these rules
deliver the expected result. However, if those conditions are not met
(large field of view and the source is located at high declination), the
sky and the projected map will both be incorrectly sampled!

Maps near Polaris (declination 87:42:04.6) where observed during the
project 219-07 (PI: P.Hily-Blant). The maps extend typically from (0,0) to
(-1500,2000) arcsec (projected offsets). We focus here on the scans 26 and
40 covering the projection center, and scans 98 and 115 observed at the
largest distance from the projection center.

Figure~\ref{fig:offsets1} shows the scans observed near the projection
center. They do not show any particular effect visible by eye. The array
receiver is still square (rotated by 9.6 degrees\footnote{See HERA Manual
  Fig.7}) on the projected map, and the coverage of the 2 scans is as
expected. Figure~\ref{fig:absolute1} shows the scan coverages in spherical
coordinates (i.e., without projection). The spherical sky is correctly sampled.

\begin{figure}[htbp]
  \centering
  \includegraphics[width=0.95\textwidth]{polaris-scan26-relative}\\
  \vspace*{1cm}
  \includegraphics[height=0.8\textwidth]{polaris-scan40-relative}
  \hspace{1cm}
  \includegraphics[width=0.5\textwidth]{polaris-HERA-0-0.png}
  \caption{Scans 26 (top) and 40 (bottom) {\bf relative coordinates
      (radio projection)} of project 219-07 observing Polaris
    (declination 87:42:04.6). The observing strategy is the usual one,
    i.e. there were 2 OTF subscans (red: first subscan, black: second
    subscan), with a 9.6 degrees derotator angle, and a slight shift
    from one subscan to another to fill the gaps.  These 2 scans cover
    the (0,0) reference position.}
  \label{fig:offsets1}
\end{figure}

\begin{figure}[htbp]
  \centering
  \includegraphics[width=0.95\textwidth]{polaris-scan26-absolute}\\
  \vspace*{1cm}
  \includegraphics[height=0.8\textwidth]{polaris-scan40-absolute}
  \caption{Same as Fig.\ref{fig:offsets1} (scans near the projection
    center) but showing {\bf absolute coordinates}. In this case, the
    uniform mapping of the projected map results in uniform mapping of
    the sky.}
  \label{fig:absolute1}
\end{figure}

On the other hand, the scans observed far away from the projection
center $(-1500'',2000'')$ are highly affected by the projection
distortions. The projected receiver array is not square anymore. It is
parallelogram-shaped in the projected map. As a consequence, the rows
scanned in declination are shifted. Figure~\ref{fig:offsets2} shows
that the vertical scanning is not correctly sampled in the projected
sky. This can be understood by looking at the scan in absolute
coordinates (Fig.~\ref{fig:absolute2}): The scan does not follow a
South-North direction suited for the 9.6 degrees derotator angle on
the sky. To first order, the angle should have been different (the
exact value is not computed here)\footnote{To second order, the
  pattern is an arc and the derotator angle could have to be modified
  along the scan.}. On the other hand, the scan along the right
ascension is correctly sampled, but the start and end points of the
rows are actually shifted in projected coordinates compared to the
scan 26 near the projection center. This leads to different edge
effects in right ascension according to the declination of the source
center.

\begin{figure}[htbp]
  \centering
  \includegraphics[width=0.95\textwidth]{polaris-scan98-relative}\\
  \vspace*{1cm}
  \includegraphics[height=0.8\textwidth]{polaris-scan115-relative}
  \hspace{1cm}
  \includegraphics[width=0.5\textwidth]{polaris-HERA-1500+2000.png}
  \caption{Same as Fig.\ref{fig:offsets1} but showing scans 98 (top)
    and 115 (bottom) {\bf relative coordinates (radio
      projection)}. The same observing strategy is used. These 2 scans
    cover the (-1500,2000) offset position, far from the reference
    position. Note parallelogram shape of the receiver array in the
    projected map, and as a consequence the unexpected coverage of the
    vertical scan.}
  \label{fig:offsets2}
\end{figure}

\begin{figure}[htbp]
  \centering
  \includegraphics[width=0.95\textwidth]{polaris-scan98-absolute}\\
  \vspace*{1cm}
  \includegraphics[height=0.8\textwidth]{polaris-scan115-absolute}
  \caption{Same as Fig.\ref{fig:offsets2} (scans far from the
    projection center) but showing {\bf absolute coordinates}. In this
    case, the attempt of uniform mapping of the projected map results
    in unexpected mapping of the sky. In particular, the 9.6 degrees
    derotator angle on the sky is obviously not suited here.}
  \label{fig:absolute2}
\end{figure}

\clearpage

\newpage
\section{Guessing ON and OFF positions}

Depending on the observing mode, the telescope is expected to observe
1 or 2 positions on the sky:
\begin{itemize}
\item Position switch: 2 positions (ON + OFF)
\item Wobbler switch: 1 or 2 positions
\item Frequency switch: 1 position
\end{itemize}

Those positions are not described directly in the IMB-FITS. Instead,
the positions of each dump is available in the \imbfitskey{LONGOFF}
and \imbfitskey{LATOFF} columns of the antenna slow table. By using
equivalence classes of the X and Y offset pairs, it should
straightforward to group the dumps by position.\\

For {\tt projection} offset system, \ie\ when offsets are described
along a sky system of coordinates (equatorial or galactic), this
approach is correct: the offsets are all strictly identical for a
given position and for both coordinates. All digits are the same,
there are no numerical round-off errors.\\

\begin{figure}[htb]
  \centering
  \includegraphics[width=\textwidth]{imbfits-longoff-latoff.png}
  \caption{\imbfitskey{LONGOFF} and \imbfitskey{LATOFF} vs
    \imbfitskey{MJD} for iram30m-fts-20150429s199-imb.fits (whole
    scan). Observing mode is tracked wobbler switch, using {\tt
      horizontalTrue} offset system. During the scan, elevation
    changes from 82.3 to 83.6 degrees.}
  \label{fig:longoff-latoff}
\end{figure}

On the other hand, for {\tt horizontalTrue} offset system, the offsets
drift. In the {\tt horizontalTrue} offset system, \imbfitskey{LONGOFF}
are actually offsets along azimuth, corrected by $cos(elevation)$, and
\imbfitskey{LATOFF} are offsets along elevation. If we look at the
example in Fig.~\ref{fig:longoff-latoff}, we can see that:
\begin{itemize}
\item \imbfitskey{LATOFF} offsets are all null: this means that all
  the dumps were sharing the same elevation as the reference (source)
  position. The telescope is tracking the source, so that the
  elevation changes (this can be seen in the \imbfitskey{CELEVATIO}
  column), but not the offset in this direction.
\item \imbfitskey{LONGOFF} offsets are drifting, slowly decreasing for
  both positions.
\end{itemize}
Because of this drift, \mrtcal\ uses a centi-arcsecond tolerance when
the equivalence classes are computed. If we reuse the example in
Fig.~\ref{fig:longoff-latoff}, this tolerance is still not enough and
results in 6 different positions:
\begin{verbatim}
  E-SUBSCAN>LIST>BUILD,  Scan has 6 positions (expected 1 or 2):
  E-SUBSCAN>LIST>BUILD,     Position #1: -119.82903552  0.00000000
  E-SUBSCAN>LIST>BUILD,     Position #2:  119.82643276  0.00000000
  E-SUBSCAN>LIST>BUILD,     Position #3: -119.81881083  0.00000000
  E-SUBSCAN>LIST>BUILD,     Position #4:  119.81592996  0.00000000
  E-SUBSCAN>LIST>BUILD,     Position #5: -119.80748135  0.00000000
  E-SUBSCAN>LIST>BUILD,     Position #6:  119.80419607  0.00000000
\end{verbatim}

\chapter{Interpolation of calibration products}

\mrtcal{} computes one set of calibration products \emph{per
  chunk}. These products are:
\begin{itemize}
\item the receiver temperature,
\item the calibration temperature,
\item the system temperature,
\item the precipitable water vapor,
\item and the zenithal opacity.
\end{itemize} Their exact values are evaluated for the frequency at
the middle of the chunk\footnote{Remember that the chunk bandwidth is
  ruled by {\tt MSET CALIB BANDWIDTH}}. The values at other
frequencies in the chunk can be evaluated using one of the two
available modes:
\begin{enumerate}
\item in the \emph{flat} mode, the central values are reused over all
  the chunk bandwidth.
\item the second mode is \emph{interpolation}. In this case, the
  chunks are reordered by ascending frequency, and the values at
  non-central frequencies are interpolated between the current chunk
  and its nearest neighbour, so that it is computed at the frequency
  of the desired channel. This has the advantage to avoid
  discontinuities along the whole bandwidth and thus to avoid
  calibration platforming. For the two half-chunks at the boundary of
  the whole bandwidth, where no neighbour chunk is available, the
  values are extrapolated from the boundary chunk and its neighbour on
  the other side.
\end{enumerate}

\chapter{Memory index}

\section{Strategy}

\mrtcal{} indexes \imbfits{} files in the so-called \emph{index files}. A
raw index file named {\tt index.mrt} can be produced with the command
\command{INDEX BUILD} (see command help for more advanced uses). When one
or more index files are then reopened for reading, the command {\tt INDEX
  OPEN} builds a \emph{memory index} named the ``Input indeX'' (IX). At
this stage, IX gathers the summary of all entries in all opened index
files. From this full set, the command \command{MFIND} can be used to make
a selection of desired entries: This is the ``Current indeX'' (CX).

While the primary purpose of the index files is to reference
\imbfits{} files, their other advantage is to store other
\emph{products} that can be derived from the \imbfits{} files. In
particular, after the calibration, the commands \command{CALIBRATE}
and \command{PIPELINE} will save new \emph{versions} of each
calibrated entry. These new versions have a modified calibration
status (from \emph{none} to \emph{failed}, \emph{empty}, or
\emph{done}), and optionally a calibration section added (storing the
calibration results). These modified versions of the entries are
themselves saved in an index file (either the original one, or a new
one, depending on the user choice), AND are implicitly appended to
IX. After the calibration process, it is then easy for the user to
select from IX the calibrated entries (or failed, or pending,
etc). The figure~\ref{fig:ix} shows a basic example of these steps.

\begin{figure}[htb]
  \centering
  \includegraphics[width=\textwidth]{mrtcal-ix}
  \caption{Basic example showing how the \imbfits{} files are first
    indexed, and then how the calibration command (here, {\tt PIPELINE})
    adds a new version of each (tentatively) calibrated \imbfits{}.}
  \label{fig:ix}
\end{figure}

\section{Contents}

When the index file is loaded, its index content is duplicated in memory
for all the entries. These elements are described in the
Table~\ref{tbl:mrtoptimize}.

\begin{table}[htb]
  \begin{center}
    \caption{Memory index elements coming directly from the index file.}
    \vspace{1ex}
    \begin{tabular}{l|l|l}
      Parameter        & Unit      & Comment \\
      \hline
      {\tt bloc}       & records   & Entry position in index file \\
      {\tt word}       & words     & Entry position its this record \\
      {\tt version}    & ---       & Entry version \\
      {\tt telescope}  & code      & Telescope code \\
      {\tt projid}     & ---       & Project id \\
      {\tt source}     & ---       & Source name \\
      {\tt dobs}       & gag\_date & Observation date \\
      {\tt ut}         & rad       & Observation time \\
      {\tt lst}        & sec       & Local Sideral Time \\
      {\tt az}         & rad       & Azimuth \\
      {\tt el}         & rad       & Elevation \\
      {\tt frontend}   & ---       & Receiver names \\
      {\tt scan}       & ---       & Scan number \\
      {\tt backend}    & code      & Backend code \\
      {\tt obstype}    & code      & Observation type \\
      {\tt switchmode} & code      & Switching mode \\
      {\tt filstatus}  & code      & Can read file or not? \\
      {\tt calstatus}  & code      & Calibration status \\
      {\tt filename}   & ---       & \imbfits{} file (no path) \\
      {\tt itime}      & nanosec   & Last indexing time (from 01-jan-1970)
    \end{tabular}
    \label{tbl:mrtoptimize}
  \end{center}
\end{table}

\begin{table}[htb]
  \begin{center}
    \caption{Memory-only index elements associated to each entry}
    \vspace{1ex}
    \begin{tabular}{l|l|l}
      Parameter  & Unit & Comment \\
      \hline
      {\tt num}  & ---  & Observation number \\
      {\tt mnum} & ---  & Entry position in the Input indeX (IX) \\
      {\tt fnum} & ---  & Entry position in the index file \\
      {\tt idir} & code & Associated index file and \imbfits{} directory \\
      {\tt sort} & ---  & Sorting array
    \end{tabular}
    \label{tbl:extras}
  \end{center}
\end{table}

However, these elements alone are not enough to deal with each \imbfits{}
file referenced in the index. Extra numbers are associated on-the-fly, at
load time. The Table~\ref{tbl:extras} summarizes those numbers. In details,
here is the exact meaning of those numbers.
\begin{description}
\item[{\tt num}] The \emph{observation number} is a unique number
  associated to each \imbfits{} file. Uniqueness is ensured by a unique
  combination of the observing date, the scan number, and the backend. In a
  memory index, there can be several {\tt version} of the same observation,
  meaning that the same \imbfits{} file is indexed several times (\eg{}, a
  first time when the index is built, a second time after the
  calibration). This number (and optionaly its version) is intended to be
  used as a frontend to the end-user. However, it is volatile: It can
  change from one session (or one INDEX OPEN) to another, depending on the
  index contents, and their number and order if several indexes are loaded
  in memory.  In practice, the observation numbers are contiguous, starting
  from 1, and ordered by observing date, then by scan number, then by
  backend code.
\item[{\tt mnum}] The \emph{memory number} is the entry position in the
  input index (IX), and thus it provides a unique identifier for each entry
  indexed in memory. This means that the array {\tt ix\%mnum(:)} runs
  contiguously from 1. Since IX is the absolute reference for all
  sub-indexes (\eg{}, the current index CX), each entry of any index can
  easily be identified thanks to this back pointer (\eg{}, {\tt
    cx\%mnum(ient)}).
\item[{\tt fnum}] The \emph{file number} is the entry position in its
  associated index file. This number can not be implicit, because several
  index files can be loaded in memory, and because the entries are
  reordered by date, scan and backend at load time.
\item[{\tt idir}] the \emph{directory code} identifies each index file
  loaded in memory, so that each entry knows easily to which index file it
  comes from. This code also identifies the directory the \imbfits{} can be
  found in (remember the index file may not be hosted in the same
  directory; this is controled by the user through the command
  \command{INDEX /FILE}).
\item[{\tt sort}] The \emph{sorting array} is an indirection array which
  would order the memory index by date, by scan, and by backend. At load
  time, the entries are actually ordered in memory, so this array is not
  needed at this time. But later on, commands like \command{CALIBRATE} or
  \command{PIPELINE} can add new versions of the entries. These new entries
  are appended at the end of the memory indexes, in order to avoid breaking
  the cross references between indexes (in particular, the {\tt
    cx\%mnum(:)} back pointers to {\tt ix}). Each application which needs
  to access the entries sorted by date, scan, and backend, should then use
  the sorting array. Typically, one has to loop on {\tt
    cx\%mnum(cx\%sort(ient))} (with {\tt ient} 1 to {\tt cx\%next-1}) to
  access the entries ordered correctly.
\end{description}

\chapter{Scan date vs observation date: Midnight issue}

When a new scan is \emph{prepared}\footnote{\emph{scan prepared} or
  \emph{scan loaded} in the NCS nomenclature.} at the telescope, the
associated \imbfits{} file name is constructed with the current date. Later
in this process (up to several minutes after), the scan is actually
\emph{started}\footnote{\emph{scan started} or \emph{subscan 1 started} in
  the NCS nomenclature.}, and the current date and time are used to build
the \imbfitskey{MJD-OBS} and \imbfitskey{DATE-OBS} in the FITS Primary
header. If those two steps are performed exactly before and after midnight
respectively, the file name and the Primary header refer to 2 different
dates, e.g.
\begin{verbatim}
iram30m-fts-20120809s323-imb.fits:
  MJD-OBS =     56149.0038078704 / MJD at observation start
  DATE-OBS= '2012-08-10T00:05:29.000'
\end{verbatim}

For several purposes, \mrtcal{} saves the \emph{observing date and time} of
each file in the index. In order to deal correctly with the above issue, it
is decided
\begin{enumerate}
\item to save the date as found in the file name, so that there is no
  difference for the user between the file name and the date exposed by
  \mrtcal ;
\item the time value saved for this file refers to the above date.
\end{enumerate}
The direct consequence of those 2 rules is that, in case of the midnight
issue exposed above, the UT value will be larger then 24h. Note that the UT
value associated to each file is saved for bookkeeping purpose, e.g. for
analysis of the calibration though date and time. It is considered as a
typical value for the whole scan. Each individual spectrum produced by
\mrtcal{} uses its own date and time.

In details, the observing date saved in the index is used
\begin{itemize}
\item to check the entry uniqueness (date, scan and backend triplet should
  be unique),
\item to sort the entries (by date, scan and backend),
\item through the command \command{MFIND /DATE},
\item in the output of the command \command{MLIST},
\item in the Sic variables \command{MDX\%DOBS} and
  \command{MHEAD\%KEY\%DOBS}
\end{itemize}
and the associated UT value is used
\begin{itemize}
\item to sort the entries,
\item in the output of the command \command{MLIST},
\item in the Sic variables \command{MDX\%UT} and \command{MHEAD\%KEY\%UT}
\end{itemize}
The user should be ready to encounter UT values larger than 24h in the
\command{MLIST} output and larger than $2 \pi$ in the Sic variables.


\chapter{Patching IMB-FITS at read time}

\mrtcal\ is divided in 2 main libraries:
\begin{enumerate}
\item the libimbfits reads the IMB-FITS files and fills a memory structure from it,
\item the libmrtcal gets the libimbfits memory structure, and use it
  to index the files and calibrate them.
\end{enumerate}
When \mrtcal\ has to deal with several versions of the IMB-FITS
(e.g. V1.35 for HERA and V2 for EMIR), the variations are
\emph{hidden} in the libimfits so that the libmrtcal does not have to
worry about those \emph{details}. This means that the memory structure
is unique while the disk structure differ\footnote{Note that this
  strategy is in use since a long time in \gildas\ to ensure the
  backward compatibility of the various formats.}. There are also
other cases when we want to fill memory structures that differ from
the file on disk. All this is described hereafter.

Note that each patch is saved as a warning in the comment field of
each element. Those warning are visible when using the command {\tt
  MDUMP}.

\section{Added elements when missing}

The purpose of adding elements in the HDUs when they are missing is
that the libmrtcal does not have to worry about this element existing
or not, and and doing different things (different code) in one case or
the other. The following elements are added:

\begin{itemize}
\item {\tt POLEX} and {\tt POLEY} are absent under V1.35: they are
  defined and default to 0,
\item {\tt SYSOFF}, {\tt XOFFSET}, and {\tt YOFFSET} columns in the
  Scan table can have 0, 1, or 2 rows. They are forced to 2 rows
  providing \emph{Nasmyth} and \emph{projection} offsets (default to 0
  if they were absent),
\item the {\tt LINENAME} column in the BackEnd HDU is built from the
  {\tt LINENAME} column in the FrontEnd HDU for IMB-FITS version lower
  than 2, or for 4MHz backend,
\item {\tt MJD\_BEG} and {\tt MJD\_END} keywords are added as
  convenient duplicates of {\tt DATE-OBS} and {\tt DATE-END} keywords
  in all HDUs of each subscan. They offer MJD values instead of ISO
  strings,
\item the column {\tt TSTAMPED} is added to BackendDATA tables in
  IMB-FITS version lower or equal to 1.35. Its defaults to 1. Remember
  that this value is sensitive (it introduces a shift on the time
  stamps) and is actually backend-dependent.
\item an {\tt IFRONT} column is added as a convenient backpointer from
  the BackEnd table to the FrontEnd table, \ie\ each chunk knows
  easily to which receiver it is associated.
\end{itemize}

\section{Modified elements}

\begin{itemize}
\item {\tt IFCENTER}, {\tt FRQOFF1}, and {\tt FRQOFF2} columns from
  the FrontEnd table are reversed for each row where {\tt IFFLIPS} is
  true. This factorizes the sign issues when dealing with IF2 instead
  ot IF1,
\item The column {\tt BAND} is read under the name {\tt PART} for
  IMB-FITS version lower than 2. Same for {\tt REC1} and {\tt REC2}
  read under the names {\tt RECEIVER} and {\tt BAND} respectively,
\item {\tt REFCHAN} is patched to correct values (instead of 1) when
  dealing with continuum backend (1 channel per chunk),
\item all the BackEnd columns, namely {\tt PART}, {\tt REFCHAN}, {\tt
    CHANS}, {\tt DROPPED}, {\tt USED}, {\tt PIXEL}, {\tt RECEIVER},
  {\tt BAND}, {\tt POLAR}, {\tt REFFREQ}, {\tt SPACING}, and {\tt
    LINENAME} are sliced into memory to more values than from the file
  in order to factorize at once the {\tt MSET CALIBRATION BANDWIDTH}
  tuning. {\tt NAXIS2} is updated accordingly. See
  section~\ref{sec:slicing} for details,
\item the columns {\tt MJD}, {\tt INTEGTIM} and {\tt ISWITCH} are
  compressed if {\tt MSET CALIBRATION BAD} is {\tt NO} (this is the
  default). In this case all rows where {\tt ISWITCH} is 0 are
  removed. Two new columns {\tt FOREPOIN} and {\tt BACKPOIN} are added
  as cross pointers between the original and compressed columns. See
  section~\ref{sec:data} for details.
\item the {\tt DATE-OBS} and {\tt DATE-END} stamps of each subscan are
  patched if {\tt MSET CALIBRATION MJDINTER} is {\tt YES}. See HELP
  for details.
\end{itemize}

\newpage
\chapter{IMB-FITS version 2.13}

For single pixel receiver, the IMB-FITS data format provides the
following FITS extensions:
\begin{enumerate}
\setcounter{enumi}{-1}
\item {\tt Primary}: general description of the observation
\item {\tt IMBF-scan}: scan description (source, antenna offsets, etc)
\item {\tt IMBF-frontend}: frontend description (which receiver was
  connected, etc)
\item {\tt IMBF-backend}: backend configuration i.e. how the chunks
  translate to sky frequency.
\end{enumerate}
Then, the 3 following extensions are repeated for each subscan
(example with FTS backend):
\begin{enumerate}
\setcounter{enumi}{3}
\item {\tt IMBF-backendFTS}: provides the data dumps through time,
\item {\tt IMBF-antenna}: provides the antenna position through time
  at slow and fast rates.
\item {\tt IMBF-subreflector}: not used by \mrtcal.
\end{enumerate}
Each of these extensions are detailed below from an example scan,
showing the output of the command {\tt MDUMP}\footnote{{\tt MDUMP}
  slightly modifies the header elements at read time for
  convenience. Some remarks are added accordingly in the output.}

\section{Primary}

{\small
\begin{verbatim}
SIMPLE   (L)  =                      T / file does conform to FITS standard
BITPIX   (I4) =                     32 / number of bits per data pixel     
NAXIS    (I4) =                      0 / number of data axes               
EXTEND   (L)  =                      T / FITS dataset may contain extensions
TELESCOP (C)  = IRAM 30m               / Telescope Name
ORIGIN   (C)  = IRAM                   / Organisation or Institution
CREATOR  (C)  = Python IRAM MBFITS Wri / Software
IMBFTSVE (R8) =   2.13000000000000     / IMBFITS version
INSTRUME (C)  = fts                    / Backend
OBJECT   (C)  = XXXXXXXX               / Source Name
LONGOBJ  (R8) =   00.0000000000000     / [deg] Source longitude in basis frame
LATOBJ   (R8) =   00.0000000000000     / [deg] Source latitude in basis frame
TIMESYS  (C)  = UTC                    / time system (TT,TAI,UTC ...)
MJD-OBS  (R8) =   57841.4547916667     / MJD at observation start
DATE-OBS (C)  = 2017-03-29T10:54:54.00 /
LST      (R8) =   83367.1998821900     / [s] Local apparent sidereal time (scan start)
PROJID   (C)  = 102-16                 / project ID
QUEUE    (C)  = 102-16                 / Observing Queue
EXPTIME  (R8) =   15.0000000000000     / Total netto integration time [s]
N_OBS    (I4) =                      3 / No of subscans defined
N_OBSP   (I4) =                      3 / No of subscans found
OBSTYPE  (C)  = calibrate              /
NUSEFEED (I4) =                      8 /
TOTANT   (I4) =                     23 / Total no. of LINES in antenna tables
TOTSUBR  (I4) =                     59 / Total no. of LINES in secondary tables
TOTBACK  (I4) =                     33 / Total no. of LINES in backenddata tables
\end{verbatim}
} % \small

\section{IMBF-scan}

{\small
\begin{verbatim}
XTENSION (C)  = BINTABLE               / binary table extension
BITPIX   (I4) =                      8 / 8-bit bytes           
NAXIS    (I4) =                      2 / 2-dimensional binary table
NAXIS1   (I4) =                     23 / width of table in bytes   
NAXIS2   (I4) =                      2 / number of rows in table   
PCOUNT   (I4) =                      0 / size of special data area 
GCOUNT   (I4) =                      1 / one data group (required keyword)
TFIELDS  (I4) =                      3 / number of fields in each row     
EXTNAME  (C)  = IMBF-scan              / name of this binary table extension
TELESCOP (C)  = IRAM 30m               / Telescope Name                     
TELSIZE  (R8) =   30.0000000000000     / [m] Telescope Diameter             
SITELONG (R8) =  -3.39875641986850     / [deg] observatory longitude (EAST) 
SITELAT  (R8) =   37.0684132670517     / [deg] observatory latitude (NORTH) 
SITEELEV (R8) =   2851.50000000000     / [m] observatory elevation          
PROJID   (C)  = 102-16                 / Project ID                         
OBSID    (C)  = XX                     / Observer initials                  
OPERATOR (C)  = Pako                   / Operator initials                  
SCANNUM  (I4) =                    138 / Scan number                        
DATE-OBS (C)  = 2017-03-29T10:54:54.00 / scan start in TIMESYS system       
DATE     (C)  = 2017-03-29T10:55:18    / Date of FITS file creation         
MJD      (R8) =   57841.4547916667     / [day] Scan date/time (Modified Julian Date)
LST      (R8) =   83367.1998821900     / [s] Local apparent sidereal time (scan start)
N_OBS    (I4) =                      3 / Number of observations in this scan          
EXPTIME  (I4) =                     15 / Total netto integration time [s]             
TIMESYS  (C)  = UTC                    / time system (TT, TAI, UTC ...)               
UT1UTC   (R8) =  0.477150000000000     / [s] UT1-UTC time translation                 
TAIUTC   (R8) =   35.0000000000000     / [s] TAI-UTC time translation                 
ETUTC    (R8) =   67.1840000000000     / [s] Ephemeris Time - UTC time translation    
GPSTAI   (R8) =   0.00000000000000     / [s] GPS time - TAI translation               
POLEX    (R8) =  2.714956614210000E-08 / [rad] Celestial Pole Offset X                
POLEY    (R8) =  1.781205464400000E-06 / [rad] Celestial Pole Offset Y                
OBJECT   (C)  = XXXXXXXX               / Source name                                  
CTYPE1   (C)  = RA-SFL                 / Basis system (longitude) --                  
CTYPE2   (C)  = DEC-SFL                / Basis system (latitude) -- XLAT-SFL          
RADESYS  (C)  =                        / additional system definition for ecliptic/equat
EQUINOX  (R8) =   2000.00000000000     / [Julian yrs] Equinox                           
CRVAL1   (R8) =   0.00000000000000     / [deg] Native frame zero in basis system (long.)
CRVAL2   (R8) =   0.00000000000000     / [deg] Native frame zero in basis system (lat.) 
LONPOLE  (R8) =   0.00000000000000     / [deg] Native longitude of celestial pole: range
LATPOLE  (R8) =   0.00000000000000     / [deg] Basis latitude of native pole            
LONGOBJ  (R8) =   00.0000000000000     / [deg] Source longitude in basis frame          
LATOBJ   (R8) =   00.0000000000000     / [deg] Source latitude in basis frame           
SWTCHMOD (C)  = totalPower             / Switch mode                                    
NOSWITCH (I4) =                      1 / no. of switch phases in a switch cycle
PHASETIM (R8) =  0.500000000000000     / [s] Integration time per phase
WOBTHROW (R8) =   0.00000000000000     / [deg] wobbler throw
WOBDIR   (I4) =                      0 / wobbler throw direction
WOBCYCLE (R8) =   0.00000000000000     / [s] wobbler period
WOBMODE  (C)  =                        / wobbler mode (SQUARE/TRIANGULAR})
NFEBE    (I4) =                      8 / \nfebe\ number of FEBEs
PRESSURE (R8) =   728.500000000000     / [hPa] Atmospheric pressure (hPa)
TAMBIENT (R8) =  -3.00000000000000     / [deg C] Outside temperature (C)
HUMIDITY (R8) =   36.6000000000000     / [%] Relative Humidity (%)
WINDDIR  (R8) =   187.300000000000     / [deg] Wind direction (deg)
WINDVEL  (R8) =   1.50000000000000     / [m/s] Wind velocity (m/s)
WINDVELM (R8) =   2.50000000000000     / [m/s] Wind max. velocity (m/s)
DATE-WEA (C)  = 2017-03-29T10:55:15.58 / Time of weather station par.
REFRACTI (R8) =   0.00000000000000     / [none] Refraction Correction, as a function of
THOT     (R8) =   295.150000000000     / [K] Hot Load Temperature in K
DATE-HOT (C)  = 2017-03-29T10:54:44.75 / Time of Hot Load Temp. meas.
TIPTAUZ  (R8) =  0.136721000000000     / [ ] Tau meter zenith opacity
TIPTAUC  (R8) =  0.999226000000000     / [ ] Tau meter goodness of fit
DATE-TIP (C)  = 2017-03-29T10:54:50.85 / Time of Tau from Taumeter
SYSOFF   (C)  = Nasmyth        projection      / WARNING! Added a dummy 'projection' value
XOFFSET  (R4) = -1.9150140E-04   0.000000      / WARNING! Added a dummy 'projection' value
YOFFSET  (R4) =  2.6664753E-05   0.000000      / WARNING! Added a dummy 'projection' value
\end{verbatim}
} % \small

\section{IMBF-frontend}

{\small
\begin{verbatim}
XTENSION (C)  = BINTABLE               / binary table extension
BITPIX   (I4) =                      8 / 8-bit bytes           
NAXIS    (I4) =                      2 / 2-dimensional binary table
NAXIS1   (I4) =                    139 / width of table in bytes   
NAXIS2   (I4) =                      1 / number of rows in table   
PCOUNT   (I4) =                      0 / size of special data area 
GCOUNT   (I4) =                      1 / one data group (required keyword)
TFIELDS  (I4) =                     22 / number of fields in each row     
EXTNAME  (C)  = IMBF-frontend          / name of this binary table extension
SCANNUM  (I4) =                    138 / Scan number                        
DATE-OBS (C)  =                        / observing date (Y2K format with time) in TIMESY
DEWRTMOD (C)  =                        / Dewar tracking system                          
DEWANG   (R8) =   0.00000000000000     / [Deg] Dewar angle                              
FEBEBAND (I4) =                      1 / \nbd\ number of basebands for this febe        
FEBEFEED (I4) =                      8 / \nfd\ total number of feeds                    
NUSEFEED (I4) =                      8 / \nch\ Number of feeds in use.                  
VELOSYS  (R8) =   6.70000000000000     / [km/s] Source Radial Velocity in Reference Fram
SPECSYS  (C)  = LSR                    / Reference Frame                                
VELOCONV (C)  = optical                / Convention for doppler correction              
EMIRBEAM (C)  = right                  / EMIR beam used                                 
RECNAME  (C)  = E230                             / Receiver Name
LINENAME (C)  = L220510                          / Line Name
RESTFREQ (R8) =   220.510000000000               / [GHz] Rest frequency of line
BEAMEFF  (R4) =  0.9200000                       / Beam efficiency
ETAFSS   (R4) =  0.9200000                       / Forward efficiency
GAINIMAG (R4) =  5.0119001E-02                   / (spectral line) Gain ratio image/signal sideban
SIDEBAND (C)  = LI                               / Side Band
SBSEP    (R8) =   12500000000.0000               / [Hz] Sideband separation
WIDENAR  (C)  = NULL                             / WARNING! Column not found in table
TCOLD    (R4) =   32.82200                       / Cold Load Temperature
THOT     (R4) =   292.6630                       / Hot Load Temperature
IFEED    (I4) =              1                   / selected feed
NOFEEDS  (I4) =              1                   / No. of feeds
POLA     (C)  =                                  / Feed orientation
DOPPLER  (C)  = Doppler                          / Frequency tracking
IFCENTER (R4) =   6.250000                       / IF center Frequency
BANDWID  (R4) =   4.000000                       / [Hz] Bandwidth for this receiver
IFFLIPPS (L)  =        F                         / IF flipps frequency scale
SPECLO   (L)  =        F                         / Special LO used
TSCALE   (C)  = antenna                          / Temperature scale selected
FRQTHROW (R4) =   0.000000                       / [GHz] Frequency switching throw
FRQOFF1  (R4) =   0.000000                       / [GHz] Frequency offset 1
FRQOFF2  (R4) =   0.000000                       / [GHz] Frequency offset 2
\end{verbatim}
} % \small

\section{IMBF-backend}

{\small
\begin{verbatim}
XTENSION (C)  = BINTABLE               / binary table extension
BITPIX   (I4) =                      8 / 8-bit bytes           
NAXIS    (I4) =                      2 / 2-dimensional binary table
NAXIS1   (I4) =                     81 / width of table in bytes   
NAXIS2   (I4) =                     24 / number of rows in table
PCOUNT   (I4) =                      0 / size of special data area
GCOUNT   (I4) =                      1 / one data group (required keyword)
TFIELDS  (I4) =                     12 / number of fields in each row
EXTNAME  (C)  = IMBF-backend           / name of this binary table extension
SCANNUM  (I4) =                    138 / Scan number
DATE-OBS (C)  = 2017-03-29T10:54:54.00 / observing date (Y2K format with time) in T
FEBEBAND (I4) =                      1 / \nbd\ number of basebands for this febe
FEBEFEED (I4) =                      8 / \nfd\ total number of channels
NUSEFEED (I4) =                      8 / \nch\ Number of channels in use.
PART     (I4) =              2      ...                   6    / Identification
REFCHAN  (I4) =              1      ...              376833    / Refrence Channel
CHANS    (I4) =          16384      ...               16384    / Number of Channels
DROPPED  (I4) =           1843      ...                1843    / Channels dropped
USED     (I4) =          12494      ...               12494    / Channels used
RECEIVER (C)  = E2HUI               ...      E2VLO             / Receiver connected
IFRONT   (I4) =              1      ...                   1    / Backpointer to frontend table (memory only)
BAND     (C)  = E2HUI               ...      E2VLO             / Sub-band specification
POLAR    (C)  = NONE                ...      NONE              / Polarization
PIXEL    (I4) =              1      ...                   1    / Pixel No. (arrays)
REFFREQ  (R4) =   7200.000          ...       -8480.000        / [MHz] IF reference frequency
SPACING  (R4) = -4.8828125E-02      ...      -4.8828125E-02    / [MHz] Spacing
LINENAME (C)  = L220510             ...      L220510           / Additional Line Name
\end{verbatim}
} % \small

\section{IMBF-backendFTS}

The {\tt DATA} column is omitted in this output:
{\small
\begin{verbatim}
XTENSION (C)  = BINTABLE               / binary table extension
BITPIX   (I4) =                      8 / 8-bit bytes           
NAXIS    (I4) =                      2 / 2-dimensional binary table
NAXIS1   (I4) =                1572888 / width of table in bytes
NAXIS2   (I4) =                     11 / number of rows in table
PCOUNT   (I4) =                      0 / size of special data area
GCOUNT   (I4) =                      1 / one data group (required keyword)
TFIELDS  (I4) =                      5 / number of fields in each row
EXTNAME  (C)  = IMBF-backendFTS        / name of this binary table extension
SCANNUM  (I4) =                    138 / Scan number
OBSNUM   (I4) =                      1 / Observation number
BASEBAND (C)  =                        / [--] Baseband number
DATE-OBS (C)  = 2017-03-29T10:54:54.41 / observation start in TIMESYS system
MJD_BEG  (R8) =   57841.4547964699     / Duplicate of DATE-OBS (memory only)
DATE-END (C)  = 2017-03-29T10:54:58.42 / observation end in TIMESYS system
MJD_END  (R8) =   57841.4548428241     / Duplicate of DATE-END (memory only)
CHANNELS (I4) =                 393216 / \nch\ Number of channels for this baseband
NPHASES  (I4) =                      1 / no. of switch phases in a switch cycle
PHASEONE (C)  = ON                     / First phase is ON or OFF source
TSTAMPED (R8) =   1.00000000000000     / Where the time stamps apply
MJD      (R8) =   57841.4547911713     ...    57841.4548490417 / [day] MJD at integration start
INTEGTIM (R8) =  0.498954000000000     ...   0.498954000000000 / [s] Integration time
ISWITCH  (I4) =              1      ...                   1    / Integration type
FOREPOIN (I4) =              1      ...                  11    / Forward pointer to compressed columns
BACKPOIN (I4) =              1      ...                  11    / Backward pointer to uncompressed columns
\end{verbatim}
}

\section{IMBF-antenna}

\mrtcal\ splits the antenna extension in the \emph{antenna slow} and
\emph{antenna fast} descriptions. Antenna slow:

{\small
\begin{verbatim}
XTENSION (C)  = BINTABLE               / binary table extension
BITPIX   (I4) =                      8 / 8-bit bytes           
NAXIS    (I4) =                      2 / 2-dimensional binary table
NAXIS1   (I4) =                   6228 / width of table in bytes   
NAXIS2   (I4) =                      8 / number of rows in table   
PCOUNT   (I4) =                      0 / size of special data area 
GCOUNT   (I4) =                      1 / one data group (required keyword)
TFIELDS  (I4) =                     19 / number of fields in each row     
EXTNAME  (C)  = IMBF-antenna           / name of this binary table extension
SCANNUM  (I4) =                    138 / Scan number                        
OBSNUM   (I4) =                      1 / Observation number                 
DATE-OBS (C)  = 2017-03-29T10:54:54.41 / observation start in TIMESYS system
MJD_BEG  (R8) =   57841.4547964699     / Duplicate of DATE-OBS (memory only)
DATE-END (C)  = 2017-03-29T10:54:58.42 / observation end in TIMESYS system  
MJD_END  (R8) =   57841.4548428241     / Duplicate of DATE-END (memory only)
OBSTYPE  (C)  = calibrate              / Observation type                   
SUBSTYPE (C)  = calSky                 / Subscan type                       
SUBSTIME (R8) =   5.00000000000000     / [s] Subscan time                   
SYSTEMOF (C)  = projection             / System for offsets
SUBSXOFF (R8) = -2.908882000000000E-03 / [rad] Subscan x offset
SUBSYOFF (R8) =   0.00000000000000     / [rad] Subscan y offset
SETYPE01 (C)  =                        / Segment type
SETIME01 (R8) =   0.00000000000000     / [s] Segment time
SEXOFF01 (R8) =   0.00000000000000     / [rad] Segment x offset
SEYOFF01 (R8) =   0.00000000000000     / [rad] Segment y offset
SEXSTA01 (R8) =   0.00000000000000     / [rad] Segment x start
SEYSTA01 (R8) =   0.00000000000000     / [rad] Segment y start
SEXEND01 (R8) =   0.00000000000000     / [rad] Segment x end
SEYEND01 (R8) =   0.00000000000000     / [rad] Segment y end
SESPES01 (R8) =   0.00000000000000     / [rad/s] Speed at start of segment
SESPEE01 (R8) =   0.00000000000000     / [rad/s] Speed at end of segment
DOPPLERC (R8) =  0.999881490337000     / Doppler correction factor
OBSVELRF (R8) =   28.8325140071000     / [km/s] Observer velocity in rest frame
MJD      (R8) =   57841.4547916667     ...    57841.4548726852 / [Julian day] MJD at integration start
LST      (R8) =   83367.2657212536     ...    83374.2848861940 / [s] Local apparent sidereal time (integration s
LONGOFF  (R8) = -2.908881986513734E-03 ...  -2.908881986513734E/ [rad] long. offset from source in user native f
LATOFF   (R8) =   0.00000000000000     ...    0.00000000000000 / [rad] lat. offset from source in user native fr
CAZIMUTH (R8) =   1.31577241984522     ...    1.31600523032287 / [rad] Commanded Azimuth
CELEVATI (R8) =  0.630288975389516     ...   0.630683181202881 / [rad] Commanded Elevation
TRACEFLA (I4) =              0      ...                   0    / Trace Flag
\end{verbatim}
} % \small

Antenna fast:
{\small
\begin{verbatim}
XTENSION (C)  = BINTABLE               / binary table extension
BITPIX   (I4) =                      8 / 8-bit bytes
NAXIS    (I4) =                      2 / 2-dimensional binary table
NAXIS1   (I4) =                   6228 / width of table in bytes
NAXIS2   (I4) =                      8 / number of rows in table
PCOUNT   (I4) =                      0 / size of special data area
GCOUNT   (I4) =                      1 / one data group (required keyword)
TFIELDS  (I4) =                     19 / number of fields in each row
EXTNAME  (C)  = IMBF-antenna           / name of this binary table extension
DATE-OBS (C)  = 2017-03-29T10:54:54.41 / observation start in TIMESYS system
MJD_BEG  (R8) =   57841.4547964699     / Duplicate of DATE-OBS (memory only)
DATE-END (C)  = 2017-03-29T10:54:58.42 / observation end in TIMESYS system
MJD_END  (R8) =   57841.4548428241     / Duplicate of DATE-END (memory only)
TRACERAT (I4) =                    128 / No. of fast traces per slow trace
MJDFAST  (R8) =   57841.4547569444     ...    57841.4548494466 / [Julian day] MJD at integration start
AZIMUTH  (R8) =   1.31550400027680     ...    1.31576626838095 / [rad] Encoder Azimuth
ELEVATIO (R8) =  0.631753369365499     ...   0.632199246447845 / [rad] Encoder Elevation
\end{verbatim}
} % \small

\end{document}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

