\documentclass{article}

\usepackage{graphicx}
% \usepackage{pdfpages}
\usepackage{fancyhdr}

\makeindex{}

\makeatletter
\textheight 625pt
\textwidth 460pt
\oddsidemargin 0pt
\evensidemargin 0pt
\marginparwidth 50pt
\topmargin 0pt
\brokenpenalty=10000

% Makes fancy headings and footers.
\pagestyle{fancy}

\fancyhf{} % Clears all fields.

\lhead{\scshape Bardeau et al., 2015}
\rhead{\scshape \nouppercase{\rightmark}}   %
\fancyfoot[C]{\bfseries \thepage{}}

\renewcommand{\headrulewidth}{0pt}    %
\renewcommand{\footrulewidth}{0pt}    %

\renewcommand{\sectionmark}[1]{%
  \markright{\thesection. \MakeLowercase{#1}}}
\renewcommand{\subsectionmark}[1]{}

\fancypagestyle{firststyle}
{
   \fancyhf{}
   \fancyhead[C]{Original version at \tt http://iram-institute.org/medias/uploads/class-herschel-fits.pdf}
   \fancyfoot[C]{\bfseries \thepage{}}
}

\newcommand{\ie} {{\em i.e.}}
\newcommand{\eg} {{\em e.g.}}
\newcommand{\cf} {cf.}

\newcommand{\gildas} {\mbox {\bf GILDAS}}
\newcommand{\sic}    {\mbox {\bf SIC}}
\newcommand{\greg}   {\mbox {\bf GREG}}
\newcommand{\class}  {\mbox {\bf CLASS}}

\newcommand{\hh}  {Herschel-HIFI}
\newcommand{\hhf} {Herschel-HIFI FITS}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%

\begin{document}

\thispagestyle{firststyle}

\begin{center}
  \includegraphics[width=5cm]{gildas-logo}\\[2\bigskipamount]
  \huge{}
  IRAM Memo 2015-3\\
  Importing Herschel-FITS into \class\\[3\bigskipamount]
  \large{}
  S. Bardeau$^1$, D.Teyssier$^2$, D.Rabois$^3$, J. Pety$^{1,4}$\\[0.5cm]
  1. IRAM (Grenoble)\\
  2. ESAC (Madrid)\\
  3. IRAP (Toulouse)\\
  4. Observatoire de Paris\\[\bigskipamount]
  \normalsize{}
  October, 28$^{st}$ 2016\\
  Version 1.0.1
\end{center}

\begin{abstract}

  Herschel/HIFI data were already readable by \class\ after conversion
  into specific FITS file by the HiClass task from HIPE. These FITS
  files are however not the ones served by the Herschel Science
  Archive as result of the standard data processing. The \class\
  community thus requested the possibility to fill Herschel/HIFI
  archive science products directly into \class.

  In a nutshell, the new filler automatically recognizes the
  Herschel/HIFI FITS level 2.0 and 2.5. The standard \class\ sections
  ({\tt General, Position, Spectroscopy, Calibration}) are filled from
  the FITS header parameters. A dedicated Herschel section is added to
  the \class\ Data Format in order to attach HIFI specific
  metadata. The data are filled as one or several \class\ spectra
  depending of the FITS level and/or context (On-The-Fly, spectral
  survey, etc). The associated FLAG and LINE arrays are filled as
  associated arrays (new \class\ feature documented in a separate
  memo) to each spectrum data. This memo describes all these points in
  details.\\

  Keywords: \class\ Data Format

  Related documents: \class\ documentation, \class\ Associated Arrays.
\end{abstract}

\newpage{}
\tableofcontents{}

\newpage{}

\section{Quick start guide}

You can import \hhf\ files if you are using a \class\ version equal or
newer than oct15. Older versions of \class\ will warn you about
unsupported data and will give you hand back without importing
anything.

\subsection{Supported HIPE versions}

The filler is able to import data created by versions\footnote{The
  version can be found in the header card {\tt CREATOR}.} of the HIPE
software equal or newer than 12. Older versions are rejected by
\class. However, it is recommended to retrieve the latest \hhf\ files
(version 14, expected mid-2016) from the Herschel Science Archive, as
there were several fixes (\eg\ missing keywords) added since the
versions 12 and 13. This means the resulting spectra in \class\ will
be incomplete (\eg\ null values, in particular in the Herschel-HIFI
section) or approximate for versions 12 and 13. If this happens,
\class\ will issue warnings when it had to choose the default value
for a parameter. Defaults are documented in the look-up tables in the
Section~\ref{txt:lut}.

\subsection{Importing \hhf}

Once you have installed a correct version of \class, importing \hhf\
is as simple as opening a \class\ output file and calling the command
{\tt FITS}, \eg
\begin{verbatim}
LAS> file out myfile.hifi single
LAS> fits read herschel-hifi.fits
\end{verbatim}

Depending on the observing mode and on the pipeline level used in the
\emph{Herschel Common Science System} (HCSS), \class\ will produce one
or more spectra from a single FITS file. Once this is done, you can
retrieve the spectra as usual in \class, \eg:

\begin{verbatim}
LAS> file in myfile.hifi
LAS> find
LAS> list
LAS> get first
\end{verbatim}

The first spectrum is stored in the {\tt R} buffer and is ready for
reduction with the \class\ analysis tools.

\subsection{Observation header}

The relevant metadata have been exported from the \hhf\ to the \class\
observation header, namely in:
\begin{itemize}
\item the General section in {\tt R\%HEAD\%GEN\%},
\item the Position section in {\tt R\%HEAD\%POS\%},
\item the Spectroscopic section in {\tt R\%HEAD\%SPE\%},
\item the Frequency Switch section in {\tt R\%HEAD\%FSW\%} if relevant,
\item a subset of the Calibration section in {\tt R\%HEAD\%CAL\%}, and
\item the new Herschel-HIFI section in {\tt R\%HEAD\%HER\%}, where the
  metadata specific to this instrument can be found.
\end{itemize}
A raw overview of the whole header can be displayed with the command
{\tt DUMP}. Each section can be listed (name and values) with the
command {\tt EXAMINE}. We present in the section~\ref{txt:lut} how
these sections were filled from the \hhf\ headers.\\

\subsection{Associated Arrays}

The spectra imported from \hhf\ are also one of the firsts to use the
so-called \emph{Associated Arrays} in the \class\ observation. These
arrays can be found under the structure {\tt R\%ASSOC\%}. \class\ can
understand those special arrays (see below where the dedicated
documentation can be found). In practice, the \hh\ spectra provides
the usual intensity array (\ie\ in the {\tt RY} variable), but also 2
arrays of the same size (same number of channels):
\begin{itemize}
\item the {\tt BLANKED} Associated Array is an integer flag array
  indicating if the {\tt RY} values should be blanked out (1) or not
  (0), under the user choice. This array is similar the usual
  \emph{bad} values in \class, except that the actual values (\ie\
  before blanking) are still available. The typical use of this array
  is:
\begin{verbatim}
  LAS> LET RY R%HEAD%SPE%BAD /WHERE R%ASSOC%BLANKED%DATA.EQ.1
\end{verbatim}
\item the {\tt LINE} Associated Array is an integer flag indicating if
  the {\tt RY} values are in (1) or out (0) a spectral line window. If
  the option {\tt /ASSOCIATED} is invoked with {\tt SET WINDOW}, the
  command {\tt BASE} will use this array (\ie\ ignore channels in a
  spectral window) for baselining (see {\tt HELP SET WINDOW} for
  details). Typical use is:
\begin{verbatim}
  LAS> SET WINDOW /ASSOCIATED  ! Will use the LINE Associated Array
  LAS> SHOW WINDOW             ! Check you will use the LINE Associated Array
  LAS> DRAW WINDOW             ! Draw the LINE Associated Array on the spectrum
  LAS> BASE                    ! Use the LINE Associated Array during baselining
\end{verbatim}
\end{itemize}

Note that the \emph{Associated Arrays} are removed by the commands
that do not know how to process them. In this case, a warning is
issued. This is being solved as the \emph{Associated Arrays} are more
and more integrated in \class{}. In order to know the current status,
please check the dedicated documentation for your current \class{}
version in the widget menu $>$ Help $>$ CLASS $>$ Associated Arrays.

\subsection{Setting the source velocity}
\label{txt:velocity}

The spectra are imported assuming a zero-valued velocity of the source
in the LSR frame. This most probably implies that the line frequencies
will appear redshifted. The standard use of \class\ is to deliver the
frequency axis in the source frame. To stick to this use, you have to
modify the source velocity for each spectrum early in the reduction
process with the command {\tt MODIFY VELOCITY} (see associated {\tt
  HELP} for details). In order to help you, the source systemic
velocity of the source provided in the HIFI observing setup by the
user is imported into the variable {\tt R\%HEAD\%HER\%VINFO}. If its
value is correct, the command
\begin{verbatim}
LAS> MODIFY VELOCITY R%HEAD%HER%VINFO
\end{verbatim}
will shift the frequency axis to the rest frame of this velocity.

\subsection{Going from $T_A^*$ to $T_{\mathrm{mb}}$}

In \class{}, the brightness scale is assumed to be expressed in
$T_A^*$ when the forward and beam efficiencies are equal. To check
this, you can type
\begin{verbatim}
LAS> EXAMINE R%HEAD%CAL%BEEFF R%HEAD%CAL%FOEFF
R%HEAD%CAL%BEEFF = 0.9600000    ! Real    GLOBAL RO
R%HEAD%CAL%FOEFF = 0.9600000    ! Real    GLOBAL RO
\end{verbatim}
In this case, if the source properties are better represented by the
main temperature scale ($T_{\mathrm{mb}}$), you can convert the scale
from $T_A^*$ to $T_{\mathrm{mb}}$ with
\begin{verbatim}
LAS> EXAMINE R%HEAD%HER%ETAL R%HEAD%CAL%FOEFF  ! Check that they have identical values
R%HEAD%HER%ETAL = 0.9600000     ! Real    GLOBAL RO
R%HEAD%CAL%FOEFF = 0.9600000    ! Real    GLOBAL RO
LAS> EXAMINE R%HEAD%HER%ETAMB                  ! Check that the beam efficiency is sensible
R%HEAD%HER%ETAMB =   0.6223738  ! Real    GLOBAL RO
LAS> MODIFY BEAM_EFF R%HEAD%HER%ETAMB          ! Scale conversion
I-MODIFY,  Former beam efficiency: 0.9600, new: 0.6224
\end{verbatim}
The {\tt MODIFY BEAM\_EFF} command will multiply the spectrum by {\tt
  R\%HEAD\%CAL\%FOEFF / R\%HEAD\%HER\%ETAMB} and change the associated
header parameters accordingly, \eg\ baseline rms and $T_{\mathrm{sys}}$.

\section{Look-up tables}
\label{txt:lut}

We describe here, section by section, how the \class\ observation
header is filled from the \hhf\ header and data. You can refer to the
\class\ or \hh\ documentations for the exact specification of each
component. The concept of FITS cards and metacards is explained in the
section \ref{txt:cards}.

\begin{table}[hb]
\small
\centering
\caption{Look-up table between the \class\ General section and the
  \hhf. Depending on the FITS structure (pipeline level), some
  elements found in a FITS header or in a FITS binary table. FITS
  names are case sensitive. The elements not described here are set
  to their usual undefined value in \class.}
\vspace{2ex}
\begin{tabular}{lclc}
  \hline
  \hline
  Name           & Type & Value                                                                      & Default      \\
  \hline
  {\tt num}      & I*8  & Automatic                                                                  &              \\
  {\tt ver}      & I*4  & 1                                                                          &              \\
  {\tt teles}    & C*12 & {\tt HIF-LL-SP-BB} where:                                                  &              \\
                 &      & - {\tt LL} is {\tt 00} (level 2.5) or the subband number in the column {\tt flux\_*} &    \\
                 &      & \hspace{1ex} name (level 2.0),                                             & {\tt 00}     \\
                 &      & - {\tt S} is the spectrometer, {\tt W} for WBS or {\tt H} for HRS (1st letter of the &    \\
                 &      & \hspace{1ex} card {\tt BACKEND}, else of the metacard {\tt polarization})  & Blank        \\
                 &      & - {\tt P} is the polarization, {\tt V} or {\tt H} (5th letter of the card {\tt BACKEND}, else & \\
                 &      & \hspace{1ex} of the metacard {\tt polarization})                           & Blank        \\
                 &      & - {\tt BB} is the band name (2 first letters of the card {\tt BAND})       & Blank        \\
  {\tt dobs}     & I*4  & Integer part of average of card {\tt DATE-OBS} and card {\tt DATE-END}     & Current day  \\
  {\tt dred}     & I*4  & Current day                                                                &              \\
  {\tt kind}     & I*4  & \class\ internal code for spectroscopic data                               &              \\
  {\tt qual}     & I*4  & \class\ internal code for unknown quality                                  &              \\
  {\tt scan}     & I*8  & Metacard {\tt bbtype} (level 2.5) or column {\tt bbtype} (level 2.0)       & 0            \\
  {\tt subscan}  & I*4  & Metacard {\tt bbnumber} (level 2.5) or column {\tt bbnumber} (level 2.0)   & 0            \\
  {\tt ut}       & R*8  & Fractional part of average of card {\tt DATE-OBS} and card {\tt DATE-END}  & Current time \\
% {\tt st}       & R*8  & 0.d0 & \\
% {\tt az}       & R*4  & 0.0  & \\
% {\tt el}       & R*4  & 0.0  & \\
  {\tt tau}      & R*4  & 0.0  & \\
  {\tt tsys}     & R*4  & Metacard {\tt tsys\_median} (level 2.5) or column {\tt tsys\_median} (level 2.0) & 0 K    \\
  {\tt time}     & R*4  & Metacard {\tt integrationTime} (level 2.5) or                              &              \\
                 &      & column {\tt integrationTime} (level 2.0)                                   & 0 sec        \\
  \hline
\end{tabular}
\label{tab:gensection}
\end{table}

\subsection{Intensity array}

The intensity array ({\tt RY}) is extracted from one or more columns
in the binary table. There are 2 possibilities:
\begin{itemize}
\item the {\tt flux\_*} columns (where {\tt *} is the subband number),
  each of them producing a different spectrum (level 2.0). In this
  case, one row of the column contains all the channel values. The
  associated frequency array is found in the column {\tt
    lsbfrequency\_*} or {\tt usbfrequency\_*}, else an error is
  raised.
\item the {\tt flux} column, producing a single spectrum (level
  2.5). In this case one row of the column contains 1 channel
  value. The associated frequency array is found in the column {\tt
    frequency} if available, else {\tt wave}, else an error is raised.
\end{itemize}

All spectra (flux and frequency arrays) are always sorted in ascending
frequency (if needed), before setting the spectroscopic section as
described in the Table~\ref{tab:spesection}. The Not-a-Number ({\tt
  NaN}) values (if any) in the flux array are patched to the {\tt bad}
value defined in the \class\ header.

\subsection{General section}

A subset of the general section is filled. The elements \emph{sidereal
  time}, \emph{azimuth}, and \emph{elevation} are left to the \class\
internal defaults as they have no meaning for space-based
observations. Note that for spectral scans (frequency surveys), the
{\tt integrationTime} is set by HIPE to the integration time of the
individual tunings multiplied by the median redundancy accross the
frequency axis.

\subsection{Spectroscopic section}

\begin{table}[htb]
\small
\centering
\caption{Look-up table between the \class\ Spectroscopic section and the
  \hhf. Depending on the FITS structure (pipeline level), some
  elements found in a FITS header or in a FITS binary table. FITS
  names are case sensitive.}
\vspace{2ex}
\begin{tabular}{lcl}
  \hline
  \hline
  \class\        & Type & Value \\
  \hline
  {\tt line}     & C*12 & {\tt DECON\_SSB} if Card {\tt OBS\_MODE} contains {\tt SScan} and \\
                 &      & if Card {\tt CLASS\_\_\_} contains {\tt Spectrum1d}, else \\
                 &      & {\tt XXXXXXXX\_YYY} where {\tt XXXXXXXX} is the {\tt lofreq} in GHz and \\
                 &      & {\tt YYY} is the metacard {\tt sideband} value ({\tt GHZ} if not defined) \\
  {\tt nchan}    & I*4  & Card {\tt NAXIS2} (level 2.5) or \\
                 &      & Card {\tt TDIM*} for column {\tt flux\_*} (level 2.0) \\
  {\tt restf}    & R*8  & Value at {\tt rchan} in column {\tt frequency} or {\tt wave} (level 2.5) or \\
                 &      & value at {\tt rchan} in column {\tt lsbfrequency\_*} or {\tt usbfrequency\_*} (level 2.0) \\
  {\tt image}    & R*8  & 2*{\tt lofreq}-{\tt restf} if {\tt lofreq} is defined, \\
                 &      & \class\ internal code for undefined value otherwise \\
  {\tt doppler}  & R*8  & 0.d0 \\
  {\tt rchan}    & R*8  & ceiling({\tt nchan}+1)/2 (\ie\ middle of spectrum) \\
  {\tt fres}     & R*8  & Absolute of spacing in column {\tt frequency} or {\tt wave} (level 2.5) or \\
                 &      & absolute of spacing in column {\tt lsbfrequency\_*} or {\tt usbfrequency\_*} (level 2.0) \\
  {\tt vres}     & R*8  & $-c \times$ {\tt fres} / {\tt restf} \\
  {\tt voff}     & R*8  & 0.d0 \\
  {\tt bad}      & R*4  & -1000.0 \\
  {\tt vtype}    & I*4  & code for LSR referential if Metacard {\tt freqFrame} is {\tt LSRk}, \\
                 &      & code for unknown referential if Metacard {\tt freqFrame} is {\tt source}, \\
                 &      & error otherwise. \\
  \hline
  Not in header: &      & \\
  {\tt lofreq}   & R*8  & Metacard {\tt LoFrequency} else {\tt LoFrequency\_measured} (level 2.5) or \\
                 &      & Column {\tt LoFrequency} else {\tt LoFrequency\_measured} (level 2.0), \\
                 &      & 0.d0 otherwise. \\
  \hline
\end{tabular}
\label{tab:spesection}
\end{table}

The exact status of the spectroscopic axes after HIFI calibration is
available here:\\
{\tt
  http://herschel.esac.esa.int/twiki/pub/Public/HifiCalibrationWeb/freq\_vel\_1.1.pdf}\\

In the context of \class\, we can note that the frequencies found in
the \hhf\ are expressed in the LSR frame\footnote{LSR frame, except
  for the Solar System Objects for which \hhf\ uses the source frame
  $(v=0)$; in this case, the velocity type is set as \emph{unknown} in
  \class.}  (hence the null Doppler factor). The source velocity is
assumed to be 0 at import time; this is consistent with the null
Doppler factor. It is the responsibility of the user to apply the
needed {\tt MODIFY VELOCITY} correction to apply the frequency shift
implied by the source systemic velocity (see
section~\ref{txt:velocity}), \ie, to get the frequency axis in the
source frame.

Moreover, at import time, the default rest frequency of the spectrum
is arbitrarily set to the middle of the spectrum bandpass. Here again,
it is the responsibility of the user to apply the needed {\tt MODIFY
  FREQUENCY} correction to center the velocity axis around the line
rest frequency under consideration.

\subsection{Position section}

The position section is described in the
Table~\ref{tab:possection}. Note that the source position is described
as the offsets between the actual (reconstructed) position ({\tt RA}
or {\tt longitude}, {\tt DEC} or {\tt latitude}) and the nominal
(commanded) position ({\tt RA\_NOM}, {\tt DEC\_NOM}). This also means
that for Solar System Objects (SSO), the offsets are not expressed in
the comoving frame, but from a fixed position on sky.

\begin{table}[htb]
\small
\centering
\caption{Look-up table between the \class\ Position section and the
  \hhf. Depending on the FITS structure (pipeline level), some
  elements found in a FITS header or in a FITS binary table. FITS
  names are case sensitive.}
\vspace{2ex}
\begin{tabular}{lclc}
  \hline
  \hline
  \class         & Type & Value                                                           & Default       \\
  \hline
  {\tt sourc}    & C*12 & Card {\tt OBJECT}                                               & {\tt UNKNOWN} \\
  {\tt system}   & I*4  & \class\ internal code for Equatorial system                     &               \\
  {\tt equinox}  & R*4  & Card {\tt EQUINOX}                                              & Error         \\
  {\tt proj}     & I*4  & \class\ internal code for Radio projection                      &               \\
  {\tt lam}      & R*8  & Card {\tt RA\_NOM}                                              & Error         \\
  {\tt bet}      & R*8  & Card {\tt DEC\_NOM}                                             & Error         \\
  {\tt projang}  & R*8  & 0.d0                                                            &               \\
  {\tt lamof}    & R*4  & Offset in current projection between card {\tt RA\_NOM} and     & 0, \ie\ actual position \\
                 &      & card {\tt RA} (level 2.5) or column {\tt longitude} (level 2.0) & defaults to nominal \\
  {\tt betof}    & R*4  & Offset in current projection between card {\tt DEC\_NOM} and    & 0, \ie\ actual position \\
                 &      & card {\tt DEC} (level 2.5) or column {\tt latitude} (level 2.0) & defaults to nominal \\
  \hline
\end{tabular}
\label{tab:possection}
\end{table}

\subsection{Calibration section}

The \class\ calibration section is activated and partially filled if
the metacard {\tt temperatureScale} is defined. The rules are detailed
in Table~\ref{tab:calsection}.

\begin{table}[htb]
\small
\centering
\caption{Look-up table between the \class\ Calibration section and the
  \hhf. Depending on the FITS structure (pipeline level), some
  elements found in a FITS header or in a FITS binary table. FITS
  names are case sensitive. The elements not described here are set
  to their usual undefined value in \class.}
\vspace{2ex}
\begin{tabular}{lclc}
  \hline
  \hline
  \class      & Type & Value                                                                                 & Default \\
  \hline
  {\tt beeff} & R*4  & Metacard {\tt forwardEff} if metacard {\tt temperatureScale} is {\tt T\_A*}, else     & 0.0 \\
              &      & metacard {\tt beamEff} if metacard {\tt temperatureScale} is {\tt T\_MB}, else error  &     \\
  {\tt foeff} & R*4  & Metacard {\tt forwardEff}                                                             & 0.0 \\
  {\tt gaini} & R*4  & Metacard {\tt *sbGain} ({\tt usb} or {\tt lsb})                                       & 0.0 \\
  {\tt tchop} & R*4  & Metacard {\tt hbbTemp} (level 2.5) or 1st value in Column {\tt hot\_cold} (level 2.0) & 0.0 \\
  {\tt tcold} & R*4  & Metacard {\tt cbbTemp} (level 2.5) or 2nd value in Column {\tt hot\_cold} (level 2.0) & 0.0 \\
  \hline
\end{tabular}
\label{tab:calsection}
\end{table}

\subsection{Frequency Switch section}

This section is enabled and filled in the \class\ observation header
if the FITS card {\tt OBS\_MODE} contains the string {\tt FSwitch} and
if a non-null LO throw value is found in the FITS data.

\begin{table}[htb]
\small
\centering
\caption{Look-up table between the \class\ Frequency Switch section
  and the \hhf. Depending on the FITS structure (pipeline level),
  some elements found in a FITS header or in a FITS binary table. FITS
  names are case sensitive. Some values are not read from the FITS but
  explicitly set.}
\vspace{2ex}
\begin{tabular}{lclc}
  \hline
  \hline
  \class          & Type & Value                                                              & Default        \\
  \hline
  {\tt nphas}     & I*4  &  2                                                                 &                \\
  {\tt swmod}     & I*4  & Metacard {\tt isFolded} (T/F), translated to \class\ internal code & code for False \\
  {\tt decal[1]}  & R*8  &  0.d0                                                              &                \\
  {\tt duree[1]}  & R*4  &  1.0                                                               &                \\
  {\tt poids[1]}  & R*4  &  +0.5                                                              &                \\
  {\tt decal[2]}  & R*8  & Column {\tt LoThrow} else {\tt lothrow} (level 2.0), or            &                \\
                  &      & Metacard {\tt LoThrow} else {\tt loThrow} (level 2.5)              &                \\
  {\tt duree[2]}  & R*4  &  1.0                                                               &                \\
  {\tt poids[2]}  & R*4  &  -0.5                                                              &                \\
  \hline
\end{tabular}
\label{tab:fswsection}
\end{table}

\subsection{\hh\ section}

The \hh\ section is filled from the FITS according to the
table~\ref{tab:hersection}. Some of the parameters here are available
for bookkeeping purpose, some others are essential for data
analysis. In particular, the sideband gain coefficients\footnote{See
  HIPE documentation for details.} will be used for double-side band
deconvolution.

\subsection{Associated Arrays}

\class\ imports each \hh\ spectrum together with the 2 Associated
Arrays {\tt BLANKED} and {\tt LINE}. They are arrays of 0 or 1, set
according to the rules exposed in Table~\ref{tab:associated}. More
information about these arrays is available in the dedicated \class\
documentation.

\begin{table}[htb]
\centering
\caption{How the \hhf\ flags are translated into the corresponding \class\ 
  Associated Arrays. Some flags apply to the whole spectrum, others per channel
  (marked \emph{ichan}).}
\vspace{2ex}
\begin{tabular}{ll}
\hline
\hline
\class\ name         & Value \\
\hline
{\tt BLANKED[ichan]} & 1 if bit 20 (IGNORE\_DATA) is set in metacard {\tt rowflag} (level 2.5), else \\
                     & 1 if bit 20 (IGNORE\_DATA) is set in column {\tt rowflag} (level 2.0), else \\
                     & 1 if bit 7 (SPUR\_CANDIDATE) is set in column {\tt flag[ichan]} (level 2.5), else \\
                     & 1 if bit 7 (SPUR\_CANDIDATE) is set in column {\tt flag\_*[ichan]} (level 2.0), else \\
                     & 1 if bit 30 (IGNORE\_DATA) is set in column {\tt flag[ichan]} (level 2.5), else \\
                     & 1 if bit 30 (IGNORE\_DATA) is set in column {\tt flag\_*[ichan]} (level 2.0), \\
                     & 0 otherwise. \\

{\tt LINE[ichan]}    & 1 if bit 28 (LINE) is set in column {\tt flag[ichan]} (level 2.5), else \\
                     & 1 if bit 28 (LINE) is set in column {\tt flag\_*[ichan]} (level 2.0), else \\
                     & 1 if bit 29 (BRIGHT\_LINE) is set in column {\tt flag[ichan]} (level 2.5), else \\
                     & 1 if bit 29 (BRIGHT\_LINE) is set in column {\tt flag\_*[ichan]} (level 2.0), \\
                     & 0 otherwise. \\
\hline
\end{tabular}
\label{tab:associated}
\end{table}

\begin{table}[htb]
\small
\centering
\caption{Look-up table between the \class\ \hh\ section and the
  \hhf. Depending on the FITS structure (pipeline level), some
  elements found in a FITS header or in a FITS binary table. FITS
  names are case sensitive.}
\vspace{2ex}
\begin{tabular}{lclcl}
\hline
\hline
\class           & Type & Value                                        & Default   & Description \\
\hline
{\tt obsid}      & I*8  & Card {\tt OBS\_ID}                           & 0         & Observation id \\
{\tt instrument} & C*8  & Card {\tt INSTRUME}                          & Blank     & Instrument name \\
{\tt proposal}   & C*24 & Card {\tt PROPOSAL}                          & Blank     & Proposal name \\
{\tt aor}        & C*68 & Card {\tt AOR}                               & Blank     & Astronomical Observation Request \\
{\tt operday}    & I*4  & Card {\tt ODNUMBER}                          & 0         & Operational day number \\
{\tt dateobs}    & C*28 & Card {\tt DATE-OBS}                          & Blank     & Start date \\
{\tt dateend}    & C*28 & Card {\tt DATE-END}                          & Blank     & End date \\
{\tt obsmode}    & C*40 & Card {\tt OBS\_MODE}                         & Blank     & Observing mode \\
{\tt vinfo}      & R*4  & Metacard {\tt vlsr} if card {\tt REDSHFT} is & 0.0       & Informative source velocity in LSR \\
                 &      & ``optical'' or ``radio''                     &           & \\
{\tt zinfo}      & R*4  & Metacard {\tt vlsr} if card {\tt REDSHFT} is & 0.0       & Informative target redshift \\
                 &      & not ``optical'' or ``radio'', else 0.0       &           & \\
{\tt posangle}   & R*8  & Card {\tt POSANGLE}                          & 0.d0      & Spacecraft pointing position angle \\
{\tt reflam}     & R*8  & Card {\tt RAOFF}                             & 0.d0      & Sky reference (a.k.a. OFF) lambda \\
{\tt refbet}     & R*8  & Card {\tt DECOFF}                            & 0.d0      & Sky reference (a.k.a. OFF) beta \\
{\tt hifavelam}  & R*8  & Metacard {\tt longitude\_cmd} (level 2.5) or & 0.d0      & HIFI ON average H and V lambda \\
                 &      & Column {\tt longitude\_cmd} (level 2.0)      &           & \\
{\tt hifavebet}  & R*8  & Metacard {\tt latitude\_cmd} (level 2.5) or  & 0.d0      & HIFI ON average H and V beta \\
                 &      & Column {\tt latitude\_cmd} (level 2.0)       &           & \\
{\tt etamb}      & R*4  & Card {\tt ETAMB}                             & 0.0       & Main beam efficiency used when \\
                 &      &                                              &           & applying doAntennaTemp \\
{\tt etal}       & R*4  & Card {\tt ETAL}                              & 0.0       & Forward efficiency used when \\
                 &      &                                              &           & applying doAntennaTemp \\
{\tt etaa}       & R*4  & Card {\tt ETAA}                              & 0.0       & Telescope aperture efficiency \\
{\tt hpbw}       & R*4  & Card {\tt HPBW}                              & 0.0       & Azimuthally-averaged HPBW \\
{\tt tempscal}   & C*8  & Card {\tt TEMPSCAL}                          & Blank     & Temperature scale in use \\
{\tt lodopave}   & R*8  & Card {\tt LODOPPAV}                          & {\tt lofreq} & Average LO frequency Doppler \\
                 &      &                                              & (Table~\ref{tab:spesection}) & corrected to freqFrame (SPECSYS) \\
{\tt gim0}       & R*4  & Metacard {\tt *sbGain\_0} ({\tt usb} or {\tt lsb}) & 0.0 & Sideband gain polynomial coeff 0 \\
{\tt gim1}       & R*4  & Metacard {\tt *sbGain\_1} ({\tt usb} or {\tt lsb}) & 0.0 & Sideband gain polynomial coeff 1 \\
{\tt gim2}       & R*4  & Metacard {\tt *sbGain\_2} ({\tt usb} or {\tt lsb}) & 0.0 & Sideband gain polynomial coeff 2 \\
{\tt gim3}       & R*4  & Metacard {\tt *sbGain\_3} ({\tt usb} or {\tt lsb}) & 0.0 & Sideband gain polynomial coeff 3 \\
{\tt mixercurh}  & R*4  & Metacard {\tt MJC\_Hor} (level 2.5) or       & 0.0       & Calibrated mixer junction current \\
                 &      & Column {\tt MJC\_Hor} (level 2.0)            &           & (horizontal polarization) \\
{\tt mixercurv}  & R*4  & Metacard {\tt MJC\_Ver} (level 2.5) or       & 0.0       & Calibrated mixer junction current \\
                 &      & Column {\tt MJC\_Ver} (level 2.0)            &           & (vertical polarization) \\
{\tt datehcss}   & C*28 & Card {\tt DATE}                              & Blank     & Processing date \\
{\tt hcssver}    & C*24 & Card {\tt CREATOR}                           & Blank     & HCSS version \\
{\tt calver}     & C*16 & Card {\tt CALVERS}                           & Blank     & Calibration version \\
{\tt level}      & R*4  & Card {\tt LEVEL}                             & 0.        & Pipeline level \\
\hline
\end{tabular}
\label{tab:hersection}
\end{table}
\clearpage

\section{Technical details}
\label{txt:details}

\subsection{Detecting an \hhf}

A FITS file fed as input of the command {\tt LAS$\backslash$FITS} is
assumed to be an \hhf\ if the Primary HDU:
\begin{enumerate}
\item contains the card {\tt HCSS\_\_\_\_}, and
\item the value of the card {\tt TYPE} is not {\tt HICLASS} nor {\tt
    `Class formatted fits file'}.
\end{enumerate}
The 2nd rule discriminates \hhf\ and \class\ FITS produced by {\tt
  HICLASS}.

\subsection{FITS level}

The command {\tt LAS$\backslash$FITS} traverses one by one all the
HDUs in the FITS file \emph{independently of their names}. If a binary
table is found and:
\begin{itemize}
\item if it contains a column {\tt frequency} or {\tt wave}, it is
  assumed to be a level 2.5 \hhf\ and the specific decoding is
  started.
\item if if contains at least one column {\tt lsbfrequency\_*} or {\tt
    usbfrequency\_*}, it is assumed be a level 2.0 \hhf\ and the
  specific decoding is started.
\end{itemize}
The levels 2.0 and 2.5 refer to the reduction pipeline level in
HCSS. The resulting FITS provides (almost) the same data but under a
different form. For our needs, the main differences come from values
which are found in columns of the binary table in level 2.0, and in
metacards (see below) of the header in level 2.5.\\

\subsection{Cards and metacards}
\label{txt:cards}

In this document, we refer to 2 concepts in the FITS context:
\begin{itemize}
\item header \emph{cards}. In the FITS standard, an header card has the
  form:
\begin{verbatim}
  CALVERS = 'HIFI_CAL_22_0'      / HIFI calibration version
\end{verbatim}
  with an 8-characters key, a value, and a comment. In this document,
  we abusively refer to \eg\ ``the card {\tt CALVERS}'' as ``the value
  in the card which key is {\tt CALVERS}''.

\item header \emph{metacards}. Not in the FITS standard, we refer as
  metacards when encoutering in the FITS header a pair of lines
  (contiguous or not, ordered or not) with the form:
\begin{verbatim}
META_10 =    62.39999999999999 / [s]
HIERARCH  key.META_10='integrationTime'
\end{verbatim}
  \ie\ a standard card and a {\tt HIERARCH} (free-form) line. We can
  see that these 2 lines provide an indirection through a {\tt
    META\_XX} key ({\tt XX} being a non-ambiguous number). In this
  case, we abusively refer to \eg\ ``the metacard {\tt
    integrationTime}'' as ``the value in the card which key ({\tt
    META\_10}) has the value {\tt integrationTime} in the associated
  {\tt HIERARCH} line''. Thanks to the metacards, case-sensitive
  keywords not limited to 8 characters can be used. There is also no
  need to refer to the exact {\tt META\_XX} name in the subsequent
  specifications, \ie\ {\tt XX} can vary from one file to another.
\end{itemize}

\section{Acknowledgements}

This work was partly founded by CNES. We thanks Claudia Comito and
Jonathan Braine for useful comments during this work.

% This document is not included in the GILDAS distribution because it
% weights more than 6 MB => we comment all this section here
% \appendix
% \includepdf[addtotoc={1,section,0,D.Teyssier test report,sec:report},pages={-}]{class-herschel-fits-validation.pdf}

\end{document}

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
