\section{IMAGER in short}

\imager\ is an interferometric imaging package in the \gildas{} 
software, tailored for usage simplicity and efficiency for 
multi-spectral data sets.

The main goals of \imager{} are
\begin{enumerate}
\item to offer a proper implementation of imaging in case of
wide relative bandwidth, where the natural angular resolution
changes with frequency.
\item to implement a simple and efficient scheme to process Mosaics,
including short spacings from single dish data
\item to take advantage of improved capabilities of \NOEMA{} and 
\ALMA{}, by offering new tools like self-calibration or wide bandwidth 
analysis.
\item to simplify user interfaces, by providing sensible defaults.
\item to minimize image sizes 
\item to minimize processing time by using parallel programming as much 
as possible and reducing Input/Output to the strict minimum.
\end{enumerate}

\imager{} was developed and optimized to minimize Input/Output that
are the bottleneck of current computers. 
Therefore, \imager{}\  works mostly on internal buffers and avoids as 
much as possible saving data to intermediate files. File saving is done 
ultimately once the data analysis process is complete, which offers an 
optimum use of the disk bandwidth. 

\imager{} also includes advanced display and image analysis tools, such as
simple overlaying of different data cubes, etc...


\section{IMAGER for you...}

If you do not know how to do something, just use the \com{HOW\_TO}
command to express your question in a (semi-) natural way, e.g.

\texttt{how to combine alma and aca}\\

Use the \com{HELP} command for more details on commands.
Use the \com{TIP} command for useful suggestions.


\subsection{IMAGER for beginners}
If you have never done any interferometry before, or if you do not know 
what is a \uv{} table, we (\textbf{strongly}) suggest you read Sections \ref{sub:structure},
\ref{sec:uvtables} 
and \ref{sec:single} before doing anything at all. These sections are short 
enough to give you the basic tools (\com{HELP} plus the 5 main 
commands: \com{READ}, \com{UV\_MAP}, \com{CLEAN}, \com{VIEW} and 
\com{WRITE}), and will give you hints on why \imager{} behaves like 
this.

Remember two things: look at the results (command \com{VIEW}) at 
all steps, and ask more experienced users if these results make sense if
any doubt arise.

\subsection{IMAGER for previous users of \mbox{\mapping{}} }
\imager{} offers a number of advantages against \mapping{} : higher
speed, simpler and more consistent user interface, integrated 
pipeline processing, and a number of improved image analysis tools.
However, the principles have not changed. 
Most commands remain quasi identical in \imager{} and \mapping{},
with basically the same control parameters.


Actions which were driven by widgets have often been replaced by simple 
commands (with intuitive names). A quick look at the \com{HELP} will 
guide you to the commands that will do what widgets were doing before. 
So you can skip over the fundamentals, and focus on the new tools 
offered by \imager{}. In practice, for you, commands will make fairly 
reasonable choices of default parameters. Just do not forget to 
\com{WRITE} your data.

In short, the biggest changes for  \mapping{} users are\\
- the lack of use of the \com{GO} command, 
and thus of the \sicvar{NAME} and \sicvar{TYPE} variables\\
- the suppression of \sicvar{MAP\_RA} \sicvar{MAP\_DEC} \sicvar{MAP\_ANGLE} and \sicvar{UV\_SHIFT}
which have been replaced by \sicvar{MAP\_CENTER} \\
- the replacement of all tasks by simpler, standard commands.

Changing the defaults may be needed only for practical reasons:
\begin{itemize}\itemsep 0pt
\item Optimizing computation time. This includes adjusting
the field of view (variable \sicvar{MAP\_FIELD}), resampling
in velocity (command \com{UV\_RESAMPLE}) and time averaging
(command \com{UV\_TIME})
\item Comparing strictly different images (on a pixel per pixel
basis). Adjusting \sicvar{MAP\_SIZE} and \sicvar{MAP\_CELL} will
become necessary in such a case.
\end{itemize}

The names of variables and most commands have been kept from 
\mapping{}, old names appear in the \com{HELP} whenever they have been 
replaced. 

\subsubsection{Forget \com{INPUT} and  \com{GO} command}

However, the \com{GO} command is totally obsolete. In general
any \qom{GO SOMETHING} is simply replaced by the equivalent
command \qom{SOMETHING}: e.g. type \com{VIEW} instead of \qom{GO VIEW}.
\comm{GO}{PLOT}  (or its variants \comm{GO}{BIT}, \comm{GO}{NICE} and \comm{GO}{MAP}) and 
\comm{GO}{UVSHOW} are replaced by the \com{SHOW} command, which offers
many more possibilities.

The capabilities given by the \com{INPUT} \mapping{} command have in
general be replaced by the more flexible use of the question mark
(qom{?}, or \qom{??}, or even \qom{???}) as arguments to individual \imager{} commands.
Most commands display their control parameters with such arguments.

Take a little time to browse through \comm{HELP}{SHOW} and \comm{HELP}{VIEW}
to get familiar with \imager{} display capabilities.


\subsection{IMAGER for huge data sets: 100 000 channel imaging with \NOEMA{} or \ALMA{} }

For \NOEMA{}, with the advent of PolyFiX, the paradigm has changed: a spectral window
may contain a substantial number of interesting spectral lines, not
just one as with the previous narrow band spectral correlator. The same
can happen (although to a lesser extent) with \ALMA{}.

That is perhaps where \imager{} is most helpful. It contains new tools 
to help you automating this tedious part.  The primary tool is command 
\com{UV\_PREVIEW} which, provided one or more spectral line catalogs 
have been defined (see command \com{CATALOG}), allows to semi-automate 
the line identification among many UV tables. 

The script \qom{@ }\script{noema-scan} used after a \com{UV\_PREVIEW} on a 
wide band UV table will help you identify who is who among the 64 
spectral windows that PolyFix may provide. 

A widget (driven by script \script{imager\_init}) also allows simple 
image creation from
any UV table, including continuum subtraction, referencing to 
different frequencies and velocities, and
spectral resampling. Script \com{@}\script{imager-one} offers another 
alternative. Both can be useful examples for more advanced processing.

A much more advanced tool is the \com{PIPELINE} command 
that streamlines and even fully automates the imaging process of
a whole ensemble of UV tables. See Section \ref{sec:all} for details.

\subsection{IMAGER for lazy or overbooked ones}

If you have many spectral lines, and need results very quickly, 
the suite of \script{all-*.ima} scripts is made for you. It can provide
you one image for each spectral line specified in your line catalog
that fall into the bandwidth of your data.

The default will do a pretty good job on most data sets, but
can be customized for a better result. 
See Section \ref{sec:all} for details.


\subsection{IMAGER for ALMA data}

\imager{} implements in CASA the \texttt{casagildas()} tool that automates
the transfer of calibrated visibilities from a CASA Measurement Set
to UVFITS files that can be readily imaged with \imager{}.


\subsection{IMAGER for image analysis and publication plots}

\imager{} also includes general data-cube handling tools, such as
spectral or spatial resampling, re-projection, image combinations,
and several elaborate viewers that can be used to explore data cubes
and/or prepare publications quality plots. See Sections \ref{sec:visual}

The display commands (but not the data handling tools) are also 
separately available in the \viewer{} program.




\section{IMAGER principles}

\subsection{Overview of the data reduction and analysis}
\label{sub:overview}

Once the data has been acquired by an interferometer such as the 
NOrthern Extended Millimter Array  
(\NOEMA{}) or \ALMA{}, two different approaches may
be used for its reduction and analysis:
\begin{itemize}
\item The first possibility is to clearly separate 1) the calibration, 2)
  the imaging and deconvolution and 3) the analysis.
\item The second possibility is to merge in a single step calibration and
  imaging. This possibility is known as self-calibration.
\end{itemize}
While \casa{} uses the second paradigm, \gildas{} mainly implements 
the first approach, as the program and the data format used 
for each step is different. The calibration of \NOEMA{} data is
done inside \clic{} on the \NOEMA{} raw data format and the outcome is a
\uv{} table, which contains only calibrated visibilities of the
astronomical source.  The imaging and deconvolution is done inside
\imager{} on the calibrated \uv{} table and deliver mainly an \lmv{}
spectral cube (2 axes of coordinates and 1 axis of velocity/frequency).
\imager{} includes \greg{} for the  visualization, and provides additional
image analysis functionalities,  which are not specific to an interferometric use 
(\eg\ they can be used with the \IRAM{} 30\,m spectral cubes as well).

The choice of clearly separating calibration and imaging+deconvolution was
taken at start of the Plateau de Bure Interferometer, 
when the limiting number of antennas prevented 
the use of self-calibration. 
While many points of the calibration algorithms
inside \clic{} are specific to \NOEMA{} data (in particular its range of
Signal-to-Noise ratio), the algorithms of imaging+deconvolution can be used
in many different contexts and the visualization and analysis of spectra
cubes is mainly independent of the instrument that delivered the data. 
This last point implies that users can import data (mainly through FITS format)
in \imager{} for imaging and deconvolution, and in \greg{} for
visualization and analysis. But the reverse is also true. While calibration
of \NOEMA{} data should be done inside \clic{}, imaging+deconvolution and
visualization+analysis can be done in other softwares (\eg\ \miriad{},
\aips{}, \casa{} for the imaging and deconvolution
and \karma{} for the visualization and analysis).

With the improvement of \NOEMA{} (increase of the number of antennas 
and better receiver sensitivities) and with the advent of a new 
generation of interferometer (\ALMA{}), an additional step of 
self-calibration may improve the consistency of the final results by 
imposing additional consistent constraints on the calibration. This 
step is further presented in chapter 4.

\subsection{The structure of the \imager{} program}
\label{sub:structure}

\subsubsection{Structure and recommendations}

The \imager{} program supports
\begin{itemize}
\item The manipulation (\eg\ resampling), visualization and flagging of
  \uv{} tables;
\item The imaging of \uv{} tables in dirty maps and beams;
\item The deconvolution of dirty maps;
\item The inclusion of short-spacings; 
\item The visualization and analysis of spectra cubes;
\item The self-calibration.
%\item A simulator of ALMA continuum observations.
\end{itemize}

It consists in a collection of commands, either dedicated to image display
  (the \lang{DISPLAY} language), UV handling and imaging with deconvolution
  (the \lang{CLEAN} language), implementing basic functionalities (the
  \lang{SIC}, \lang{GREG}, or \lang{CALIBRATE}  families of languages),
  advanced methods and image analysis tools (\lang{ADVANCED} language)
  or even complex suites of operations (the \lang{BUNDLES} language).
  
A few additional widgets are grouped in the \imager{} main menu to
provide more integrated interface to the above possibilities, or
more elaborate control. As of today\footnote{01-Jan-2022}, some
of these widgets may still evolve. On the contrary, commands
are in general very stable, though minor syntax adjustments may occur
(usually to implement a more convenient syntax, but keeping as much as
possible backward compatibility as a major constraint). 


\subsubsection{Implementation issues}

The new implementation of \imager{} and in particular of the 
\com{UV\_MAP} and \com{CLEAN} commands uses most of the 
older code, but re-arranged such that ensembles of contiguous channels 
(``chunks'') are treated at once and share the same synthesized beam. 
Deconvolution with \com{CLEAN} then proceeds by using the synthesized 
beam with the appropriate frequency for each channel. The user can 
control the ``chunk'' size, and hence the precision of the process 
given the desired field of view.

As a result of the new concept, beams (whether primary or synthesized) 
can be 4-D arrays, as they may depend on Frequency and Field.

\subsection{Imaging/deconvolution: a brief sequence of commands}
For the user, \imager{} reduces the number of actions to the strict minimum. 
The imaging sequence is always the same: 
\begin{verbatim}
    1- Reading data
       READ UV MyData.uvt /RANGE Min Max Type
       ! here, optionally use UV_TIME, UV_COMPRESS, UV_BASELINE to average data
       ! or UV_FILTER, UV_CONT to filter lines or remove continuum
    2- Imaging
       UV_MAP         
    3- Deconvolving
       CLEAN           
       ! here, optionally use UV_RESTORE
    4- Looking at the result
       VIEW CLEAN      ! or SHOW CLEAN     
    5- Writing the result 
       WRITE * MyData  
\end{verbatim}

\begin{itemize}
\item \textbf{Step 1:} Reading  the  specified  internal  buffer (here 
UV) from the input file (\file{.uvt} file type), loading only the channels falling in 
the range defined by the variables Min and Max, of Type \qom{CHANNEL}, 
\qom{VELOCITY} or \qom{FREQUENCY}. \imager{} recognizes whether the UV 
table is for a single field or a mosaic. The only difference between 
the single field and mosaic cases is that \imager{} yields a Sky 
brightness image for Mosaics, while the computed sky brightness of a 
single field is not automatically corrected for the primary beam 
attenuation. Imaging for multiple fields will be presented in Section~\ref{sec:mosaic}.
Single Dish data can also be loaded  in the following way : 
\comm{READ}{SINGLE}~\file{File}.
\item \textbf{Step 2:} Computing a  dirty  map  and  beam from a UV data. 
\com{UV\_MAP} processes  single fields as well as Mosaics.
\item \textbf{Step 3:} Deconvolving the \sicvar{DIRTY} image map (a Single-field or Mosaic) 
using the dirty \sicvar{BEAM} with the current \sicvar{METHOD}. The default 
for the SIC variable \sicvar{METHOD} is \com{HOGBOM},  the other 
supported methods being \com{CLARK}, \com{MRC}, \com{MULTI} and 
\com{SDI}. See \qomm{CLEAN}{?} for the other SIC variables 
controlling the deconvolution process. The outputs are the \sicvar{CLEAN} and 
\sicvar{RESIDUAL} images, and the Clean Component Table \sicvar{CCT}, all being stored 
in dedicated SIC variables.
\item \textbf{Step 4:} Plotting the result in the specified internal 
buffer (\sicvar{CLEAN}). Optionaly, the user can restrict the plot to a 
subset of channels through the optional arguments First and Last. 
\comm{SHOW}{CLEAN} can also be used instead, and produces a different 
type of plot.
\item \textbf{Step 5:}  Writing all modified image-like buffers (not 
the UV  tables) under the common file name ''\file{MyData}''. In the case of the 
present example, the following files are produced: \file{MyData.lmv, 
MyData.lmv-clean, MyData.cct, MyData.beam}, which correspond to the 
buffers: \sicvar{DIRTY}, \sicvar{CLEAN}, \sicvar{CCT}, and \sicvar{BEAM}, respectively. 
\qomm{WRITE}{UV}~\file{MyData} would only write the internal buffer (\sicvar{UV}) in the 
file \file{MyData.uvt} (the default extension corresponds to the specified 
buffer data type).
\end{itemize}    

\subsection{Getting Help: the HELP command and the ? token}
\label{sub:help}

As with any \sic{} based program, simple call to \com{HELP} will 
display the various languages (e.g., \lang{SIC}, \lang{GREG}, 
\lang{CALIBRATE}, \lang{CLEAN}) accessible to the help documentation 
and list some commands with available documentation. Note that 
\qom{CLEAN} is a command and \lang{CLEAN} a language (i.e., a family of 
commands). The language of a given command is recalled in the help of 
each command. 

Example: the command \qom{APPLY} belongs to the language \lang{CALIBRATE}, 
it has one argument which can be \qom{AMPLI} or \qom{PHASE} exclusively, 
one optional argument \qom{gain}, and one option \qom{/FLAG}.
\begin{verbatim} 
IMAGER> help apply
        [CALIBRATE\]APPLY [AMPLI|PHASE [gain]] [/FLAG]
\end{verbatim} 
In the provided description, arguments within \texttt{[]} are optional. Upper case 
arguments are fixed keywords that can (in general) be abbreviated. A $|$ character separates
the allowed keyword values. Lower-case arguments are expected to be numbers, text, filenames, SIC variables or
expressions that can be evaluated.

A brief description of the imager program can be obtained through:
\begin{verbatim} 
IMAGER> help imager
USER\IMAGER = "@ welcome.ima"
 
      IMAGER is a interferometric imaging package, tailored for usage
  simplicity and efficiency for multi-spectral data sets.
 
     The basic concept of IMAGER is the use of a simple
         READ data  - ACTION(s) - [SHOW or VIEW] - WRITE
  sequence of commands, minimizing the data I/O as much as possible.
  Automatic guesses of appropriate default values for the ACTIONs
  parameters is implemented whenever possible.
 
Additional Help Available:
 Actions      BEAM_Handlin MAP_Handling UV_Handling  WebPage      
 HOW_TO       MAPPING      
\end{verbatim} 

In addition, finding documentation and help for the \imager\ commands 
can be done in three different ways:  
\begin{itemize}
\item a simple call to the \com{HELP} command provides a description of 
the command and its arguments and options
\item the command name followed by one or more questions marks will 
display some partial help on the command and its most useful 
parameters (''\qom{Command ?}''), its second level parameters for advanced 
users (''\qom{Command ??}''), all its parameters (''\qom{Command ???}''). 
%\item \com{INPUT Command} provides a list of default values for the most commonly used parameters.
\end{itemize}

Documentation on subtopics of a given command (e.g., Variables, 
Arguments, or Results) can be obtained though: \comm{HELP}{command subtopic}.
The list of available subtopics is found at the bottom of the documentation of each command: 
\begin{verbatim} 
IMAGER> help uv_map
[...] 
Additional Help Available:
 Mosaics      /FIELDS      /RANGE       /TRUNCATE    
 Variables:
              BEAM_STEP    MAP_CELL     MAP_CENTER   MAP_CONVOLUT MAP_FIELD
 MAP_POWER    MAP_PRECIS   MAP_ROBUST   MAP_ROUNDING MAP_SHIFT    MAP_SIZE
 MAP_TAPEREXP MAP_TRUNCATE MAP_UVTAPER  MAP_UVCELL   MAP_VERSION  
 Old_Names:
              convolution  map_angle    map_dec      map_ra       mcol
 uv_cell      uv_shift     uv_taper     taper_expo   wcol         weight_mode  
\end{verbatim} 
Subtopics that appear in UpperCase in the \texttt{Additional Help Available}
list are case insensitive. Subtopics that are in LowerCase of mixed case are
strictly case sensitive.

Example: the following command will list the control variables of the 
\com{UV\_MAP} function and describe the associated parameter(s): 
\begin{verbatim} 
IMAGER> help uv_map Variables
UV_MAP Variables
                [CLEAN\]UV_MAP ?
        Will list all MAP_* variables controlling the UV_MAP parameters.
 
    The  list  of  control  variables  is (by alphabetic order, with the old
    names used by Mapping on the right)
    New names       [   unit]       -- Description --    % Old Name
    BEAM_STEP       [       ]  Channels per dirty beam   % MAP__BEAM__STEP
    MAP_CELL        [ arcsec]  Image pixel size
    MAP_CENTER      [ string]  RA, Dec of map center, and Position Angle
    MAP_CONVOLUTION [       ]  Convolution function     % CONVOLUTION
    MAP_FIELD       [ arcsec]  Map field of view
    MAP_POWER       [       ]  Maximum exponent of 3 and 5 allowed in MAP_SIZE
    MAP_PRECIS      [       ]  Fraction of pixel tolerance on beam matching
    MAP_ROBUST      [       ]  Robustness factor        % UV_CELL[2]
    MAP_ROUNDING    [       ]  Precision of MAP_SIZE
    MAP_SIZE        [       ]  Number of pixels
    MAP_TAPEREXPO   [       ]  Taper exponent           % TAPER_EXPO
    MAP_TRUNCATE    [      %]  Mosaic truncation level
    MAP_UVCELL      [      m]  UV cell size             % UV_CELL[1]
    MAP_UVTAPER     [m,m,deg]  Gaussian taper           % UV_TAPER
    MAP_VERSION     [       ]  Code version (0 new, -1 old)
 
    NAME is no longer used, and WEIGHT_MODE is obsolete.
    
    MAP_RA          [  hours]  RA of map center
    MAP_DEC         [    deg]  Dec of map center
    MAP_ANGLE       [    deg]  Map position angle
    MAP_SHIFT       [Yes/No ]  Shift phase center
      are obsolescent, superseded by MAP_CENTER.  They are provided  
    only  for compatibility with older scripts.
\end{verbatim} 

A more detailed description (type, size) of a given variable can be obtained 
through \texttt{help command variable}, as in this example:
\begin{verbatim} 
IMAGER> help uv_map map_uvtaper
UV_MAP MAP_UVTAPER
 
      MAP_UVTAPER[3]  Real
 
    Parameters of the tapering function (Gaussian if MAP_TAPEREXPO = 2): ma-
    jor axis at 1/e level [m], minor axis at 1/e level [m], and position an-
    gle [deg].
\end{verbatim} 
\sicvar{MAP\_UVTAPER} is an array that requires 3 values of type Real.
\vspace{0.5cm}

The default values of the useful parameters are checked through 
\qomm{Command}{?}\footnote{For users familiar with \mapping{}, the
question mark replaces the capabilities of the  \qomm{INPUT}{Command} although the output format may be slightly 
different.}
\begin{verbatim} 
IMAGER> uv_map ?
 UV_MAP makes a dirty image and a dirty beam from the UV data
 
 Behaviour is controlled by a number of SIC Variables
 - BEAM_STEP and MAP_PRECIS control the dirty beam precision
 - MAP_CENTER controls shifting ang rotation
 - MAP_CELL[2], MAP_SIZE[2], MAP_FIELD[2] control the map sampling
 - MAP_UVTAPER[3], MAP_UVCELL and MAP_ROBUST
     control the beam shape and weighting scheme
 
---- Basic parameters 
  Map Size (pixels)                 MAP_SIZE        [ 0 0 ]
  Field of view (arcsec)            MAP_FIELD       [ 0 0 ]
  Pixel size (arcsec)               MAP_CELL        [ 0 0 ] 
  Map center                        MAP_CENTER      [  ]      
  Robust weighting parameter        MAP_ROBUST      [ 0 ]
  UV cell size (meter)              MAP_UVCELL      [ 0 ]   
  UV Taper (m,m,deg)                MAP_UVTAPER     [ 0 0 0 ] 
\end{verbatim} 

For commands that support the \qom{?} mechanism, \qomm{Command}{?}~\sicvar{ControlVariable} and, in most cases,
\com{Command}~\sicvar{ControlVariable}\qom{ ?} are shortcuts to \qomm{HELP}{Command}~\sicvar{ControlVariable}.

\subsection{IMAGER versus CASA: controlling the number of synthesized beams, \sicvar{BEAM\_STEP}}
\label{sec:sub:beamstep}

In aperture synthesis, the angular resolution scales with the frequency: in fact the whole Fourier plane scales with 
this frequency, so that the angular scaled defined by the baselines varies across the frequency coverage. Accordingly, 
when using  sufficiently wide bandwidths (and/or imaging sufficiently large areas), it becolmes important to account 
correctly for this effect.

The CASA and \imager{} approaches to this problem differ. CASA systematically produces one synthesized beam per
spectral channel, leading to a frequency variable spatial resolution.   In this approach, CASA can handle different
weighting for individual channels, optimizing signal to noise, but the interpretation of the data becomes more
difficult since there is no unique angular resolution after deconvolution.

The approach offered by \imager{} is different, and can be controlled by variable \sicvar{BEAM\_STEP}.
\begin{itemize}
  \item \sicvar{BEAM\_STEP} = 0 instructs \imager{} to produce a single synthesized beam for all channels. This is 
appropriate for narrow bands and spectral channels with very similar noise (ideally identical). Unavoidably, the
synthesized beam is only an approximation, exact only at the reference frequency.
  \item \sicvar{BEAM\_STEP} = N instructs \imager{} to ensure that N consecutive channels share the 
same synthesized beam. This can be used when channels have the same weights (same effective UV coverage) but
the overall bandwidth is large. 
  \item \sicvar{BEAM\_STEP} = -1 instructs \imager{} to derive the numnber N of consecutive channels
  that can shared the same beam to within a certain precision in the imaged field of view. This precision
  is given by \sicvar{MAP\_PRECIS}. The derived N will depend of this number, on the relative bandwidth and on
  the size of the synthesized field (hence \sicvar{MAP\_FIELD} or \sicvar{MAP\_CELL} $\times$ \sicvar{MAP\_SIZE}).
  \item \sicvar{BEAM\_STEP} = 1 mimicks to some extent the behaviour of CASA. 
\end{itemize}

However, contrary to CASA,  \imager{} can use a common clean beam in all cases. This is achieved through the use
of the so-called \textit{\bf JvM} factor, which approximately scales the undeconvolved residuals to match the flux
scale of the Clean components. This behaviour is only obtained through the \com{UV\_RESTORE} command.

\sicvar{BEAM\_STEP} is a key control parameter for \imager{}.  The \imager{} pipelines (see Section\ref{sec:all}
use sensible default for each telescope, but users should be aware of this possible difference when comparing
with CASA results.
