.. C-Munipack - User's manual

   Copyright 2012 David Motl

   Permission is granted to copy, distribute and/or modify this document under the
   terms of the GNU Free Documentation License, Version 1.2 or any later version published
   by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and
   no Back-Cover Texts.

   $Id: frame_merging.rst,v 1.2 2016/02/27 09:14:01 dmotl Exp $

.. index::
   pair: Frame; merging

.. _frame-merging:
   
Frame merging
=============

The frame merging makes one or more resulting CCD frames by combining a set of source
frames. This function is usable for example when the very faint objects are observed,
thus demanding very long exposure durations to obtain desired accuracy. The exposure
duration is limited in practice due to inaccuracy of the telescope driving. The merging
can virtually extend the attainable exposure time, because the image shift can be corrected.

The implementation used in the current version of the C-Munipack project requires
calibratted source frames and corresponding photometry files with information about frame
offsets. These are produced by the matching process, no special option is required.

Corresponding pixels are summed and the result is divided by the total number of source frames. 
Pixels, which are not covered by all source frames or which contain bad pixels, are set to zero. 
The exposure duration is computed as sum of the durations of source frames. The observation time 
is computed as arithmetic mean of times of source frames. The Muniwin program allows automatized 
or manual splitting of the set of input files and making a single or a set of result frames.

Before you start
----------------

Before you start the reduction of your own CCD frames, you may
need to perform several pre-processing steps. Though it isn't necessary,
combining the several correction frames into so-called "master" ones
is advisory, because it reduces the noise and makes the result more
precise. The method of making master correction frames is described in
separate chapters.
   

Creating a new project
----------------------

First of all, we will create a new :ref:`project <projects>`. To begin with processing of a variable star observation, 
we create a new project. To do so, open the :guilabel:`Project` menu and activate the :guilabel:`New` item. A new dialog 
appears.

.. figure:: images/newproject_merge.png
   :align: center
   :alt: "New project" dialog
   
   The dialog for making a new project.
   
Fill in a name that will be assigned to the project. Because the name of the file that keeps track of the data related
to the project, the project file, is derived from the name, some characters cannot appear in the project name, do not 
use: / \ ? % * : | " < and >.

The field :guilabel:`Location` shows a path to the directory where a new project will be created. Edit the path to 
change the location, you can also click the :guilabel:`Browse` button to select a directory in a separate dialog.

Select a profile (3) to specify an intial set of configuration parameters into a new project. When you confirm the dialog, 
you should be in the main window again now. The table of input files shown there is empty.

.. figure:: images/main.png
   :align: center
   :alt: Main application window
   
   The main application window with the table of input files, now empty.

.. seealso:: :ref:`new-project-dialog` and :ref:`main-window`

   
Input files
-----------

Now, we are going to tell the program which files we are going to work on. These files are called the input 
files. Their list is displayed in the table in the main application window. When the application is closed,
the list of files are saved to the disk and it is restored back when the program is launched again.

Supposing that the table now consists of files from your previous task, let's get rid of them. Please, use 
:menuselection:`Files --> Clear files` to start a new task instead of just removing the files from 
the table. Besides the clearing the table of input files, this function resets all internal variables, too.

Now, we need to populate the table with the CCD frames we're going to reduce. There are two methods how to 
achieve that - adding a individual files or adding all files from a folder. Which way is the best for you 
depends on organization of your observations on the disk. I'd suggest you to make a folder for each year, 
a folder for each night in it, the a subfolder for a name of object or another view field identification and 
finally a subfolder named upon the color filter (if you use more of them). In this case, the "Add frames from a folder" 
method is more convenient.

Click on :menuselection:`Files --> Add frames from folder` in the main menu. A new dialog appears. In the dialog, 
find a folder where the inputs files are stored in. Click on an entry in the :guilabel:`Places` pane to go to one 
of a preselected folders, double click in the middle pane enters the folder. The buttons in the upper part 
of the dialog shows your current position in the directory tree, you can use them to go to one of the parent
folders. Enter the folder with the input files - you should see them in the middle pane. Then, click on the 
:guilabel:`Add` button to add files to the table of input files. The program shows the number of added files in
the separate dialog. The :guilabel:`Add frames from folder` dialog is not closed automatically and allows a user 
to continue. Click on the :guilabel:`OK` button to close the dialog and return to the main window.

.. figure:: images/addfolder.png
   :align: center
   :alt: "Add folder" dialog
   
   The "Add folder" dialog with the place selection box (left), the file
   selection box (middle) and the preview panel (right).

If you want to reduce only a subset of files from a folder, click on :menuselection:`Files --> Add individual frames` 
in the main menu. A new dialog appears, similar to the previous one. In the dialog, find a folder where the inputs 
files are stored in. Click on an entry in the :guilabel:`Places` pane to go to one of a preselected folders, double
click in the middle pane enters the folder. The buttons in the upper part of the dialog shows your current position 
in the directory tree, you can use them to go to one of the parent folders. In the middle pane, select the files 
using the :kbd:`Ctrl` modifier to include and exclude a single file and the :kbd:`Shift` modifier to include a range 
of files. Then, click on the :guilabel:`Add` button to add selected files to the table of input files. The program 
shows the number of added files in the separate dialog. The :guilabel:`Add individual frames` dialog is not closed 
automatically and  allows a user to continue. Click on the :guilabel:`OK` button to close the dialog and return to 
the main window.

.. figure:: images/addfiles.png
   :align: center
   :alt: "Add files" dialog
   
   The "Add files" dialog with the place selection box (left), the file
   selection box (middle) and the preview panel (right).

.. seealso:: :ref:`add-frames-from-folder-dialog` and :ref:`add-individual-frames-dialog`

   
Frame reduction
---------------

.. tip:
   If the frames have been already aligned (transformed) and/or calibrated, you can skip the steps of calibration,
   photometry and matching and go directly to "Making combined frames" section below.
   

Reduction of CCD frames is a process that takes source CCD frames, performs their conversion and calibration,
detects stars on each frame and mearures their intensity and finally finds correlation (match) between objects 
that were found in the data set. The process of reduction prepares the data that are necessary for making a light 
curve or a variable star.

The reduction consists of several steps - conversion, calibration, photometry and matching. They can be invoked
step-by-step manually. The preferred way is to use the :guilabel:`Express reduction` dialog that allows to perform 
these steps in a batch. Using the menu, activate the :menuselection:`Reduce --> Express reduction` item.
A new dialog appears. The dialog has several options aligned to the left, Each of them relates to an optional 
step in the reduction process. 

.. figure:: images/express.png
   :alt: "Express reduction" dialog
   :align: center
   
   The dialog for setting parameters of the reduction process

.. rubric:: Fetch/convert files
      
Check the :guilabel:`Fetch/convert files`. In this step, the program makes copy of the source CCD frames. 
This is necessary, because the following calibration steps will modify them and we don't want the program 
to change our precious source data.

.. figure:: images/express_convert.png
   :alt: "Express reduction" dialog
   :align: center

.. rubric:: Dark-frame correction

A raw CCD frame consists of several components. By the calibration process, we get rid of those 
which affect the result of the photometry. In some literature, the calibration is depicted as the 
peeling of an onion. There are three major components which a raw frame consists of - the current 
made by incident light, current made thermal drift of electrons (so-called dark current) and 
constant bias level. In standard calibration scheme, which we will demonstrate here, the dark-frame 
correction subtracts the dark current and the also the bias. Because of the nature of the dark current, 
it is necessary to use a correction frame of the same exposure duration as source files and it must 
be carried out on the same CCD temperature, too. Thus, the properly working temperature regulation 
on your CCD camera is vital.

.. figure:: images/express_dark.png
   :alt: "Express reduction" dialog
   :align: center

.. rubric:: Flat-frame correction

Then, we have to compensate the spatial non-uniformity of a detector and whole optical
system. These non-uniformities are due to the fabrication process of a CCD chip and they are
also natural properties of all real optical components, lenses in particular. The flat-frame
correction uses a flat-frame to smooth them away. The flat-frame is a frame carried out 
while the telescope is pointed to uniformly luminous area. In practice, this condition is
very difficult to achieve, the clear sky before dusk is usually used instead.

.. figure:: images/express_flat.png
   :alt: "Express reduction" dialog
   :align: center
   
.. rubric:: Photometry

The photometry is a process that detects stars on a CCD frame and measures their brightness. Unlike the previous 
steps, the result is saved to a special file, so-called the photometry file. There are a lot of parameters which 
affect the star detection and also the brightness computation. In this example, the default values work fine, 
but I would suggest you to become familiar with at least two of them - FWHM and Threshold - before
you start a real work. Check the :guilabel:`Photometry` option.

.. figure:: images/express_photometry.png
   :alt: "Express reduction" dialog
   :align: center


FWHM
	The FWHM parameter specify the expected width of stars on a frame.
	The value is the Full Width at Half Maximum in pixels. The parameter
	controls the behavior of the low-pass digital filter, which is used
	in the star detection algorithm.

Threshold
	The Threshold parameter specify the lowest brightness of detected
	stars. Fainter objects are considered to be background artifacts and thus
	sorted out. The value is dimensionless coefficient.

Once you tune up the parameter for your environment, usually it is not
necessary to adjust them for every task, unless the quality of your images
varies considerably. In the first iteration, you can use the default values
(FWHM = 3.0 and Threshold = 4.0) and do the photometry. Click on the
:menuselection:`Reduce --> Photometry` item in the main menu and confirm the new dialog by the :guilabel:`OK`.
Then, by double click on a frame in the main window open the preview window
and check the results. If there are stars which have been detected as a close
binary although it is not true, you should increase the *FWHM*
value. If the stars you are interested in are not detected, try decrease the
*Threshold* value. If it doesn't help, decrease the
*FWHM*. If there is a lot of background artifacts detected
as a real stars, increase the *Threshold*. By several
iterations, adjust the parameters, so all the stars you are interested in are
detected and there are no false binaries.

.. rubric:: Matching
   
The previous command treated all source files independently. As a result of this, a star #1 
in one file is not necessarily the same as a star #1 in another file. The matching is a process 
which finds corresponding stars on source frames and assigns an unique identifier. Check the
:guilabel:`Matching` option.

It is necessary to select one frame from the set of source frames that all other frames are matched 
to, this frame is called a reference frame. In my experience, the frame with the greatest number of 
stars works the best. Back to our example, let's pick up the first one.

.. figure:: images/express_matching.png
   :alt: "Express reduction" dialog
   :align: center

.. rubric:: Invoking the reduction process

In previous steps, we have configured parameters of the reduction process and we are ready to start
it. Click the :guilabel:`OK` button. During the execution a new window appears displaying the state 
of the process; all the information is also presented there. This window will be automatically 
closed after finishing the process. Wait for the process to finish. 

.. figure:: images/progress.png
   :alt: Progress dialog
   :align: center
   
   The dialog displayed during time demanding operations.

After finishing, the icon in the file table changes; the information about the time of observation, 
the length of the exposition and the used filter is filled in. In case some of the frames could not be 
processed successfully, the entry is be marked with a special icon and in the :guilabel:`Status` column 
the error message is indicated.

.. figure:: images/inputfiles3.png
   :alt: Main application window
   :align: center
   
   The main application window after the recution.
   
.. seealso:: :ref:`express-reduction-dialog`

   
   
Making combined frames
----------------------

Open :guilabel:`Make` menu, select the :guilabel:`Merge frames` item.
A new dialog window appears. If you are going to make a set of resulting frames, select
the :guilabel:`Split frames and process each group separately` option. Set the parameters
for automatic splitting frames to groups; one resulting frame will be made for each particular
group. Fill the maximum number of frames in a group to the :guilabel:`Merge every ... frames`
field. Set the maximum time span between the first and the last frame in a group. Enter the
minimum number of frames (:guilabel:`Min. frames`). Confirm the dialog by
the :guilabel:`OK` button. The standard dialog for saving files is opened.
Find a directory, where the output files shall be stored to and enter the file name
prefix. For example, if you enter the crcas prefix, the resulting files will be named
:file:`crcas00001.fts`, :file:`crcas00002.fts`, etc. Close the dialog by the :guilabel:`Save` to start the merging.

It is possible to split the files manually. In the `Input files` table, select the
frames, which shall be combined to a single resulting frame. In the :guilabel:`Make`
menu, select the :guilabel:`Merge frames` item. The same dialog window as
in previous case is opened. In the upper part of the dialog, select the :guilabel:`Process
selected files only` option and in the lower part, select the :guilabel:`Merge
all source frames to a single output frame` item. Confirm the dialog by the
:guilabel:`OK` button. The standard dialog for saving files is opened. Find
a directory, where the output file shall be stored to and enter the its name. No numbers
are added to the name in this case. Close the dialog by the :guilabel:`Save`
to start the merging. Repeat the action subsequently for all input files.

If the frames have been aligned (transformed) already, you can run this step after the 
frame conversion. You have to check the option "Do not transform the source frames". If this option
is checked, the software does not attempt to do any alignment of the frames, so if they do 
not match, the objects will be deformed or copied on the combined frames.


.. seealso:: :ref:`merge-frames-dialog`


Processing merged frames
------------------------

Merged frames can be processed in the same way as common CCD frames with an exception
that no calibration (bias, dark and flat correction) is applied to them. Follow instructions
in the :ref:`light-curve` section.

.. seealso: :ref:`light-curve`
