/**
\if LALPULSAR

\defgroup lalapps_pulsar_SFTSpecification SFT Data Format Version 2 Specification (T040164-01)
\ingroup lalapps_pulsar
\authors The Continuous Waves Search Group

# Changelog

<table>
  <tr><td>June 27, 2005</td><td>SFT data should be finite() [not IEEE +/- Inf or NAN].</td></tr>
  <tr><td>November 18, 2004</td><td>All times in SFT filenames should be in seconds; simplify SFTtype description.</td></tr>
  <tr><td>September 22, 2004</td><td>Added new section: SFT File Naming Convention</td></tr>
  <tr><td>September 17, 2004</td><td>After discussion with the pulsar search group, added detector type to SFT header.</td></tr>
  <tr><td>September 14, 2004</td><td>After discussions with the pulsar search group, updated spec to require that all SFT blocks share some common header info.</td></tr>
  <tr><td>September 9, 2004</td><td>Release 2.3 of reference library.  Spec updated as per comments in SCCB_2004_09_09</td></tr>
  <tr><td>August 17, 2004</td><td>First public release of version 2.2 of reference library.</td></tr>
</table>

# SFT Data Format Version 2 Specification

An SFT is stored in a file. See below for file name convensions. The
file is composed of concurrent SFT BLOCKS. Each SFT BLOCK is organized as follows,
in \ref SFTRL_summary_struct "this table".

\anchor SFTRL_summary_struct
<table>
  <caption>Summary of SFT BLOCK structure.</caption>
  <tr><td>HEADER</td><td>48 bytes</td></tr>
  <tr><td>ASCII COMMENT</td><td>8*n bytes, where n is a non-negative integer</td></tr>
  <tr><td>DATA</td><td>8*N bytes, where N is a positive integer</td></tr>
</table>

The total length of the SFT BLOCK is 48+8*n+8*N bytes.  The SFT BLOCK
may be written in either big-endian or little-endian ordering.  All
floats and doubles follow the IEEE-754 floating-point conventions.

The HEADER contains 48 bytes as follows, in \ref SFTRL_header "this table".

\anchor SFTRL_header
<table>
  <caption>Summary of HEADER structure.</caption>
  <tr><td>8 bytes</td><td>REAL8</td><td>version</td></tr>
  <tr><td>4 bytes</td><td>INT4</td><td>gps_sec;</td></tr>
  <tr><td>4 bytes</td><td>INT4</td><td>gps_nsec;</td></tr>
  <tr><td>8 bytes</td><td>REAL8</td><td>tbase;</td></tr>
  <tr><td>4 bytes</td><td>INT4</td><td>first_frequency_index;</td></tr>
  <tr><td>4 bytes</td><td>INT4</td><td>nsamples;</td></tr>
  <tr><td>8 bytes</td><td>UNSIGNED INT8</td><td>crc64;</td></tr>
  <tr><td>2 bytes</td><td>CHAR</td><td>detector[2];</td></tr>
  <tr><td>2 bytes</td><td>CHAR</td><td>padding[2];</td></tr>
  <tr><td>4 bytes</td><td>INT4</td><td>comment_length;</td></tr>
</table>

The SFT blocks in a given SFT file are required to:

<ol>
  <li> come from the same instrument, and have:</li>
  <li> identical version numbers</li>
  <li> monotonically increasing GPS times</li>
  <li> identical values of tbase</li>
  <li> identical values of first_frequency_index</li>
  <li> identical values of nsamples</li>
</ol>

NOTE: SFT blocks in a given SFT file are in general NOT contiguous.
In other words the GPS start time of a given block may or may not
equal the GPS start time of the previous block plus the time baseline.

Note that the HEADER corresponds to the C structure below
\code
struct SFTtag {
  REAL8	version;
  INT4	gps_sec;
  INT4	gps_nsec;
  REAL8 tbase;
  INT4	first_frequency_index;
  INT4	nsamples;
  UINT8 crc64;
  CHAR	detector[2];
  CHAR	padding[2];
  INT4	comment_length;
} SFTheader;
\endcode
when the structure is packed, i.e. no zero padding between fields is
allowed.  Note that several of these quantities that might be taken as
unsigned are in fact signed.  This makes it easier and less
error-prone for user applications and code to compute differences
between these quantities.

The structure of the ASCII COMMENT is comment_length==8*n arbitrary
ASCII bytes, where n is a non-negative integer.  The following rules
apply to NULL bytes appearing in ASCII COMMENT, if n is non-zero:
<ol>
  <li> There must be at least one NULL byte in the ASCII COMMENT</li>
  <li> If a NULL byte appears in the ASCII COMMENT, all the following
    bytes in the ASCII COMMENT must also be NULL bytes.</li>
</ol>
The reason for these two rules is so that if the ASCII comment has
nonzero length then it may always be treated as a C null-terminated
string, with no information 'hidden' after the null byte.
If the SFT comes from interferometer data, then the full channel name
used will be contained in the comment block.  If a window function is
used (see below), then the window name (along with parameters if the
name is not sufficient) of the window function will also be contained
in the comment block.

The DATA region consists of N COMPLEX8 quantities.  Each COMPLEX8 is
made of a 4-byte IEEE-754 float real part, followed by a 4-byte
IEEE-754 float imaginary part.  The packing and normalization of this
data comply with the LSC specifications for frequency-domain data.
The current version of this specification may be found
[here](http://www.ligo.caltech.edu/docs/T/T010095-00.pdf).

<ul>
  <li> version: shall be 2.0:
    Note that SFTs produced before this specification will have this
    field set to 1.0.  Note that future versions of this specification
    will have version=3.0, 4.0, etc.  This field will always be an
    integer value that can be exactly represented as an IEEE754
    double.  If this field is not an exact integer in the range 1 to
    1000000, then software reading this data should assume that it is
    byte-swapped and take appropriate measures to reverse the byte
    ordering.  If byte swapping does NOT cause the version number
    to be an exact integer between 1 and 1000000, then the SFT does
    not comply with these specifications.</li>
  <li> gps_sec:
    Integer part of the GPS time in seconds of the first sample used to make
    this SFT.</li>
  <li> gps_nsec:
    GPS nanoseconds of the first sample used to make this SFT.  This
    must lie in the range from 0 to \f$10^{9}-1\f$ inclusive.</li>
  <li> tbase:
    The time length in seconds of the data set used to make this SFT.
    This must be greater than zero.
    \c Note: if the sample interval is dt, and the number of
    time-domain samples is S, then tbase=S dt.  Note that if the data
    is produced with heterodyning, tbase still refers to the total
    time length of the data set.  Note that the frequency spacing in
    between samples (df, as defined in T010095-00) is 1.0/tbase.  If df
    can not be exactly represented as an IEEE-754 double, then the closest
    IEEE-754 double to 1.0/tbase will be the closest IEEE-754 double to df.</li>
  <li> first_frequency_index:
    This is the subscript of the first complex FFT
    value that appears
    in DATA.  It's allowed range is 0 to (Nyquist_Frequency *
    tbase)/2 = S/2 inclusive.  Note: if S is odd, then in this
    document S/2 shall mean the integer part of S/2.</li>
  <li> nsamples:
    The number of complex samples in DATA.  nsamples=N.  Its
    allowed range is 1 to S/2+1 inclusive</li>
  <li> crc64:
    The 64-bit CRC checksum of the 48+8*n+8*N bytes that make up
    the SFT, with the 8 bytes labeled crc64 set to zero.  The
    CRC checksum will be evaluated using the
    polynomial D800000000000000 (base-16) =
    1101100000000000000000000000000000000000000000000000000000000000 (base-2).
    The primitive polynomial is \f$x^{64} + x^4 + x^3 + x + 1\f$.
    The CRC will be initialized to all ones (\f$\sim 0\f$ULL).</li>
  <li> detector:
    two characters of the form 'Xn' characterizing the detector,
    following the naming convention for the channel-name prefix
    in the Frame-format, cf. LIGO-T970130-F-E.
    X is a single capital letter describing the site.
    n is the detector number.
    Currently allowed names are:

    <table>
      <tr><td>"A1",</td><td>ALLEGRO</td></tr>
      <tr><td>"B1",</td><td>NIOBE</td></tr>
      <tr><td>"E1",</td><td>ET</td></tr>
      <tr><td>"G1",</td><td>GEO_600</td></tr>
      <tr><td>"H1",</td><td>LHO_4k</td></tr>
      <tr><td>"H2",</td><td>LHO_2k</td></tr>
      <tr><td>"K1",</td><td>KAGRA</td></tr>
      <tr><td>"L1",</td><td>LLO_4k</td></tr>
      <tr><td>"N1",</td><td>Nautilus</td></tr>
      <tr><td>"O1",</td><td>AURIGA</td></tr>
      <tr><td>"P1",</td><td>CIT_40</td></tr>
      <tr><td>"T1",</td><td>TAMA_300</td></tr>
      <tr><td>"V1",</td><td>Virgo (3km)</td></tr>
      <tr><td>"V2",</td><td>Virgo_CITF</td></tr>
    </table>

  <li> padding:
    These two bytes will be set to zero.  They are here so that all multi-byte
    quantities are byte-aligned with respect to the header.  This may permit
    certain efficiencies and library usages on certain platforms/architectures.</li>
  <li> comment_length:
    The number of bytes that appear in ASCII COMMENT.
    comment_length=8*n with n a non-negative integer.. Note
    that if comment_length==0 then the SFT contains no comment.</li>
</ul>

The complex quantities contained in DATA REGION are defined by the
following equations, complying with
[the specification](http://www.ligo.caltech.edu/docs/T/T010095-00.pdf).

The data set (with the native fundamental sample interval dt) is
denoted by x\f$_i\f$ for i=0, ..., S-1.  The x\f$_i\f$ are all real.  Let
\f[
\mathrm{n}_k = \sum_{j=0}^{\mathrm{S}-1} \mathrm{x}_j \exp(-2 \pi i j k/\mathrm{S})
\f]
be the values of the DFT with LSC sign conventions.  The values in
DATA REGION are
\f[
\mathrm{data}_k = \mathrm{dt} * \mathrm{n}_k
\f]
for k\f$ = \f$first_frequency_index to k\f$ = \f$first_frequency_index+nsamples-1.
[Note: the interesting range of k is from 0 to S/2 inclusive.]

The allowed range of first_frequency_index is from 0 to S/2 inclusive.

The allowed range of nsamples=N is 1 to S/2+1-first_frequency_index
inclusive.

[Here we assume that the window function is rectangular, eg each
sample is weighted by a window function whose value is 1.  If the data
IS windowed then the normalization conventions of T010095-00.pdf still
apply.]

Note that if a data stream is band-limited (for instance by filtering)
and then decimated or down-sampled, the values stored in DATA REGION
for a given set of frequency bins will be unchanged compared to those
computed with the original data set.  This is true even though the
sample interval dt' of the new downsampled data set is larger than the
original native sample time.  In fact, except for the DC (k=0) and
Nyquist (k=S/2+1) frequency bins, the power spectral density may be
written as:
\f[
\mathrm{psd}_k = (2/\mathrm{tbase}) |\mathrm{data}_k|^2
\f]
(except for k=0 or Nyquist).

# Examples
## EXAMPLE 1

Consider the case where the fundamental time-domain data set consists
of 16 samples, taken at a sample rate of 16 Hz.  All 16 samples are
\f$x_0 = \dots = x_{15} = 1\f$ which gives \f$n_0=16, n_1 = \dots = n_8 = 0\f$.  Since dt=1/16, we find
\f{align*}{
\mathrm{data}_{0} &= 1 + 0i \\
\mathrm{data}_{1} &= 0 \\
\mathrm{data}_{2} &= 0 \\
\mathrm{data}_{3} &= 0 \\
\mathrm{data}_{4} &= 0 \\
\mathrm{data}_{5} &= 0 \\
\mathrm{data}_{6} &= 0 \\
\mathrm{data}_{7} &= 0 \\
\mathrm{data}_{8} &= 0
\f}
If we store only nsamples=5 frequency bins in the SFT, then DATA
REGION will contain the 40 bytes corresponding to identical values for
\f$\mathrm{data}_k\f$:
\f{align*}{
\mathrm{data}_{0} &= 1 + 0i \\
\mathrm{data}_{1} &= 0 \\
\mathrm{data}_{2} &= 0 \\
\mathrm{data}_{3} &= 0 \\
\mathrm{data}_{4} &= 0
\f}
These values could be obtained by considering a subset of the original
SFT.  Alternatively they could be obtained by low-pass filtering the
original time series, and downsampling it, and using the previous
definitions.  For example if the downsampled time series had 8 samples
\f$x_0 = \dots = x_7 = 1\f$ with a sample time of dt=1/8, then \f$n_0 = 8\f$, and
\f$n_1 = \dots = n_4 = 0\f$.  This gives the same values as above.

## EXAMPLE 2

Consider a sinusoid function at 2 Hz, \f$x(t) = 1 *cos (2 \pi 2 t)\f$. Using again 16
samples taken at a sample rate of 16 Hz,
\f{align*}{
x_{00} &=  1.000000 \\
x_{01} &=  0.707107 \\
x_{02} &=  0.000000 \\
x_{03} &= -0.707107 \\
x_{04} &= -1.000000 \\
x_{05} &= -0.707107 \\
x_{06} &= -0.000000 \\
x_{07} &=  0.707107 \\
x_{08} &=  1.000000 \\
x_{09} &=  0.707107 \\
x_{10} &=  0.000000 \\
x_{11} &= -0.707107 \\
x_{12} &= -1.000000 \\
x_{13} &= -0.707107 \\
x_{14} &= -0.000000 \\
x_{15} &=  0.707107
\f}
giving
\f{align*}{
n_0 = n_1 &= 0 + 0i \\
n_2 &= 8 + 0i \\
n_3 = \dots = n_8 &= 0 + 0i
\f}
and
\f{align*}{
\mathrm{data}_0 = \mathrm{data}_1 &= 0 + 0i \\
\mathrm{data}_2 &= 0.5 + 0i \\
\mathrm{data}_3 = \dots = \mathrm{data}_8 &= 0 + 0i
\f}
If we down-sample the original data stream by a factor of two we get:
\f{align*}{
x_{00} &=  1.000000 \\
x_{01} &=  0.000000 \\
x_{02} &= -1.000000 \\
x_{03} &= -0.000000 \\
x_{04} &=  1.000000 \\
x_{05} &=  0.000000 \\
x_{06} &= -1.000000 \\
x_{07} &= -0.000000
\f}
giving
\f{align*}{
n_0 = n_1 &= 0 + 0i \\
n_2 *= 4 + 0i \\
n_3 = \dots = n_4 &= 0 + 0i
\f}
and
\f{align*}{
\mathrm{data}_0 = \mathrm{data}_1 &= 0 + 0i \\
\mathrm{data}_2 &= 0.5 + 0i \\
\mathrm{data}_3 = \dots = \mathrm{data}_4 & = 0 + 0i
\f}

# SFT File Naming Convention

<ol>
  <li> SFT file names are to follow the conventions of LIGO technical
    document LIGO-T010150-00-E, "Naming Convention for Frame Files which are
    to be Processed by LDAS," for class 2 frames, except with an extension
    of <tt>.sft</tt> rather than <tt>.gwf</tt>:

    > <tt>S-D-G-T.sft</tt>,

    where
    <ul>
      <li> \c S is the source of the data = an uppercase single letter designation of
        the site, e.g., G (GEO), H (Hanford), L (Livingston), T (TAMA), or V
        (Virgo).</li>

      <li> \c D is a description = any string consisting of alphanumeric characters
        plus underscore (_), plus (+), and number (\#). In particular, the
        characters dot (.), dash (-), and space are prohibited, as is any
        description consisting of a single uppercase letter, which is reserved
        for use by class 1 raw frames.</li>

      <li> \c G is the GPS time at the beginning of the first SFT in the file (an
        integer number of seconds). This is either a 9-digit or 10-digit number.
        (If the beginning of the data is not aligned with an exact GPS second,
        then the filename should contain the exact GPS second just before the
        beginning of the data.)</li>

      <li> \c T is the total time interval covered by the file, in seconds.  If only 1
        SFT is in the file, then T is tbase in the header.  For multiple SFTs,
        if the data is aligned with exact GPS seconds, then T is simply the
        number of seconds between the beginning of the first SFT and the end of
        the last SFT. If the data is not aligned with exact GPS seconds, then T
        should be calculated from the exact GPS second just before the start of
        the first SFT to the exact GPS second just after the end of the last
        SFT. Data gaps (i.e., non-contiguous SFTs within a file) are permitted,
        though the SFTs in the file must be time ordered.</li>
    </ul>

  <li> Note that even though SFTs do exist outside the LDAS diskcache,
    adopting the class 2 frame naming convention (except for the extension)
    ensures that all SFTs can be indexed by LDAS and other tools adopting
    the LDAS conventions. Note that LDAS v1.2.0 allows a list of extensions
    as a diskcachAPI resource variable. Thus LDAS v1.2.0 and higher can
    automatically index SFTs with names as specified here.</li>

  <li> SFT file names will follow these additional rules for the description
    field D:

    <ol>
      <li> The description field, D, for SFTs will be an underscore "_"
        delimited alphanumeric string with these subfields:

        <tt>D = numSFTs_IFO_SFTtype[_Misc]</tt>, where
        <ul>
          <li> \c numSFTs is the number of SFTs in the file.</li>

          <li> \c IFO is a two character abbreviation of the interferometer data used to
            generated the SFT, e.g., G1, H1, H2, L1, T1, or V1.  This field must
            always begin with an uppercase letter.</li>

          <li> \c SFTtype is the type of SFT(s) in the file, and must be a concatenation
            of tbase in seconds and SFT, e.g., 1800SFT for 30 minute SFTs, 60SFT for 60
            second SFTs.  Note that this information is redundant but required since
            it ensures that SFTs of a given type are indexed uniquely (many scripts
            and programs use the D field as an index; see also comment b below).</li>

          <li> \c Misc is an optional field that contains any other pertinent information
            about the SFTs, e.g., the run name, the input channel name, the
            calibration version, the data quality version, the frequency band,
            etc.... This field can be anything the group or individual that
            generates SFTs wants to include in the name, and should be used to
            distinguish a set of SFTs as unique from other sets for the same site,
            IFO, SFTtype, and GPS times. Possible examples of the Misc subfield are:
            <ol>
              <li> S1: The SFT is from the S1 Science run.</li>
              <li> S2v1Cal: The SFT is from S2 data using v1 calibration.</li>
              <li>  S3ASQv3Calv5DQ: The SFTs was generated from the LSC-AS_Q channel
                using v3 of the calibration and v5 of the data quality segments.  (Note
                that LIGO channel names contain prohibited characters; thus if channels
                names are included in the Misc subfield a nickname or abbreviation must
                be used.)</li>
              <li> S4hot: The SFT is from the S4 run using calibrated h(t).</li>
            </ol>
        </ul>
      </li>
    </ol>

  <li> Even though some of the information required in the description
    field, D, is redundant, many scripts and programs (such as LSCdataFind
    and the LDAS diskcacheAPI) rely on this field, plus the GPS interval and
    source letter, to find files. The required subfields of D will ensure
    that SFTs files are uniquely identified by these scripts and programs.</li>

  <li> Example SFT files names
    <ul>
      <li> A 30 minute H2 SFT:
        H-1_H2_1800SFT-735627918-1800.sft</li>
      <li> A 30 minute S2 H1 SFT:
        H-1_H1_1800SFT_S2-733467931-1800.sft</li>
      <li> A file with 1887 30 minute H1 SFTs for the 352 to 353 Hz frequency band, (with gaps in time):
        H-1887_H1_1800SFT_352to353Hzband-733467931-4622400.sft</li>
      <li> A 30 minute S3 GEO SFTs produced from h(t):
        G-1_G1_1800SFT_S3hot-732465218-1800.sft</li>
      <li> A 60 second S2 L1 SFT from the L1:LSC-AS_Q channel for v2 of the calibration and version
        6 of the data quality segments:
        L-1_L1_60SFT_S2ASQv2Calv6DQ-788901256-60.sft</li>
  </ul></li>

  <li> Thus note that Misc is left as an arbitrary subfield of the D field
    in this specification to allow flexibility.  It is up to the groups
    using SFTs to agree on the information to include in this subfield for
    the generation of SFTs for each run.</li>
  </li>
</ol>

\endif
*/
