Multi-Frame and All-Sky Reduction

If we want to determine star colors, calculate transformation coefficients to transform data to a standard system or obtain magnitudes of stars for which we don't have standards in the same field, we must reduce multiple observation frames together.

People fortunate enough to observe in photometric conditions can use a number of packages to reduce their data. For low altitude dwellers, the selection is not that large. For them, GCX implements multiple-frame reduction routines that are designed to work in less than perfect conditions.

Input data to the multi-frame reduction consists of observation reports as produced by the aperture photometry routine. For color coefficient fitting and transformation to a standard system, we need frames of the target objects taken in enough bands. For all-sky reductions, the observation reports need to have accurate time and airmass information (which implies that the original frames need to have enough information for the airmass determination).

Color Transformation Coefficients

To keep notation simple, let's assume that we reduce data taken in B and V. We'll use $B$ and $V$ for the standard magnitudes, and $b$ and $v$ for the instrumental magnitudes. The expressions for the standard magnitudes are:^8.1

$$B_j = b_j + Z_k + k_B(B_j-V_j)$$ (8.1)

$$V_j = v_j + Z_k + k_V(B_j-V_j)$$ (8.2)

The $j$ subscripts go over all individual star observations in a given band, while the $k$ subscripts iterate over the observation frames. For standard stars, the $B_j$ and $V_j$ are known while $b_j$ and $v_j$ are measured from the frames themselves. We have to fit the zeropoints $Z_k$ and the transformation coefficients $k$. Because we have chosen to express the transformation coefficients in function of the standard magnitudes, we can proceed with one band at a time. We'll assume it is V. The steps are:

1. Set an starting value of the transformation coefficient. We can start with 0 without problems, as the coefficients are generally small numbers.
2. For each V frame, fit the zeropoint using the robust algorithm in Appendix B, with the difference that the current color transformation is applied when calculating the residuals, so Equation B.5 becomes:
   

   $$\rho_j=y_j - \widetilde{Z_k} - k_V (B_j - V_j)$$ (8.3)

   
   
   Note that in this equation, the $j$ subscripts iterate over the stars in frame $k$.
3. Now, for all the stars in all the V frames, estimate the ``tilt'' of the residuals, and adjust the transformation coefficient and zeropoints accordingly. We use the weights from the individual frames' fits when estimating the tilt:
   

   $$\theta = \frac{\sum_j\rho_j(B_j-V_j) W'_j}{\sum_j(B_j-V_j)^2 W'_j}$$ (8.4)

   
   
   
   

   $$k_V = k_V + \theta$$ (8.5)

   
   
   
   

   $$Z_k = Z_k + \theta\frac{\sum_j(B_j-V_j) W'_j}{\sum_jW'_j}$$ (8.6)

   
   

4. Iterate the last two steps until the solution converges (the transformation coefficient doesn't change significantly).

We have now obtained the transformation coefficient for the V magnitudes, and also adjusted all the frame zeropoints so that their dependence on the color of the standard stars in each frame is eliminated.^8.2The above is repeated for each band we want to reduce. Note that we have obtained the transformation coefficients without assuming any relation between the zero points of various frames--just differential photometry.

We can choose any color index for a given band. For instance, there is nothing stopping us from calculating a $(B-V)$ transformation coefficient for $I$ or $R$ magnitudes. In fact, if we have more standards data in B and V, it may prove better to do so. In general, Equation (8.1) can be written for any band $M$ as:

$$M_j = m_j + Z_k + k_M(C_1^M - C_2^M)\\$$ (8.7)

where $C_1^M$ and $C_2^M$ are any bands for which we have standards data. We can fit the transformation coefficient $k_M$ using only the $M$ observations. However, when we want to transform the stars, we will need observations in $M$, $C_1^M$, $C_2^M$ and all the bands that these depend on.

Transforming the Stars

To calculate the transformed standard magnitudes of our target stars, all we have to do is to write Equation (8.7) for each band, and solve the resulting system of linear equations for the standard magnitudes. The system is very well behaved (it's matrix is close to unity) so GCX uses the simple Gauss-Jordan elimination method to solve it.

All-Sky Reduction

When the field of our intended target doesn't contain any suitable standard stars, we have to determine their magnitudes by comparing to stars in a different field. To do this, we need to determine a relation between the zeropoints of different frames.

Under photometric conditions, we can consider that the atmospheric extinction^8.3depends only on the thickness of the atmosphere along the light path. The ratio between the thickness of the atmosphere in the direction of field and it's thickness towards zenith is called the airmass of the field. The airmass depends on the zenital angle z of the field, and is close to $\sec(z)$ when far from the horison. The formula used by GCX is the following:^8.4

$$s = \sec(z) - 1;$$ (8.8)

$$A=1 + s[0.9981833 - s(0.002875+0.0008083 s)]$$ (8.9)

The zenital angle of a frame can be determined given it's equatorial coordinates, the geographical coordinates of the observing site, and time.

If the extinction is unform in all directions, we can define an extinction coefficient $E$, so that for any frame:

$$Z(A) = Z_0 - EA$$ (8.10)

where $Z_0$ is the zeropoint outside the atmosphere, and A is the frame's airmass. If we have two frames with airmasses $A_1$ and $A_2$, their zeropoints can be related by:

$$Z_2 = Z_1 - E(A_2-A_1)$$ (8.11)

Under photometric conditions, it is customary to determine the extinction coefficient by observing the same field at different airmasses and then fitting E from (8.11). This is only possible when $E$ doesn't change (or changes in a smooth, linear fashion) over a period of the order of hours.

Because the GCX all-sky routine is targeted at less-than-perfect conditions, we will choose another strategy in determining the extinction coefficient. We use several standard fields located relatively near our target fields. Then we we try to ``chop'' the extinction coefficient as much as possible by alternating between the standard and target fields.^8.5

We end up with a series of observations from different fields, all in the same general airmass range. By examining the standard fields' zeropoints variation with time and airmass, we can determine if there were any ``windows'' during which the extinction was stable.

Once a stable window was found, we can fit the extinction coefficient from the observations in that window. It is unlikely that the observations will span a wide range of airmasses, which will make the fitted value of the extinction coefficient somewhat imprecise. But this is offset by the fact that the airmass of the target frames is the same range, so the contribution of the extinction term is not very large. As long as the airmass of the standard fields brackets those of the target fields, we are interpolating rather than extrapolating the extinction.^8.6

Extinction Coefficient Fitting

Before attempting to fit the extinction coefficient, the zeropoints and color transformation coefficient of all frames must be fitted. It is highly recommended to examine plots of the resulting zeropoints versus time and airmass to see if it's worth trying to do any all-sky reduction at all (more on this below).

With these precautions, the program will proceed to fit the extinction coefficients using a variant of the algorithm described in Appendix B,^8.7with the initial weights assigned based on the calculated errors of the zeropoints. The fitted model is:^8.8

$$Z = \bar{Z} + E (A - \bar{A})$$ (8.12)

where $\bar{A}$ is the mean airmass of the standard frames, while $\bar{Z}$ is the zeropoint of a mean airmass frame.

A different extinction coefficient is fitted for each band. Frames that are outliers of the fit (their standard error exceeds the threshold set in Multi-Frame Photometry Options/Zeropoint outlier threshold) are marked as such.

Calculating Zero Points

After fitting the extinction coefficient, we can apply Equation 8.12 and calculate the zeropoint of any target frame. The program tries to filter the frames for which such a determination would likely be in error. It will only calculate a zeropoint for frames which satisfy the following:

1. The frame has to be both preceded and succeded in time by non-outlier standard frames;
2. The frame's airmass has to be in the same range as the standard frames from which the extinction coefficient was fitted.

   If $A_m$ is the minimum and $A_M$ is the maximum standard airmass, and $r$ is the value of Multi-Frame Photometry Options/Airmass range the zeropoint is only calculated for frames with airmasses between
   

   $$\frac{A_M + A_m}{2} - \frac{r}{2}(A_M - A_m)$$ (8.13)

   
   
   and
   

   $$\frac{A_M + A_m}{2} + \frac{r}{2}(A_M + A_m)$$ (8.14)

   
   

Running Multi-Frame Reduction

This section is a step-by-step tour of the multi-frame reduction tool. An realistically-sized example input file is provided in the distribution data directory (cygs-aug19.out). This file was generated by GCX aperture photometry from 143 frames taken in B, V, R and I in a single night, all in Cygnus. The standards data is from Henden sequence files, which were converted into GCX recipies with the import function. The file consists of individual aperture photometry reports appended together.

Specifying Reduction Bands

Before we can reduce data, we have to define which color indices are used for each band. The Multi-Frame Photometry Options/Bands setup option specifies this. It contains a list of specifiers of the form: <band>(<c1>-<c2>) separated by spaces. Each specifier tells the program to use the color index ``<c1>-<c2>'' to reduce frames taken in ``band''. For example, the default setting:

b(b-v) v(b-v) r(v-r) i(v-i)

will set the following transformation model:

$$B = b+Z_B+k_B(B-V)$$ (8.15)

$$V = v+Z_V+k_V(B-V)$$ (8.16)

$$R = r+Z_R+k_R(V-R)$$ (8.17)

$$I = i+Z_I+k_I(V-I)$$ (8.18)

This model is appropiate if we reduce BV, BVI, BVR or BVRI data. However, it will have to be changed if for instance we want to reduce VI data, as using it will require B, V and I observations, because of the V dependence on B. An appropiate model for VI data would be:

$$V = v+Z_V+k_V(V-I)$$ (8.19)

$$I = i+Z_I+k_I(V-I)$$ (8.20)

which is specified by setting:

v(v-i) i(v-i)

in the Bands setup option.

The same option can be used to set initial transformation coefficients and their errors, by appending ``=<coeff>/err'' to each band specifier like for example:

b(b-v)=0.12/0.001 v(b-v)=-0.07/0.02

Loading Report Files

The data to be reduced can reside in one or more files. To load data, open the the multi-frame reduction dialog using Processing/Multi-frame reduction or Ctrl-M, and select File/Add to Dataset. Select the file name and press Ok. The data from the frames contained in the file will load, and the frames will appear in the ``Frames'' tab of the dialog.

More observations can be added by using Add to Dataset repeteadly.^8.9 We'll assume the example file (cygs-aug19.out) is loaded for the next steps.

Frames

The ``Frames'' tab contains a list with all the frames in the dataset, one per line. It will display increasing amounts of information as the fit progresses. The columns that are filled up right after loading the data files should be self-explanatory. The other columns show:

Zpoint	The fitted zero point of the frame;
Err	The calculated error of the zero point;
Fitted	The number of standard stars used in the fit;
Outliers	The number of standard stars that are considered outliers of the fit (have large standard errors);
MEU	The mean error of unit weight for the zeropoint fit of the frame.

Clicking on the column headers will make the program sort the list by the respective column. Clicking again will reverse the sort order. One or more frames can be selected in the list. All operations apply to the selected frames or, if none are selected, to the whole list.

Stars

When a frame line is clicked, the stars from that frame are displayed in the ``Stars'' tab. Like the frames, the stars can be sorted on various columns by clicking on the column headers. The columns of the stars table show:

Name	The star's name. A star is identified across multiple frame by it's name;
Type	Star type (standard or target);
Band	The band the magnitudes are in;
Smag	The standard magnitude for the star. If the contents of this field are calculated by the program, as for target stars, the magnitude appears in square brackets;
Err	Error of the standard magnitude (either taken from the report file, or calculated by the program);
Imag	Instrumental magnitude in this observation;
Err	Error of the instrumental magnitude, taken from the report file;
Residual	The residual in the last fit of the frame. Only appears for standard stars;
Std Error	Standard error of the star (the residual divided by the estimated error). Only for the standard stars;
Outlier	''Y'' or ``N'' depending on whether the star has a large standard error or not;
R.A, Dec	Star catalog position;
Flags	A list of flags that apply to the star. Some are taken from the report file, some are added by the fitting routines.

Bands

The bands tab shows the currently configured bands, and the various transformation coefficients relating to these bands. They only show the fitted coefficients as they resulted from the last fit operation. If only some frames were selected in that operation, then these values may only apply to those frames.

Fitting Individual Zero Points

The simplest type of fit we can do is fit the zeropoints of each frame individually, without taking the other frames into consideration (like the last step in the aperture photometry routine). Even though the report files likely contained the individual fit information, it was discarded when the report was loaded. We need to perform at least this step before we can generate any plots for the data.

There are two variants of this command: one zeroes all the transformation coefficients before doing the fit (Fit Zero Points with Null Coefficients), while the other will apply the current transformation coefficients to the standard stars first. (Fit Zero Points with Current Coefficients).

Make sure the frames you want to fit are selected before applying the command (if no frames are selected, the command will apply to all frames).

After the fit, examine the MEU column, which will show the quality of the fit (the number should be around 1.0). Since we only fitted the zeropoint, and not the color coefficients the values are slightly larger than the best than can be obtained.

Plots

Figure 8.1: Residuals vesus standard magnitudes for all frames in the example data set, after zero point fitting without color transformation.

At this point, we can generate various plots, which are instrumental in judging the quality of the data, especially when we consider the more ellaborate fits. The program generates data file for the gnuplot utility, and will run gnuplot directly if the option File and Device Options/Gnuplot command is correctly set.^8.10 If the Plot to File option in the Plot menu is selected, the program will generate a data file instead of running gnuplot directly.^8.11

Let's select the V frames (click on the band column header twice to bring the V band at the top, then click on the first V frame, and finaly shift-click on the last V frame). Now run Plot/Residuals vs Magnitude. A plot should appear that is similar to the one in Figure 8.1.

Figure 8.2: Residuals versus standard magnitudes of one AU CYG frame.

The plot generally has the familiar shape of photon-shot noise dominated observations, with random errors increasing as the stars become fainter. An additional feature of this dataset are the ``branches'' going up starting at around mag 12 and 11. These are caused by saturated standard stars (the standards we used are not reliable above mag 12.5 or so). If the stars would be saturated in our observations, the ``branches'' would go downward.^8.12

Figure 8.3: Standard errors versus magnitudes of one AU CYG frame.

To investigate the matter further, we select a frame with a large number of outliers, which is likely to contain such a ``branch''. For example, let's select aucyg. The Residuals vs Magnitude plot for this frame is shown in Figure 8.2. The bright stars branching up are obvious in this plot. However, the importance of the errors is difficult to judge, as the ``normal'' error changes with the stars' magnitudes (and fainter stars show similar residuals). The Standard Errors vs Magnitude plot comes handy in this situation. It is similar to the previous plot, only the residuals are divided by the expected error of the respective stars. We expect all stars to show similar standard errors, all within a 6 units wide band around zero. This plot is shown in Figure 8.3. We can clearly see that the relatively large residuals to the right of the plot are within normal limits (also indicated by the value of the MEU fit parameter). The ``branch'' is clearly deviant (with standard errors going up to 30 and more).<sup>8.13</sup> Fortunately, the robust fitting algorithm has downweighted the deviant points significantly, so the ``good'' values still spread symetrically around zero.^8.14

Fitting Color Transformation Coefficients

Figure 8.4: Standard errors vs color for the V frames, before transformation coefficient fitting.

Figure 8.5: Standard errors vs color for the V frames, after transformation coefficient fitting.

With the V frames selected, let's plot the standard errors again, this time against the star's color index. For this, select Plot/Standard Errors vs Color. The output should look similar to Figure 8.4. Even given the scatter of the individual observations, the plot shows a clear sloping (making the residuals proportional to the color index).^8.15 To remove this slope and at the same time calculate the color transformation coefficient, we use Reduce/Fit Zero Points and Transformation Coefficients. After the fit is done, the slope is removed, as shown in Figure 8.5. The title of the figure shows the transformation used. In our case, the resulting tranformation coefficient is 0.062, a rather small figure indicating a good fit between the filters used and the standard ones. If we check the MEU fields for each frame, we will see that they have decreased, showing that the data more closely matches the standard magnitudes after the color transformation.

After the fit, the list in the ``Bands'' tab is updated to show the fitted transformation coefficients and their expected errors. Note that the error is quite small in our case (0.002),^8.16 even though the data seemed to spread a lot. The large number of stars used in the fit helped reduce the error considerably.

A good sanity check for the transformation coefficient fit is to run the same routine on subsets of the initial data set and compare the resulting transformation coefficients. They should match within the reported error figures.

Before proceeding, let's do the transformation coefficient fit for the whole dataset: Edit/Unselect All, then Reduce/Fit Zero Points and Transformation Coefficients.

All-Sky Reduction

Figure 8.6: Zero points vs airmass for frames with standards data.

Figure 8.7: Zero points vs time for frames with standards data.

The example data set contains BVRI frames for all fields. However, only some of the fields have R and I standards data. The night was clear, but conditions were changing. Let's see what we can do about the R and I frames that need all-sky reduction.

We can examine the frame zero points versus the airmass, expecting them to fall on a down-sloping line.^8.17 Using Plot/Zeropoints vs Airmass will produce the plot in Figure 8.6, which shows all the bands' zeropoints on the same graph. We see that most of the frames do indeed lie on down-sloping lines with a scatter consistent with their expected errors as shows by the error bars, but there are some outliers. So the conditions weren't photometric. If we now plot the same zeropoints against time (Plot/Zeropoints vs Time), Figure 8.7 we can see what has happened: the transparency has improved starting at MJD 53236.95, to the point where we can use the all-sky method for frames taken after that point.

Figure 8.8: Zero points vs airmass for all frames, after the extinction fit.

Figure 8.9: Zero points vs time for all frames, after the extinction fit.

Let's run the all-sky reduction (Reduce/Fit Extinction and All-Sky Zero Points) and generate the plots again. As we can see in Figures 8.8 and 8.9, the program has selected the frames which are bracketed by other ``good'' frames, ^8.18 and calculated their all-sky zeropoints. The all-sky frames are shown with different colors. These plots should be carefully examined and any suspicious frames removed from the all-sky reduction. In our case however, it seems that the program has made a good choice of frames.

The calculated extinction coefficients and their errors are displayed in the ``Bands'' tab. The errors of the exctinction coefficients are relatively large. In this case, this is due to the fact that the frames are taken in a narrow range of airmasses. The same narrow range of airmasses will however reduce the impact of the errors on the calculated zeropoints. this can be seen on the graphs, where the error bars of the all-sky frames, which take the extinction coefficient errors into account, are of the same order as those of the ``normal'' frames.^8.19

Reporting

After the fits are done, the complete dataset can be saved in the native format using File/Save Dataset. The native format preserves all the information in a future-proof fashion, but importing it into other applications can be a little involved.

The -rep-to-table or -T command-line option allows the native format to be converted into a table with fixed-width columns. The format and content of the columns are fully programmable by changing the File and Device Options/Report converter output format option. The following command will convert dataset.out from the native format to a table (dataset.txt) with the format as set in the option:

gcx -T dataset.out $>$dataset.txt

Alternatively, the table format can be specified on the command line. For example, to create a table with the stars' name, mjd of observation, and V magnitudes and errors use:

gcx -T dataset.out -S ".file.tab_format=name jdate $\backslash$
smag 'v' serr 'v'" $>$dataset.txt

The complete format string specification can be found in Appendix E.

Finally, it is possible to list the target stars in the AAVSO format. If a validation file location is set in the File and Device Options/Aavso validation file, it will be searched for the designation of the stars. The observer code field will be filled in from the general Observation Setup Data/Observer code option.