BHM Caveats

On this page

On this page we document a number of known issues with the BOSS data released in SDSS DR19

Targeting Caveats

Missing MOS Target Cartons and Semaphore Flag Descriptions

In DR19, there are some cartons included in the semaphore SDSS-V Targeting Flags definitions that are not included in the mos_target product or the tables loaded into the CAS. These cartons are related to SDSS-V FPS commissioning. In some data products, other (non-commissioning) SDSS-V Targeting Flags might be set for cartons that are not described in the DR19 documentation. These flags are associated with targeting cartons that will be described in future data releases.

Truncated FIRSTCARTON strings

A small number of FIRSTCARTON values in the BOSS data products have been inadvertently truncated. The list below shows the mapping from the truncated string to the originally intended carton name:

  • “bhm_spiders_agn_efeds_str” → “bhm_spiders_agn_efeds_stragglers“
  • “bhm_spiders_agn_supercosm” → “bhm_spiders_agn_supercosmos“
  • “bhm_spiders_clusters_ps1d” → “bhm_spiders_clusters_ps1dr2“
  • “bhm_spiders_clusters_lsdr” → “bhm_spiders_clusters_lsdr8“
  • “bhm_spiders_clusters_efed” → “bhm_spiders_clusters_efeds_stragglers“

Optical Spectra Caveats

As with any dataset, there are several caveats to watch out for when using the optical spectra released by SDSS. Some affect only a few spectra or a few data columns, while some others have a wider impact. Caveats are in many cases specific to this data release and will be corrected in future releases. This page contains a list of known caveats valid for SDSS Data Release 19 optical spectroscopic data. For caveats related to SDSS-IV and earlier spectra see www.sdss4.org/dr17/spectro/caveats/

FPS Commissioning and Engineering Fields

FieldIDs between 16000 and 100000 are designed as FPS commissioning and engineering fields. While valid science can be done with these fields, caution is advised.

FieldID 016181 is a commission field where the telescope position was moved each exposure, so this field should not be used for science.

NaN FIBER_RA/FIBER_DEC

For a small number of rows in the DR19 v6_1_3 spAll, spAll-epoch (plus their associated lite files), and the DR19 allspec file, the fiber_ra and fiber_dec values are NaN. Unless there is an associated ZWARNING flag of “UNPLUGGED”, then you can assume that the fiber is at the associated RACAT and DECCAT coordinates. If, however, there is an associated ZWARNING flag of “UNPLUGGED”, disregard this fiber in any analysis, as its location is likely incorrect.

FIELDQUALITY Flag

Due to the rather heterogeneous science requirements in the FPS era, the legacy FIELDQUALITY columns in the spAll and FieldList files may not capture the true quality of any given Field or spectrum. Users are advised to use alternative metrics (e.g. ZWARNING, SN_MEDIAN_ALL) to judge if BOSS spectra are suitable for their science goals.

BOSS Spectra PNG images

The line labels for the Ca II H & K lines in the BOSS Spectra PNG images are reversed.

spAll SPECPRIMARY FLAG

In DR19 BOSS summary products (i.e. spAll files) the SPECPRIMARY flag is set to 1 for the ‘best’ spectrum within each spatially associated group of spectra (grouping is via a friends-of-friends algorithm, with linking length 2arcsec); all other spectra within the group have SPECPRIMARY=0. This sky matching is done independently from the catalog cross-matching performed as part of target selection. The consequence is that a small number (<1000) of unique targets (uniqueness determined via SDSS_ID) that are represented in the spAll files, are not associated with any spectra having SPECPRIMARY=1. Similarly, a very small number of SDSS_IDs (21) in the daily and epoch coadd spAll files are associated with more than one spectrum having SPECPRIMARY=1.

spAll GAIA_G Magnitudes

In the plate era data produced by the BOSS data reduction pipeline (FieldIDs < 16000), the values listed in the GAIA_G column are actually Gaia BP magnitudes. These values have been corrected in the data visualizations tools associated with Zora

pyXCSAO Outputs

pyXCSAO is a package used to measure radial velocities for stars by cross-correlating BOSS spectra with stellar templates. Although pyXCSAO was run for all BOSS spectra, the outputs are only valid for stellar targets.

Redshift Status

The quality flags for the redshift fitting procedure are stored in the ZWARNING bit mask. A non-zero ZWARNING likely indicates a substantial problem with the data, or is an indication that the best-fit classification or redshift is not reliable (due, e.g., to low S/N, or the unusual nature of the spectrum). An exception is the MANY_OUTLIERS flag, which is set when many pixels are poorly explained in a statistical sense by the best-fit model. This bit is often set for very high signal-to-noise ratio stars (where errors are small, so χ2 is high), or galaxies with broad lines (the redshift fitting model includes only narrow lines); in such cases, the reported redshift is usually fine. Approximately 2% of non-sky BOSS spectra have some ZWARNING bit set other than MANY_OUTLIERS, and the pipeline redshift should be treated with caution in these cases. For the remaining BOSS spectra, the reported redshifts are almost always correct. After inspection, many of the spectra flagged with problems do turn out to have correctly reported redshifts and classifications. However, we recommend care before using them. Note that the defined ZWARNING flag bits for BOSS data are similar, but not identical, to those used in SDSS-I/II spectral data products.

Night sky emission lines

The night sky emission lines at 5577Å, 6300Å, 6363Å (when there is auroral activity), and in the OH forest (towards the red end of the BOSS passband) can be very strong and leave significant residuals in the spectra. The amplitude of these residuals is occasionally underestimated by the noise model. Be cautious about interpreting the reality of weak features close to strong sky lines.

Sky Subtraction Bias

The sky spectrum estimates in BOSS (and in fact in earlier SDSS spectrograph data) that are subtracted from each object are biased slightly low. This is due to the well-known bias associated with fitting an error-weighted model to data when the errors are estimated from the data itself (e.g. in the case of Poisson estimates of errors). These residuals can be detected by taking the average of the sky-subtracted sky fibers, which yield a slightly positive spectrum ranging from 7×10-20 erg/cm2/s/Å at around 8000 Angstroms to up to 10-18 erg/cm2/s/Å at the bluest and reddest end of the spectra.

Since DR13, the BOSS extraction algorithm was modified as described in Bautista et al. (2017). This new algorithm uses a constant weighing scheme, reducing considerably the sky subtraction bias. As a consequence, the extraction is slightly less optimal, yielding 5 to 10% larger errors in flux estimates per pixel.

Coadd errors are underestimated

The default BOSS spectra distributed in DR19 are coadded from several individual exposures. Each individual exposure has a slightly different relationship of pixel number to wavelength. Thus, errors in the coadded spectra have covariance between neighboring spectral bins; however, we do not calculate or track this covariance. As a result, we estimate a 10-20% “error-on-the-error” in the coadd noise model. If discrepancies at this level matter for your analysis, you should use the individual exposures, which have much better accuracy in their noise model (1-2%).

Clipped Spectral Lines

Some emission lines intrinsic to the target are erroneously clipped because they were identified as cosmic rays. In SDSS-V, our clipping algorithm was modified to prevent the clipping of sky lines used for secondary wavelength calibration. This modification limits the clipping of spectral lines, but that does not preclude the possibility that some strong (typically narrow) emission lines might still be clipped. If an emission line is so bright that it is saturated in the individual 15-minute exposures of the spectrograph, it can suffer this effect. Unfortunately, such saturated pixels are not flagged as such, although usually, that region of the spectrum has an inverse variance equal to zero. Luckily, objects with such strong emission lines are very rare, but the user should be aware of the possibility of objects with extremely strong emission lines and unphysical or unusual line ratios.

‘NOQSO’ spectral modeling

A dominant source of bad classification/redshift fits to galaxy spectra is QSO templates with unphysical parameters, e.g. negative terms so that QSO emission lines “fit” galaxy absorption lines. To correct for this, galaxy spectra also have a ZWARNING_NOQSO mask, Z_NOQSO redshift, etc. which excludes QSO templates when performing classification/redshift fits. For studies with galaxy spectra, these *_NOQSO values should be used instead of the original ZWARNING mask, Z redshift, etc.

Bad CCD column results in bogus high-z quasars

Some unmasked intermittently bad CCD columns result in spurious identifications of spectra as z>5 quasars. These affect fibers 40, 59, and 60. Users are encouraged to treat any very high pipeline redshifts with caution, especially when they were obtained using these particular fibers (determined via the FIBERID_LIST column).

QSO pipeline redshifts

The QSO pipeline redshifts are generally reliable for >95% of the quasars, with a small fraction of catastrophic redshift failures and misclassifications. The reported uncertainty in the pipeline redshift for quasars, however, is underestimated due to intrinsic velocity shifts of quasar emission lines with respect to the systemic velocity. More details about the pipeline redshift uncertainties of SDSS quasars can be found in Wu & Shen (2022).

BOSS Flux Calibration

The flux calibration of individual exposures has an observing hour-angle and fiber dependence, especially below 4200 Å. Analyses which rely upon accurate flux calibration of individual exposures should perform additional systematics cross-checks for the consistency between different exposures of the same object, and avoid data observed at large hour-angles (or Airmass). The spAll (and spAll-lite) summary files include a parameter (EXP_DISP_MED) that characterizes the dispersion of median exposure flux in multi-exposure coadds. This issue may also affect SDSS spectra, but that has not been confirmed.

BOSS Stellar Classifications

BOSS object classifications are primarily focused on the identification of galaxy vs. quasar vs. star. Although sub-classifications are provided, they are not optimized for accuracy. In particular, the CV star templates have more degrees of freedom than other stellar templates, which can result in unphysical solutions where negative PCA components of the CV templates can fit the absorption features of White Dwarfs. Fixing this has not been a high priority since the primary classification of “star” vs. “galaxy” or “qso” is still correct.

Artificial dichroic transitions at 6000 Å due to cross-talk from bright stars

A small number of spectra are affected by cross-talk from bright stars in neighboring fibers. This is often manifested in a strong break feature at the dichroic transition around 6000 Å, resulting from different levels of cross-talk between the red and blue arms of the spectrograph. These effects appear to occur less frequently at later survey dates, which would be consistent with the improvements in the focus of the BOSS spectrograph cameras that have been achieved with routine operation. We intend to mitigate these effects in future BOSS data releases through improvements in the extraction codes and to flag any spectra that remain compromised. At present, no masking of this effect is implemented, however, except to the extent that it triggers a ZWARNING bit in certain instances.

Pre-SDSS-V eBOSS, BOSS, and SDSS spectrograph reductions

The SDSS-III and -IV BOSS/eBOSS spectrograph and SDSS-I and -II SDSS plates are not reduced with the SDSS-V version of the reduction pipeline. However, the reductions are these plates are available under their respective pipeline tags (BOSS/eBOSS: v5_13_2, SDSS: 26, 103, and 104) in the current version of the SAS at https://dr19.sdss.org/sas/dr19/spectro/sdss/redux/.

Missing eBOSS plate in v5_13_2 (DR17+)

The eBOSS plate 10658 was included in DR16 (v5_13_0) and labeled ‘good’, but due to some minor changes in the reduction pipeline, ended up being labeled ‘bad’ in DR17 (v5_13_2), and was therefore not included in that data release (or any future release). The plate is however still available in DR16, with the caveat that it was close to the cut-off to being excluded from that data release too.

Back to the Black Hole Mapper home/overview page

Back to Top