The fundamental products of the MaNGA survey is the spectroscopy obtained by the hardware system composed of the SDSS 2.5m telescope, the integral-field units (IFUs) holding the hexagonally spaced optical fibers, and the BOSS spectrographs, and the derived quantities based on these spectra. These data products are described by the MaNGA Data Model. In order to both produce these data products, and make best use of them, MaNGA uses numerous metadata reference files, which range from the core description of the fiber bundle metrology and spectrograph buildout to the DRPall and DAPall catalogs that provide summary metadata about the MaNGA survey observations and global derived properties.
This page makes frequent use of certain terms that include:
|Cart||The hardware on which MaNGA fibers and pseudo-slithead are installed that gets wheeled out to the SDSS 2.5m telescope|
|Pseudo-slit||Each cart has fibers mounted onto a pseudo-slithead which aligns with the real spectrograph slithead when a cartridge is mounted on the 2.5m telescope.|
|Ferrule||The physical piece of a fiber assembly that plugs into the plate.|
|IFU||A subset of the ferrules that contain a bundle of fibers instead of a single (sky) fiber.|
|V-groove block||The linear arrangement of fibers that attaches to the spectrograph pseudo-slit.|
|Harness||An ensemble constituting an entire set of interconnected ferrules plus v-groove blocks. For science IFUs there is a single IFU ferrule and anywhere from 2 to 8 sky ferrules, for spectrophotometric standard minibundles there are 3 IFU ferrules and 3 sky ferrules. Each harness may consist of anywhere from 1 - 4 v-groove blocks. Note that the 'ma008' (etc.) part number refers to the HARNESS, and therefore might include more than one IFU ferrule.|
|plateid||The ID number of a given plate|
|designid||The ID number of a given design (prior to assignment to a specific plateid)|
|plategroup||E.g., plateid=6653 would have plategroup=0066XX. plateid=10134 would have plategroup=0101XX.|
|designgroup||E.g., designid=7870 would have designgroup=D0078XX|
MaNGA uses bitmasks (see introduction at this link) for many purposes, ranging from target selection to DRP pipeline data quality. Detailed descriptions of each of these bitmasks can be found at the links below:
MANGA_TARGET1: Galaxy targeting bits
MANGA_TARGET2: Non-galaxy targeting bits
MANGA_TARGET3: Ancillary program targeting bits
MANGA_DRP2PIXMASK: 2d DRP per-pixel mask
MANGA_DRP2PIXMASK: 3d DRP per-pixel mask
MANGA_DAPPIXMASK: DAP maps mask
MANGA_DAPSPECMASK: DAP model data cubes mask
MANGA_DRP2QUAL: 2d DRP reduction quality
MANGA_DRP3QUAL: 3d DRP reduction quality
MANGA_DAPQUAL: DAP analysis quality
We want to retain much more information about the individual exposures that constitute a DRP final RSS file or data cube for a given galaxy than can easily be put into the FITS header for the file. Therefore, key information about each exposure is stored in the OBSINFO structure that is included as a FITS binary table extension in these files. OBSINFO structures have one line per exposure.
This structure is appended to the manga-[plate]-[ifudesign]-RSS and manga-[plate]-[ifudesign]-CUBE files so that delivered data can be tracked back to individual exposures easily, and so that future simulations created for a given galaxy can read from this extension everything necessary to know about the instrument and observing configuration to simulate.
See the OBSINFO data model for details.
MaNGA must store descriptive files about slit construction, plugging, and metrology of individual IFUs. These metadata files are touched by the data flow across the entire project, from cartridge buildout to target selection, plugging, and pipeline processing. These metadata are stored in their own SVN product called mangacore, available here.
Many individual files within mangacore are effectively version controlled using the MJD date stamps incorporated into the filenames, which also allows for programmatic file selection. Files can also be grouped by their plateid (a unique number assigned to each plate) and their designid (a unique number assigned to each plate design). At most 100 plates or plate designs are assigned to each plategroup or designgroup. For example,
plateid=6653 is assigned to
plategroup=0066XX; plate and plate-design group directories can be determined using the IDL routine
All ascii metadata stored in mangacore are formatted as SDSS parameter files. These files are analogous to a simple FITS binary table with header keyword pairs, and they can be written/read using the IDL routines
mangacore contains the following directories:
|apocomplete||Files indicating plate completion.|
|bin||Version control routine.|
|calib||Calibration and throughput reference files.|
|cartmaps||Build structure of the MaNGA carts|
|drill||Plate drilling measurements and plate metrology.|
|etc||Product version management for modules.|
|fluxcal||Templates for flux calibration (deprecated; templates are now stored on the SAS)|
|hdrfix||Raw data header fix files.|
|ifuflat||Storage location for reference files on ifu flatfield throughput.|
|mapper||Storage location for plPlugMapM files produced by the plate mapper at APO.|
|metrology||Files describing metrology of individual fiber harnesses and hex ferrules.|
|platedesign||Plate design file archive (foregroundstars, plateHolesSorted, plateMags, plateTargets, targetfix files)|
|pro||Version control routine for IDL.|
|simfiles||Reference files required by the MaNGA data simulator.|
|slitmaps||Storage location for MaNGA slitmaps.|
|ups||Product directory management.|
Each plate is observed using multiple exposures and dither positions until a set of S/N criteria are met. Once a plate is determined to be complete by the scheduling software at APO, an automated routine will create an apocomplete file listing the exposures that should be combined in the final data cube; see, for example apocomp-7443.par. The existence of this file is the indication that APO regards a given plate as complete and that it should be processed accordingly by the pipeline.
apocomplete files may include a keyword
eammode. If set to
off, the DRP will not execute the extended astrometry module (EAM) during the processing of this plate. At the top level of the apocomplete directory, the file bogey.par provides information about strange terrestrial features, such as airplane/satellite trails, that have interfered with the quality of a given exposure for a single IFU.
A core piece of hardware used during MaNGA observations is the observing cart --- the interchangeable hardware unit for the SDSS 2.5m telescope on which the MaNGA fibers and pseudo-slit are installed. MaNGA has manufactured six carts such that multiple plates can be plugged and ready for observation during an observing run. The elements plugged into each MaNGA plate with the starting point of the optical train to the spectrographs are called ferrules. These ferrules can hold either an IFU, for the main science observations, or a calibration IFU or single-fiber element. The end-point of the optical train is arranged into a pseudo-slit with groups of fibers, termed v-groove blocks, originating from a single IFU or set of calibration fibers. The ensemble constituting an entire set of interconnected ferrules and v-groove blocks is called a harness. Each MaNGA cart contains many harnesses plugged into the two spectrograph slits. An example layout for the two spectrographs are shown in the two figures below:
A cartmap file contains all of the information about the arrangement of harnesses along each slit, which can be used to translate from the design-stage target selection to the plugging-stage IFU assignment. These files are constructed by hand each time the slit is built for a given cartridge and stored in mangacore. They are SDSS parameter files named cartmap-[cartid]-[mjd].par, where cartid is the cart number (e.g., 01) and mjd is the mjd on which the build was completed.
Full cartmap data model
After each MaNGA plate is drilled, the positions of holes in the plate are measured using the University of Washington CMM. A random selection of most holes are measured; all MaNGA holes are measured. The UWash measurement scheme is to produce a plaintext file listing nominal hole locations (in xFlat,yFlat coordinate frame) as well as measured locations. The global shift, rotation, and scaling that best applies to the measured locations is listed in the header.
The UWash measurements are provided as a plaintext file and converted to an SDSS parameter file using
ml_importcmm.pro and stored in mangacore under the appropriate plategroup subfolder, and they are named
plate-[plateid]-CMM.par. The routine that converts the CMM measurements from plaintext to SDSS parameter file, also performs a quality-assessment check that all MaNGA holes are within 20 microns of their nominal location. The MaNGA DRP reads the CMM data to determine offsets of individual holes from nominal locations during the astrometric routine (link).
Full drillfile data model
Occasionally, the raw data files contain incorrect header keywords. Rather than modifying the raw files themselves, the corrections are saved in hdrfix files. The corrections are applied by the routine
ml_mghdrfix.pro (called from
sdssproc.pro). Headers are also fixed when looking for MaNGA files (using
ml_selectmanga.pro), since this is needed to distinguish MaNGA from BOSS files. The hdrfix files are SDSS parameter files called
sdHdrFix-[MJD].par and placed in subdirectories organized by MJD. The most frequent creation of hdrfix files is done by DOS (the on-mountain reduction software), which flags exposures as
BAD based on criteria such as total exposure time.
Full hdrfix data model
In order to monitor the throughput performance of the MaNGA IFUs over time, the DRP produces parameter files with names
ifuflat-[cart]-[camera]-[mjd]-[exposure].par each time it reduces a flatfield. Good 'reference' files for each cart are stored by hand in mangacore to show what the distribution of throughputs should look like when the IFUs are known to be clean; deviations from this suggest that it is time to give the IFUs a cleaning. The DRP looks for the latest file for the appropriate cart/mjd/camera combination to include on the flat-field plots.
Full ifuflat data model
These plPlugMapM files are what defines the relation between location on plate and fiberID (1 through 1423) on a given slit, and are produced by scanning a laser down the slitheads and recording the location on the plate at which the laser light shows up using a video camera. MaNGA uses the same plPlugMapM files as all other SDSS spectroscopic surveys, but saves them in mangacore for convenience. These have the form
Full plPlugMapM data model
Each hexagonal ferrule into which the MaNGA fibers are packed undergoes QA testing prior to use, measuring the shape, size, wall angle, etc of the hexagonal opening. This information for all ferrules (identified by 4-digit code) is stored in the parameter file hexferrules.par
Each harness manufactured has the locations of each fiber within the IFU measured and stored as SDSS parameter files called
[harName]/[harName]-[MJD]-[version].par, where, e.g.,
harName='ma012'. The provided MJD is the date on which the harness was measured at the University of Wisconsin lab. From the Wisconsin metrology point of view, each harness is given its own data file according to the harName (e.g., ma017). Each file lists all of the fibers in the harness, in order of ascending
finblock, across all associated v-groove blocks. Fibers are flagged as either SKY or IFU. The 4-digit code etched on the ferrule associated with each fiber is provided. For IFU fibers, the
fnum fiber number within the IFU runs from 1 to the size of the IFU. For sky fibers,
fnum uses a negative indexing from -1 to -X, where X is the total number of sky fibers in the harness. All fibers should have a descriptor of their quality and, if applicable, relative position within their IFU. The DRP searches for the most recent version of the most recent mjd appropriate for the date on which the plate was plugged.
There are five sets of files created during the plate design stage that are archived in mangacore:
There is a foreground stars reference file that identifies foreground stars that fall within the field of view of MaNGA IFUs. This file is used to mask the appropriate pixels in the reconstructed data cube.
This file identifies foreground stars that fall within the field of view of the MaNGA galaxy IFU observations. There is a single file foregroundstars.par that lists the mangaid, right ascension, and declination of foreground stars identified by eye based on the broadband SDSS imaging. Anything within some distance of these locations in the data cubes is flagged with the FORESTAR bit in the MANGA_DRP3PIXMASK mask extension in the MaNGA data cubes.
Full foregroundstars data model
The plateHolesSorted files are generated along with the plPlugMapP files during the SDSS phase of the plate design process (i.e., after a design has left the MaNGA team). They follow the overall SDSS-IV data model. When the DOS pipeline is run it looks for the plateHolesSorted file for a given plate in mangacore; if not found, it copies it there from platedb. Subfolders within platedesign/plateholes/ are ordered by plategroup, so that the individual files are platedesign/plateholes/[plategroup]/plateHolesSorted-[plateid].par where plateid is a 6-digit string.
Full plateHolesSorted data model
PlateMags files give the expected fiber2mag magnitudes expected in g,r,i,z bands for a fiducial 127-fiber IFU pointed at each target galaxy. If full preimaging data cannot be found, then this information is used by DOS to calculate SN2 as a function of fiber magnitude for determining plate completeness. The plateMags-[DESIGNID].par files are archived in repo/manga/mangacore/tags/v1_6_1/platedesign/platemags with designgroup subfolders and be of SDSS Yanny-style format. There should be 127*Nifu entries, where Nifu is the number of IFUs on the plate. For each IFU, there is a line with magnitude information at the location of each of the 127 theoretical fibers (127 lines per IFU regardless of what size IFU is actually assigned). fnumDesign, raoff, and decoff values are for a theoretically perfect 127-fiber IFU defined by the reference file manga_simbmap_127.par.
Magnitudes at these fiducial positions are interpolated to magnitudes at an actual set of positions using the routine ml_evalplatemags.pro.
Full plateMags data model
The plateTargets files list information about each galaxy that is assigned to a plate design. There is a plateTargets file for each catalog used to select targets for observation; for instance, most MaNGA galaxies were selected from catalogid=1, and can be found at plateTargets-1.par. A plaintext description of the different catalogs can be found at catalog_ids.dat.
Formally the plateTargets file is used to contain ancillary information about each galaxy (e.g., redshift, mass, etc.) in a centrally located place that can be queried easily by the summary file generation code and used to build the DRPall file associated with each set of reduced data.
Note that everything starting with nsa_ is taken from the relevant NSA input target catalog.
Full plateTargets data model
These files provide corrections to the information contained in the plateHolesSorted or plateMags files, without the need to remake these files. Corrections in the targetfix files are propagated into the slitmaps when the slitmaps are created, and from there into the data file FITS headers. For example, the first few MaNGA plates had incorrect targeting bits set, so these files fix those bits after-the-fact so that the slitmap and other other parts of the MaNGA DRP use the correct values.
Full targetfix data model
The slitmap is MaNGAs version of the BOSS plugmap, in the sense that it contains all important data relating to an individual exposure for what fiber goes where. The slitmap is generated using the routine ml_makeslitmap.pro, which combines information from plPlugMapM, plateHolesSorted, targetfix, cartmap, and IFU metrology files. Note that this code does not actually REQUIRE plateHolesSorted to be sorted; information is matched between plugmap and plateHolesSorted files based on the xfocal/yfocal hole position. The slitmaps are stored in mangacore/slitmaps/slitmap-[plate]-[mjd]-[scan].par, where the [plate]-[mjd]-[scan] combination is identical to that in the corresponding plPlugMapM file.
Full slitmap data model