SBN Comet Data Review 14 October 2004 Attending: Hal Weaver Tony Farnham Ed Grayzeck Stephanie McLaughlin Beatrice Mueller Dave Rabinowitz Marina Fomenkov Ludmilla Kolokolova Anita Cochran Tyler Brown Anne Raugh Mike A'Hearn Preliminary Discussion of Overarching issues: Ludmilla notes that some of these data sets contain poor quality data that is not clearly indicated as such, leading to substantial wasted time in review. Ludmilla had to refer back to the supplier to determine that certain data were known to be of poor quality. Mike notes that we have in the past rejected parts of data sets that were not of minimal acceptable quality. Anne remarked that if the data provider tells us that there are data of questionable quality, we will document it in the DATA_SET catalog object and perhaps in the labels themselves. There seems to be no consistent method for naming data files. IN particular, sometimes data files are named by the date/time of creation, rather than observation. Suggestion was made that the Archive Preparation Guide should contain recommendations for file naming conventions. Tony notes that there seems to be no consistency in designators for calibration observations and minor planet names. A big part of this is trying to control OLAF input, which will have to be worked with Mark, et al. Archiving ground-based cal data: Hal notes that while it is valuable to have the data preserved, there is a risk of exhausting budgets and disk space for pre-flight data of limited use. Mike notes that the DI volume is small, so all of it is being archived, whereas in larger missions it would more likely be only selected calibration observations that would be preserved. Marina notes that the Stardust reference list was clearly incomplete, since the first science papers are out and are not included. Anita notes that the reference list is not intended to be exhaustive, but rather to supply only the specific references needed for interpreting the data, or understanding the instrument description, etc. Continuing the discussion of the amount of effort involved in collecting and documenting the thermal vac data, Hal notes the effort involved seems to be large. Stef notes that, so far at least, it has paid off in smoother planning and analysis of later operations. Hal suggests that perhaps these early documentation and calibration data handling techniques should also be included in the archive preparer's guide. Very General Notes ------------------ - Some systems (Macs) have problems with FTP transfers. In future, use (or at least offer) HTTP links for ZIP file download. - IRAF has problems with directory names that are numbers with leading zeroes. Perhaps this should also be mentioned in the Archive Preparation Guide (APG). - Tony suggests that we should make a concerted effort to preserve intact (as far as possible) the web sites created by various teams working on the mission data, as well as those describing the hardware. - Once the review data are posted and the site opened to the reviewers, THE DATA MUST NOT CHANGE. If it is changed for some reason, or - more appropriately - if a problem is discovered after posting, reviewers MUST be notified immediately. DI Thermal Vac Data =================== General Liens: - The document set should include the requirements document. - The Ball calibration plans need to be ITAR-cleared and added to the archive. - All the FITS files indicate an OBJECT of "ground_cal" which is not particularly helpful. Is it possible to put the experiment name (EXPRMNT) there instead? - In the mission.cat file, there are no units in the tables. - There are "n/a" entries in the look-back phase table, which contradicts the text preceding that says all three instruments will continue to operate for 60 hours. Fix this inconsistency. - In inst_hriir.cat there are typos in the units of the overview tables. Also, the information here should be compared to that available on the instrument web site and edited/improved accordingly. - In inst_hriir.cat, move the data in text into tables as far as possible. - The catalog and info.txt files need to be proofed. "first thermal vac," which actually occurred during the second thermal vac run. This should be changed to avoid ambiguity. - HRIIR data needs a discussion of the manganen wire temperature correction for the focal plane array. clearly not true in at least three cases. - There are two different things called "templog", which is misleading. One name or the other should be changed (and the change propagated to info.txt and other documentation). [One is a temperature log, one is actually an IDL program.] - IN catalog files, "principle" is regularly (and improperly) used where "principal" is intended. Thermal Vac 1 ------------- Liens: Thermal Vac 2 ------------- Liens: - In the index directory there are two files in the directory that are not listed in the info.txt file. (Content duplicates other files.) - In the document directory, the docinfo.txt file contains typos. Also, it is confusing terminology to talk about the "first thermal vac," which actually occurred during the second thermal vac run. This should be changed to avoid ambiguity. Thermal Vac 3 ------------- Liens: Thermal Vac 4 ------------- Liens: - report_20030222.asc is a blank file. - In some DATA_SET catalog files there are single and strings of question marks that seem to look like information is missing. - Double-check tv4 document/image_log(s) directory to make sure the right version of the right files were preserved. - We need a discussion of the amplifier gain problem added to the documentation, with a corresponding note in the DATA_SET catalog file. - The bad pixels have bloomed in these data, but there is no warning or explanation of this in the documentation or catalog files. - In dataset.cat there is a reference to "multiple temperatures," but in fact there was only one temp. for TV4. SOHO ==== Liens: - In the data set catalog there is a reference to an "approximate conversion" to V magnitude, but no estimate of the quality of the approximation is given. It is needed. - In the DATA_SET catalog file (under the "Data" subheading), the date range for this data is incorrect (data only cover through 2000). - There are data in the document directory under "raw_astrom". They should be properly formatted and documented as data tables. - In the raw astrometry table, the relationship between the different time values quoted is not clear. All times should be well-defined. - The RA & Dec values for the comet are always zero. Why? Is it possible to add the comet positions? Is it possible to add errors? - It was not easy, or perhaps not possible to determine the pixel scale. This information should be prominently displayed in the instrument catalog file. - There are undefined abbreviations in the raw astrometry tables that no one could translate (e.g., "HIC" after RA and Dec). Both PS and PDF versions of one image file, but not the other. This should be consistent. - There is no consistency in time standards between, for example, the times listed in calibration files and those contained in the FITS headers. Time in general is not well-defined. The various time scales require detailed documentation, and the relationship between the time standards used in the various parts of the archive should be documented in the appropriate catalog and document files. - 2000_xy.txt (from the document/comet_pos subdirectory) records, as identified by the time tags, do not seem to have an obvious or consistent correspondence to the FITS files in the data directory. The relationship should be discovered and documented. This problem likely extends to similar files in this directory. It is not at all obvious that this is the case. - A note should be added to the DATA_SET description to warn novice users that many of the comets are faint. Adding the comet position to the label and headers will go a long way to help this problem. - The DATA_SET_ID is not compliant. Three principal problems with this dataset: - We don't have a clear correspondence between file names and records in various documentation and calibration tables. - There is a pervasive issue with times being undocumented and inconsistent between files and various tables. There are at least two potential reference frames involved, but never specified. - Comet astrometry is missing. Once these issues are dealt with, this dataset should come back for a full review. WILD2_KECK ========== Liens: - Is information on the photometric quality of the night available? If so, please add it. (Or a general comment in the DATA_SET description, if appropriate.) - Please document the aperture used for the standard star observations (if this is not already present). - Every standard star has a TARGET object that doesn't include the basic RA and Dec of the star. If these TARGET objects stay, that minimal information should be added. If not, either provide a table with the full set of star parameters (RA, Dec, type, etc.), or omit the information entirely and refer users to the star catalogs. - The CALIBRATION catalog files are completely wrong. Fix it and propagate back through OLAF. - The instrument catalog file never gives the pixel scale. - The instrument catalog file does not give particularly good descriptions of the various instrument modes. - In the data directory the processed data are in a subdirectory whereas the raw data are not. There should be both a "raw" and "processed" subdirectories. - The DATA_SET_ID is not compliant. - In the FITS headers, gain and read noise are absent. The exposure time has no unit. Is it possible that the FITS normalizer expunged info from comment fields? - Need a processing description for the DATA_SET description, especially in view of its impact on Stardust. - Need a glossary for the FITS keywords. - The Wild 2 catalog file presented here is not consistent with the catalog file provided for this object in other data sets. - The processed esi0082.fit file (from 2003dec20) gave Ludmilla problems in trying to open it, where other files worked. Investigate this and make sure there is nothing pathologically wrong with the file. - Include comet centroid information in a separate table at the level of the astrometry table. - Investigate including thumbnail images for browsing in a systematic way. ========================================================================