SOHO 1996 --------- Reviewers: K. Battams, L. Feaga General ------- - Need specific provenance for comets in this archive. - Strongly consider which data products are archived at PDS. Is a fully-calibrated level 1.0 image required here? All current publications use level 0.5 data. These data are well understood by the scientific community and are already available in an archive. M. Knight's work is the intermediate product for generating light curves. Ultimately, archive the light curves. This is the most crucial data. - Recommend taking level 1.0 data product and write a simple routine to rescale for our work, then we don't need all the calibraiton products, etc. - For absolute completeness of the SOHO comet archive at PDS, consider archiving: o Comets discovered by SOHO SWAM instrument o Data from observations of comets by the SOHO UBCS instrument - Smaller images, 512x256, give a comet position is not on the image, e.g. data/1996a/1996a2_c3_****.fit. These images should be identified. This may occur on any image that does not have one dimension that is 1024. May need to manually look at these images to see if position is outside the frame. Affects comet post files, not astrometry files. After 1997-1998, this happens infrequently since nearly all data are full-frame. See note for data/1996a2/ in L. Feaga's list appended below. - Verify documentation correctly describes what is on the volume. - Implement changes and corrections noted by K. Battams' (appended below). Most of these items are duplicated in this section and the following sections. - Make corrections noted by L. Feaga (appended below). List includes typos and inconsistencies in voldesc.cat, aareadme.txt, /document, /calib, /index, and /data. - Make corrections noted by A. Raugh (appended below, from "problems.acr"). Images ------ - Problems remain with the LASCO images in the PDS SOHO archive. As they stand, they are not suitable for release/archiving. M. Knight and K. Battams are still working on the processing issues (scaling for dynamic range is not correct). C2 pretty good. C3 needs work because comet was "erased" in nearly all of the of these images. Labels ------ - Problem with image start and stop times: o LASCO is accurate to only to one tenth of a second. o Absolute times (SOHO clock) are accurate to better than 15 seconds. Consider removing stop time? See Chapters 7 & 8 of the LASCO C3 calibration paper which discusses this topic in detail. /document/comet_pos/sub-directory --------------------------------- pos_1996.tab - Missing constants are incorrect. Document claims they are of the form -9999.99 but in table, just ----'s. - Three different image times are given; reconsider the inclusion of these time formats in favor of a single corrected start time - Add the corrected exposure time to this file - Missing "T" in ISO-formatted dates pos_1996.txt - Add note that post Aug 2004, the astrometry files contain extra information in the first row (SOHO#, Camera, Discoverer) - Clarify that the column "know" differentiates comet positions (know=0) from star positions (know=1) - Remove the comment "For images which do not have an identified comet, these values are replaced to -9999.99 of the same width as the corresponding field."? The astrometry file does not have entries with this value. /document/comet_elements.txt ---------------------------- - Remove inconsistencies in spacing for the name of comet discoverer. References ---------- - Review and add to list ""Calibration of the SOHO/LASCO C3 White Light Coronagraph", Morrill et al, Solar Physics (2006) 233: 331-372. calib/filter/*.tab ------------------ - Some values are negative, do not change to 0; note this in description/ index.lbl --------- - Add column descriptions. - Describe how SOLAR_NORTH_POLE_CLOCK_ANGLE is measured in an image. ______________________________________________________________________________ ______________________________________________________________________________ Karl Battams' List of Corrections --------------------------------- Images (Processing) ------------------- - Problems remain with the LASCO images in the PDS SOHO archive. As they stand, they are not suitable for release/archiving. There is ongoing correspondence between myself and Matthew Knight to resolve the processing issue(s), so unless requested (...??), they will not be discussed further at this point (with the exception of a brief mention in the summary). Labels ------ - Image "Start" and "Stop" time: Within the .lbl files for each image (also the "keyword_glossary" file), the beginning and end time of each individual exposure is given. (The "STOP_TIME" value is calculated by "adding the exposure duration value to the start time as extracted from the FITS file". Additionally, the exposure duration is corrected using the exposure correction factor.) The "start","stop" and "exposure duration" values are formatted to 1000ths of a second. It must be noted in the documentation that after all appropriate corrections: 1. LASCO exposure durations are accurate to better than 1/10th second 2. Absolute times (SOHO clock) are accurate to better than 15 seconds - The current scheme implies (to me, at least) a precision in time recording that does not exist within any SOHO data product! - Consider removing "STOP_TIME" value in data label files (and associated documentation). - Chapters 7 & 8 of the LASCO C3 Calibration paper (see below) discuss this in detail. Documentation ------------- /document/comet_pos/ sub-directory pos_1996.tab - Missing constants are incorrect. The documentation claims they are of the form -9999.99 (or variations thereof), but in the actual pos_1996.tab file, they are just ----'s. - Three different image times are given: 1. Corrected (calibrated) start time 2. Mid-time (corrected UTC time of s/c at midpoint of image) 3. "Old" time (uncorrected UTC time of s/c at midpoint of image) - Bearing in mind the aforementioned uncertainties, reconsider the inclusion of these three time formats in favor of a single (corrected) start time _ Add the corrected exposure time to this file pos_expl.txt - Note that post Aug 2004, the astrometry files contain extra information in the first row (SOHO#, Camera, Discoverer) _ Maybe clarify that the column "know" differentiates comet positions (know = 0) from star positions (know = 1) - Regarding the statement "For images which do not have an identified comet, these values are replaced by -999.999 of the same width as the corresponding field." -- there are no such lines in the astrometry files as they exist now. Only images with an identifiable comet were used for SOHO astrometry measurements. If this "missing" data is to be subsequently added to the astrometry files, it will need to be made very clear that the new measurements were performed "after the fact", on a different machine/OS, with a different operator (SOHO measurements can be subjective...), and potentially with a different version of software. /document/comet_elements.txt - Inconsistency in spacing for the name of comet "discoverer". Example: "X.Leprette" and "X. Leprette". This could cause issues for someone who has written a whitespace-dependent program of some kind... References ---------- _ Review (and add to list) "Calibration of the SOHO/LASCO C3 White Light Coronagraph", Morrill et al, Solar Physics (2006) 233: 331-372 Bottom line ----------- - The processed images continue to be a problem and are still not suitable for the PDS archive. Again, I will restate my previous recommendation that the use of a 'Level-1 variant' data product be reconsidered in favor of the well- established and widely available level-0.5 data (upon which all current SOHO comet data and papers are based). Additionally/alternatively, consider the use of the easily available NRL LASCO Level-1data product, which would only required a conversion to the desired units. - Aside from the images, and the few notes described earlier, the SOHO data archive appears to be in very good shape. Developers should keep in mind the possible future addition of (the currently incomplete/unpublished) SOHO comet photometry data. For absolute completeness of the SOHO comet dataset, additional consideration should be given to including: 1. comets discovered by the SOHO SWAN instrument, and 2. data from observations of comets by the SOHO UVCS instrument ______________________________________________________________________________ ______________________________________________________________________________ Lori Feaga's List of Corrections -------------------------------- voldesc.cat - MISSION_CATALOG entry not correct, the file name was shortened. aareadme.txt - Missing comet_elements.txt and eventual label should data directories be listed separately? - Change Puneet as contact person document/docinfo.txt - Be consistent in "PNG", "png", "pdf" as all caps or all lower case document/capture_dates.txt - Missing a lot of files that I believe are downloaded from LASCO: o document/spacecraft_info/time_and_rotation_axis.txt o document/calib_docs/filter/ 13 plots o document/calib_docs/quantum_eff/ 2 plots o document/sun_position/sc_body_axis_avg_roll.tab o document/sun_position/sunpos.txt o document/comet_pos/soho*.txt (26 files) o document/telemetry_stats/tm_plot_1996_* (12 files) o calib/filter/*.tab (11 files) o calib/quantum_eff/*.tab (2 files) o calib/vignetting/*.fit (3 files) document/comet_elements.txt - M3 and M2 are "out of order", make sure the rows have the correct data. YES, Karl said in perihelion order document/keyword_glossary.txt - Compared with an actual data label, the following keywords are not defined here, should they be included? o PDS_VERSION_ID o RECORD_TYPE o RECORD_BYTES o FILE_RECORDS document/inst_docs/handbook.txt - Spelling error from old liens: near the end of the file, "accessable" should be "accessible" document/inst_docs/temperature/*.png files - Not of greatest quality document/comet_pos/astrometry.txt - "pos_1996.*" should just be "pos_1996.tab" document/comet_pos/pos_1996.tab and .lbl - Missing data values are "-----" in the .tab and are specified as "-999.99" in .lbl - Space between date and time, should be one single column w/ "T" - Impose format for values in table especially sun_comet_dist and spacecraft_comet_dist - Label values for RECORD_ BYTES, FILE_RECORDS, ROW_BYTES, ROWS, BYTES, FORMAT, MINIMUM, MAXIMUM, START_BYTE need to be checked/corrected document/comet_pos/soho*.txt - All 26 files need labels document/sun_position - Some bad quality images: o pes_a.png o bs.png o 960101sc_roll.png calib/filter/*.tab - Small negative numbers (-1<#<0) in the tables for transmittance values, but in the label, there is a note that these run from 0-100, so note there is noise index/index.tab - Many columns do not have correct description and fix all BYTES, START_BYTE, FORMAT, etc. data/1996*/*.lbl - Check FILE_RECORDS value data/1996a2/ - All data are 512x256, comet is listed as being in a y pixel out of the appropriate range in the pos_1996.tab ______________________________________________________________________________ ______________________________________________________________________________ Anne Raugh's List of Corrections (from the problems.acr file) ------------------------------------------------------------- 30 March 2006 ============= Known problems with SOHO 1996 update (PDS Standards) GENERAL o wrong DSID throughout aareadme.txt o Refers to this as separate data set. o Sentence describing the DATA/ directory makes no sense o Does not mention document/keyword_glossary voldesc.cat o Does not list target catalog files (these are missing) calib/vignetting o All files missing labels catalog/ o Missing all TARGET files catalog/dataset.cat o Syntax errors o Missing DATA_SET_MISSION catalog/lasco.cat o Syntax errors catalog/reference.cat o Syntax errors o Incomplete REFERENCE_TEXT for BIESECKERETAL2002 o Incorrect REFERENCE_TEXT for FLECKETAL1995 document/ o Unlabelled files: comet_elements.col, comet_elements.txt document/docinfo.txt o Information in here does not match file list in the actual directory. document/capture_dates.lbl o RECORD_TYPE, RECORD_BYTES and FILE_RECORDS are incorrect document/keyword_glossary.lbl o Invalid RECORD_TYPE o Incorrect RECORD_BYTES document/calib_docs/exp_fac/expfac.lbl o RECORD_BYTES, FILE_RECORDS, RECORD_TYPE incorrect document/* o SAME ERRORS AS ABOVE FOUND IN ESSENTIALLY ALL LABELS IN THE DOCUMENT/ TREE index/index.lbl o ROWS, ROW_BYTES, RECORD_BYTES, FILE_RECORDS all incorrect. o index.tab is improperly formatted. o Column definitions don't work