2022-02-09 NH KEM and Phaethon Review Notes New Horizons ALICE KEM1 Encounter Raw/Calibrated Data V5.0 Reviewers: John Noonan, Lori Feaga John No issues, major or minor. Lori No problems. Certification: Yes New Horizons Pluto Encounter Atmosphere Data v2.0 Lori Collection of documents here is different from all other datasets. Some may not be applicable for derived data (level 5), but nh_met2utc, nh_fov, and payload_ssr should be useful. All these documents exist and are in great shape, just don't know why they aren't in this documentation collection. Jillian: I think we just dropped them, but we can add them back in. John: Easy thing to do would be to move them over, in case someone downloads this single dataset (and doesn't need the rest). Lori: A lien to add those back in, then. No data liens John No major or minor issues Certification: Yes, w/ one minor lien New Horizons ALICE Interplanetary Medium Scans Lori --> Major formatting lien on data. Error using readpds in IDL. Error in object SPREADSHEET: ROWS or ROW_BYTES or FIELDS <= 0. Possible error in data label. NASAView gives error on Field 2 bytes exceed 5 (but defined as a 5-byte field). Field formatting not identical for fields 3-90. Data are readable by Excel and by eye. All defined as F10.6, but some sig figs (trailing 0s) are omitted. Tilden: Did you use the fixed label I sent for NASAView? Lori: No, just with readpds. Tilden: I think one of those problems is a label error. index.lbl --> Add "the" to "this parameter identifies (the) target..." ref.cat --> Has one additional reference for PEPSSI that other ALICE KEM1 V2 data do not have. Should ref.cat be uniform for all deliveries? dataset.cat --> Should give full wavelength range. Just says "we integrate over lyman alpha". Spell out exact wavelength range. Joel Parker: We do not impose a wavelength cutoff when doing IPM scans. It is a full UV detector, but lyman alpha dominates signal. Lori: Okay, make sure that's clear in the dataset that it's integrating over the whole detector. John Ran into issue w/ definition of integration space. lbl file says "total flux", aareadme.txt says lyman alpha, dataset has clarification that total counts are assumed to be lyman alpha. Joel: Clarification important because we don't want people to think these are h alpha bright stars. It's the total count. Tilden: Richard Chen found a few oddities w/ data, but probably not the label issue here. Jillian: I will make ref.cat the same across all L5 datasets. Joel: So we just need the right statement to force those trailing 0s to appear. Certification Lori: clarification that it's across entire wavelength range is important. certified w/ liens John: I agree. New Horizons PEPSSI KEM1 Encounter Raw/Calibrated Data V5.0 New Horizons PEPSSI Derived Plasma Fluxes and Rates Reviewers: Steve Joy, Rudy Frahm Steve voldesc.cat --> Address for swri different in -4- file than -2- and -3-. Joel: Ste 300 is correct now. Steve: Make it consistent. ref.cat --> -4- dataset has kollmaneatal2019 reference, but -2- and -3- don't. Let's keep them consistent so 2 and 3 have it, too. dataset.cat (plasma) --> Does not provide column naming convention for data in files. Labels in files have cryptic names for columns/fields. Would be helpful to users if there was a clear naming convention. --> Incorrect statements on units. Looks like a copy/paste error. Labels list correctly, but ds.cat does not. Documents Bulk of files have not changed (but soc_inst_icd has) since Dec 2020 review. nh-x-pepssi-4-plasma-v1.0 is mising a few documents that are included in older datasets. Most of this is fine, but you might consider including nh_mission_trajectory as well as bad time intervals (bti) and command history files (seq). Comment and not a lien. -3- data NASAView has some display issues with these data/labels. Change of character from end of 2020 to beginning of 2021. Reason for difference is unexplained. -4- data --> Meaning of time column is not provided. start, center, or end should be spelled out here or in ds.cat. Rudy raw/cal datasets first Documentation --> no errors found Data --> Quick look spectrograms. Why doesn't v5 new data look like v4? Why so different from other time periods? Difference appears with heavy ions but not others. Rudy: I found issues w/ swap ICD, not pepssi. Steve: The file has changed, though. Sampled Data Documentation nh_kem.cat --> Mission stop time given as tbd, but keyword has a 2021 data which has already passed. Jillian: We do have a contract extension through 2022, so we can update that. --> "Names and times chosen for this mission phase are still in flux.." is that true? Jillian: We can update that pepssi.cat --> "Please refer to ds.cat for use of primary HDU and extensions..." But data are in CSV format, not FITS. Remove statement or see ds.cat in l2, l3 data. dataset.cat --> Dataset overview description of time range does not match keywords for stop_time --> "Data File Contents" should describe what is in the data files, as opposed to data file headers. --> Label files that go along with ds.cat are cryptic. columns need to be explained. not sure what "allall" means Data motly looks good Certification Steve: We both agree data are in good shape. Documentation is the issue. Rudy: Not sure a reasonable scientist can understand the data w/ doc present. Steve: Yeah, not being able to understand columns or timing is important. Would be happy w/ being sent updated files so we can verify changes have been made rather than a full delta review. Rudy: That's reasonable. (csv vs fits) Tilden: pepssi.cat should only be describing instrument, not data. Steve: True, but it shouldn't be incorrectly describing the data. Rudy: For l2 and l3, just check to make sure quicklook spectrograms are correct. Larry Brown: Yes, those look weird and i'd like to understand them. Jillian/Larry: Fine for us to just fix up the documentation and have you guys look it over. Tilden: Good to have column names spelled out in labels. Certification: l2 and l3 are certified (w/ minor liens). l4 data needs delta review of documentation. New Horizons SWAP KEM1 Encounter Raw/Calibrated Data V5.0 New Horizons SWAP Solar Wind Derived Data v2.0 Reviewers: Rudy Frahm and Steve Joy Rudy raw/cal datasets Documentation swap_cal.pdf --> Please indicate which curve is which between primary, secondary, and coincidence. (cem voltage graph) soc_inst_icd.pdf --> Error in equation 3 trajectory.tab --> Fill values throwing automated scaling off, but important to show what those values are. Data histogram data --> Some files (last 7) do not process correctly. HDU doesn't match what's in the ICD. Label files do decribe these data files, but ICD does not. fv plots data fine. Rest of the histogram and science data look fine. Rudy: Withhold certification to deal with histogram issues. Heather: Documentation findings look correct. Software update to change onboard software so that we could store more of the non-normalized histograms to get 30 min resolution instead of 1 day resolution. So we've removed the normalized spectra. We've read those files in and checked them and have code that reads them in properly, we just haven't put that info in the ICD. I did not realize files from this new collection type were being reviewed. Easier to take those 7 files and move them to the next archive delivery. File format is correct and as intended. SWAP Derived Dataset Documentation nhkem.cat --> Same date issues as with pepssi swap.cat --> Link in calibration file works as of 2022, not 2007. --> Sections describing level 3 data, not the level 5 this is Data example spectrum Heahter: To that he+ label on top, they've drawn a vertical line where it would occur. but also interstellar hydrogen+ would also occur there. If we didn't see this cutoff shape, then you would expect it's the he+ from the solar wind. Looks like most of that is interstellar pickup. Steve Catalog Directory nhsc.cat --> 5/20 review comments about KBO observations not addressed, but okay. ref.cat --> l5 and l2, l3 should have the same kollmaneatal2019 paper in ref.cat. dataset.cat --> don't recall there being a v1 for this to be a v2. Rudy: I remember a v1. Data Lack of continuity from late 2020 to early 2021. not sure whether that's because data still onboard or because instrument wasn't on or not facing solar wind. Steve: Agree with option to just remove those 7 files from this release and not include until ICD is updated. Brian: I agree the best thing to do is remove the files. Software changes on the spacecraft is going to affect a lot of things, so I'd rather handle all of these together and pay close attention to them. Lori: According to tilden, no problem with validation, but in trajectory files, there's a missing constant value that's been changed. Wondering why it's been changed. -0 might cause problems on some systems. and the value is between some actual values, not outside the data range. Brian: Don't remember exactly why changed. Guess is that's a field where the entire range of data is valid. So we had to get creative as to a missing data constant. "-0" is a value you wouldn't see and no other value would work. Heather: Haven't changed the spice code. Brian: We should look into to make sure it's intentional and won't cause issues. It's a lien. Certification: Certified (with 7 files removed) w/ minor doc liens New Horizons REX KEM1 Encounter Raw/Calibrated Data V4.0 Reviewers: Dustin Buccino/Ashok Verma Dustin no issues with data. documentation appears complete. Ashok newrex_py.py --> Written according to python 2. I've provided the python 3 version, which will have longer usability. pyfits is deprecated in python3. Data --> Did not find new reconstructed spice kernels, so not able to compare computed values with raw values, although raw values seem to be valid. --> As with last review, did not find uplink data for dss 24, 36, and 63. But probably beyond provider's control. Ludmilla: Is spice data for this part of the mission not ready? Ashok: I did not see a new spice trajectory which covers the data points for this delivery. Ludmilla: (in reference to python2 vs python3) We usually don't archive code. Does this work without this change? Ashok: Works without warning in python2, can work in python3. Tilden: Any code is there for documentation purposes. So we would hope it's well documented and could be updated for newer version/different language. Ludmilla: Is there a code description somewhere which says which version of python it's created for? Don't change code, just document. Just a statement saying "this code written with this version of python." Certification: yes Day 2 (2022-02-10) New Horizons LORRI KEM1 Encounter Raw/Calibrated Data V5.0 Reviewers: Tony Farnham, Xiao-Duan Zou Tony Overall, both data sets are in great shape. well documented with lots of description and information available. Added new targets. dataset.cat --> several references to MU69 rather than 2014 MU69 (and in nh_kem.cat). Should be corrected. Data --> number of files are discrepant because 68 in raw not included in calibrated. These images all in one directory, have pixel values of 0 except for the 38 header pixles. One target of 2011jx31. Is that missing? no explanation for what these are, or why they are included. Is there a plan for giving a purpose or reasoning behind the sequence of images so users know what to look for or whether they can be used for a particular purpose? Should they be left out of raw as well? Hal: I agree this seems funny. Brian Encke: I will take a look at them. Too many for packets not received yet and waiting. SPICE --> Can't comment on accuracy of geometry in headers and labels because not updated since last review. Tony: Sometimes in past reviews we get unreviewed spice files to look at. Team: We could do that. Tony: Pipeline is mature enough that I don't really find errors, so I'm not that concerned. just something I check on. Xiao-Duan dataset.cat --> Description added to v5 discussing window_mismatches. This extension is in all the v4 images as well (no file is affected). description should be edited to accurately reflect that. --> When Nw=1row, is window_mismatch meaningless? Team: Yes, it is meaningless and we should update the description to reflect that. Tilden: This text was added in part in response to LEISA data. Data --> In most folders there are only a few images, which means a lot of folders. Would it be easier to arrange them in fewer folders, one per day for example? --> Will there be a browse folder? Hal Weaver: Folder convention comes from how data is organized at the SOC. Brian Encke: Yeah, it's a naming convention put in long ago. It does get stretched out when there are so many images. Many folders does have the advantage that if you know the date you're looking for, you can find data really quickly. Team: Browse thumbnail images are a possibility. Something to include for the next dataset. Certification: yes w/ minor liens New Horizons MVIC KEM1 Encounter Raw/Calibrated Data V5.0 Reviewers: Xiao-Duan Zou, Tony Farnham Xiao-Duan dataset.cat --> same window_mismatch problem. And all images of level 1 mvic have this binary table. --> concept of N1=1row window_mismatch should be explained somewhere. Tony dataset.cat/nh_kem.cat --> same MU69 issue Tilden: No comments on LORRI or MVIC from me or Richard Chen. Certification: Yes New Horizons LEISA KEM1 Encounter Raw/Calibrated Data V5.0 Reviewers: Mike DiSanti, Adam McKay Mike LEISA Intensity Calibration --> Tried per-pixel calibraiton formula in calinfo.txt but I get a ridiculously large number 10^12, which doesn't make sense. May be a matter of checking procedure rather than using data. Because calibrated data are much cleaner. Summary/Suggestions --> While vega scan direction as expected (target moves up through lambda), mu69 scan direction is reversed. In each case it was ~1 row per step in lambda. Modified software to account for this. Spatial Registration --> Both vega and mu69 show some wobble in the x-direction. --> Difficult to extract spectra for them due to smearing. --> Unable to reproduce flux calibration. maybe more documentation required. Adam Data --> Image in folder 20181231_040854 doesn't seem to have any data (same as previous reviews) --> Image described as having mu69, but not apparent. probably just too faint? --> Last image (for arcturus data) star position drifts from right to left rather than vertically. Probably just a difference in how scan mode was executed. Tilden: No comments from EN or anything that needs discussion from me. Mike: Anything I would question is clarity of the flux calibration. Probably quite detailed to lay out. Ludmilla: A lien? Mike: No, just a curiosity. Me trying to understand the data better. But it's still usable. Ludmilla: The team should check the calibration formula. Typos have been found before. Cathy Olkin: It's a good suggestion. We should check the equation in calinfo. Mike: Even prodide an example like alpha lyrae. (image orientation problem) Cathy: For arcturus going the other way, reason we do that is we want some calibration to tie points the other direction across some individual scans. Additionally for object you said you couldn't find, we couldn't either, just too faint. For previous image that was just 3 white lines, I think we just brought down a few wavelengths and haven't brought down the rest because we don't believe there's anything to be gained and we want to be good shepherds of the DSN. Took a leisa spectrum in case we get lucky. Cathy: Because arrokoth so faint, we wanted to increase integration time, but that would mean more smear. And want to constrain to smaller portion of chip to get less dead band. So tests were run. Bea: Is that stuff documented? Cathy: I don't remember what made it into the documentation. We could add that. I don't think it matters too much why we did that with arcturus. Bea: Say "some images scanned in a different direction" so users knos it's not an error. Lori: I looked and in some documentation (maybe leisa.cat) it says that scans can go in any direction based on instrument team's discretion. Certification: Yes (on 68 mystery files) Brian: Howard and I have been looking at the 68 mystery files. 64 of them are "zero dark" files. our intention was to just hold the raw ones out (but didn't). 4 others of them with target jx31 do belong in calibrated (and don't have 0s). We'll add a note to ds.cat about them. Tilden: Is there a better target name than n/a? Brian: Probably a good name. They're calibration, so that might be a target name. Tilden: There is a keyword target description and you can give a brief explanation of what this really is. Brian: At the time we thought there was some value in keeping the zero darks. Brian: Hal - We only grabbed the bias columns for those images. New Horizons Arrokoth Encounter Surface Composition Maps New Horizons Arrokoth Encounter Geology and Geophysical Maps Reviewers: Trent Hare, Carolyn Ernst Trent Documentation --> Minor spelling issues --> Figures could be crisper --> Any NB comments can be removed. Labels --> Case issue with filenames in PDS3. they'are all caps in the labels, but not the actual files themselves. Data --> Appreciate warning that we don't care about FITS headers, but then there is discussion of FITS header. --> "third axis (band) is image number and could be thought of as time". Not sure this is useful, don't understand what it means. Carolyn: I understood the statement and just understood it as a stack going from earlier to later in time. Maybe warrants a parenthetical explaining that. Suggestion --> No attempt to mask or define Nodata in the label. Just a little nicer for the user. --> No keyword or comment in lable pointing to *_wavelength.lbl --> Is 64-bit warranted? Higher bit-type, less useable. MVIC cube --> No keyword or comment in label mapping the bands.Pperhaps just a simple description listing wavelengths, blue (400-550 nm), red, near, etc. --> With long tail histogram, can values be extracted and set to nodata? ca06_stack_albedo --> Label defines saturation values but not used. Browse image labels --> ^png_document is maybe valid, but not actually read by any pds tools. Same for spice. --> It would be great if objs had a texture applied. could that be supplied? Leslie Young: I saw file names in slide that said _albedo rather than _normalreflect. Is that a mistake? Trent: Probably Carolyn Documentation --> leisa.cat document does not mention the new name for the LEISA acronym. --> a few typos Data --> Comment on IMG files for surface composition maps. Note for SBN that NASAView doesn't work for these (but it and readpds.pro are called out to be used here). readpds.pro reads in -1s for data and it's not a data problem. Tilden: Carolyn, what version of mac os using to read img files? Tony: I was not able to read these img files either, and i have been able to with readpds previously. Leslie: I had trouble reading an earlier version of these img files as well. Lev sent me a new version of readpds that gave reasonable float values with correct min, max, and mean. but I was not convinced it was doing the flip correctly. Wasn't until 2 weeks before review that I got the same review files and was not able to read them in. Still not sure if the flip is done right (and we can go with Lev's new readpds). Trent: I did the same trick, using qjs because it's using gdal. Tilden: Was something missing from pds label accounting for it not being read? Trent: The label is simple enough for gdal, which has minimal support for pds3. Ludmilla: We don't want users to have to use tricks to read the data. Gerbs: Carolyn - how faithful is the orientation conversion compared to other things? Carolyn: Didn't do any flipping. obj just existed. Used same ordering in files. Tony: Is it a readpds problem or label problem? Tilden: Looking at the label, there are some keywords I don't normally see for images, so readpds might not be correctly reading the label. Have to talk to lev to see if he's fixing or making a special change to read it. Trent: My recommendation, if this is a problem, I don't see a problem converting to a fits or cube and having label point to that. EN node pds3 read tool also fails. Ludmilla: Lien is that this needs to be checked to see if it's a label issue or readpds issue. (certification to be discussed for entire geophys dataset) New Horizons Arrokoth Encounter Shape Models Reviewers: Boris Semenov & Marc Costa, Tony Farnham Marc Documents aareadme.txt --> Directory structure not outlined appropriately. Suggest updated provided file. dataset.cat --> minor typos mvic.cat --> missing tab/spaces Shape Models mu69_fr2kf_lopoly.obj --> Format diverges from other OBJs. Documented properly in label, but might lead to confusion. obj and dsk --> Model is rotated -90 deg wrt to +x axis. Should be aligned properly as others are. reference frame --> Cannot find coordinate system document agreed upon for the object. --> Reference frame mu69_fixed not defined anywhere in current nh spice data set. --> Spice toolkit n67 includes iau_arrokoth frame and is provided in nh spice dataset. dsk surface names --> Surfaces do not have "surface names." Has ID but not name. Should be defined in auxiliary FK or defined at creation of dsk. Recommend do both. missing orientation information --> Rotation model only present in referenced article which is not generally accessible. Not available in current spice dataset. Either explicitly cite the spice kernel dataset and indicate info will be available there, or include updated/corrected pck in the document collection. file names (recommendation) --> Drop *_spice suffix and instead include file version. dsk comment area --> Binary kernels typically have comment area that is filled out with certain sections and used by naif. The whole thing should be filled in. pds labels --> Description field of label is too poor. Tony shape models --> asp model is rotated 90 degrees from other models. Needs to be fixed before certification. dataset.cat --> Pole orientation and spin rate should be included somewhere in the dataset for reference and for orienting the shape model. Spice kernels only contain pole orientation. --> Table that defines latitude information. They specify a julian date with 0 decimal places, so they should specify noon to remove ambiguity. --> Note that points to raw and calibrated lorri and mvic data version 4. Is that specific to v4 (what was used to produce data) or just the latest? Because v5 would now be latest. Documents nh_ralph_v100_ti.txt --> Only a small piece of relevant info, the rest is numerous warnings about excess material that should be avoided. Better to just pull out relevant info into a separate document so as not to confuse user. --> Found some typos in documentation for albedo and temperature maps. Certification (for entire geophys dataset): delta review for shape models and geo phys. for composition maps - carolyn says yes, trent says yes. (trent willing to help fix labels if needed.) New Horizons SDC KEM1 Encounter Raw/Calibrated Data V5.0 Reviewers: Andrew Poppe, Jamey Szalay Andrew documents --> One file that has not been updated since 2021-02-06. Edwin Bernardoni: The v3 file might be deprecated because the newer version is there. Andrew: Then it's as simple as just removing it and cleaning up the folder. Jamey Data --> Distribution of grain radii/mass is not power law. Is that okay? --> Possiblity of noise sources not currently accounted for. Andrew: My initial guess is that these are thruster firings and removing coincidence hits would account for that. Edwin: We have been thrusting more recently. Jamey: Quality flags are in the data, so that's adequate to assess if it's dust or not. Certification: Yes Lowell Discovery Telescope Observing Campaign of Near-Earth Asteroid (3200) Phaethon Reviewers: Tony Farnham, Adam McKay Tony LMI Data --> Some reduced data files cannot be read via readpds4. Error occurs with data_type SignedMSB2 labels. Changing the format doesn't work, gets end of file error. DeVeny Data --> Raw data need value_offset of 32768. --> For reduced data, some are 32-bit, some are 16-bit. Need to make all data consistent or set the data_type attribute to proper value for each file. Not sure why some reduced data differ in precision, not stated in documents. --> Reduced data need value offset added as well. --> field_length/format needs to be 13.6 rather than 8.6 for deveny extracted flux at specified wavelengtgh. Documentation --> Include LMI and deveny instrument manuals for reference. --> May want to include a copy of the phaethon paper collection.xml --> start and stop times don't correspond to dates when images were obtained. description.pdf --> in table 1, AO calibrator stars are introduced in association with the BC images, but there is reference for them and no mention of how they are used. Adam Data --> Difference in shape of flats between different filters. Tony: That's how LDT works. Why we should include the manuals. Documentation --> Documentation should discuss which is blue and which is red. Explain why lack of darks. --> More detail for reductions/extractions Bea: Would like to plug OLAF. If used, most of label problems would not have shown up at all. Certification Tony - Data are certifiable, but xml labels need to be fixed (pointed out). Adam - Spectra at the bluest wavelength has some weird thing going on with the reduction. I would expect to be able to use the data across the whole range documented. So either fix that, or say that data below 3800 angstroms (or whatever) is not useable. Mike Kelley: We truncated the spectra at the long wavelengths, right? 3800-5500, so we didn't look at that. QZ: I will take a closer look at that. I tried different ways to reduce to get rid of noise. Tony: My first thought is the CCD dies at both ends, so you're just amplifying noise. Processed data very noisy beyond die off for raw data. Mike: With flats, you're seeing the dichroic being moved in, which vignettes a smaller area than the BC filter, so it creates a weird donut and doubles. Agree that we should have the manuals in there so users know that's a thing. Tony: Might want to flag that issue so users know there will be extra structure to sky background with dichroic, etc. Ludmilla: Do reviewers need to see the data again? Tony: No, as long as the labels are fixed. But would be interested to see the DeVeney again. Adam: Yeah, either fix the reduction or say we don't trust it beyond this point. Mike: Just have a mask column in the tables? Certification: No, but may require delta review if reduction changed, or not if just a comment added. And label fixes. Mike: Are we trying to archive the data presented in the paper or improve it? QZ: Preference is to have all things available, even if there are caveats for some of it. Because it could still be used.