=============================================== NOTE: (1) This is the 2022-09-14 data delivery presented at the Oct 2022 peer review. (2) Pre-review liens and responses may be found below when still relevant, and should still be taken as liens. Reviewers comments will be marked by their initials and sometimes a slide number to point you to more clarification if you need it. The reviewers' presentations can be found linked into the agenda (https://sbnreview.astro.umd.edu/202210_NH_KEM/agenda.shtml) or in the review's presentations folder (https://sbnreview.astro.umd.edu/202210_NH_KEM/notes-presentations/). =============================================== Datasets Validated: NH-A-ALICE-2-KEM1-V6.0 NH-A-ALICE-3-KEM1-V6.0 NH-A-LEISA-2-KEM1-V6.0 NH-A-LEISA-3-KEM1-V6.0 NH-A-LORRI-2-KEM1-V6.0 NH-A-LORRI-3-KEM1-V6.0 NH-A-MVIC-2-KEM1-V6.0 NH-A-MVIC-3-KEM1-V6.0 NH-A-PEPSSI-2-KEM1-V6.0 NH-A-PEPSSI-3-KEM1-V6.0 NH-A-REX-2-KEM1-V5.0 NH-A-REX-3-KEM1-V5.0 NH-A-SDC-2-KEM1-V6.0 NH-A-SDC-3-KEM1-V6.0 NH-A-SWAP-2-KEM1-V6.0 NH-A-SWAP-3-KEM1-V6.0 =============================================== Errors for: GLOBAL (all datasets) file: 'catalog/nh_kem.cat' --> RF (pepssi slide 5): The mission phases have been completed at this stage of the mission and in some places it is reported correctly, but in others it is not. --> --> line 148-161: There is an inconsistency with the paragraph on lines 160-161 and the text on line 153. Assuming that the mission phase name and times are now static, I recommend removing the paragraph on lines 160-161. --> --> line 169: We assume the mission phase end time for KEM 1 Encounter is known, so please replace the TBD. Assuming that the mission phase name and times are now static, I recommend removing the paragraph on lines 183-184. files: 'data/*/*.lbl' --> NEW 9/16: I noticed that the PRODUCT_CREATION_TIME keyword value in general has been updated in every (or nearly every) PDS data label, even though no other changes were made to the label or data parts of the product (which is more important). I would expect the PRODUCT_CREATION_TIME to only change when the product is created or updated. If this is easily possible, can we limit it to when changes are made? This unnecessary update swamps the difference checking that is done. file: 'document/soc_inst_icd.pdf' --> NOTE that there may be liens against this file in the following instrument lien sections for liens specific to that instrument. --> RF (pepssi slide 7-10): There are a lot of formatting issues with this document, examples given in the slides. It is assumed there was a problem converting the source to PDF/A-1b. You may need to do the conversion on the source document's source machine (perhaps a problem between Windows vs Mac vs Linux). --> --> Similar issues were found XZ for LORRI and MVIC. file: 'index/index.{lbl,tab}' --> NEW 9/16: Many updates were done in the label, but the label forgot to update RECORD_BYTES and ROW_BYTES from 407 to 408. Please correct. --> NEW 9/16: Due the update in the PRODUCT_CREATION_TIME values in every data label, COLUMN 6 of this file likewise needs to be updated. I see that the value "2022-09-01" was plugged in, but the actual value should probably match what is found in the data labels, namely "2022-09-01T00:00:00". The column is wide enough to fit this expected value. file: 'voldesc.cat' --> Note that the PUBLICATION_DATE found in this file will be updated to the date PDS releases the data to the public. PDS-SBN will make this update when it is known. =============================================== Errors unique to: ALICE Certification Status: CERTIFIED, with minor liens. files: 'data/*/*' --> JN (slide 10): 12/4/2019 data should have Target_Name as "SUN", given header keyword NEWHORIZONS:OBSERVATION_DESC = "APPROACH SOLAR UNOCCULTED SOCC OBSERVATION PIXELLIST". --> --> LF: "N/A" can be dealt with (for migration) for wake ups and high voltage checks, but if Sun really is target because it's an occultation, then should be "Sun". --> --> SBN-TB: Please consider if this adjustment is correct and should be made and make it if it is appropriate. If you have questions, please reach out to SBN. =============================================== Errors unique to: LEISA Certification Status: CERTIFIED, with minor liens. [NO UNIQUE ISSUES FOUND] =============================================== Errors unique to: LORRI Certification Status: CERTIFIED, with minor liens. TARGET_NAME = "MILKY WAY BULGE RR LYRAE TEST" --> Review Panel: Consider renaming this to what was observed or attempted to be observed (aka star field that had variable stars). The current value is an okay description, but not really a target. For PDS3, we can live with it, but better if it is changed. For PDS4, it must be different. If changed, please run the name by SBN and then consult with SBN about producing a catalog file for the new target name. Existing target names in PDS3 are not recommended based on our understanding of the target. file: 'document/soc_inst_icd.pdf' --> There is mention of two new ApIDs 0x63C/D for co-added LORRI data in the DS.CAT file, but they are not mentioned in the ICD. This may not be a concern at this stage, but I mention it as an issue that might come up. --> --> 9/15 [BLE] It would be great if we could hold these ICD updates off until the next release, when there is actual data for the new ApIDs. So far, the ICD has NOT been updated yet. --> --> SBN-TB: This is probably fine at this stage. file: 'document/seq_lorri_kem1.{lbl,tab}' --> XZ (pg 2): KBO "P2156486" has been corrected to b3e "2020 KH42", but this data table still maintains the old designation. --> --> Review Panel: The data table should not be corrected/updated, but the label should add a note about this change in case there is any confusion in the future. --> --> SBN-TB: I suggest putting the note in the SEQUENCE_TABLE DESCRIPTION keyword. =============================================== Errors unique to: MVIC Certification Status: CERTIFIED, with minor liens. file: 'catalog/dataset.cat' --> Review Panel: If the only change to this data set from the prior version was some of the windowed frames are now full frames, please specifically call this out in the version history for v6.0. file: 'document/soc_inst_icd.pdf' --> There is mention of two new ApIDs 0x54e/f for co-added MVIC data in the DS.CAT file, but they are not mentioned in the ICD. Should they be? --> --> 9/15 [BLE] Ditto with above comment. Hopefully we can hold the ICD updates off for a while until we know exactly what the PDS updates will look like. Note – the real issue here is that the FITS header info (and therefore the labels) for post-FSW FITS files may need to include more info, and that will affect (probably) how we describe the co-adding process within the ICD. We don’t have all those answers yet. I’ll keep working this with LORRI, Howard, and Carly. --> --> SBN-TB: This is probably fine at this stage. =============================================== Errors unique to: PEPSSI Certification Status: L2 CERTIFIED, with minor liens. L3 NOT CERTIFIED, requires a delta review. This is due to the L3 data having their proton flux increased by orders of magnitude with the new calibration compared to the previous calibration. file: 'catalog/dataset.cat' --> NEW 9/16: STOP_TIME matches START_TIME = 2018-08-13T23:59:59.011. Please update the STOP_TIME value to equal 2022-04-10T23:59:59.318. --> SJ (slide 7): the -3- dataset description in the dataset.cat file is missing the detailed discussion of the file contents and naming conventions that were added for the V5.0 delta review. I expect that now that the delta review is complete, this text will make it into the V6.0 version of dataset.cat. --> --> SBN-TB: I believe SJ is wanting a similar solution as to what was done during or as a result of the delta review for the L4 data set or text that was added in the reviewed V5.0 data set in response to a lien. files: 'nh-a-pepssi-3-kem1-v6.0/calib/calpars/*tab' --> NEW 9/16: I noticed that the PRODUCT_CREATION_TIME keyword value in general has been updated in every (or nearly every) PDS label, even though no other changes were made to the label or data parts of the product (which is more important). I would expect the PRODUCT_CREATION_TIME to only change when the product is created or updated. If this is easily possible, can we limit it to when changes are made? This unnecessary update swamps the difference checking that is done. files: 'nh-a-pepssi-3-kem1-v6.0/data/*/*' --> SJ (slide 7): During the early part of the review, there was some confusion regarding flux columns with different names but identical descriptions (e.g L01S00FLUX and L01S00HFLUX in the EXTENSION_FLUX_TABLE). It turns out that this is explained in a line in the ICD. However, it might be easier on users if the explanation (best guess species, in this case Helium) was either in the label (best) or discussed in the dataset.cat file. -- >--> SJ (slide 15): If the team is going to fix the -3- label generator, you might consider changing the text in the flux column descriptions that don’t contain a species identifier (no H, P, Q) so that they say something like "unknown, but assumed to be Helium", rather than "L01S00H Helium ..." If this is out of scope, then maybe you could add some words to the dataset description in the dataset.cat file. --> --> SBN-TB: I believe SJ is wanting a similar solution as to what was done during or as a result of the delta review for the L4 data set. --> SJ (slide 7): Some of the uncertainties associated with the counts/sec columns in the "EXTENSION_FLUX_TABLE" list the UNIT = "1/(cm^2 sr s keV)" – units of flux. The labels for the -3- data files should have the UNITS values corrected for some of the uncertainty columns in the EXTENSION_FLUX_TABLE (e.g. B00S03UNC). --> RF (slides 32-39): Comparing the L3 calibration between v5 and v6 of this data set shows the shape of the data looks good, but the amplitude is at least an order of magnitude greater across the board. The team should look into this. file: 'document/soc_inst_icd.pdf' --> Other instruments that changed their data format had sections in the ICD updated, but not PEPSSI. Do any of data format changes outlined in the DS.CAT version history sections need to be included in this document? We can wait for Peer Reviewer input if desired. --> --> 9/15 [BLE] I think we can hopefully wait on this one too (or update it if LORRI/MVIC are updated). The reasoning is the same for all three instruments, i.e. we don’t have the new post-FSW data in K5. There will be a lot of ICD updates for PEPSSI eventually. --> --> 9/16 SBN-TB: I think the difference for PEPSSI is that some of the changes (L3 additions in particular) have been implemented in the current delivery, unlike with LORRI and MVIC. I'll bring this up at the peer review for comment. --> --> SJ (slide 9): The PEPSSI chapter of the soc_icd should be updated at some point to describe the new processing and data file structure of the -3- dataset. These changes are briefly described in the dataset.cat file in the dataset versioning discussion but additional detail is probably warranted. file: 'index/index.lbl' --> NEW 9/16: Similar as the ds.cat file, the START/STOP_TIME values mentioned herein are incorrect as well. file: 'voldesc.cat' --> NEW 9/16: Similar as the ds.cat file, the START/STOP_TIME values mentioned in the DESCRIPTION keyword value are incorrect. =============================================== Errors unique to: REX Certification Status: CERTIFIED, with minor liens [NO UNIQUE ISSUES FOUND] =============================================== Errors unique to: SDC Certification Status: CERTIFIED, with minor liens file: 'data/20210728_049070/sdc_0490701167_0x700_eng.*' --> JS (slides 3-5): One set of telemetry values = 0 at 2021-212T11:51:01.000. Please investigate. =============================================== Errors unique to: SWAP Certification Status: CERTIFIED, with minor liens file: 'catalog/dataset.cat' --> NEW 9/16: In the ABSTRACT_DESC, the START/STOP_TIME values were updated, but the START_TIME value still excludes two files in with a START_TIME = 2018-08-14T18:08:02.011 (L2 only) and 2018-08-14T20:43:46.011 (L2 and L3). Please update/correct. files: 'data/*/*' --> RF (slides 38-41): In the L3 0x584 data, there are missing energy steps starting in the latter part of 2021. This should be investigated. --> --> Heather: This is due to the data no longer being collected. We will need to find the date it was switched and document it. directory: 'document/data_summary_plots/' --> SJ (slide 7): Plots available for all days when calibrated data are available. Directory listing is overwhelmed by 1 day files, consider subfolders. files: 'document/traj/traj_2006_2028_1d.*' --> SJ (slide 7): They are okay, but unclear where the reconstructed vs. predicted boundary occurs in the file based on the kernels listing. Please document this. file: 'document/soc_inst_icd.pdf' --> RF (slide 8): In section 14.5.4 Errors (currently page 156), the first paragraph has a "Error! Bookmark not defined." Please fix. --> RF (slide 9): In section 14.5.4 Errors (currently page 156), Equation 3 is not correct. The first term on the right the equal sign ( C^2 / t^4 ) is incorrect since delta R_C = (-C / t^2 ) * delta t. --> --> RF: Equation 4 is correct. --> --> RF: Note that was not corrected from version 5. --> --> SJ: Note that Equation 3 was fixed in v5 of the data sets. --> --> Review panel: Should check that any other updates from v5 to v6 were not lost. file: 'index/index.lbl' --> NEW 9/16: Similar as the ds.cat file, the START/STOP_TIME values mentioned herein are incorrect as well. file: 'voldesc.cat' --> NEW 9/6: Similar as the ds.cat file, the START/STOP_TIME values mentioned in the DESCRIPTION keyword value are incorrect.