Data, Meet Compute: NASA’s Cumulus Ingest Architecture Patrick Quinn [email protected] The material is based upon work supported by the National Aeronautics and Space Administration under Contract Number NNG15HZ39C DM_PPT_NP_v01 NASA Cumulus WHERE WE’RE GOING DM_PPT_NP_v01 3 DM_PPT_NP_v01 4 (Image is licensed CC0, free for commercial use, no attribution required: https://pixabay.com/en/network-server-system-2402637/) DM_PPT_NP_v01 NASA Cumulus CUMULUS: DATA ➙ CLOUD DM_PPT_NP_v01 U s e r s Data Operator Scientist Ops Metrics Direct OPeNDAP AppEEARS Dashboard Reports Download Distribution Earthdata Ops API API Login EMS CMR Ingest Ingested Workflow Metadata Data Workflow Workflow On-Prem Archive Inventory SIPS Cumulus 6 DAAC-in-the-Cloud External Systems DM_PPT_NP_v01 DAAC-Owned Workflows Discover PDR Validate PDR Generate Download Validate Files PDRD Files Run MODIS Generate Processing PAN Submit to CMR 7 DM_PPT_NP_v01 8 DM_PPT_NP_v01 Output ??? Input 9 DM_PPT_NP_v01 Output UNIX Pipe Input 10 DM_PPT_NP_v01