1

NISDC UWG Annual Meeting Minutes for August 9-10, 2016

1.0 Summary The NSIDC User Working Group met on August 9-10, 2016 at the University of Colorado in Boulder, Colorado. There were 27 participants including 6 UWG members and 21 guests representing NSIDC, NASA ESDIS and other DAACs. The participants represented a wide variety of disciplines including Arctic climate, sea and land ice, mountain snow and DAAC user services and data stewardship. The discussions focused on how UWG members are using data sets in their research and where there are usability challenges, on new airborne data sets for the snow community, and on new NSIDC activities to better engage and serve the users. Key Points

Key accomplishments in the past year include development of data system ingest capabilities for ICESat-2, moving ECS and other NSIDC infrastructure equipment to a new university facility on east campus, and transition of processing of high value sea ice products from the failed DMSP F17 to F16/F18 operational satellites.

High priority projects in the next 6 to 12 months include development of ICESat-2 data services, release of SMAP enhanced resolution products and SnowEx campaign data management.

Feedback received from the UWG members was that the topics discussed were very informative and they valued the opportunity to present their data access and workflow experiences with the DAAC.

Areas for improvement suggested by the UWG members were to have a presentation by the DAAC Manager or NSIDC Director on the vision of the DAAC going forward, and provide a more focused meeting outline with clear goal statements in each section.

Key Actions

NSIDC to provide a “lesson’s learned” for data management of airborne campaigns based on experience with from OIB and SnowEx experience to other DAACs and the ESDSWG airborne working group.

Make use of the UWG to assess usability and functional capabilities of the polar map application, the DAAC’s internal Dataviewer tool and AppEEARS.

Facilitate a workshop meeting with mountain snow researchers and applications users at the university, NSIDC, NCAR, NOAA and other local organizations to discuss science community needs for new or existing data sets and data services needs.

As the SnowPex snow products inter-comparison activity wraps up this summer, NSIDC should provide a news article on its website summarizing information about the relevant data products.

2.0 UWG Charter The NSIDC DAAC User Working Group (UWG) plays a user representational and strategic advisory role in the DAAC’s continued operations and development. Members of the UWG represent the interests and needs of the varied groups of DAAC (and NSIDC) users. Responsibilities of the UWG include:

Reviewing the progress and performance of the DAAC relative to the NSIDC DAAC’s missions;

Representing the user community in the development and operation of the NSIDC DAAC products and services;

Recommending specific data products and services to NASA and the NSIDC DAAC;

2

Providing strategic guidance and advice to NASA and NSIDC on the NSIDC DAAC objectives, priorities, target user communities, implementation approaches, and other issues.

UWG Members

Name Affiliation Discipline Expertise Email

Chris Derksen Environment Canada

Sea ice, snow, GCM modeling, snowmelt hydrology, passive microwave, in situ

Chris.Derksen@ec.gc.ca

Gina Henderson

United States Naval Academy

Sea ice, surface, snow, GCM modeling, snowmelt hydrology, numerical modeling, passive microwave

ghenders@usna.edu

Jesse Johnson University of Montana

Ice sheets, glaciers, GCM modeling, numerical modeling, visible

johnson@cs.umt.edu

John Kimball University of Montana

Surface, snowmelt hydrology, carbon cycle, passive and active microwave

johnk@flbs.umt.edu

Nathan Kurtz NASA GSFC Sea ice, surface, snow, altimetry, passive and active microwave, visible, infrared

nathan.t.kurtz@nasa.gov

Nettie LaBelle-Hamer (ex officio)

Alaska Satellite Facility (ASF)

Director ASF nettie@asf.alaska.edu

Ben Livneh University of Colorado, Boulder

Surface, paleoclimate, numerical modeling ben.livneh@colorado.edu

Walt Meier NASA GSFC Sea ice, surface, active and passive microwave, scatterometry, visible, infrared

walt.meier@nasa.gov

Anne Nolin Oregon State University

Atmosphere, snow, glaciers, surface energy balance, altimetry, snowmelt hydrology, numerical modeling, active and passive microwave, visible, infrared, hyperspectral, in situ

nolina@geo.oregonstate.edu

Axel Schweiger

University of Washington, APL

Atmosphere, sea ice, surface, surface energy balance, numerical modeling, active and passive microwave, visible, infrared, sounder, in situ

axel@apl.washington.edu

Leigh Stearns University of Kansas

Ice sheets, surface, glaciers, active and passive microwave, visible, in situ

stearns@ku.edu

Nick Steiner City College of New York

Ice sheets, surface, snow, snowmelt hydrology, active and passive microwave, scatterometry, visible, in situ

nsteiner@ccny.cuny.edu

3.0 Meeting Goals for 2016

The annual face-to-face National Snow and Ice Data Center User Working Group meeting was held at the University of Colorado campus in Boulder on August 9 and 10, 2016. The primary goals were to:

Update UWG on NASA ESDIS initiatives and EOSDIS evolution roadmap Update UWG on key DAAC accomplishments and activities Solicit UWG input and recommendations on planned activities and priorities

Themes

3

Finding new ways to engage with our user community to improve services

Improving discovery and usability of our data sets

Areas for Feedback & Questions from UWG:

Help the DAAC identify priority User groups (profiles) to focus our work on improving data

discovery, access and usability.

How does the UWG see the broader polar science and snow communities interacting with and

benefiting from NASA satellite data in the commercial cloud?

Are there cross-DAAC collaborations that would be particularly beneficial to the cryospheric

research community?

4.0 NSIDC UWG Attendees UWG members Gina Henderson (Chair) Chris Derksen Jesse Johnson John S. Kimball (not able to attend) Nathan Kurtz Ben Livneh (not able to attend) Walt Meier Anne Nolin Axel Schweiger (not able to attend) Leigh Stearns (not able to attend) Nick Steiner NISDC Mark Serreze (NSIDC Director) Brian Johnson Amanda Leon Renea Ericson NASA ESDIS Dawn Lowe (ESDIS Project Manager) Jeanne Behnke (ESDIS Deputy Project Manager/Operations – webexed in) Drew Kittel (ESDIS, Deputy Manager Science Operations) Frank Lindsay (ESDIS, DAAC Operations) Christopher Lynnes (ESDIS System Architect) Ex Officio Nettie Labelle-Hamer (ASF) Other DAAC Representatives Beth Huffer (ASDC) Shania Sanders (ASDC)

4

Jeanne Laurencell (ASF) Bhaskar Ramachandran (LAADS) James Acker (GSFC, 10th only) Sherry Harrison (GHRC) Chris Torbert (LP DAAC) Bob Chen (SEDAC) 5.0 Meeting Minutes: 5.1 Schedule Tuesday (Aug. 9) 8:00 a.m. – 8:30 a.m. Meet & Greet 8:30 a.m. – 8:45 a.m. Welcome Brian J. 8:45 a.m. – 9:00 a.m. Introduction Gina H. 9:00 a.m. – 9:30 a.m. NASA ESDIS Updates Drew K. 9:30 a.m. – 9:45 a.m. NSIDC DAAC Updates Brian J. 9:45 a.m. –10:15 a.m. Action Item Review Brian J. 10:15 a.m. –10:30 a.m. Break 10:30 a.m. –12:00 p.m. UWG and NSIDC Scientific Data Uses

Gina Henderson (15 min) Mark Serreze (15 min) Walt Meier (15 min) Nick Steiner (15 min) Mahsa Moussavi (15 min)

12:00 a.m. –1:15 p.m. Break for Lunch 1:15 p.m. –1:45p.m. JPL Airborne Snow Observatory and SnowEX campaigns (Amanda) 1:45 p.m. –2:15 p.m. “Google Earth Engine version of the Snow Cover Frequency product” (Anne N.) 2:15 p.m. –2:45 p.m. Metrics (Paul) 2:45 p.m. –3:00 p.m. Break 3:00 p.m. –3:30 p.m. Improving Data Access, Discovery and Usability (Amanda) 3:30 p.m. –4:00 p.m. Review Actions/Recommendations 4:00 p.m. –5:00 p.m. Old Main Heritage Center tour 6:30 p.m. - Group Dinner -- Cantina Laredo, Twenty Ninth Street Boulder Wednesday (Aug. 10) 8:30 a.m. – 8:45 a.m. Logistics update 8:45 a.m. –10:15 a.m. UWG and NSIDC Scientific Data Uses

Anne Nolin (15 min) Nathan Kurtz (15 min) Chris Derksen (15 min) John Kimball (15 min) Jeff Thompson (15 min)

10:15 a.m. –10:30 a.m. Break 10:30 a.m. –11:15 a.m. Data Analysis Services (Data viewer Demo, ESDIS CATEES example) 11:15 a.m. –11:45 a.m. Condensate Project – an analytics example (Dave G.) 11:45 p.m. –1:00 p.m. Break for Lunch 1:00 p.m. –1:30 p.m. Tools Evaluation Project (Shannon)

5

1:30 p.m. –2:00 p.m. Status of Passive Microwave Satellite Transition Processing (Donna) 2:00 p.m. –2:45 p.m. User Services: new activities to better engage and serve users (Lisa) 2:45 p.m. –3:15 p.m. "Using image reconstruction to enhance spatial resolution of the satellite

passive microwave historical record" (Mary Jo) 3:15 p.m. –3:30 p.m. Break 3:30 p.m. –4:00 p.m. UWG Membership/next meeting discussion 4:00 p.m. –4:30 p.m. Review Actions/Recommendations 4:30 p.m. –5:00 p.m. UWG Wrap-up/Adjourn 5.2. Minutes Summary Day 1 Summary a.) ESDIS Update Opening remarks made by Brian Johnson to welcome attendees and introduce the agenda and meeting logistics. Gina Henderson made introductory remarks requesting active dialogue and participation by UWG members. This is the UWG members time to speak up about issues they are passionate about and also that their colleagues may be concerned about. Frank Lindsay and Chris Lynnes gave updates to ESDIS activities and a brief review of ESDIS history. Frank gave an historical overview of ESDIS. It became operational in 1994 with first heritage data sets. The first EOS mission in 1997. NASA developed an Open data policy. With all users having the same access and provided for free. New ESDIS is opening up the system so that people can build tools on it and interoperate with other data archives. ESDIS develops and applies standards and practices. ESDIS facilitates collaboration across DAACs. Manages Earth Science Data Systems Working groups to bring together funded scientists, SIPS, DAACs. 1.42 Billion data products over 2.6 million users. There are 13 upcoming missions. The 2020 NISAR mission is expected to produce 15-20 Petabytes in a year. In general the EOSDIS archive volume is increasing along with distribution volume. EOSDIS distributes about the same amount of volume as is archived. An overview of ESDIS cloud prototyping activities was given. The Cloud Prototypes presentation notes the different types of archives and the type of outcomes they are hoping to achieve. Hoping for a longtime paradigm shift where more analysis is done at the data. Chris Lynnes noted that cloud analytics is the main benefit of cloud storage enabling analysis at scale – makes supercomputing capability to average researcher. Need to reorganize the data in a form that is more susceptible to cloud analytics rather than a file-based storage approach. Data usually needs to be broken up and spread across multiple nodes. (Analytics optimized storage). Another paradigm shift is that data are not local to user’s computer. Users work with data where the data live (in the cloud) instead of bringing the data to local storage. Also, working on an end-user analysis toolbox as part of the CATEES prototype. Would have simple read routines, but also some analytics routines. There is some gains for hosting EOSDIS clients (Earthdata Search, CMR, GIBS) in the cloud. Biggest gains are in analytics support. Other services include Google, Amazon. Chris says they are bringing down large quantities of data. They are “essentially power users”. Seems to define power users as users pulling down and working with large quantities of data. ASF has been working with Web Object Storage. Distribute Sentinel radar data from Amazon storage. Doing an A/B test against a storage at LP. GIBS in the cloud – testing out some new cloud services that look promising. One is LAMDA, which is server-less architecture. Can radically decrease the archive management codebase size. An Ingest and

6

archive management prototype called Cumulus is being developed. NASA Marshall funded to lead for this effort. Phase 1 complete some time in early Fall. NSIDC will contribute to Phase 4 of this project. The cloud analytics prototypes (CATEES) focus is currently on python as a language. Working on Jupyter notebooks too. Can interleave text and images with executable code. Jupyter hub – focus on use cases and workflows that are common across DAACs. The main benefits of a common toolbox in the cloud will be:

1. Analyze data at scale 2. Analyze datasets together 3. Avoid downloading data and local management.

Support for users of cloud analytics: 1. Community open source tools 2. DAAC-developed tools 3. Cloud analytics examples and recipes.

Questions or comments:

Nathan – Question on Jupyter notebooks. Would users have to learn a new coding language? Chris – they are currently focused on Python flavor, but there is also an R flavor. Supposedly they can make both work together. Gina – coming from an undergrad university – python isn’t used. Pretty much matlab Nick and Anne says all students are learning Python. Chris L – at Millersville, they paired Comp Sci and Met to use python Walt – for older scientists like myself it is a challenge. Goddard started a python group. Younger is python and older struggle Nettie – we need to bring this discussion in to the user needs. Chris – with the cloud, it might be easier for early career scientists Mark S – so much of the challenge is human nature. Older scientists want the data on their computer. Walt – if I download the data and I have it and produce a paper, they can hand off the data if someone questions. How does that get preserved in the cloud. Nettie – it goes back to reproducible Frank – the notebooks could solve it Chris – but it’s still just pointers to the data Nettie – think that we have to think about the later career. Walt, Nettie, Chris – some kind of record of what is done Gina – paradigm shift of doing analysis non-locally would take a big shift for her. If moving workflow to cloud is that going to decide what languages you use to do your analysis. Chris – we’re going to make it easy to use python, but you could use C or other codes. The toolbox is intended to be language independent, but started with Python. Gina – security issues are a concerned because she works on a base. She can’t access NSIDC unless using IE because it’s the only way you can turn off passive FTP. Chris – we’ve heard this use case. Folks using the ASF edge server don’t know they are using the cloud. Mary Jo – back to Jupyter work – is the vision to set up Jupyter hub so that someone on the hub has access to the DAAC data. Chris – not really, it’s more about using python notebooks to access the data. Initially this is a showcase. It’s not really intended to be your analysis engine.

7

Dawn notes that cloud prototypes are just that – prototypes. Nettie notes that it makes planning the out years difficult.

b.) NSIDC DAAC update Update contained a brief overview of accomplishments, and status of ICESat-2 development. Dawn asks where the requirements come from – general requirements cone form mission Level 1 requirements but detail requirements are derived by DAAC with science team input. Steve notes that requirements reflect end-user needs and they come from science definition team and application group – several points of feedback.

Dawn – have you gotten an adequate amount of sample data for Raytheon? Amanda – no Steve – and we have noted that making services ready is contingent on test data.

General DAAC Challenges in the near-term:

Differing expectations of stakeholders

Balancing ongoing core activities like mission and user support. Cross function skills required.

New potential challenges w/ cross-DAAC collaboration projects

Timing of ESDIS cloud technology studies and adoption.

SMAP released validated L2-L4 data products this past year.

Amanda – enhanced products are the replacement products to accommodate for the radar failure Dawn – have you seen samples of those? Amanda – a whole suite just this week.

The DAAC has been trying to clarify our discussion of users and their categorization. How should we focus priorities? Are we covering our Cryospheric scientists well enough and should we be focusing on supporting interdisciplinary researchers?

Anne: To what extent do you also consider the international community? At this point the DAAC has not folded consideration of internato user community needs into our data and user services requirements.

Progress on last years UWG Action Items were reviewed. Noah Molotch suggestion for capturing mountain snow needs is to hold working group meetings with INSTAAR, CU, NCAR, and NOAA mountain snow researchers for recommendations. NSIDC expressed a need to identify the role of a DAAC Scientist as a critical liaison between the data system engineers and software developers, and the community scientist; someone who has been in the community and has done research, but is interested in the Data Science. Nettie: have a chief scientist who she pays and then taps in to his post-doc. It’s tough. You have to constantly evolve the concept.

Brian: we certainly have science oversight. This is more a science in practice. Amanda: Probably need to target a specific person.

c.) Scientific Data Uses presentations

8

A number of presentations from UWG members and NSIDC scientists were given during the mornings of both days describing their specific examples with i.) Gina Henderson -- Quantifying the impact of the MJO on the state of the Arctic (3 year NSF) Scale – global from tropics to Arctic. Trying to connect MJO convection to the Arctic. Hypothesis: MJO convection excites a poleward wave train. This affects atmospheric variables in the Arctic – sea ice conc. and terrestrial snow extent/depth. The primary goal is to investigate snow and sea ice modulation by the MJO. Looked at MJO in connection to sea ice extent and looked at MJO related to snow depth and SWE. Data sets used: MJO Multivariate MJO index- a tropical index, NCEP/NCAR REanalyis 2 for atmosphere, NOAA CDR – chose this one because she was familiar, well documented, netCDF, all files were individual which was tough for students since it was large.

- downloaded once and have it locally. Student then binned it together. - Big learning curve for student was downloading all of the files and catting it altogether. - If you have a missing day, leave out the file or leave it in? She prefers that it’s left in with missing

data codes so that it’s easier to find. - Binned and composited data by phase of MJO - Created a daily change in sea ice concentration (day 2-day1) - Kept data in it’s own grid.

MJO and NH snow depth **Spin up for students is challenging. She usually only gets students for a year.** In a perfect world, speeding up the ingesting and subsetting would allow her to make better use of her students. If they had tools to improve that, I could ask more of them. **A large challenge was dealing with data on different grids. Harmonizing could have been a whole project on it’s own. Ultimately subset regions to compare rather than dealing with the varying grids.**

Chris T asks how big of a difference is the challenge of regridding vs reprojecting Gina: We regrid, but we rarely reproject because of the problems. Have tended to stay with more visual analysis because the projection issue is hard to overcome.

ii.) Mark Serreze -- Death of the St. Patrick Bay Ice Caps. NE Ellesmere Island, Nunavut. Very small ice caps. Digitized an air photo and created an outline. Then have an area from another year where students walked the permimeter with GPS (2001 and a few other years). Then used ASTER to find clear shots that could be used to map the permiter. Also used some of the new LANDSAT 8 data. Ultimately came up with a 55-year record. Looked at summer warming using MODIS. The little ice cap shrunk drastically in 2014-2015. Used NCEP/NCAR reanalysis data to understand atmospheric temperatures. It’s an older data set, but it’s easy to use. Would rather use MERRA or MERRA2, but the tools aren’t available to do the quick analysis that NCEP/NCAR has. Also looked at timeseries of 850 mb temperature from radiosondes at Alert. Used Integrated Global Radiosonde Archive. In order to look at anomalies back in the little ice age (the source of the ice cap), he had to look at the paleoclimate information. Thinks the little ice cap will be dead by 2020.

Often the best way to find the data is to personally contact scientists. iii.) Walt Meier -- Remote Sensing Lab Goal – develop a simple lab for participants with sea ice experience but limited remote sensing data. Approach was to compare sea ice concentration from several products. Focus was in region around

9

barrow. Used: PM TBs to show how to derive sea ice; 0081; AMSR2 NASA Team 2; Bootstrap AMSR2; ASI from AMSR2 (Bremen); MASIE and MODIS; Noting the wide range of grids. A lot of pre-processing is required. The lab would have limited bandwidth and time for downloading and processing. Used ImageJ for processing. Converted everything to a simple flat-binary and Hemispheric 10 km EASE2 w/ no interpolation. Beaufort Sea subset gridded at 1km resolution. Subset region was chosen to match a modis tile. Would have preferred a different region but wanted to avoid stitching tiles. Graphed the results. Used Worldview to find regions and dates for the case study. (Walt gave an example of how worldview would have a more helpful workflow.) Challenges/Recommendation:

– Getting data in a consistent grid, subset. o -offer flexibility in how it gets done.

– NetCDF, HDF are state of the art byt still can be difficult to work with. o -Panoply is ok for viewing but not analysis o -IDL tools ok, but require license. o -Python tools very good and open source but may not be familiar to all

– Would been easier if not used for a workshop lab where participant has limited knowledge and tools.

iv.) Nick Steiner -- Microwave Radiometer processing for carbon in the arctic reservoirs experiment (CARVE). Used AMSR-E, AMSR2 and SSM/I(SSMIS) data sets. Wanted to use an AMSR instrument, but AMSR-E died just before the beginning of CARVE mission.

- Using 4 channels, ASC and DSC, and two polarizations. - Many channels to coordinate. - Swath to grid processing - account for the characteristics of the instrument - antennae footprint size, etc.

Data Access: Their question drove the data access – needed multi-frequency obs; Getting the data as a continues pull. They were processing the data as the aircraft missions were going. Used and processed NRT data and looked for areas of transition. Gave to flight team for planning. Friction points:

- 3 instruments - high data volume - basic data access – pull from basic FTP at JAXA - needed a way to obtain swath programmatically

Strip metadata and ingest data locally in to a local database. Helped him track what was already processed, what the version was, location in the file system. Wrote python code that treated individual DB entries as an object. -Query return object with read/write functions for associated data (HDF5) Challenges: Preprocesing, Correct bad scan, RFI correction, and Water-body and coastline filtering. Swath to grid aggregation was also challenge. Subset raw obs by location and time (asc/dsc), Grid based on physical parameters of instrument, Footprint matching make processing difficult. A solution to the challenges was implementing a Multidemensional Database (SciDB). Recommnedations for the DAAC:

– Python notebooks – very handy – Libraries for common task (community based software) – Programmatic access to data (download->archive)

10

– selecting and filtering – read/write and visualize – reporjection and gridding – Packaged collections of Remote sensing libraries – Numpy, Sci-kit Learn, Pandas, GDAL – Anaconda (python) – Virtual machines (Data and software kits); e.g. Providing a virtual machine with data and code.

(Mary Jo mentions and effort launching in September that might be what Nick is talking about.) d.) Airborne Snow Observatory and SnowEX campaigns: Amanda gave a brief overview of ASO and NSIDC DAACs data management role. The instruments are flying Uncompaghre Basin, and Toulumne basin (CA) routinely. All ASO data is coming to NSIDC. Working with ASO to define the product list. There are 16 distributed products, and working on volume estimates. Noted, from a data storage estimate point of view, that the number of basins and ASO flight repeats expected in the future is rapidly outgrowing the flying time available.

Karl asked about the instrument type – will it change? (Missed the answer) Mary Jo: Are all data coming to NSIDC, or just here forward. Amanda: All. We’re working to define the project list. Bhaskar: What formats? Jeff: Most are geotiffs. Some are ENVI. Some are LASZ. Chris D: Where do the SCAG products fit, because they are hard to get our hands on Amanda: They are going through our accession process.

SnowEx: Still ongoing discussions on instruments. ASO will be one of the planes flying campaigns this year. No campaigns next year (year 2). Will leverage Jeff, Chris D, and Anne’s knowledge as they are involved. Jeff: Airborne and ground coincident measurements are most important. We are in a position to help influence decisions.

Very dynamic project at this stage. Instrument decisions still being made. Grand Mesa and Senator Beck in CO. ASO will fly one of the campaigns this year. Year 1 campaign, and forming science definition team to guide year 3-5. There is no campaign in year two.

It is a follow on CLPx project. Successful model, so will have data managers in the field to help work with PIs in the field to ensure complete data.

Lessons learned from CLPx and OIB would be helpful to other centers who are battling the same thing.

Defining user communities - applications and research science

The original plan was a snowoff for a few weeks, measuring veg. Then snow on for 3 weeks in Feb. Nothing in between. We have capacity to put out weather stations to collect data.

Flights in between to ensure the algorithms

need to clarify the bounds of the snow data group. There will likely be measurements that are taken not under the banner of snowex.

The full value isn't just the 3 weeks. There is modeling before or after that. The whole experiment runs all winter, with a 3 week suborbital campaign.

May do research across longer period but use this snowex experiment as part of the research

Create lagging indicators for past, and leading indicators to forecast the future.

11

If monitoring April 1 snow from a snotel site, it doesn't tell you what is happening at higher eleveations, hence the need for remote sensing data.

Using MOD10A1 product.

UNESCO colleagues in Chile are interested in this b/c communities depend on snowmelt for water. They went from 8 year drought to El Nino event.

Anne: Diff between CLPX and SnowEx is the focus on forested snow. Able to look at the relationship between snow and vegetation without convolving it with topography. Walt: some discussion about Alaska. Jeff: Not this year, but other years could. Chris D: There was talk about covering 4 types of seasonal snow, but there is also value to repeating. Nick: What frequency are the instruments? Chris D: X and Ku. Mark: Mention a parallel thing going on – we have submitted a proposal of a series of snow schools. Intended for training on snow surveys and snow techniques for mid-career. Jared Entin said it will be funded. Dawn: asks if there is overlap between IceBridge and SnowEx? So we don’t repeat the same mistakes of IceBridge and learn about tools like metgen? Amanda: Jeff Deems is the biggest on our team. Not sure about the science team overlap. Nettie: Would love to learn from your efforts on IceBridge and SnowEx to help us with UAV program. Anne: Summed up well. SnowEx will have more of a research community. They will be testing instruments and techniques. What are the limits so that we can discovere synergy and limitations. Amanda: are there any particular data services? Anne: One of the concerns we have had…the original plan is snow off to measure veg and snow on in Feb and nothing in between. In my mind we have a lot of capacity for AWS to collect energy balance and snow depth that we can use in models to show the evolution of snow. It’s low cost and high value. No one has said they want that. The data would be important. Chris has expressed interest in flights in inbetween to help calibrate algorithms. Amanda: Needs to clarify what the bounds of the snowex data asset group. There could be measurements made not under the band of snowex. With CLPX, we would take data overlap the campaign only. What I hear is we need to archive all of the met tower data, not just the overlap. Nettie: Doesn’t that go back to what Walt was saying about reproducibility. If you are encapsulating layers of data as a certain place in time Amanda: Versioning is essential when you are capturing a snapshot. The product (like MODIS) move on, but the Nettie: You don’t want the snapshot because that’ Walt: IF version 1 was used at the time, that’s what you want to archive. Chris D: the full value of Snowex isn’t just the 3 week campaigns but more the whole winter. Jeff: one thing we discussed is brokering the online data that covers the area. Would be different from archiving the campaign, but I’m going to want access to a long record, not just the snowex period if I’m running a model. IT’s beyond our mandate to snag a data set. Amanda: we have seen challenges with a site no longer being maintained. Brian: Do you want to pull that data as part of the bundle. Anne: Might be helpful for the user to had a bundle. Gina: Goes back to no reinventing the wheel. People are going to repeatedly use a bundle, so why not provide it up front.

12

e.) Anne Nolin presented “Google Earth Engine version of the Snow Cover Frequency product” Develop new snow metrics for a warming world. Lagging indicators and leading indicators. Both remotely sensed and modeled. In a maritime snow climate, and April 1 SWE is challenging. There may not be snow, or snow may be at high elevations. The holy grail is SWE, but we may not get that. Snow cover frequency and snow disappearance date. They have a number of other parameters that they are working on. Currently using MOD10A1. They have a number of stakeholders – forecast runoff for ag, winter recreation. Figured they would create a snow cover frequency data set with monthly and seasonal. A grad student decided to implement the snow cover frequency algorithm in Google Earth Engine. It represents a new paradigm for producing data. IT’s so easy. Can subset the data. Group thinks about stakeholders and fit for purpose. State of Oregon is interested in statewide, but you might have others who want to subset based on a shapefile. They don’t have a fancy user interface yet because that is not how Google Earth Engine works. Code is written in JavaScript but python works as wsell. Might end up working in python when we get more advanced. “Even code dummies like me can figure it out.”

Frank: Are you visualizing or generating the product in Google Earth Engine? Anne: I am generating it. The code runs on the fly for whatever is in that window (which ever region. She ran the code for June 1 to April 1 2015 at the standard native resolution. Showing the polygon for SnowEx (ish) If you want to generate metadata, go to the inspector which shows the value. It shows the lat/lon value of the center of the pixel. The GEE is really fluid. They just changed names of things which caused them to have to change their code. Use case: if you have someone in the Elke Valley in Chile, they rely totally on snow melt for water. They don’t have other forecast assets, so they want to use this. There are downsides to creating in GE: documenting, the interface. Chris L: How much of that interface is coming from Google Earth Engine? Anne: This is all the GE developer interface. Amanda: what capabilities will you provide to users? Anne: We want to offer subsets in the US, import a shapefile and use that as a boundary and to draw a polygon. You can define a polygon and export all of the data as CSV with it’s lat lon **Users want to work with CSV files.** Anne: User gives the engine a time frame and area. Date range has to be Mary Jo: So the data the user will be getting is the data google earth engine has already sucked in? Anne: Yes. Gina: How does MOD10A1 data deal with clouds? Anne: It has a cloud mask. If the pixel has a value of 200, it’s snow. They just look at snow or snow free land. Frequency is discounting cloudy days. They are looking at an algorithm that assumes snow days as a gap fill. Nettie: do you have temperature variance? Anne: Nope, just mod10a1 right now. Gina: Really cool and exciting. Ways you can subset by state because already in GE. Could you subset by elevation? Anne: Yes, you can. The SnowEx region is for areas above 1500 meters. Anne: Would like to use MODSCAG, but it’s a

13

Jeff: Seems like a good example of a cloud development environment. Getting your code to run on a different machine is challenging. This provides a dev environment to short-cycle. Could this run more complex code? Anne: There are some buffer limits on GE that you can run in to. Another problem is that it’s someone else’s thing and they are changing the environment. They had to change all the calls. Right now code is really simple, would have to switch to Python. Amanda: what about versioning up? Anne: They haven’t put 6 in yet, but there is nothing stopping you from doing it. Amanda: Do the assets get shared across all GE users? Anne: I don’t know. I’m still pretty new.

f.) Metrics: Note MEaSUREs has a research page. We need to work on the citations counts.

Chris L: intern is working on it. Walt: track DOI citations over time. Chris L: our intern started doing this manually. Only 8% were DOI citations.

g.) Improving Data Access, Discovery and Usability presentation & discussion Worldview provides a primary visualization capability for EOSDIS data sets. On demand data services - we are hearing there is a workflow challenge. We hear about what is needed, but then we don't see the metrics that needs are being utilized.Reprojection vs. regridding based on earlier conversations.Gina: reprojection is a can of worms b/c you are creating new data. Regridding is not as big of an issue, as going up and down in resolutions. Reprojecting moving from stereographic to linear. Walt: I would be more worried about how the grid was changed.Chris Lynnes: Kevin has asked that we just explain to the users what was done.Amanda: Need a recipe for how to. This is no an easy service, especially as our data set list grows. Testing and verifying is bigger than we expected for SMAP for instance. How do we do this in a economical and sustainable way.

Dawn: Is the ability to drop users to preloaded Worldview something other DAAC can do? Amanda: Yes, it’s just using Worldview’s parameterized URL. Roger: Can you get a power point? Amanda: No, but you can get a png Chris B: We put GIBSGen in to the ECC. Amanda: and that works off Opendap Chris B: yes, we utilize opendap.

On Demand Data Services Amanda notes: Will be looking at the workflow for accessing the On demand services. Also notes that we are focusing on reprojection, but not so much regridding. We need to take a closer look at that.

Gina: It think that it’s a can of worms because your creating new data. Regridding is less can of worms. There are a few commonly used regridding techniques. NSIDC just needs to be transparent. Mary Jo: How are you distinquishing the terms? Gina: Regridding is up and down in resolution where reproject is changing projections. Walt: I would want to be a lot more careful with regridding because you might not want to average and smooth as a default. He would want more control over regridding than reprojecting. Chris L: Kevin Murphy suggests some way of describing what data loss or what artifacts were introduced. Karl: Easy enough to say it’s nearest neighbor or not. Or offer options.

14

Gina: It all depends on your application. If you calculate the snow area of one product and then re-project or re-grid and calculate again, it won’t be the same. Amanda: Almost like a data recipe or a tutorial and state the buyer beware. Brian: you could probably quantify the change for selected “validation” regions or conditions

Programmatic access Chris L: Opendap can do most of the subsets that are being discussed, but you have to be verys epcific. Put an easier to use GUI in front of it and deliver a “cube” of data with the appropriate space and time. Also looking at an API portal with a description of things like the Opendap API. If you are technically inclined, you could start by hitting the API until the front end can be developed. A lot more resources for massaging the data but they just aren’t easy to find. Polar Map Application Effectively replacing the functionality of the mapserver. Focusing on functionality replacement rather than the application. Chose the GeoServer technology, and will leverage its capabilities for other work in the DAAC down the road. Tool can be leveraged by others at NSIDC. AI: Can schedule a demo at an upcoming UWG telecon. AppEEARS - LP and NSDIC DAAC CollaborationApplication for Extracting and Exploring Analysis Ready Samples Developed at LPDAAC. Chris T - as good as OPeNDAP is, it can be complicated. It is a point sampler, and allows to pick pionts thru time. In future exposing subsets instead of points. We have a proof of concept, so next step is to try and create an operational instance. Looking to extend beyond MODIS to SMAP. MJ: Back when we were developing Polaris we tapped UWG for usability testing. Are there any plans to do that again? We have used UWG since polaris. UWG thinks valuable use of their time.

Mary Jo asks: Is this just MODIS data? Chris Torbert says WELD is available as well. Amanda: For NSIDC, just MODIS, but we would look at extending to SMAP as well. What sets this apart is aggregating the data in to a time series. Chris L: Have you looked at the time aggregation for Opendap Amanda: We are looking at it for the user side. (Without the need for appears) Bhaskar: How do you handle the bit QA? Chris: not an easy answer. Every product implements QA in a different way. First step is to document the relationship within CMR. Mary Jo: I’ve been out of the DAAC loop. Asked about usability testing. Nettie: It’s worked well except in one situation where the developers were in the room.

5.3 Day 2 Summary Action Item Review from Day 1:

NSIDC should document the lessons learned from OIB (i.e. managing airborne data). Make sure Nettie is in the loop.

Polar Map, AppEEARS and Data Viewer projects – show these to the UWG folks. General consensus is to send it cold or demo it depending on the type of feedback wanted/needed. Gina – send it out a little before a telecon and then we can give feedback at the telecon.

15

Mountain Snow – NSIDC host a short (e.g. ½ day) workshop to discuss mountain snow data sets and services.

Evaluate AMSR2 products from LANCE to ensure geolocation information is correct. o Walt noted that sea ice from LANCE is not geolocating properly in Panoply. Sherry noted

that they can take a look, but hopefully the issue will be corrected when standard product algorithms are implemented in LANCE.

NSIDC to write an article about the results of SNOWPEX intercomparisons o How to best present SNOWPEX results to users? Intercomparison results may help users

understand the data. Chris D thinks the JAXA snow products are the only ones we don’t archive at NSIDC.

Data accession web site needs improvement in clarity for users NSIDC should consider Thorsten’s melt onset data set for accession into the DAAC holdings.

o Walt notes that Goddard has a page of data that advertises data sets out of Goddard, particularly Thorsten’s melt onset data. Donna states that she has discussed with Thorsten and it’s on him to get the process going. Walt notes that it’s widely cited. Donna thinks it can move through the process quickly since the product review already stated that we should get it.

a.) Scientific Data Uses presentations i.) Anne Nolin – project trying to determine how snowpack impacts Dahl Sheep. IceTrendr – big data mining system to map changes in glacier with Landsat. Uses Google Earth as a plug in display. Had 5(?) glaciers that each presented a challenge. A lot of glaciologist use NDSI. Uses expert knowledge combined with automated processing to label points on a glacier. Proposing to NSF to map all glaciers.

Brian asks: Is this something you couldn’t do without Google Earth? Anne: Google Earth is no longer supported. This also uses Adobe Flash. We’re moving to Google Earth Engine. Also working with ESRI to understand what they might do. Wasn’t done in the cloud. Did have to download the landsat data, which was a huge pain. Would prefer to have had the data in the cloud. Amanda: What thought have you given to data citation and data providence? Not a classic model of data distribution. Anne: I don’t even know. We just want people to use it. We would love to partner with you on the next phase as we Frank: Wouldn’t you cite the tool. Anne: But we’re creating a data set – the semantics are a data set. Amanda notes that there is working in the data stewardship community about citing dynamic data that changes. Nettie: Isn’t DOI scary because it’s forever? Amanda: Yes, and ideas are to mint new DOIs for every new data.

ii.) Nathan Kurtz -- Main goal to use Cryosat 2 and cobined the records of Cryosat 2 and OIB. Derive freeboard from altimetry data using airborne data from OIB and sat from Cryosat2. Develop sea ice thickness product from merged data set. Matlab was the only software package that could incorporate all elements of his model. Had to do a lot of regridding of cryosat2 and pm data to merge. Then added all the OIB data in and put all data on a common grid. Used many observations of snow depth on sea ice available were combined using data assimilation. Again, regriddign a big issue. Everything is built to create the product, but need

16

to automate the download of input the data. Notes a community desire for both swath level data and gridded data. Data assimilation people want swath. People looking at trends want the gridded product. Cryosat are not user friendly (he says it’s the poster child for unfriendly). He defined this as data in a weird format (text+binary) – he was able to read by using a community developed reader. When they changed the format and his reader broke, he had to look for an updated reader. 100+ fields at different resolutions (only a hand of which are useful). Need to read a large handbook to underatnd how to use. No subsetting of the data is avalaible and must download different modes (SAR and SARin) to maintain geographic coverage. Needs subsetting, regridding, and reprojections to better combine data. Cloud computing could have improved his processing.

Gina asked which grid he used - Using 25 km polar stereo. Mostly because it’s what his code is set to run. Amy S asks how he found the code: ESA did not provide the code, but ESA linked to the code. Cryosat 2 site.

iii.) Chris Derksen -- Perspectives of accessing NSIDC data from Environment and Climate Change Canada. Snow (Ross Brown, Libo Wang, Chris Derksen): Use an observational ensemble approach (IMS, Measures, Globsnow and other SWE, Daily TB, MODIS snow cover. Use FTP pull initiated by user. Issues: undetected missing IMS files. Can NSIDC automatically flag when an ftp failed from NIC and generate a request for replacement? Caveats/issues with data product not clearly indicated and tend to be buried in documentation or not mentioned by dataset PI. Provide netDCF version of MODIS snow cover products. They convert to netCDF formats first. Data push capability would be good. For Sea Ice applications (josh.king@canada.ca), use DMS, ATM, POS/AV, Sea ice Freeboard, Snow data products. Pull whenever updated. An ftp pull initiated by user fine. Issues: No geo-tools to subset the data into manageable volumes and the variety of file types and format. For Pan Arctic Sea Ice applciations (Stephen Howell) data used includes Sea ice concentration (NT2/Bootstrap). It’s an FTP pull. Issues: must regrid the data. Climate model (Michael Sigmond) uses MEaSUREs and Torsten’s melt onset with Sea ice concentration (NT2/Bootstrap). Need data daily for operational forecasts (update them monthly). FTP pull is fine for data access. Sea Ice Motion (Mike Brady) applciations use Sea ice age (TSchudi), AVHRR APP-x, Concentration (NRT NASA Team2), and Other data sets from NOAA. Data access through ftp pull works fine. Subsetting is a pain – download and subset locally. Map projections: would like to see a single common standard across all NSIDC-hosted data sets, either EASE-Grid2 or Polar Stereo. SMAP FT (Chris Derksen) -- Validation of SMAP freeze/thaw using SMAP L3_FT_A, Cal/val field data sets. Tend to grab data straight through JPL. General comments:

Positive feedback. Very professional interactions in all regards.

Most common issues related to projections, subset, communication of data quality/uncertainty

Most issues/recs are under the category of “nice to have”

The ECCC user community wants straightforward access to data, DAAC-side data preparation tools, efficient data transfer

17

iv.) Jeff Thompson -- Land surface phenology Jeff uses data from different DAACs. For example: Used MOD10a1, MOD11a1, MOD13q1 A lot of regridding required. Had code to use the MRT to stitch the tiles. Not very efficient though. Tried to assemble 3-d grids (space and time) but ran in to trouble with processing. Ended up with an indexed space over time grid. MRT is slow. MODIS data products are not stored consistently (projections, tiled/not tiled). How do we make it easier to facilitate analysis across products? A Wish list for capabilities:

– Spatial Queries – find data near locations (point, shapefile) – Amanda: did you use a tool to find your data? – Jeff: no, not for this. Went to Earth Explorer for another – Amanda: Might be good for Earthdata Search usability – Compare image with non-image data in a moveable window – List all available data for a specific region (table) – Quick visualizations for data (e.g. ATM data, photon counts)

v.) Karl Rittger -- Snow cover from MODIS, VIIRS, Landsat Melt partitioning research work using information about snow cover, albedo, reanalysis, forest canopy. Processing: Resampling and regriding. JPL MODSCAG and VIIRSCAG Geotiff is not a correctly geolocaed. Needs HDF or NETCDF format.

Chris D: For the OCO2 community, snow is noise since they can’t use ozone measurements over snow. Surface grain size is important, and Chris’s group pointed them to MODSCAG, but tiles are limited. Karl: More tiles for the historical data is based on receiving funding to produce them. It’s easier to add tiles to the NRT product. Mary Jo: Would they just want to know whether snow is there or not, or is grain size necessary. Chris D: they need both. They needed gap-filling and complete hemispheric coverage. Karl: has a gap-filled product for 12 tiles that he would hope to submit to NSIDC.

b.) Data Analysis Services (Data viewer Demo, ESDIS CATEES example) Changed from presentation to a quick discussion. Dataviewer – tool that was initially internal for verifying data, but extended to users to consolidate IDL tools. CATEES Objectives are for several DAACs to develop and deploy ipython/jupyter notebooks with data tools.NSIDC idea is to provide a tool that Integrates data sets retrieved from more than one DAAC thru a notebook. NSIDC has a framework for a notebook. Data prep to regrid the 4 data sets being looked at.

Walt: People are starting to use python and share notebooks. Just heard about modelingguru.nasa.gov – modeling focus, but people can post code and ask questions. There are a lot of these communities. Have you thought of tying in to a bigger community? Brian: we have not thought about bigger picture. We thought more about incorporating these in to data recipes. Anne: What does the extract function create? Can you extract data from a region and export to a CSV. Chris L: you’re extracting it to memory. Nick: Are you going to give processing capabilities to users? Brian: We’re just exploring it. Not a real vision.

18

Chris L: we’ve talked about it, but right now just using Jupyter hub as a showcase. For serious analysis, providing a docker container or AMI, could be done. Nick: is it mostly Amazon? Chris L: yes, but google earth engine is an interesting option.

c.) Tools Evaluation Project Frank was surprised by the number of tools. (These include tools develop under other programs at NSIDC and EOSDIS-wide tools like Reverb, Earthdata Search, Worldview) Discussion started among the group around maintaining tools with the lack of ongoing funding.

Anne: To what extent do you encourage external contributors of software to post it Github. Answer: we don’t. Frank: Support is supposed to be built in to the plan. Chris L: Involvement of the DAAC in the ACCESS proposals usually means it will go well. Nathan: Is the tool list available for users to see? Shannon: The software inventory is likely to be more internal. Amanda talked about the workflow for users finding these tools. Brian: Maybe the lab space can be an off ramp where we assess the use.

d.) Status of Passive Microwave Satellite Transition Processing Walt is hearing that F20 will not happen. Metops option from European EUMETSAT operational satellite for PM follow on.

Brian: Defense might be looking to go forward with their own sensor. Walt: F20 was built in around 1995, so it’s old technology. There has been talk of doing one more AMSR follow-on. Japan doesn’t have a good reason to launch another AMSR. NASA is trying to encourage JAXA to launch. Chris D: The newest PM sensor is on MetOp-B which is European. Is that something NSIDC could consider? Walt: Most things are looking at 2020 or so to come online. He seems skeptical that Met-Op B would be launched on time because if others are working they might hold it back much like what happened with the F series. Donna: Walt, Donna, and Brian will be talking with scientists about what the future looks like.

e.) USO (Lisa)Key priorities: Building stronger connections, proactive in gathering user needs, creating better user support thru data recipes and help topics.Created a tiered request support. One person now managing request queue. Requests requiring detailed data set knowledge will be escalated to the product specialists. Moving FAQs and data recipes closer to the data, utilizing support tab. Learning how to make video tutorials from Jennifer Brennan. User forums - participating in ESDSWG on the Atm Science user forum. Renaming this to Earth Data User Forum.

ACSI - we do evaklaute the results. Targeted surveys - creating surveys for AGU attendees at booth, creating surveys to take to new mission app workshops, early adopter groups.

Nettie: I thought we couldn't do surveys Amanda: can have optional surveys.Chris L: AI: This is a Jeanne question probably.

Data reviews through testing data and new services.Creating a stronger connection to our user community, and broader communities

19

f.) Mary Jo presented here recent work on MEaSURES passive microwave products titled: "Using image reconstruction to enhance spatial resolution of the satellite passive microwave historical record" Mary Jo is aiming for May 2017 for her Measures product. Produced a python notebook for coordinate system conversion. This product should supersede 4 other NSIDC data products. g.) UWG Membership/next meeting discussion Do we have gaps in membership?

Increased desire for an applications focus.

A member from developing country, perhaps high mtn Himalaya. Could be a U.S. person but works in these countries.

NSIDC scientist

Human dimensions representation and tying those projects into input from the DAAC.

What about a journalist, and some of the educational issues.

Someone tied to NISAR

Early career people. Anne wonders how many users are from developing countries. Bob Chen notes an educator could bring in educational issues? Walt suggests early career Brian: Where do we see value in overlapping with the DAAC. From Jeanne's perspective, get more attendance by HQ. Nice to have in convenient location for the HQ people.Amanda: What about considering joint DAAC effort to help solve pain points related to the issues we heard UWG discuss in their talks. Anne: If do go back to GSFC, then could have contact with Snowex.Chris: When here you DO get DAAC participation. When you go to DC you MIGHT get HQ. Amanda: Depending where you go will drive what you accomplish in the meeting. h.) Review Actions/Recommendations Donna will follow up with Walt about Thorsten’s Melt Data Visibility of data accession information at NSIDC Investigate Mary Jo’s comments about supersede products. Anne notes that Oregon state maintains a “scholars archive” for data. She believes that most universities have that. E.g. CUAHSI – hydrologic information system archives small scale data if it’s hydrology related.