breakout room 4

[alee]

Shankar Sankaran, Isaac Ullah, Bruce Edmonds, Quillon Harpham, Ross Hammond

Bruce: Open Source should be the norm - it's not always possible though
Isaac: MatLab script can be open source, but the platform is not
Quillon: Look to the Essawy et al paper in IEMSS on a taxonomy for reproducible and replicable research in environmental modeling

Reusable black box model vs reusable library component

It's rare to use a model or model component for the exact same purpose

Reuse as a library is facilitated by clean code that meets community norms

Ross: describe incentives - less an imposition than a good practice that has internal benefits for a team as well that provides institutional knowledge

[cpritcha]

Interoperability

• models should include synthetic datasets for easy running and testing
• Bruce - models often have to be adjusted to be useful (often need to be able to change the model itself)

Reusability

• Quillon
  • https://doi.org/10.1016/j.envsoft.2020.104810
  • A taxonomy for reproducible and replicable research in environmental modeling
• Isaac - should there be clean code standards?
  • Allen - mostly out of scope but may be covered in peer review process
• Isaac - should we support somewhat proprietary platforms with open code (like Matlab)?

Documentation

Ideal Documentation Standards

• Quillion
  • Towards standard metadata to support models and interfaces in a hydro-meteorological model chain
  • A simple taxonomy for describing the spatio-temporal structure of environmental modelling data
• Make use of relevant existing metadata standards like
  • https://schema.org
  • https://dublincore.org/ (more suitable for accessibility/findability than documentation)
• Isaac: make use of tools like doxygen for documentation
• Ross: Need separate documentation of model, code, experiments

Accessibility

• Repositories curating model source code should make that code discoverable by humans and machine search algorithms.
  • should also include metadata here for search
• Scope
  • do we include conceptual models?
    • Marco and Michael - no we'll focus on computational models
• Isaac - take existing library search as inspiration - subject index, keyswords
• Marco - accessibility requires open source software if you want to containerize the model
• Isaac and Marco - facilitate model mashup videos and learning resources (like StatQuest)
• Isaac - reduce platform dependency
  • possible solutions
    • web services
    • containerization
    • static linking or fixed rpaths (like go or nix)
• Marco - training for students and later career scholars for making work available
• Marco - need accessibility checklist
• Marco - standards need to be future proof

[platipodium]

models should include synthetic datasets for easy running and testing

Software systems such as CESM already do this, they usually have three representations of a component

1. The scientific model code
2. The (dead) data stub replacing the scientific model
3. The clean API of the model code

[platipodium]

Isaac - should we support somewhat proprietary platforms with open code (like Matlab)?

@isaacullah I think we should for both

• including legacy models
• nudging
  reasons

[platipodium]

Reusable black box model vs reusable library component

No difference.

In an ideal world, I would like to be able to use my fully open source code in the same way as a library component as a totally obscure black box model: I use the exposed and documented API