|
Dataset Persistent ID
|
doi:10.26165/JUELICH-DATA/GRXBJX |
|
Publication Date
|
2026-05-02 |
|
Title
|
Evolving Scientific Software in Long-Running Observatories: Lessons from the TERENO Sensor Management Migration
|
|
Author
|
Loup, Ulrich (Forschungszentrum Jülich GmbH) - ORCID: 0009-0005-1370-6226
Küpper, Werner (Forschungszentrum Jülich GmbH)
Lorenz, Christof (Karlsruhe Institute of Technology) - ORCID: 0000-0001-5590-5470
Gasche, Rainer (Karlsruhe Institute of Technology)
Kunkel, Ralf (Forschungszentrum Jülich GmbH) - ORCID: 0000-0003-2022-0011
Gründling, Ralf (Helmholtz Centre for Environmental Research)
Groh, Jannis (Forschungszentrum Jülich GmbH) - ORCID: 0000-0002-1681-2850
Brinckmann, Nils (GFZ Helmholtz Centre for Geosciences) - ORCID: 0000-0001-8159-3888
Buberger, Jan (Helmholtz Centre for Environmental Research) - ORCID: 0000-0003-3780-8663
Hanisch, Marc (GFZ Helmholtz Centre for Geosciences) - ORCID: 0000-0001-5272-4674
Kuhnert, Tobias (Helmholtz Centre for Environmental Research) - ORCID: 0009-0002-3854-3417
Moorthy, Rubankumar (Forschungszentrum Jülich GmbH)
Obersteiner, Florian (Karlsruhe Institute of Technology) - ORCID: 0000-0002-7327-8893
Schäfer, David (Helmholtz Centre for Environmental Research) - ORCID: 0000-0003-4517-6459
Schnicke, Thomas (Helmholtz Centre for Environmental Research) - ORCID: 0000-0001-9868-4671
|
|
Contact
|
Use email button above to contact.
Loup, Ulrich (Forschungszentrum Jülich)
|
|
Description
|
Scientific software in the geosciences often evolves organically, growing from small, purpose-built tools into long-lived infrastructure. Using the example of the TERENO observatories and their integration into the Earth & Environment DataHub, this contribution reflects on the challenges and opportunities of transforming legacy systems into sustainable, community-ready platforms. A central task was the migration of sensor metadata from a tightly coupled legacy system to the Helmholtz Sensor Management System, requiring a clear separation of data and metadata, adaptation to new API-based technologies, and reconciliation of differing data models and vocabularies. The scale (thousands of devices and tens of thousands of parameters) and the presence of heterogeneous, project-specific edge cases further complicated the process. To address these challenges, we developed the Python-based ODM2SMS tool, enabling configurable, incremental, and reversible migration workflows. Automated processing was complemented by validation routines, staged testing environments, and targeted manual inspection to ensure scientific plausibility. The approach emphasized modularity, explicit documentation, and the enrichment of source data to improve migration quality. From this experience, we derive practical lessons for sustainable scientific software development: prioritize conceptual design over rapid prototyping, involve users early, and avoid tightly coupled database designs. More broadly, the work highlights how incremental refactoring, open tools, and community engagement can turn organically grown systems into robust, reusable infrastructure for future research.
|
|
Subject
|
Earth and Environmental Sciences
|
|
Related Publication
|
Loup, U., Küpper, W., Lorenz, C., Gasche, R., Kunkel, R., Gründling, R., Groh, J., Brinckmann, N., Bumberger, J., Hanisch, M., Kuhnert, T., Moorthy, R., Obersteiner, F., Schäfer, D., and Schnicke, T.: Evolving Scientific Software in Long-Running Observatories: Lessons from the TERENO Sensor Management Migration, EGU General Assembly 2026, Vienna, Austria, 3–8 May 2026, EGU26-23282, https://doi.org/10.5194/egusphere-egu26-23282, 2026. doi: 10.5194/egusphere-egu26-23282 https://doi.org/10.5194/egusphere-egu26-23282
|
|
Depositor
|
Loup, Ulrich
|
|
Deposit Date
|
2026-05-02
|