yadg: yet another datagram
yadg is a set of tools and parsers aimed to process raw instrument data. Given an experiment represented by a dataschema, yadg will process the files and folders specified in this dataschema, and produce a datagram, which is a unified data structure containing all measured (“raw”) data in a given experiment. The parsers available in yadg are shown in the sidebar. As of yadg-5.0, the datagram is stored as a NetCDF file. The produced datagram is associated with full provenance info, and the data within the datagram contain instrumental error estimates and are annotated with units. You can read more about yadg in our paper: [Kraus2022b].
Some of the yadg parsers are exposed via an extractor interface, allowing the user to extract (meta)-data from individual files without requiring a dataschema. Several file formats are supported, as shown in the sidebar. You can read more about this extractor interface on the MaRDA Metadata Extractors WG website, as well as in the Usage: Extractor mode section of this documentation.
Warning
All of the post-processing features within yadg have been removed in yadg-5.0, following their deprecation in yadg-4.2. If you are looking for a post-processing library, have a look at dgpost instead.
Contributors
Acknowledgements
This project has received funding from the following sources:
European Union’s Horizon 2020 programme under grant agreement ID 957189.
DFG’s Emmy Noether Programme under grant number 490703766.
The project is also part of BATTERY 2030+, the large-scale European research initiative for inventing the sustainable batteries of the future.