meascsv: Legacy MCPT log file parser
This parser handles the reading and processing of the legacy log files created by the LabView interface for the MCPT instrument. These files contain information about the timestamp, temperatures, and inlet / process flows.
DEPRECATED in yadg-4.0
As of yadg-4.0, this parser is deprecated and should not be used for new data.
Please consider switching to the basiccsv parser.
Usage
Available since yadg-3.0. Deprecated since yadg-4.0. The parser supports the
following parameters:
- pydantic model dgbowl_schemas.yadg.dataschema_5_0.step.MeasCSV
Legacy file parser for
measurement.csvfiles from FHI.Note
This parser is deprecated, and the
BasicCSVparser should be used instead.Show JSON schema
{ "title": "MeasCSV", "description": "Legacy file parser for ``measurement.csv`` files from FHI.\n\n.. note::\n\n This parser is deprecated, and the :class:`BasicCSV` parser should be\n used instead.", "type": "object", "properties": { "tag": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Tag" }, "parser": { "const": "meascsv", "title": "Parser" }, "input": { "$ref": "#/$defs/Input" }, "extractor": { "$ref": "#/$defs/NoFileType" }, "parameters": { "$ref": "#/$defs/Parameters" }, "externaldate": { "anyOf": [ { "$ref": "#/$defs/ExternalDate" }, { "type": "null" } ], "default": null } }, "$defs": { "ExternalDate": { "additionalProperties": false, "description": "Supply timestamping information that are external to the processed file.", "properties": { "using": { "anyOf": [ { "$ref": "#/$defs/ExternalDateFile" }, { "$ref": "#/$defs/ExternalDateFilename" }, { "$ref": "#/$defs/ExternalDateISOString" }, { "$ref": "#/$defs/ExternalDateUTSOffset" } ], "title": "Using" }, "mode": { "default": "add", "enum": [ "add", "replace" ], "title": "Mode", "type": "string" } }, "required": [ "using" ], "title": "ExternalDate", "type": "object" }, "ExternalDateFile": { "additionalProperties": false, "description": "Read external date information from file.", "properties": { "file": { "$ref": "#/$defs/dgbowl_schemas__yadg__dataschema_5_0__externaldate__ExternalDateFile__Content" } }, "required": [ "file" ], "title": "ExternalDateFile", "type": "object" }, "ExternalDateFilename": { "additionalProperties": false, "description": "Read external date information from the file name.", "properties": { "filename": { "$ref": "#/$defs/dgbowl_schemas__yadg__dataschema_5_0__externaldate__ExternalDateFilename__Content" } }, "required": [ "filename" ], "title": "ExternalDateFilename", "type": "object" }, "ExternalDateISOString": { "additionalProperties": false, "description": "Read a constant external date using an ISO-formatted string.", "properties": { "isostring": { "title": "Isostring", "type": "string" } }, "required": [ "isostring" ], "title": "ExternalDateISOString", "type": "object" }, "ExternalDateUTSOffset": { "additionalProperties": false, "description": "Read a constant external date using a Unix timestamp offset.", "properties": { "utsoffset": { "title": "Utsoffset", "type": "number" } }, "required": [ "utsoffset" ], "title": "ExternalDateUTSOffset", "type": "object" }, "Input": { "additionalProperties": false, "description": "Specification of input files/folders to be processed by the :class:`Step`.", "properties": { "folders": { "items": { "type": "string" }, "title": "Folders", "type": "array" }, "prefix": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Prefix" }, "suffix": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Suffix" }, "contains": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Contains" }, "exclude": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Exclude" } }, "required": [ "folders" ], "title": "Input", "type": "object" }, "NoFileType": { "additionalProperties": false, "properties": { "filetype": { "const": "None", "default": "None", "title": "Filetype" }, "timezone": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Timezone" }, "locale": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Locale" }, "encoding": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Encoding" } }, "title": "NoFileType", "type": "object" }, "Parameters": { "additionalProperties": false, "properties": { "timestamp": { "anyOf": [ { "$ref": "#/$defs/Timestamp" }, { "$ref": "#/$defs/TimeDate" }, { "$ref": "#/$defs/UTS" } ], "default": { "timestamp": { "format": "%Y-%m-%d-%H-%M-%S", "index": 0 } }, "title": "Timestamp" } }, "title": "Parameters", "type": "object" }, "TimeDate": { "additionalProperties": false, "description": "Timestamp from a separate date and/or time column.", "properties": { "date": { "anyOf": [ { "$ref": "#/$defs/TimestampSpec" }, { "type": "null" } ], "default": null }, "time": { "anyOf": [ { "$ref": "#/$defs/TimestampSpec" }, { "type": "null" } ], "default": null } }, "title": "TimeDate", "type": "object" }, "Timestamp": { "additionalProperties": false, "description": "Timestamp from a column containing a single timestamp string.", "properties": { "timestamp": { "$ref": "#/$defs/TimestampSpec" } }, "required": [ "timestamp" ], "title": "Timestamp", "type": "object" }, "TimestampSpec": { "additionalProperties": false, "description": "Specification of the column index and string format of the timestamp.", "properties": { "index": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Index" }, "format": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Format" } }, "title": "TimestampSpec", "type": "object" }, "UTS": { "additionalProperties": false, "description": "Timestamp from a column containing a Unix timestamp.", "properties": { "uts": { "$ref": "#/$defs/TimestampSpec" } }, "required": [ "uts" ], "title": "UTS", "type": "object" }, "dgbowl_schemas__yadg__dataschema_5_0__externaldate__ExternalDateFile__Content": { "additionalProperties": false, "properties": { "path": { "title": "Path", "type": "string" }, "type": { "title": "Type", "type": "string" }, "match": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Match" } }, "required": [ "path", "type" ], "title": "Content", "type": "object" }, "dgbowl_schemas__yadg__dataschema_5_0__externaldate__ExternalDateFilename__Content": { "additionalProperties": false, "properties": { "format": { "title": "Format", "type": "string" }, "len": { "title": "Len", "type": "integer" } }, "required": [ "format", "len" ], "title": "Content", "type": "object" } }, "additionalProperties": false, "required": [ "parser", "input" ] }
- Config:
extra: str = forbid
- pydantic model Parameters
Show JSON schema
{ "title": "Parameters", "type": "object", "properties": { "timestamp": { "anyOf": [ { "$ref": "#/$defs/Timestamp" }, { "$ref": "#/$defs/TimeDate" }, { "$ref": "#/$defs/UTS" } ], "default": { "timestamp": { "format": "%Y-%m-%d-%H-%M-%S", "index": 0 } }, "title": "Timestamp" } }, "$defs": { "TimeDate": { "additionalProperties": false, "description": "Timestamp from a separate date and/or time column.", "properties": { "date": { "anyOf": [ { "$ref": "#/$defs/TimestampSpec" }, { "type": "null" } ], "default": null }, "time": { "anyOf": [ { "$ref": "#/$defs/TimestampSpec" }, { "type": "null" } ], "default": null } }, "title": "TimeDate", "type": "object" }, "Timestamp": { "additionalProperties": false, "description": "Timestamp from a column containing a single timestamp string.", "properties": { "timestamp": { "$ref": "#/$defs/TimestampSpec" } }, "required": [ "timestamp" ], "title": "Timestamp", "type": "object" }, "TimestampSpec": { "additionalProperties": false, "description": "Specification of the column index and string format of the timestamp.", "properties": { "index": { "anyOf": [ { "type": "integer" }, { "type": "null" } ], "default": null, "title": "Index" }, "format": { "anyOf": [ { "type": "string" }, { "type": "null" } ], "default": null, "title": "Format" } }, "title": "TimestampSpec", "type": "object" }, "UTS": { "additionalProperties": false, "description": "Timestamp from a column containing a Unix timestamp.", "properties": { "uts": { "$ref": "#/$defs/TimestampSpec" } }, "required": [ "uts" ], "title": "UTS", "type": "object" } }, "additionalProperties": false }
- Config:
extra: str = forbid
- field parser: Literal['meascsv'] [Required]
- field parameters: Parameters [Optional]
- field extractor: NoFileType [Optional]
Schema
The parser is used to extract all of the tabular data in the input file, storing
them in the same format as basiccsv, using the column headers
as keys.
- yadg.parsers.meascsv.process(*, fn, encoding, timezone, parameters, **kwargs)
Legacy MCPT measurement log parser.
This parser is included to maintain parity with older schemas and datagrams. It is essentially a wrapper around
yadg.parsers.basiccsv.main.process_row().DEPRECATED in
yadg-4.0For new applications, please use the
basiccsvparser.- Parameters:
fn (
str) – File to processencoding (
str) – Encoding offn, by default “utf-8”.timezone (
ZoneInfo) – A string description of the timezone. Default is “localtime”.parameters (
BaseModel) – Parameters forMeasCSV.
- Returns:
A
xarray.Datasetcontaining the timesteps, metadata, and full date tag. No metadata is returned. The full date is always provided inmeascsvcompatible files.- Return type: