xrdtrace: X-ray diffractogram parser

This module handles the reading and processing of X-ray diffraction data.

Usage

The usage of xrdtrace can be specified by supplying xrdtrace as an argument to the parser keyword of the dataschema. The parser supports the following parameters:

pydantic model dgbowl_schemas.yadg.dataschema_4_1.step.XRDTrace.Params

Show JSON schema
{
   "title": "Params",
   "type": "object",
   "properties": {
      "filetype": {
         "title": "Filetype",
         "default": "panalytical.csv",
         "enum": [
            "panalytical.xy",
            "panalytical.csv",
            "panalytical.xrdml"
         ],
         "type": "string"
      }
   },
   "additionalProperties": false
}

field filetype: Literal['panalytical.xy', 'panalytical.csv', 'panalytical.xrdml'] = 'panalytical.csv'

Formats

The currently supported file formats are:

Provides

xrdtrace loads X-ray diffraction data, determines reasonable uncertainties of the signal intensity (y-axis), and explicitly populates the angle axis (\(2\theta\)), if necessary. This raw data is stored, for each timestep, using the following format:

- raw:
    traces:
      "{{ trace_number }}":   # number of the trace
        angle:
          {n: [!!float, ...], s: [!!float, ...], u: "deg"}
        intensity:
          {n: [!!float, ...], s: [!!float, ...], u: "counts"}

The uncertainties "s" in "angle" are taken as the step-width of the linearly spaced \(2\theta\) values.

The uncertainties "s" of "intensity" are currently set to a constant value of 1.0 count as all the supported files seem to produce integer values.

Metadata

Warning

The metadata collected from the raw data file depends on the filetype, and is currently not strictly defined.

Submodules

yadg.parsers.xrdtrace.common.panalytical_comment(line)

Processes a comments from the file header into a dictionary.

Parameters

line (str) – A line containing the comment.

Returns

A dictionary containing the processed comment.

Return type

dict

yadg.parsers.xrdtrace.common.snake_case(s)

Converts Sentence case. and camelCase strings to snake_case.

From https://stackoverflow.com/a/1176023

Parameters

s (str) – The input string to be converted.

Returns

The corresponding snake_case string.

Return type

str

yadg.parsers.xrdtrace.main.process(fn, encoding='utf-8', timezone='UTC', parameters=None)

Unified X-ray diffractogram data parser.

This parser processes X-ray diffractogram scans in intensity(angle) format.

Parameters

  • fn (str) – The file containing the trace(s) to parse.

  • encoding (str) – Encoding of fn, by default “utf-8”.

  • timezone (str) – A string description of the timezone. Default is “UTC”.

  • parameters (Optional[BaseModel]) – Parameters for XRDTrace.

Returns

(data, metadata, fulldate) – Tuple containing the timesteps, metadata, and full date tag. The currently implemented parsers all return full date.

Return type

tuple[list, dict, bool]

panalyticalcsv: Processing of PANalytical XRD csv files

File Structure

These files are split into a [Measurement conditions] and a [Scan points] section. The former stores the metadata and the latter all the datapoints.

Warning

This parser is fairly new and untested. As a result, the returned metadata contain all the entries in the [Measurement conditions] section, without any additional filtering.

Structure of Parsed Timesteps

- fn:  !!str
- uts: !!float
- raw:
    traces:
      "{{ trace_number }}":  # Number of the trace.
        angle:               # Diffraction angle.
          {n: [!!float, ...], s: [!!float, ...], u: "deg"}
        intensity:           # Detector counts.
          {n: [!!float, ...], s: [!!float, ...], u: "counts"}
yadg.parsers.xrdtrace.panalyticalcsv.process(fn, encoding='utf-8', timezone='UTC')

Processes a PANalytical XRD csv file. All information contained in the header of the csv file is stored in the metadata.

Parameters

  • fn (str) – The file containing the trace(s) to parse.

  • encoding (str) – Encoding of fn, by default “utf-8”.

  • timezone (str) – A string description of the timezone. Default is “UTC”.

Returns

(data, meta) – (data, metadata, fulldate) : tuple[list, dict, bool] Tuple containing the timesteps, metadata, and the full date tag. For .csv files tag is specified.

Return type

tuple[list, dict]

panalyticalxrdml: Processing of PANalytical XRD xml files

File Structure

These are xml-formatted files, which we here parse using the xml.etree library into a Python dict.

Note

The angle returned from this parser is based on a linear interpolation of the start and end point of the scan, and is the \(2\theta\). The values of \(\omega\) are discarded.

Warning

This parser is fairly new and untested. As a result, the returned metadata contain only a subset of the available metadata in the XML file. If something important is missing, please contact us!

Structure of Parsed Timesteps

- fn:  !!str
- uts: !!float
- raw:
    traces:
      "{{ trace_number }}":  # Number of the trace.
        angle:               # Diffraction angle.
          {n: [!!float, ...], s: [!!float, ...], u: "deg"}
        intensity:           # Detector counts.
          {n: [!!float, ...], s: [!!float, ...], u: "counts"}
yadg.parsers.xrdtrace.panalyticalxrdml.etree_to_dict(e)

Recursively converts an ElementTree.Element into a dictionary.

Element attributes are stored into “@”-prefixed attribute keys. Element text is stored into “#text” for all nodes.

From https://stackoverflow.com/a/10076823.

Parameters

e (Element) – The ElementTree root Element.

Returns

ElementTree parsed into a dictionary.

Return type

dict

yadg.parsers.xrdtrace.panalyticalxrdml.process(fn, encoding='utf-8', timezone='UTC')

Processes a PANalytical xrdml file.

Parameters

  • fn (str) – The file containing the trace(s) to parse.

  • encoding (str) – Encoding of fn, by default “utf-8”.

  • timezone (str) – A string description of the timezone. Default is “UTC”.

Returns

(data, metadata, fulldate) – Tuple containing the timesteps, metadata, and the full date tag. For .xrdml tag is always specified

Return type

tuple[list, dict, bool]

panalyticalxy: Processing of PANalytical XRD xy files

File Structure

These files basically just contain the [Scan points] part of PANalytical csv files yadg.parsers.xrdtrace.panalyticalcsv. As a consequence, no metadata is recorded, and the format does not have an associated timestamp.

Structure of Parsed Timesteps

- fn:  !!str
- uts: !!float
- raw:
    traces:
      "{{ trace_number }}":  # Number of the trace.
        angle:               # Diffraction angle.
          {n: [!!float, ...], s: [!!float, ...], u: "deg"}
        intensity:           # Detector counts.
          {n: [!!float, ...], s: [!!float, ...], u: "counts"}
yadg.parsers.xrdtrace.panalyticalxy.process(fn, encoding='utf-8', timezone='UTC')

Processes a PANalytical XRD xy file.

Parameters

  • fn (str) – The file containing the trace(s) to parse.

  • encoding (str) – Encoding of fn, by default “utf-8”.

  • timezone (str) – A string description of the timezone. Default is “UTC”.

Returns

(data, metadata, fulldate) – Tuple containing the timesteps, metadata, and the full date tag. For .xy files tag is never specified.

Return type

tuple[list, dict, bool]