Developing tomato drivers

As of tomato-1.0, all device drivers are developed as separate Python packages with their own documentation and versioning. To ensure compatibility of the Manager between the tomato-driver process and the implementation of the driver, an abstract class ModelInterface is provided. A class inheriting from this abstract class, with the name DriverManager, has to be available when the selected driver module is imported.

Note

The ModelInteface is versioned. Your driver should target a single version of this Manager by inheriting from only one such abstract class. Any deprecation notices will be provided well in advance directly to driver maintainers. Support for driverManager_1_0 introduced in tomato-1.0 is guaranteed until at least tomato-3.0.

Bootstrapping a driver process

When the driver process is launched (as a tomato-driver), it’s given information about how to connect to the tomato-daemon process and which device driver to spawn. Once a connection to the tomato-daemon is established, the driver settings are fetched, and the DriverManager is instantiated passing any settings to the constructor. Then, all components on all devices of this driver type that are known to tomato-daemon are registered using the dev_register() function.

Note

Each driver creates a separate log file for each port tomato has been executed with. The logfile is stored in the same location as the tomato-daemon logs, i.e. as configured under the jobs.storage option. The verbosity of the tomato-driver process is inherited from the tomato-daemon process.

Communication between jobs and drivers

After the driver process is bootstrapped, it enters the main loop, listening for commands to action or pass to the ModelInterface. Therefore, if a job needs to submit a Task, it passes the Task to the tomato-driver process, which actions it on the appropriate component using the task_submit() function. Similarly, if a job decides to poll the driver for data, it does so using the task_data() function.

In general, methods of the ModelInterface that are prefixed with dev deal with managing devices or their components on the driver, methods prefixed with task deal with managing Tasks running or submitted to components, and methods without a prefix deal with configuration or status of the driver itself.

Note

The ModelInterface contains a sub-class DriverManager. In general, the ModelInterface acts as a pass-through to the (abstract) methods of the DriverManager; e.g. ModelInterface.dev_get_attr() is a passthrough function to the appropriate DriverManager.get_attr().

We expect most of the work in implementing a new driver will actually take place in the DriverManager class.

Currently, when a Task is submitted to a component, a new Thread is launched on that component that takes care of preparing the component (via DriverManager.prepare_task()), executing the Task (via DriverManager.task_runner()), and periodically polling the hardware for data (via the abstract DriverManager.do_task()). As each component can only run one Task at the same time, subsequently submitted Tasks are stored in a task_list, which is a Queue used to communicate with the worker Thread. This worker Thread is reinstantiated at the end of every Task.

Note

Direct access to the DriverManager.data object is not thread-safe. Therefore, a reentrant lock (RLock) object is provided as DriverManager.datalock. Reading or writing to the DriverManager.data with the exception of the get_data() and do_task() methods should be only carried out when this datalock is acquired, e.g. using a context manager.

Note

The DriverManager.data object is intended to cache data between get_data() calls initiated by the job. This object is therefore cleared whenever get_data() is called; it is the responsibility of the tomato-job process to append or store any new data.

To access the status of the component, the DriverManager provides a status() method. The implementation of what is reported as component status (including e.g. returning latest cached datapoint) is up to the developers of each driver.

Best Practices when developing a driver

• We follow the usual Python (PEP-8) convention of _-prefixed methods and attributes considered to be private. However, there is no way to enforce such privacy in Python.

• The DriverManager.attrs() defines the variable attributes of the component that should be accessible, using Attr. All entries in attrs() should be present in DriverManager.data. There should be no entries in data that are not in returned by attrs().

• Each DriverManager contains a link to its parent ModelInterface in the DriverManager.driver object.

• Internal functions of the DriverManager and ModelInterface should be re-used wherever possible. E.g., reading component attributes should always be carried out using get_attr().

ModelInterface ver. 1.0

class tomato.driverinterface_1_0.ModelInterface(settings=None)

An abstract base class specifying the driver interface.

Individual driver modules should expose a DriverInterface which inherits from this abstract class. Only the methods of this class should be used to interact with drivers and their devices.

class DeviceManager(driver, key, **kwargs)

An abstract base class specifying a manager for an individual component.

This class should handle determining attributes and capabilities of the component, the reading/writing of those attributes, processing of tasks, and caching and returning of task data.

driver: ModelInterface

The parent DriverInterface instance.

key: tuple

The key in self.driver.devmap referring to this object.

task_list: Queue

A Queue used to pass Tasks to the worker Thread.

thread: Thread

The worker Thread.

data: dict[str, list]

Container for cached data on this component.

datalock: RLock

Lock object for thread-safe data manipulation.

run()

Helper function for starting the self.thread.

task_runner()

Target function for the self.thread.

This function waits for a Task passed using self.task_list, then handles setting all Attrs using the prepare_task() function, and finally handles the main loop of the task, periodically running the do_task() function (using task.sampling_interval) until the maximum task duration (i.e. task.max_duration) is exceeded.

The self.thread is re-primed for future Tasks at the end of this function.

prepare_task(task: Task, **kwargs: dict)

Given a Task, prepare this component for execution by settin all Attrs as specified in the task.technique_params dictionary.

abstract do_task(task: Task, **kwargs: dict)

Periodically called task execution function.

This function is responsible for updating self.data with new data, i.e. performing the measurement. It should also update the values of all Attrs, so that the component status is consistent with the cached data.

stop_task(**kwargs: dict)

Stops the currently running task.

abstract set_attr(attr: str, val: Any, **kwargs: dict)

Sets the specified Attr to val.

abstract get_attr(attr: str, **kwargs: dict) → Any

Reads the value of the specified Attr.

get_data(**kwargs: dict) → dict[str, list]

Returns the cached self.data before clearing the cache.

abstract attrs() → dict[str, Attr]

Returns a dict of all available Attrs.

abstract capabilities() → set

Returns a set of all supported techniques.

status(**kwargs) → dict

Compiles a status report from Attrs marked as status=True.

reset(**kwargs) → None

Resets the component to an initial status.

CreateDeviceManager(key, **kwargs)

A factory function which is used to pass this instance of the ModelInterface to the new DeviceManager instance.

devmap: dict[tuple, DeviceManager]

Map of registered devices, the tuple keys are component = (address, channel)

settings: dict[str, Any]

A settings map to contain driver-specific settings such as dllpath for BioLogic

dev_register(address: str, channel: str, **kwargs: dict) → Reply

Register a new device component in this driver.

Creates a DeviceManager representing a device component, storing it in the self.devmap using the provided address and channel.

The returned Reply should contain the capabilities of the registered component in the data slot.

dev_teardown(key: tuple, **kwargs: dict) → Reply

Emergency stop function.

Should set the device component into a documented, safe state. The function is to be only called in case of critical errors, or when the component is being removed, not as part of normal operation (i.e. it is not intended as a clean-up after task completion).

dev_reset(key: tuple, **kwargs: dict) → Reply

Component reset function.

Should set the device component into a documented, safe state. This function is executed at the end of every job.

dev_set_attr(attr: str, val: Any, key: tuple, **kwargs: dict) → Reply

Set value of the Attr of the specified device component.

Pass-through to the DeviceManager.set_attr() function. No type or read-write validation performed here!

dev_get_attr(attr: str, key: tuple, **kwargs: dict) → Reply

Get value of the Attr from the specified device component.

Pass-through to the DeviceManager.get_attr() function. Units are not returned; those can be queried for all Attrs using self.attrs().

dev_status(key: tuple, **kwargs: dict) → Reply

Get the status report from the specified device component.

Iterates over all Attrs on the component that have status=True and returns their values in the Reply.data as a dict.

task_start(key: tuple, task: Task, **kwargs) → Reply

Submit a Task onto the specified device component.

Pushes the supplied Task into the Queue of the component, then starts the worker thread. Checks that the Task is among the capabilities of this component.

task_status(key: tuple, **kwargs: dict) → Reply

Returns the task readiness status of the specified device component.

The running entry in the data slot of the Reply indicates whether a Task is running. The can_submit entry indicates whether another Task can be queued onto the device component already.

task_stop(key: tuple, **kwargs) → Reply

Stops a running task and returns any collected data.

Pass-through to DriverManager.stop_task() and task_data().

task_data(key: tuple, **kwargs) → Reply

Return cached task data on the device component and clean the cache.

Pass-through for DeviceManager.get_data(), with the caveat that the dict[list] which is returned from the component is here converted to a Dataset and annotated using units from attrs().

This function gets called by the job thread every device.pollrate, it therefore incurs some IPC cost.

status() → Reply

Returns the driver status. Currently that is the names of the components in the devmap.

reset() → Reply

Resets the driver.

Called when the driver process is quitting. Instructs all remaining tasks to stop. Warns when devices linger. Passes through to dev_reset(). This is not a pass-through to dev_teardown().

capabilities(key: tuple, **kwargs) → Reply

Returns the capabilities of the device component.

Pass-through to DriverManager.capabilities().

attrs(key: tuple, **kwargs: dict) → Reply

Query available Attrs on the specified device component.

Pass-through to the DeviceManager.attrs() function.