Developing tomato drivers

Since tomato-1.0, all device drivers are developed as separate Python packages with their own documentation and versioning. To ensure compatibility between the tomato-driver process (implemented in the tomato.daemon.driver module) and the functionality of the driver (implemented in the driver packages), a set of abstract classes is provided by tomato in versioned modules, see supported driver interface modules.

Note

The tomato.driverinterface_X_Y modules are versioned, with X being the major and Y the minor version of the corresponding tomato release. Your driver should target a single version of this interface. The version of your driver should match the version of the driver interface, with a point release appended, if necessary:

Warning

The tomato.driverinterface_2_1 has been deprecated in tomato-3.0 and will be removed in the next release.

Note

  • Support for driverinterface_2_0 was removed in tomato-3.0.

  • Support for driverinterface_1_0 was removed in tomato-2.2.

Driver-specific settings

There are two ways to specify driver-specific settings:

  • The Settings class at the top level of the driver package. This class should inherit from a tomato.driverinterface_X_Y.Settings class (i.e. in the driver interface module). It describes the type and purpose of each setting, and can be used to supply a default value of each setting.

  • The drivers section of the settings file can be used to supply user-specified overrides to the above settings, or any settings required to instantiate the driver, e.g. drivers.picovna.dllpath in the tomato-picovna driver.

The following settings are defined by the tomato.driverinterface_3_0.Settings class and are therefore reserved for use by tomato:

  • idle_measurement_interval: Specifies the interval (in seconds) after which a measurement is triggered on each component registered on the driver. This is done by calling the cmp_measure() function, which checks that components are idle, i.e. without a running Task. Defaults to None, meaning that no idle measurements will be performed.

  • lpp_timeout: Specifies the timeout (in seconds) after which a communication attempt with this driver will be aborted. When unset, defaults to tomato.daemon.lpp.REQ_TIMEOUT (in seconds).

The driver-specific settings are validated against the respective Settings classes whenever the settings file is parsed (i.e. during tomato start or tomato reload).

Bootstrapping a driver process

When the driver process is launched (as a tomato-driver), it’s given information about how to connect to the tomato-daemon process and which device driver to spawn. Once a connection to the tomato-daemon is established, the driver-specific settings discussed above are fetched, and the DriverInterface is instantiated passing any settings to the constructor. These settings are then stored locally as a Settings object under DriverInterface.settings.

Note

Each tomato-driver process creates a separate log file for each port tomato has been executed with. The logfile is stored in the same location as the tomato-daemon logs, i.e. as configured under the logdir option in the settings file. The verbosity of the tomato-driver process is inherited from the tomato-daemon process.

Finally, all components on all devices of each driver type, as defined in devices file, are registered via tomato_driver_bootstrap() function, calling the DriverInterface.cmp_register() function. In simpler drivers, this means a Component class (which should inherit from the ModelComponent abstract class) is instantiated by the DriverInterface. The individual Components are stored in DriverInterface.devmap; each Component is also given a reference to the parent DriverInterface object, so that per-driver objects stored on the DriverInterface (Python modules, serial ports, etc.) can also be accessed.

Communication between jobs and drivers

After the driver process is bootstrapped, it enters the main loop, listening for commands to action or pass to the DriverInterface on its port. Therefore, if a job needs to submit a Task, it passes the Task to the tomato-driver process, which actions it on the appropriate component using the task_submit() function. Similarly, if a job decides to poll the driver for data, it does so using the task_data() function.

Warning

The processing of the commands received via the driver port is within a try - except block. Any Exceptions raised by the DriverInterface are caught, logged in the per-driver log, and otherwise ignored.

A Reply with success=False is returned back to the command source (i.e. tomato-daemon, tomato-job, or functions from tomato.passata module).

In general, methods of the ModelInterface class that are prefixed with cmp_* deal with managing device components on the driver, methods prefixed with task_* deal with managing Tasks running or submitted to these components, and methods without a prefix deal with configuration or status of the driver itself.

Note

In general, the DriverInterface(ModelInterface), that you have to implement, acts as a pass-through to the (abstract) methods of the Component(ModelComponent) tbat you also have to implement. For example:

  • ModelInterface.cmp_get_attr(name, ...) calls ModelComponent.get_attr() on component name;

  • ModelInterface.task_stop(name, ...) calls ModelComponent.stop_task() and ModelComponent.get_data() on component name.

We expect most of the work in implementing a new driver will actually take place in implementing the Component(ModelComponent) class.

Task handling

When a new Component(ModelComponent) is instantiated, e.g. when a component is registered on a driver, a new Thread is launched on that component (stored under ModelComponent.thread) running the ModelComponent.task_runner() function. When a Task is submitted to this component, that Thread takes care of:

  • preparing the component by setting any ModelComponent.attrs to the appropriate values via ModelComponent.prepare_task(),

  • managing the Task via ModelComponent.task_runner(),

  • periodically polling the hardware for data using this Thread via ModelComponent.do_task(),

  • storing the last measured datapoint under ModelComponent.last_data via the abstract ModelComponent.do_measure(),

  • concatenating any new data into ModelComponent.data via ModelComponent.do_task().

Tip

Idle measurements, triggered by the tomato-driver process, as well as measurements requested via ModelInterface.cmp_measure() use the same Thread, but submit a special “task” that does not touch ModelComponent.attrs.

As each component can only run one Task at the same time, subsequently submitted Tasks are stored in an ModelComponent.task_list, which is a Queue used to communicate between the main thread of the driver and the Task worker Thread.

Warning

Direct access to the ModelComponent.data object is not thread-safe. Therefore, a reentrant lock (RLock object) is provided under ModelComponent.datalock. Reading or writing to the ModelComponent.data object should be only carried out within the ModelComponent.get_data() and ModelComponent.do_task() methods, which use the datalock.

If you need to touch the ModelComponent.data object outside these two methods, you should acquire the datalock first, e.g. using a context manager.

Note

The ModelComponent.data object is intended to cache running Task data between ModelComponent.get_data() calls initiated by a job. This object is therefore cleared whenever ModelComponent.get_data() is called; it is the responsibility of the tomato-job process to append or store any new Task data.

Driver and component status

To access the current status of the driver, the ModelInterface class provides a ModelInterface.status() function, which returns the names of the registered components when successful.

To access the current status of a component, the ModelInterface.cmp_status() function can be used, calling the ModelComponent.status() method. This function returns a Status class, indicating connection state, component state, values of selected attrs, and any running task.

To access the latest data of the component, the ModelInterface.cmp_last_data() function can be used, calling ModelComponent.get_last_data() method.

Best practices when developing a driver

  • The ModelComponent.attrs() defines the variable attributes of the component that should be accessible, using Attr. All entries in attrs() should also be present in ModelDevice.data, as the data likely depends on the settings of these Attrs.

  • Internal functions of the ModelComponent and ModelInterface should be re-used wherever possible. E.g., reading component attributes should always be carried out using ModelComponent.get_attr().

Supported driver interface modules