Changes in salobj 4: the dds version

Data Management DM Notifications
#1

ts_salobj 4 is coming. This is a complete rewrite that uses OpenSplice dds libraries directly, instead of using SALPY libraries generated by ts_sal. This posting is intended to give an overview of what changed and some help with converting code.

Overview of changes:

  • salobj 4 uses OpenSplice dds libraries directly, rather than SALPY libraries. salobj 4 gets its topic information from IDL files, which are generated by ts_sal.
  • More topic fields are visible than with SALPY. All but one of these field names start with the prefix “private_” and salobj sets the values of these fields automatically. The fields are:
    • component_nameId: the SAL index of the component. Example field names include TestId and ATPtgId.
    • private_seqNum: the command ID. This is generated by the sender (e.g. salobj.RemoteCommand) and copied by the receiver (e.g. salobj.ControllerCommand) to the ackcmd samples it sends in response.
    • private_rcvStamp: the time the sample was received
    • private_sndStamp: The time the sample was sent
    • private_host: an integer specified by environment variable LSST_DDS_IP; intended to be the sender’s IP address
    • private_origin: the process ID of the sender
  • Command acknowledgement uses the topic “ackcmd”.
    • There is only one ackcmd topic per SAL component, and it is used for (shared by) all commands for that component. SAL differentiates between different commands by using a different range of private_seqNum for each command.
    • The field private_seqNum is the value salobj formerly called “command ID” or “cmd_id”.
    • Familiar fields include “ack”, “error”, “reason”
  • DDS code should use a single “domain participant” per process; this saves time and memory, as the domain participant contains the cache of topic data. salobj encapsulates the domain participant in a new class Domain.
  • I have changed salobj to expose as much of DDS as I felt we could manage.

Backward Incompatible Changes:

  • Remote's constructor no longer blocks until historical data is read. Instead it has a new attribute start_task that is set done when historical data is read and the remote is reading new data.
  • The CommandIdData class is gone because command ID is is available directly in the data as field private_seqNum. do_command methods and callbacks now simply receive the command data (as specified by the IDL, with the additional formerly hidden fields mentioned in the introduction). To update your code:
    • Change id_data.data to data
    • Change id_data.cmd_id to data.private_seqNum (rare)
    • Change id_data to data
    • If you have unit tests that create CommandIdData instances, just make the data instead.
  • The CommandIdAck class is gone. RemoteCommand.start and next_ackcmd both receive ackcmd topic data. To update your code:
    • Change id_ack.cmd_id to ackcmd.private_seqNum
    • Change id_ack.ack to ackcmd
    • Change id_ack to ackcmd
  • The command acknowledgement object is now called ackcmd instead of ack,
    to match the topic name and to eliminate the confusion of ack.ack.
  • The AckError object now has just one field: ackcmd, instead of two fields cmd_id and ack.
    • Change code that raises AckError (rare)
    • Change access of error fields from .ack to .ackcmd and .cmd_id to .ackcmd.private_seqNum.
  • SalInfo.makeAck is now SalInfo.makeAckCmd and it has one new required parameter: private_seqNum.
  • If your CSC has method start it now takes no arguments, because BaseCsc.start now accepts no arguments; you no longer pass in initial_simulation_mode.
  • If your CSC implements stop, change it to close_tasks and in that method call await super().close_tasks() and close your own extra tasks. BaseCsc.stop has been replaced by BaseCsc.close which calls BaseCsc.close_tasks and close_tasks is normally all a CSC will have to override.
  • salobj now uses IDL files instead of SALPY libraries. These files are built by ts_sal and kept in new package ts_idl. To make the IDL files using make_idl_files.py for example make_idl_files.py Test Script ScriptQueue.
    We plan to have ts_sal generate modules for the enums found in the XML, but that is not yet supported.
  • Make ts_idl a dependency of your package, instead of ts_sal
  • Read topics (RemoteEvent, RemoteTelemetry, ControllerCommand) do not have the data property; use get instead.
  • You cannot set an array field to a scalar; with SALPY salobj this had the effect of setting every element
    to the specified value, but it will raise an exception in dds salobj when sending the data.
  • If you try to send an array with too many values the extra items are silently ignored.
  • Construct a Controller with (name, index) instead of (SALPY_name, index).
  • Construct a Remote with (domain, name, index) instead of (SALPY_name, index). Get domain from controller.domain or make one explicitly. I made this change because best practice is to have only one dds domain participant per process.
  • Shutdown requires more care, especially in unit tests. Everything is cleaned up eventually
    if you just shut down the process, but it takes some time, and if too many things are being cleaned up at once in a unit test you may start running out of resources. Both Controller and Domain are async context managers, so you can use them as follows:
async with Controller(...) as controller: 
    # or async with Domain() as domain:
    # ...do stuff here
  • SalInfo.start() must be awaited after all topics have been created. Controller and Remote do this automatically, but it is important for code that has neither, such as some unit tests and Jupyter scripts. If you forget then read topics will not read any data.
  • The Logger class is gone; its functionality has been incorporated into the Controller class.
  • CSCs and Scripts take several seconds to close, which is slower than it used to be. This may require small changes to existing unit tests.

Other changes:

  • New method ControllerCommand.set_start allows you to specify command data and start a command using one call.
  • The default log level is now INFO