| YEP: | 107 |
|---|---|
| Title: | Using Apache Avro-RPC |
| Authors: | Kyle Sunden |
| Status: | accepted |
| Tags: | standard |
| Post-History: | 2020-07-02, 2020-07-05, 2020-10-29, 2020-12-09 |
yaq uses Apache Avro to communicate schema information and serialize RPC calls/responses.
Using explicit schemas communicated during the handshake makes type information very concrete and well determined, which we hope will make implementations outside of Python easier.
The primary motivation behind using Avro is to have a standardized RPC format for the yaq ecosystem. Important considerations were having a well defined, flexible type system, support for binary data, and having a self describing protocol format. Specifically, we wanted a description that is explicitly static at time of writing the daemon, which limits dynamic behavior, making client interactions much more predictable. Avro fit that description very well, and its JSON protocol format provides an easy way to add in yaq ecosystem specific behavior in an otherwise valid Avro RPC protocol. We use the Avro protocol definition to provide a full description of the external behavior of the daemon: configuration, RPC calls, and state saving. Additionally documentation is included in the Avro protocol.
The Avro protocol is not tied to any specific implementation of a yaq daemon. We have provided a tool, yaq-traits to make writing the Avro protocol easy.
As we are using an outside specification, we will not copy the full specification here. The full specification can be found at Avro's Website
yaq uses Avro as an RPC over TCP sockets (without http headers) with stateful handshaking supported.
List of string trait names.
This can be used by implementations to validate that expected RPC calls are present. This can also be used by clients to make UI choices.
Each config parameter is a record with the name of the parameter as the key within the config map with the following keys:
type (required)
Avro type definition, commonly a string such as "int" or "ndarray", but may be a table representing collection types or a record or an array representing a union of types.
doc (optional)
A string description of the config parameter
default (optional)
The default value if the config parameter is omitted in the daemon configuration. If no default is given, it is considered required.
A daemon may override the default value defined by a trait (or provide one where none was given).
Additionally, rather then overwriting the doc, a key addendum may be added to provide additional context to the variable
(e.g. communicating that a parameter is required despite a null default value being defined in the trait).
For more information on configuration, see YEP-102.
The state is configured exactly the same as config, except that all state variable must have a default value.
For more information on configuration, see YEP-103.
The links are a map of arbitrary keys to string URLs. In general links to documentation, source, and a bugtracker are good to have. Additionally links to the manufacturer/library used are common.
Installation is just like links, except with the specific goal of pointing to installable package references. For python, this likely includes a link to PyPI and/or conda-forge.
A list of supported hardware, formatted as 'make:model'. Supported hardware should also appear in known-hardware on yaq.fyi Used only for building documentation.
This document is placed in the public domain or under the CC0-1.0-Universal license, whichever is more permissive.
built 2023-10-10 23:52:42 CC0: no copyright