Updating userguide for data streams

Draft for updating the userguide for datastreams.
Url for datastreams are found in the “Managing asset” user guide found here

Added the last part from example and down to end.
I’ll add urls for filter examples too

Anything else that should be added? @paul121

Data Streams

Data Streams are records that represent sets of data streamed from sensors or other real-world devices.


Each Data Stream must have a type. All Asset types have a common set of attributes and relationships. Specific Data Stream types may also add additional attributes and relationships (collectively referred to as “fields”). Data Stream types are defined by modules, and are only available if their module is enabled. The modules included with farmOS define the following Data Stream types:

  • Basic
  • Listener (Legacy)


Each Data Stream will be assigned two unique IDs in the database: a universally unique identifier (UUID), and an internal numeric ID.

The UUID will be unique across farmOS databases. The internal ID will only be unique to a single farmOS database. Therefore, the farmOS API uses UUIDs to ensure that IDs pulled from multiple farmOS databases do not conflict. Internally, farmOS modules use the internal IDs to perform CRUD operations.


Data Streams have a number of attributes that serve to describe their meta information. All Data Stream have the same standard set of attributes. Modules can add additional attributes.

Standard attributes

Attributes that are common to all Data Stream types include:

  • Name
  • Private key
  • Public


Data Streams must have a name that describes them. The name is used in lists of Data Streams to easily identify them at quick glance.

Private key

A Data Stream’s private key is the password used to post data to (and get data from) its API endpoint.


A Data Stream may be marked as “public” to allow read-access from the API without a private key. This is useful for loading data into public third-party apps or scripts, for graphing or other purposes. Data Streams are not public by default.


Data Streams can be related to other records in farmOS These relationships are stored as reference fields on Data Stream records.

All Data Streams have the same standard set of relationships. Modules can add additional relationships.

Relationships that are common to all Data Streams types include: da

  • Assets


Data Streams can reference one or more Assets to indicate that the data they collect is directly relevant to them.

For example, if a soil moisture sensor that is installed in a field can be represented with a Sensor Asset (for the soil moisture sensor device itself), with a Data Stream (soil moisture data readings), that reference a Land Asset (the field it is installed in). This makes it possible for a single device to be moved/reused for monitoring multiple Assets, by moving the Sensor Asset and creating new Data Streams.

Type-specific fields

In addition to the fields that are common to all Data Stream types described above, some types add additional type-specific fields. These include:

Basic Data Streams

Basic Data Streams do not define any type-specific fields.

Listener (Legacy) Data Streams

Listener (Legacy) Data Streams have an additional “public key” attribute, which is used in the Data Stream’s API endpoint for posting/getting data. This was used in farmOS 1.x to provide a unique ID for the sensor, separate from the Sensor Asset ID that housed the data. This is no longer needed in farmOS 2.x, because each Data Stream has its own UUID, which is used in the API endpoints instead. The public key is retained for Legacy (Listener) Data Streams to ensure that existing sensors can continue to push data without needing to be reconfigured.


Once the data stream is created within the sensor asset, you can reveal the developer info and get all the information you need to start posting sensor data. bilde

Data from the stream can also be pulled out of farmOS with a GET request using the same URL. The following filters can be used to limit the amount of data:

Filter Type
start timestamp Filter data to timestamps greater than or equal to this start timestamp.
end timestamp Filter data to timestamps less than or equal to this end timestamp.
name string Filter to data with a matching name.
offset integer The value to start at.
limit integer The number of results to return. See “Limits” below for details.

Filter examples:


The limit parameter can be used to specify how many data points should be returned. Setting limit to 0 will return the maximum number of points, which is capped at 100000 (to prevent memory exhaustion). If more data is desired, multiple requests should be made using the offset parameter. Setting limit to a value greater than the max will result in a response code of 422 Unprocessable Entity: exceeds max limit of 100000.


Thanks for starting this @pat ! Maybe a couple things you could add:

  • Mention the public/private configuration and how to pass the query parameter
  • Describe sending/POST data to the data stream

Also you’re correct, it looks like we only have a data model page for data streams: Data Streams | farmOS

I think what you’re describing might be more of a “user guide” for data streams or maybe “Sensor data” more generally. It would be nice to have a page that explains how to create a sensor asset + data stream, send data to it, and also query data back out. Maybe include a few suggested use-cases to demonstrate how this can be used but without going too technical. What do you think?


I’ll try to add that, and also make a userguide with examples.

1 Like