Skip to main content

ZeroMQ-based Network Protocol

Defines the formats of output data transmitted using ZeroMQ.

Data Publishing Endpoint (PUB-SUB)​

Nimble uses a data socket to continuously publish real-time video analytics results to subscribers.

  • ZeroMQ socket pattern: PUB
  • Endpoint type: TCP (e.g. tcp://localhost:51983)
  • Port: dynamic (see the Data port property in the config JSON)

Consumers must use a SUB socket to receive video analytics results in the form of JSON metadata and internal performance data of Nimble.

Topics​

There are three data categories: Channel, FPS, and Latency.

Nimble uses "topic prefixes" to specify which data category each sent message belongs to. See ZeroMQ PUB-SUB Topics for how ZeroMQ implements topics and multipart messages.

Multipart Message Formats​

"channel/{id}" {metadata}
"channel/{id}" {metadata} {timestamp}
"channel/{id}" {metadata} {jpeg}
"channel/{id}" {metadata} {timestamp} {jpeg}

"fps" {fps}

"latency" {latency}

Channel topic​

A Channel message is sent when Nimble finishes processing a new video frame.

First, the channel topic is appended using this string format: channel/{id}

{id} is a placeholder for a unique channel ID.

Custom JSON Metadata​

Next, custom JSON metadata is encoded as UTF-8 and appended to the multipart message. The format of this metadata is dependent on the type of the running Use Case and contains inferred analytics data extracted from the video content. This metadata can be used to render overlays on the source video stream.

Important: Any position or size in the metadata is normalized from 0 to 1 relative to the dimensions of the video. 0,0 indicates top-left. For example, if the video dimensions are 1920x1080 pixels, then 0.5,0.2 encodes the position 960,216 in pixels. This allows the video to be transformed without requiring the metadata to be parsed and transformed too.

Media Presentation Timestamp (and/or JPEG image)​

A Channel message can be composed in 4 different ways depending on the static configuration or dynamic runtime state of the channel:

"channel/{id}" {metadata}
"channel/{id}" {metadata} {timestamp}
"channel/{id}" {metadata} {jpeg}
"channel/{id}" {metadata} {timestamp} {jpeg}

If the Channel message only contains 2 parts, then the channel may have a static configuration to only output metadata.

If the Channel message contains exactly 3 parts, then either a media presentation timestamp or a JPEG-encoded image has been appended to the multipart message. If the channel is configured with a non-null Stream ID, then this third part will be a timestamp, otherwise it will be a JPEG-encoded image.

If the Channel message contains 4 parts, then the channel is configured to output both a media timestamp, and a JPEG-encoded image.

Consumers should NOT make assumptions about the format of messages received from the same channel - it is allowed to change at runtime. For example, a Use Case could choose to only append a JPEG-encoded image on a conditional trigger. In this situation, the Channel messages could switch between the 1st and 3rd format, or the 2nd and 4th format.

The media presentation timestamp identifies a processed frame from the input video source. It's relative to the beginning of the video, starts at zero, and is measured in seconds. It's encoded as a JSON number primitive in UTF-8 when it's appended to the multipart message.

FPS and Latency topic​

An FPS and Latency message is sent when Nimble computes a new estimate for these performance metrics.

Both the fps and latency topics use a simple name: fps and latency, respectively.

The fps and latency values are encoded as JSON number primitives in UTF-8 and appended to the multipart message. Latency values are measured in milliseconds.