Skip to main content

Elements

A Pipeline Element is the fundamental work item of Nimble. A Pipeline describes an ordered list of Elements and accepts metadata frames from connected Sources. Each pipeline metadata frame is passed through the elements from top to bottom while maintaining order. As metadata is passed from one element to the next, the metadata is processed (and potentially modified) by each element.

Element Initialization​

Each element has a unique name for identification (name) and a set of initialization arguments (args).

name: element-name
args:
  key: value

Deduplication​

Nimble may avoid creating duplicates of the same element as an optimization - this is referred to as "element deduplication". An element's name and args is used to determine whether it's deduplicated or not. The element itself can also override whether it's a candidate at all for deduplication.

Element Configuration​

Some elements support a runtime configuration (config). An element's config is different than its initialization arguments (args); it cannot be used for deduplication purposes since it's not available during element initialization. Multiple configurations can be specified for a single element, up to one for each source. Nimble resolves and associates the configuration to a specific channel and makes the configuration available to the element at runtime.

The format of an element's config is defined in its manifest as a JSON Schema (Draft 7) under the config_schema property. An element's manifest data can be accessed programmatically using the element catalog API endpoint.

REST API Format​

The config property contains a mapping from a source ID to a custom runtime configuration. The JSON format supports an empty string and comma-separated IDs in a single mapping key. An empty string sets the default config for the mapping as a fallback.

{
  "name": "element-name",
  "args": {
    "key": "value"
  },
  "config": {
    "": {
      "key": "value"
    },
    "0": {
      "key": "value0"
    },
    "1,2": {
      "key": "value12"
    }
  }
}

Configuration File Format​

Since the "Configuration File" format requires sources to be directly embedded in the pipeline definition, the config property does not use a key-based mapping.

Instead, the format supports either a single configuration object:

name: element-name
args:
  key: value
config:
  key: value

Or, a list of configuration objects that align with the list of sources for the pipeline:

name: element-name
args:
  key: value
config:
  - key: value0
  - key: value1

Pipeline Examples​

These are complete pipeline examples that use several pipeline elements to construct line-crossing use cases:

With One Source​

note

This example is in the "Configuration File" format. The REST API format will be slightly different.

elements:
  - name: infer
    args:
      model: person-detection-nano
      mode: a
      score_threshold: 0.3
      iou_threshold: 0.3
  - name: object-tracking-OH
    args:
      max_age: 3
      min_hits: 3
      history: 50
      region: bottom
      ratio: 0.2
      moving_average: 3
  - name: line-crossing
    args:
      directions: [-1]
      history: 5
      region: bottom
      ratio: 0.2
    config:
      lines:
        - name: Entrance
          coords:
            - x: 0.4864583
              y: 0.4731481
            - x: 0.3583333
              y: 0.7861111
usecase: line-crossing-example
sources:
  - type: file
    address: video.mp4

With Two Sources​

note

This example is in the REST API format and therefore requires two sources be created beforehand.

File Source: 0
{
  "id": 0,
  "type": "file",
  "address": "video.mp4"
}
Video Source: 1
{
  "id": 1,
  "type": "video",
  "address": "rtsp://10.2.3.4:554/media.amp"
}
Create Pipeline
{
  "elements": [
    {
      "name": "infer",
      "args": {
        "model": "person-detection-nano",
        "mode": "a",
        "iou_threshold": 0.3,
        "score_threshold": 0.3
      }
    },
    {
      "name": "object-tracking-OH",
      "args": {
        "max_age": 3,
        "min_hits": 3,
        "history": 50,
        "region": "bottom",
        "ratio": 0.2,
        "moving_average": 3
      }
    },
    {
      "name": "line-crossing",
      "args": {
        "directions": [-1],
        "history": 5,
        "region": "bottom",
        "ratio": 0.2
      },
      "config": {
        "0": {
          "lines": [
            {
              "name": "Entrance",
              "coords": [
                {"x": 0.4864583, "y": 0.4731481},
                {"x": 0.3583333, "y": 0.7861111}
              ]
            }
          ]
        },
        "1": {
          "lines": [
            {
              "name": "Exit",
              "coords": [
                {"x": 0.451, "y": 0.471},
                {"x": 0.755, "y": 0.467}
              ]
            }
          ]
        }
      }
    }
  ],
  "usecase": "line-crossing-example",
  "sources": [0, 1]
}

Built-in Elements​

Model Inference Element​

The infer element provides access to Nimble's powerful deep learning backend.

name: infer
args:
  model: model-name
  hw: cpu
  mode: a
  score_threshold: 0.3
  iou_threshold: 0.3

See the Models documentation to learn more.

Control Elements​

This section describes the various "control" elements; these are designed to change the fundamental flow of data within the pipeline.

Demux​

The demux element splits a metadata frame of inferences into individual metadata frames for processing one-at-a-time.

This element requires one argument:

  • type: The inference type to demux on. Options: detections, poses, classifications, embeddings
name: demux
args:
  type: detections

Mux​

The mux element takes split metadata frames and reconstructs the original metadata frame. This element is designed to occur after a demux.

This element requires one argument:

  • type: The inference type to mux on. Options: detections, poses, classifications, embeddings
name: mux
args:
  type: detections

Custom Elements​

Follow the Custom Element documentation for a guide on creating your own custom pipeline elements.