Orchestrating CGs by Kubernetes

The NR1™ AI Inference Appliance is designed with Kubernetes integration in mind. Discover how to orchestrate Compute Graphs (CGs) using Kubernetes, the leading container orchestration platform.

Compute Graphs (CGs) are exposed to Kubernetes via CG pods. CG pods are a kind of sidecar to the CG that runs on an NR1™ AI Inference Module. Unlike the sidecar container pattern, where a sidecar container runs in the pod with the main application container, the CG pod as a whole is a “sidecar pod” that runs in parallel with the Compute Graph. Deploying a CG pod triggers provisioning of the CG, while deleting the pod triggers unprovisioning. You can use your favorite Kubernetes tools to work with NeuReality Computation Graphs, be it kubectl, Helm or ArgoCD.

NR1™ AI Inference Modules expose metrics in Prometheus text-based format, per CG and per whole server. The metrics are correlated with CG pods using the Prometheus Adapter and can be used for Horizontal Pod Autoscaling.

NeuReaility Node Problem Detector detects problems on NR1™ nodes and reports them as Kubernetes Events and Node Conditions. With the help of NeuReality Calico Dataplane Driver, CGs are addressable using CG pod IPs on a cluster pod-to-pod network. You can define Kubernetes Services to perform load balancing between CGs deployed on different NR1™ AI Inference Modules.

The CG operator facilitates the creation and operation of CGs using CG CRD.

Basic Kubernetes knowledge required

The examples on this website assume that the reader has basic Kubernetes knowledge, including familiarity with Kubernetes concepts like Pod, Deployment, Service, CNI. Familiarity with kubectl and YAML is also assumed.

A Kubernetes cluster with NeuReality AI Inference Modules is required

The examples on this website assume that the reader has access to a Kubernetes cluster and one or more NR1™ nodes.