Before you begin

These examples assume that you have a Kubernetes cluster with one or more NR1™ nodes. The NeuReality CNI must be deployed in the cluster.

The required tools must be installed on the machine you run examples:

You must have at least one NR1™ AI Inference Module joined to your cluster, and each node must have the neureality.ai/arch=nr1 label. You can check the nodes as follows:

  1. Check your nodes:

    $ kubectl get node
NAME                                            STATUS   ROLES           AGE    VERSION
nr1m-110                   Ready    <none>          1h    v1.30.2
demo2-k8s-clients-node-1   Ready    <none>          411d   v1.30.2
demo2-k8s-master-1               Ready    control-plane   411d   v1.30.2
demo2-k8s-worker-1               Ready    <none>          411d   v1.30.2

  2. If you do not see your NR1™ nodes in the output, contact your cluster administrators and ask them to add NR1™ nodes to the cluster.

  3. Check that your NR1™ nodes have the neureality.ai/arch=nr1 label:

    $ kubectl get node nr1m-110 --show-labels

NAME                              STATUS   ROLES    AGE     VERSION         LABELS
nr1m-110   Ready    <none>   1h   v1.30.2   beta.kubernetes.io/arch=arm64,beta.kubernetes.io/os=linux,kubernetes.io/arch=arm64,kubernetes.io/hostname=nr1m-110,kubernetes.io/os=linux,neureality.ai/arch=nr1,nr-performance-demo-role=server,performance-demo-node-type=nr1

  4. If the NR1™ nodes do not have the neureality.ai/arch=nr1 label, contact your cluster administrators and ask them to add the label.

  5. Download one of the compute graphs for Llama-3-8B CG from NeuReality Customer downloads center, under /repository/NR1/ai-pipelines/llama-3-8B/. Store it either in your NFS sever or in your AWS S3 storage, network-accessible from your NR1™ modules.

  6. Create an image pull secret for NeuReality customer container registry. The examples on this site assume that the image pull secret exists in the namespace to which you deploy containers, and that its name is neureality-container-registry-credentials.

  7. Verify that the NeuReality CNI is installed in the cluster. Check by running the following command:

    $ kubectl get pod -n calico-system -l app.kubernetes.io/name=nr-calico-dataplane-driver

NAME                               READY   STATUS    RESTARTS   AGE
nr-calico-dataplane-driver-7npb7   1/1     Running   0          38h
nr-calico-dataplane-driver-v8zlx   1/1     Running   0          38h

    You should see NeuReality Dataplane Driver pods ready, one pod per NR1™ node.

    If you do not see any pods in the output of the command above or if the pods are not ready, contact your cluster administrators and ask them to setup CNI for NeuReality.