Management API
Overview
The Management API is composed of the following sections:
Server Status Endpoints
Compute Graph Endpoints
Health Check Endpoints
Server Maintenance Endpoints
All types of Endpoints operate using the HTTP and the HTTPS protocols. Each protocol is served over a configurable port. By default, port 5200 is used for HTTP, port 4321 is used for HTTPS.
The Server Status Endpoints are used to obtain server status information, such as provisioned compute graphs and server compute capacities.
The Compute Graph-level Endpoints are utilized for provisioning, un-provisioning, and obtaining information and status updates on compute graphs.
The Health Check Endpoints provide various health checks for http server, provisioner, and the NIF.
The Server Maintenance Endpoints are utilized for restarting the provisioner or NR1™ module, configuring logging aggregation, and Kubernetes cluster join and unjoin actions.
Compute Graphs and Provisioning
The Provisioning API uses the the following URL address: v1/cgs/
The state of a CG is retrieved using the GET method which returns the status (pending/error/provisioned) along with additional information like the CG name, CG URI, namespace, input/output tensors information, etc.
Provisioning is carried out using the POST method with the following parameters: the identifier of the compute graph in the CG store, such as its name, an optional CG ID (if not provided, it will be assigned by the server), and an optional profiler log size (for profiling). The CG ID is returned by the API. This is an asynchronous API that responds immediately, even if the compute graph’s status is marked as pending provisioning by the AI-Server. To confirm successful provisioning, the GET method should be used.
When CG store is an NFS server, the URI is of the form
nfs://<server ip or the hostname>/<path to the artifact>?uid=<uid>&gid=<gid>,
for example nfs://my-nfs-server.com/mycgs/llama_3_8b.tar.gz?uid=1000&gid=1000. uid and gid parameters are optional.
When CG store is S3 storage, the URI is of the form s3://<bucket_name>.s3.<region>.amazonaws.com/<object_prefix>/<object_name>, for example s3://my-cg-store.s3.eu-central-1.amazonaws.com/mycgs/llama_3_8b.tar.gz.
Credentials for S3 and Container registry are provided withing the body of the provisioning request.
Unprovisioning is done using the DELETE method.
These APIs are usually used by K8S components (CG-Pods) to provision and un-provision compute graphs, and the deployment is done by K8S.
Persistent Provisioning
By default provisioning requests are persistent i.e. will recover after a reboot without manual intervention.
Persistent CGs will be re-provisioned from the same URI as in original request, so it must be accessible after reboot.
This functionality can be disabled by setting persist flag in provisioning request to false.
Information about specific CG persistency can be retrieved via GET method using the URL v1/cgs/
Health Monitoring for Server and Compute Graphs
The server has three health checks: one for the HTTP server, one for the Provisioner, and one for the NIF (network interface). The health status is retrieved with the GET method using the URLs ‘v1/health’ ‘vi/health/api’ and ‘v1/health/nif’ respectively.
The CG health status can be retrieved with the GET method using the URL v1/cg/health/
Certificate management for HTTPS
During the first boot, a self-signed certificate is generated with a 90-day lifespan. We recommend that you replace the self-signed certificate as soon as possible, using your company’s PKI.
To replace the certificate and the private key, securely copy them to /etc/nginx/certs, using file names ai-server.cert and ai-server.key respectively.
scp ai-server.cert root@<NR1's Management IP>:/etc/nginx/certs
scp ai-server.key root@<NR1's Management IP>:/etc/nginx/certs
After the copy, you must run the following command using ssh:
ssh [email protected] systemctl reload nginx