Logo

User Guide (1.3)

  • Introduction
    • Scope
    • Audience
  • System Overview
    • NR1 AI Inference Appliance
    • NR1 AI SoC (AI-CPU)
    • NR1 Module
    • NR1 Platform Software
    • NeuReality SDK
      • Deployment API
      • Management API
      • Serving API
      • Kubernetes Plugins
  • AI Server
    • Setup
    • BMC Configuration
      • Connecting to BMC console
      • BMC Network Configuration
    • NR1 Module Configuration
      • Network Configuration
      • Firmware Upgrade
      • Factory Reset
    • Metrics
      • Overview
      • Types
      • Format
      • Aggregation
      • Definition
      • Enable/Disable
      • Getting metrics
      • Configuring Prometheus
    • Logging
      • Overview
      • Modes and Parameters
  • Deployment API
    • System Requirements and Setup
    • Model/Pipelines Deployment and Compile
      • Initialization - Original Application Code
      • Refactor the pipeline
      • Moving to NeuReality Programming Model
      • Pre-compilation (run the reference code on X86)
      • Compile, Simulate, Validate and Upload
    • Simulate, Validate and Upload
      • Simulate (only for simulator mode)
      • Validate (only for building to simulator mode)
    • CG Provisioning and Inference on NR1™ module
    • Profiling
      • Build Example Pipeline
      • Set AWS credentials
      • Upload Artifacts to CG storage
      • Run Profiler
    • Deployment API Reference
    • NeuReality Operators Reference
    • Workload samples
  • LLM Tutorial
    • 🛠️ Step 1: Create QPC
      • 🔧 Create the QPC Entrypoint
      • 🐳 Docker Setup
      • 🧱 Build Docker Image
      • 🚀 Run Docker Container to Create QPC
    • 🧩 Step 2: Generate NR1 Deployment Binary
      • 🏗️ Build and Compile with app.py
      • 🏃 Run Script
      • Available Flags
    • 🧪 Optional: Compile to Simulator
    • ⚠️ Notice About model_name_or_path
      • 1. Hugging Face Model
      • 2. Local Path
    • ☁️ Upload to Local SSD or S3 (NR1 CG Store)
  • Management API
    • Overview
      • Compute Graphs and Provisioning
      • Persistent Provisioning
      • Health Monitoring for Server and Compute Graphs
    • Certificate management for HTTPS
    • API Reference
  • Serving API
    • AIOF
    • OpenAI-compatible API
      • Supported Endpoints
      • 1. Completions (/completions)
      • 2. Chat Completions (/chat/completions)
      • 3. Embeddings (/embeddings)
      • 4. Moderations (/moderations)
      • Common Error Structure
      • Notes
      • API Reference
    • Quick Start with the NeuReality AI Playground API
  • AI Frameworks
    • LlamaIndex Integration
      • LLM - the first option
      • Embedding model - the second option
    • LangChain Support
      • Prerequisites
      • Running Embedding Model Inference
      • Running LLM Model Inference
      • Running RAG (Retrieval-Augmented Generation)
  • OpenAI API
    • Supported Endpoints
      • 1. Completions (/completions)
      • 2. Chat Completions (/chat/completions)
      • 3. Embeddings (/embeddings)
      • 4. Moderations (/moderations)
      • Common Error Structure
      • Notes
  • Orchestration
    • Orchestrating CGs by Kubernetes
    • Before you begin
    • CG pod
    • Providing OpenAI API
    • Resource Management for CGs
    • Load Balancing
      • Before you begin
      • Walkthrough
    • Horizontal autoscaling
      • Before you begin
      • Deploying the server and the client, observing metrics
      • Deploying the Prometheus Adapter
      • Deploying the Horizontal Pod Autoscaler
      • Watch an increase in server pods
      • Cleanup
    • CG operator
      • Installing CG Operator
      • Using the CG Operator
      • Autoscaling with the CG Operator
    • Node Problem Detector
      • Installing the NeuReality Node Problem Detector
      • Using the NeuReality Node Problem Detector
    • Adding NeuReality AI Inference Modules to Kubernetes clusters
      • Joining a node
      • Removing a node
    • CNI setup
    • Prometheus Exporters setup
    • Network Policies
      • Before you begin
      • The story
      • Initial setup
      • Defining a Network Policy
      • Validate the policy
      • Cleanup
  • Acronyms

Release Notes

  • Release Notes 1.3
    • Release Date
    • Target Audience
    • What’s new in this release?
    • Delivery Packages and Upgrades
    • Documentation
    • Known Limitations
    • How to Get Help
  • Release Notes 1.2
    • Release Date
    • Target Audience
    • What’s new in this release?
    • Delivery Packages and Upgrades
    • Documentation
    • Known Limitations
    • How to Get Help
  • Release Notes 1.1
    • Release Date
    • Target Audience
    • What’s new in this release?
    • Delivery Packages and Upgrades
    • Documentation
    • Known Limitations
    • How to Get Help
  • Release Notes 1.0
    • Release Date
    • Target Audience
    • What’s new in this release?
      • NeuReality NR1-S AI Inference Solution
      • NeuReality SDK
    • Delivery Packages and Upgrades
    • Documentation
    • Known Limitations
    • How to Get Help
  • Search

© Copyright 2025, NeuReality.

Built with Sphinx using a theme provided by Read the Docs.