Stop Leaking Data to Cloud AI! Xinity AI Is the Fix

What if using ChatGPT for your company's most sensitive work was actually illegal? Not against policy—actually against the law. For thousands of enterprises across Europe and beyond, this isn't hypothetical. Every time an employee pastes a contract, patient record, or trade secret into a cloud AI tool, your organization could be violating GDPR, banking secrecy laws, journalistic source protection, attorney-client privilege, or trade secret regulations. The fines aren't theoretical either: GDPR penalties alone can reach 4% of global annual revenue.

But here's what stings worse than the legal risk. Your developers want to use AI. Your teams are begging for Copilot, for automated contract analysis, for intelligent customer support. You've been forced into an impossible choice: innovation or compliance. Most organizations compromise with half-measures—data loss prevention tools, usage policies, employee training—that treat the symptom while the disease spreads.

What if you didn't have to choose? What if you could run GPT-4-class AI inside your own data center, with zero data egress, full audit trails, and an OpenAI-compatible API that makes migration effortless? That's not a fantasy. That's Xinity AI—the open-source platform that's already deployed in production by regulated European enterprises. And it's about to change how you think about AI infrastructure entirely.

What Is Xinity AI?

Xinity AI is a complete, open-source AI operations platform designed for enterprises that cannot send data to the cloud—not by preference, but by legal necessity. Built by a Vienna-based team and proudly made in Europe, it delivers model orchestration, inference serving, management dashboards, fine-tuning pipelines, and multi-node scaling that runs entirely on your own hardware.

The project emerged from a stark realization: existing open-source AI tools solve pieces of the on-premise puzzle, but none deliver the complete operational layer enterprises actually need. Ollama makes running models easy. vLLM squeezes performance from GPUs. LocalAI offers API compatibility. But production deployment requires so much more: authentication, multi-tenancy, usage tracking, load balancing, audit trails, and governance controls.

Xinity's philosophy is captured in its badge: "Sovereign by architecture—not by contract." This distinction matters profoundly. Cloud vendors offer "data residency" promises, but your data still traverses their networks, sits on their disks, and falls under their legal jurisdictions. Xinity's architecture guarantees sovereignty structurally: nothing leaves your infrastructure. Not to a region. Not to a cloud. Not at all.

The project uses a dual-license model that balances openness with sustainability. The core engine—gateway, daemon, CLI, database layer, and shared libraries—ships under Apache 2.0, free for any use including commercial. The SvelteKit-based management dashboard uses Elastic License v2, with a generous free tier (one organization, one node) and paid unlocks for multi-node orchestration and multi-tenant features.

Key Features That Make Xinity AI Production-Ready

Xinity AI distinguishes itself through eleven enterprise-critical capabilities that competing tools simply don't offer:

🔒 True Data Sovereignty Unlike cloud AI or even "private cloud" deployments, Xinity guarantees zero data egress through architectural design. Your prompts, completions, model weights, and usage logs never leave hardware you control.

⚡ OpenAI-Compatible API Drop-in replacement for OpenAI's API. Point existing SDKs, LangChain applications, or custom integrations at your Xinity gateway with a single configuration change. No code rewrites required.

🖥️ Full Management Dashboard A SvelteKit-based web interface with role-based access control (RBAC), SSO/SAML integration, two-factor authentication, and multi-organization tenant isolation. Manage models, API keys, and review recorded LLM calls from a single pane.

🚀 Multi-Model & Multi-GPU Orchestration Deploy and serve multiple models simultaneously across distributed GPU nodes. Automatic load balancing and intelligent request routing optimize hardware utilization.

📊 Usage Tracking & Observability Comprehensive analytics on token consumption, latency patterns, and error rates. Essential for capacity planning, cost allocation, and compliance reporting.

🧠 Fine-Tuning & Distillation Pipelines Built-in support for adapting foundation models to your proprietary data without ever exposing that data externally.

🤖 MCP Server for AI-Managed Infrastructure Let AI assistants like Claude or Cursor manage your Xinity deployment through natural language, via the Model Context Protocol.

🇪🇺 EU Governance & Audit Trails Purpose-built for European regulatory frameworks: GDPR, EU AI Act, NIS2, banking secrecy, and journalistic source protection.

💰 The Utilization Inversion Here's the economic secret: cloud pricing assumes bursty ~15% utilization. Always-on AI agents run at 80–90%. At that utilization, dedicated on-prem infrastructure delivers roughly 80% cost savings versus equivalent cloud capacity.

🔍 Fully Auditable Source Code Every line visible, reviewable, and verifiable. No black-box components, no hidden telemetry.

📈 Performance That Exceeds Bare Metal Counterintuitively, Xinity's gateway layer improves performance under extreme load—18× higher completion rates at 512 concurrent requests compared to raw vLLM, through intelligent admission control and transparent retry mechanisms.

Real-World Use Cases Where Xinity AI Dominates

Banking & Financial Services

Swiss and German banks face strict banking secrecy laws (Bankgeheimnis) that prohibit transmitting customer data to third parties. Xinity enables automated credit risk analysis, AML document processing, and customer service automation while keeping all data within bank-controlled data centers. The audit trail features satisfy regulatory examination requirements.

Healthcare & Medical Research

HIPAA, GDPR health data provisions, and national healthcare laws create a compliance labyrinth. Hospitals use Xinity for clinical note summarization, medical literature analysis, and research data processing—without the legal impossibility of cloud AI patient data processing agreements.

Legal & Professional Services

Attorney-client privilege and work product doctrine make cloud AI usage professionally irresponsible for many matters. Law firms deploy Xinity for contract analysis, due diligence automation, and legal research, maintaining privilege protection while gaining AI efficiency.

Media & Journalism

Source protection laws (Pressegeheimnis in Germany, similar statutes across Europe) prohibit revealing journalistic sources. News organizations use Xinity for transcription, translation, and content analysis without exposing source materials to cloud providers subject to subpoena.

Manufacturing & Industrial IP

Trade secret protection for proprietary processes, formulations, and designs. Manufacturers deploy AI for quality control analysis, predictive maintenance documentation, and R&D acceleration while keeping crown-jewel IP entirely air-gapped.

Government & Public Institutions

European public institutions face layered requirements: GDPR, national data protection laws, public procurement rules, and security classifications. Xinity's EU governance features and fully auditable code satisfy procurement and security review processes that exclude proprietary cloud solutions.

Step-by-Step Installation & Setup Guide

Xinity AI offers three deployment paths. Here's the fastest path to production using the Xinity CLI:

Prerequisites

Linux server with systemd (for CLI deployment)
GPU-capable hardware for inference nodes
Docker^{↗ Bright Coding Blog} and Docker Compose (for local development)
Bun ≥ 1.3 (for source builds)

Production Deployment (CLI Method)

# One-line CLI installation
curl -fsSL https://get.xinity.ai/install.sh | bash

# Deploy complete stack on target server
# Replace with your SSH-accessible GPU server
xinity up all --target-host gpu-server.yourcompany.internal

# Configure dashboard access URL
xinity configure dashboardUrl https://ai.yourcompany.internal

# Create admin account directly from terminal
xinity act onboarding.cli

Local Development Setup

For contributing or evaluation, set up the full development environment:

# Clone repository
git clone https://github.com/xinity-ai/xinity-ai.git
cd xinity-ai

# Install dependencies (Bun handles everything)
bun install

# Initialize environment files from examples
find . -name 'example.env' -not -path '*/node_modules/*' | while read -r f; do
  target="${f%/example.env}/.env"
  [ -f "$target" ] || cp "$f" "$target"
done

# Generate secure auth secret for dashboard
cd packages/xinity-ai-dashboard
openssl rand -base64 33
# Copy output into .env as BETTER_AUTH_SECRET

Infrastructure Startup

# Start core services (Postgres 17, Redis 7, Mailhog)
docker compose up -d

# For full feature evaluation including web search and S3 storage
docker compose --profile full up -d

# Verify health
docker compose ps

Database Preparation

# Apply migrations across all packages
bun run --cwd packages/common-db migrate

Service Startup (Critical Order)

Xinity has strict service dependencies. Start in this exact sequence:

# 1. Infoserver FIRST (model registry, must be running)
bun run --cwd packages/xinity-infoserver dev

# 2. Gateway (API layer)
bun run --cwd packages/xinity-ai-gateway dev

# 3. Dashboard (management UI)
bun run --cwd packages/xinity-ai-dashboard dev

# 4. Daemon (on GPU hardware only)
bun run --cwd packages/xinity-ai-daemon dev

First Model Deployment

# Deploy Phi-3 Mini as quick validation
xinity act deployment.create '{
  "name": "Phi-3 Mini",
  "publicSpecifier": "phi-3-mini",
  "modelSpecifier": "phi3:mini",
  "enabled": true
}'

# Monitor until status shows "ready"
xinity act deployment.list '{"withStatus": true}'

# Verify system health
xinity doctor

REAL Code Examples from the Repository

Let's examine production-ready code patterns from Xinity's actual implementation.

Example 1: OpenAI-Compatible API Usage

The entire point of Xinity's gateway is seamless migration from OpenAI. Here's exactly how your existing code works unchanged:

from openai import OpenAI

# Only change: point to your Xinity gateway instead of OpenAI
client = OpenAI(
    base_url="https://your-dashboard/v1",  # Your internal Xinity instance
    api_key="sk_..."  # Dashboard-generated API key
)

# Identical API to OpenAI—zero code changes
response = client.chat.completions.create(
    model="qwen3.5",  # Your deployed model name
    messages=[
        {"role": "user", "content": "Summarize this contract."}
    ]
)

print(response.choices[0].message.content)

Why this matters: The base_url parameter is the only modification needed. All existing LangChain, LlamaIndex, or custom OpenAI integrations migrate instantly. The gateway handles model routing, request validation, rate limiting, and response streaming transparently.

Example 2: Complete CLI Deployment Sequence

This is the actual production deployment flow from Xinity's README, annotated with operational context:

# Install the CLI—single binary, no dependencies
curl -fsSL https://get.xinity.ai/install.sh | bash

# Deploy entire stack: Postgres, inference engine, dashboard
# The --target-host flag enables remote deployment to GPU servers
xinity up all
# Alternative: xinity up all --target-host gpu-server.internal

# Configure dashboard URL for CLI operations
# This enables subsequent commands without browser interaction
xinity configure dashboardUrl http://localhost:3100

# Create admin account from terminal—critical for air-gapped environments
# where browser-based onboarding is impossible
xinity act onboarding.cli

# Deploy first model with full specification
xinity act deployment.create '{
  "name": "Phi-3 Mini",
  "publicSpecifier": "phi-3-mini",  # API-facing model name
  "modelSpecifier": "phi3:mini",     # Ollama/vLLM model identifier
  "enabled": true
}'

# Poll deployment status until model downloads and loads
xinity act deployment.list '{"withStatus": true}'
# Expected: status transitions "pulling" → "loading" → "ready"

# Comprehensive health check across all services
xinity doctor

Operational insight: The xinity act command exposes the entire dashboard API through CLI, enabling GitOps workflows, CI/CD integration, and automated disaster recovery.

Example 3: Direct API Verification

Once deployed, verify functionality with raw HTTP—essential for debugging and monitoring:

# Test OpenAI-compatible endpoint directly
curl http://localhost:3000/v1/chat/completions \
  -H "Authorization: Bearer sk_..." \
  -H "Content-Type: application/json" \
  -d '{
    "model": "phi-3-mini",
    "messages": [
      {"role": "user", "content": "Hello from on-prem."}
    ]
  }'

Response structure matches OpenAI exactly, including streaming format (stream: true), usage statistics, and error codes. This compatibility extends to tool calling, function calling, and vision models where supported by the underlying inference engine.

Example 4: MCP Server Configuration for AI-Managed Infrastructure

Xinity's most futuristic feature: let AI assistants manage your AI infrastructure. Configuration for Claude, Cursor, or compatible clients:

{
  "mcpServers": {
    "xinity-ai": {
      "url": "https://your-dashboard/mcp",
      "headers": {
        "Authorization": "Bearer sk_..."
      }
    }
  }
}

What this enables: Natural language infrastructure operations—"Scale the Qwen deployment to 4 GPUs," "Show me yesterday's token usage by organization," or "Roll back the model to version 2.1." The MCP server translates these to precise API calls, with confirmation prompts for destructive operations.

Example 5: Development Environment Automation

For teams maintaining Xinity itself, the environment setup script demonstrates sophisticated shell patterns:

# Find all example.env files, excluding node_modules
find . -name 'example.env' -not -path '*/node_modules/*' | while read -r f; do
  # Construct target path by removing /example.env suffix
  target="${f%/example.env}/.env"
  
  # Idempotent: only copy if .env doesn't already exist
  # Preserves existing configurations during updates
  [ -f "$target" ] || cp "$f" "$target"
done

Pattern value: This idempotent approach prevents accidental configuration overwrites during git pulls or branch switches, a common source of developer environment drift.

Advanced Usage & Best Practices

Performance Optimization: Xinity's benchmark data reveals a counterintuitive truth. The gateway's ~56ms TTFT overhead at single-stream latency is more than compensated by superior behavior under load. For production deployments, configure bounded request admission to keep underlying vLLM/Ollama instances in their efficient operating ranges. The gateway's transparent retry layer for stream_parse errors eliminates client-visible failures that would crash bare inference deployments.

Security Hardening: Generate BETTER_AUTH_SECRET with openssl rand -base64 33—never use default values. Deploy WireGuard tunnels for gateway-to-daemon communication across network boundaries. Enable SSO/SAML integration rather than local authentication for production dashboards.

Multi-Node Scaling: The daemon must run on each GPU node, but centralizes through the gateway. For Kubernetes environments, consider the Docker Compose deployment route with custom orchestration, or await the upcoming native Helm chart.

Cost Optimization via Utilization Inversion: Size your GPU fleet for 80-90% sustained utilization, not cloud-style burst capacity. The economics invert dramatically: always-on workloads favor owned hardware, while truly bursty workloads might still benefit from cloud burst capacity.

Monitoring Integration: The gateway exposes OpenAPI schemas at /openapi.json and live documentation at /docs. Integrate these with your existing API monitoring and SIEM tools for unified observability.

Comparison with Alternatives

Capability	Xinity AI	Ollama	LocalAI	vLLM
OpenAI-compatible API	✅	✅	✅	✅
Multi-model orchestration	✅	❌	Partial	❌
Multi-GPU orchestration	✅	❌	❌	✅
Load balancing	✅	❌	❌	❌
Web dashboard with RBAC	✅	❌	❌	❌
Enterprise auth (SSO/SAML/2FA)	✅	❌	❌	❌
Multi-org tenant isolation	✅	❌	❌	❌
Usage tracking & data collection	✅	❌	❌	❌
Fine-tuning / distillation pipeline	✅	❌	❌	❌
MCP server (AI-managed infra)	✅	❌	❌	❌
EU Governance & Audit trail	✅	❌	❌	❌
Fully auditable source code	✅	✅	✅	✅

The verdict: Ollama and vLLM are exceptional inference engines—and Xinity uses them under the hood. LocalAI provides API compatibility but lacks operational depth. Xinity is the only complete operations layer for production enterprise deployment. Choose Ollama for personal experimentation, vLLM for raw performance research, LocalAI for simple API translation, and Xinity when compliance, governance, and scale matter.

FAQ: Common Developer Concerns

Is Xinity AI actually free to use? Yes. The core engine—gateway, daemon, CLI, infoserver, database layer—is Apache 2.0 licensed with zero restrictions. The dashboard free tier supports one organization and one inference node indefinitely. Paid tiers unlock multi-node and multi-organization features at startup-friendly pricing.

How does Xinity compare to running Ollama + a reverse proxy? That DIY approach provides inference and basic routing, but misses load balancing, RBAC, audit trails, usage tracking, multi-tenancy, fine-tuning pipelines, and the retry mechanisms that deliver 18× better reliability under load. Xinity is what you build after learning why the DIY approach fails in production.

Can I migrate from OpenAI without rewriting code? Absolutely. Change base_url and api_key in your OpenAI SDK initialization. That's it. All standard endpoints—chat completions, embeddings, streaming, function calling—work identically.

What hardware do I need? Minimum: any Linux server with Docker for the control plane. Inference requires CUDA-capable GPUs (NVIDIA recommended; AMD via ROCm experimental). The DGX Spark (128GB) handles 35B parameter models efficiently. Scale GPU nodes horizontally for larger deployments.

Is the code really fully auditable? Every line. The repository contains complete source for all components. The dual-license model (Apache 2.0 for engine, Elastic License v2 for dashboard) ensures transparency while funding continued development.

How do I get support? Community support via Discord and GitHub Discussions. Enterprise support contracts are available for production deployments requiring SLA guarantees.

What's on the roadmap? Distributed split inference (Mixture-of-Experts across nodes with privacy-preserving encoding), one-click OpenAI/Azure migration tools, advanced cost attribution analytics, and EU compliance reporting templates for GDPR, EU AI Act, and NIS2.

Conclusion: Infrastructure Is Reality

Xinity AI exposes a fundamental truth that cloud vendors hope you'll ignore: contracts are paper, but infrastructure is reality. No amount of legal assurances, data processing agreements, or "sovereign cloud" marketing changes the physical fact that your data traverses networks and rests on disks outside your control.

For organizations where that reality creates legal impossibility—not inconvenience, but actual prohibition—Xinity AI isn't a nice-to-have alternative. It's the only viable path to AI adoption. And it comes with unexpected gifts: 80% cost savings at high utilization, superior performance under extreme load, and the peace of mind that comes from genuine architectural sovereignty.

The European enterprises already running Xinity in production—media companies, manufacturers, public institutions—aren't luddites resisting cloud progress. They're pragmatists who recognized that data sovereignty and AI capability aren't trade-offs. They're prerequisites that must coexist.

Your move. The repository is open, the code is auditable, and the CLI installs in seconds. Deploy your first model this afternoon, prove it to your security team this week, and never again explain to a regulator why sensitive data left your perimeter.

🚀 Get started now: github.com/xinity-ai/xinity-ai

💬 Join the community: Discord · DeepWiki Documentation

Built in Vienna. Open to the world. Sovereign by design.