BharatMLStack: India's ML Infrastructure Revolution

Building machine learning systems that serve a billion users isn't just hard—it's a completely different game. Most ML platforms crumble under the weight of true internet scale, leaving engineers wrestling with infrastructure instead of innovating. Enter BharatMLStack, the battle-tested machine learning infrastructure platform that powers Meesho's e-commerce empire across India. This isn't another toy framework; it's a production-ready, cloud-agnostic beast designed to handle 1M+ queries per second with sub-10ms latency while slashing infrastructure costs by 60-70%. Ready to transform how you deploy ML? Let's dive deep into the stack that's redefining scale.

What is BharatMLStack?

BharatMLStack is an open-source, end-to-end machine learning infrastructure platform engineered by Meesho's ML team to solve the hardest problems in production ML at scale. Born from the trenches of one of India's largest e-commerce platforms, this stack orchestrates real-time feature serving, model inference, and embedding search for hundreds of millions of users across diverse network conditions and device capabilities.

The name itself carries weight—"Bharat" is the Hindi word for India, representing the platform's origins and its design philosophy: building for scale, diversity, and resource efficiency. While hyperscaler solutions like AWS SageMaker or Google Vertex AI lock you into expensive, rigid ecosystems, BharatMLStack runs anywhere—public cloud, on-premises data centers, and even edge locations. It's Kubernetes-native, vendor-agnostic, and optimized for both CPU and GPU workloads.

What makes it truly revolutionary is its four-core architecture that prioritizes workflow integration, economic efficiency, and enterprise-grade reliability. The stack isn't just a collection of tools; it's a cohesive platform where each component—TruffleBox UI, Online Feature Store, Inferflow, Numerix, Skye, and Horizon—plays a specific role in the ML lifecycle. With 99.99% uptime across clusters and performance metrics that dwarf conventional solutions, BharatMLStack is rapidly becoming the go-to choice for organizations serious about ML at scale.

Key Features That Define Bharat Scale

1. Workflow Integration & Productivity Acceleration

Ship ML models 3x faster—this isn't marketing fluff; it's a measurable outcome. BharatMLStack achieves this through TruffleBox UI, a web console that centralizes feature registry, cataloging, and approval workflows. Data scientists can register features once and serve them everywhere, eliminating the duplicate work that plagues traditional ML pipelines. The 95% reduction in model onboarding time comes from standardized SDKs (Go and Python) that abstract away infrastructure complexity. Engineers focus on model logic, not boilerplate code.

2. Cloud-Agnostic & Zero Vendor Lock-In

Run anywhere. Own your stack. This tenet resonates deeply in today's multi-cloud world. BharatMLStack's Kubernetes-native design means you can deploy identical infrastructure across AWS, GCP, Azure, or your own data centers. The Horizon control plane orchestrates all services uniformly, whether you're managing 10 nodes or 10,000. This flexibility translates to massive negotiation leverage with cloud providers and the freedom to optimize for cost, latency, or compliance without architectural rewrites.

3. Economic Efficiency Through Smart Architecture

60–70% lower infrastructure costs versus managed services isn't magic—it's engineering. The Online Feature Store uses custom caching strategies and efficient serialization to minimize compute overhead. Numerix, the Rust-powered math engine, delivers 10x better performance per dollar on matrix operations compared to Python-based solutions. Inferflow's DAG-based orchestration minimizes redundant computations, while Skye's pluggable vector search backends let you choose between speed (FAISS) and accuracy (HNSW) based on your budget.

4. Enterprise-Grade Availability & Scalability

99.99% uptime with 1M+ QPS capacity defines Bharat scale. The Online Feature Store achieves 2.4M QPS for batched lookups through horizontal sharding and intelligent request routing. Embedding search hits 500K QPS using Skye's distributed index architecture. Feature retrieval latency stays sub-10ms via multi-level caching (Redis for hot data, ScyllaDB for warm data). Every component is designed for graceful degradation—if a node fails, requests automatically reroute without dropping.

Real-World Use Cases That Showcase Raw Power

Personalized Candidate Generation at 2.4M QPS

Imagine generating personalized product recommendations for 100 million active users in real time. BharatMLStack's Online Feature Store retrieves user behavior features, purchase history, and contextual signals in under 10ms. Skye then performs vector similarity search across a 10-billion-item catalog at 500K QPS, ranking candidates by relevance. This powers Meesho's homepage personalization, where every user sees a unique feed tailored to their preferences, driving 30%+ conversion lifts.

Fraud Detection in Milliseconds

Financial fraud moves at the speed of light. BharatMLStack's Interaction Store, backed by ScyllaDB, ingests user interaction signals—clicks, payments, device fingerprints—in real time. Inferflow orchestrates a DAG that enriches these signals with historical features from the Online Feature Store and runs them through ensemble models. The entire pipeline—from signal ingestion to fraud verdict—completes in <50ms, blocking fraudulent transactions before they complete.

Visual Search for E-Commerce

Users upload images to find similar products. This seemingly simple feature crushes traditional systems. BharatMLStack handles it by: (1) Numerix performing GPU-accelerated feature extraction from images, (2) Skye indexing billion-scale embeddings with HNSW, and (3) Inferflow orchestrating the end-to-end pipeline. The result: 500K QPS visual search with 95% accuracy, enabling Meesho's image-based product discovery that serves millions of queries daily.

LLM-Powered Recommender Systems

Next-generation recommenders use large language models to understand user intent. BharatMLStack's Inferflow orchestrates complex DAGs where: (1) features are retrieved from the store, (2) prompts are constructed dynamically, (3) LLM inference runs on GPU clusters, and (4) outputs are post-processed and cached. This architecture supports mixture-of-experts deployments, where smaller models handle simple queries and large models tackle complex ones, optimizing cost and latency simultaneously.

Step-by-Step Installation & Setup Guide

Getting started with BharatMLStack takes minutes, not hours. The quick-start directory contains everything you need for a local development environment.

Prerequisites

• Kubernetes cluster (minikube for local, EKS/GKE for production)
• Helm 3+ for package management
• Docker and Docker Compose
• kubectl configured
• Go 1.21+ or Python 3.9+ for SDK development

Clone and Configure

# Clone the repository
git clone https://github.com/Meesho/BharatMLStack.git
cd BharatMLStack/quick-start

# Set component versions (always pin versions in production)
export ONFS_VERSION=v1.2.0
export HORIZON_VERSION=v1.3.0
export TRUFFLEBOX_VERSION=v1.3.0
export NUMERIX_VERSION=v1.0.0
export INFERFLOW_VERSION=v1.0.0
export SKYE_VERSION=v1.0.0

Deploy with Docker Compose

# Start the entire stack
./start.sh

# The script performs:
# 1. Pulls container images for all components
# 2. Spins up PostgreSQL for metadata
# 3. Deploys Redis clusters for caching
# 4. Starts ScyllaDB for interaction store
# 5. Launches Horizon control plane
# 6. Initializes TruffleBox UI on localhost:8080

Verify Installation

# Check component health
curl http://localhost:8080/health  # TruffleBox UI
curl http://localhost:9090/health  # Online Feature Store
curl http://localhost:9091/health  # Inferflow

# View logs
docker-compose logs -f horizon
docker-compose logs -f online-feature-store

Production Deployment on Kubernetes

# Add the BharatMLStack Helm repository
helm repo add bharatml https://meesho.github.io/BharatMLStack/charts
helm repo update

# Install with custom values
helm install bharatml-stack bharatml/bharatml-stack \
  --namespace ml-platform \
  --set onlineFeatureStore.replicas=5 \
  --set inferflow.gpu.enabled=true \
  --set skye.backend=faiss

For detailed configuration options, see the Quick Start Guide →.

REAL Code Examples from the Repository

Example 1: Feature Retrieval Using Python SDK

This snippet demonstrates how to fetch user and item features for real-time inference.

from bharatml.feature_store import FeatureStoreClient
from bharatml.types import FeatureQuery, FeatureView

# Initialize client with endpoint from Horizon
cilent = FeatureStoreClient(
    horizon_url="http://horizon.ml-platform.svc:8080",
    api_key=os.getenv("BHARATML_API_KEY")
)

# Define feature query for batch retrieval
# Fetching user behavior features and product metadata
query = FeatureQuery(
    feature_views=[
        FeatureView(name="user_engagement", version=2),
        FeatureView(name="product_catalog", version=1)
    ],
    entity_keys={
        "user_id": ["user_123", "user_456", "user_789"],
        "product_id": ["prod_001", "prod_002"]
    }
)

# Execute query with sub-10ms latency guarantee
features = client.get_features(query)

# features structure:
# {
#   "user_123": {"click_count_7d": 45, "purchase_value_30d": 1299.00},
#   "prod_001": {"category": "electronics", "price": 25000, "rating": 4.5}
# }

# Use features for model inference
model_input = features.to_model_input(format="tensorflow")
predictions = my_model.predict(model_input)

Explanation: The Python SDK abstracts gRPC calls to the Online Feature Store. The FeatureQuery object constructs a batch request that retrieves features for multiple entities simultaneously, crucial for high-throughput scenarios. The to_model_input() method handles serialization into TensorFlow/PyTorch tensors automatically.

Example 2: Real-Time Inference DAG with Inferflow

Inferflow uses DAG definitions to orchestrate complex ML pipelines declaratively.

# inferflow_dag.yaml
apiVersion: inferflow.bharatml.io/v1alpha1
kind: InferenceDAG
metadata:
  name: fraud-detection-pipeline
  namespace: ml-platform
spec:
  nodes:
    - name: fetch-user-features
      type: feature_store
      config:
        feature_views: ["user_risk_profile"]
        entity_key: "{{ .user_id }}"
    
    - name: fetch-transaction-features
      type: feature_store
      config:
        feature_views: ["transaction_patterns"]
        entity_key: "{{ .transaction_id }}"
    
    - name: enrich-features
      type: python_function
      config:
        image: "my-registry/feature-enrichment:v1.2"
        handler: "enrich_risk_signals"
      dependencies: ["fetch-user-features", "fetch-transaction-features"]
    
    - name: fraud-model
      type: model_inference
      config:
        model_name: "fraud_detector_v3"
        framework: "tensorflow"
        gpu: true
      dependencies: ["enrich-features"]
    
    - name: post-process
      type: python_function
      config:
        image: "my-registry/post-process:v1.0"
        handler: "format_fraud_verdict"
      dependencies: ["fraud-model"]
  
  output: "{{ .post-process.output }}"

Explanation: This DAG defines a fraud detection pipeline where each node represents a computation step. Dependencies ensure sequential execution: features are fetched first, enriched next, passed to the model, and finally post-processed. The {{ .variable }} syntax enables dynamic parameter injection at runtime. Deploy this with kubectl apply -f inferflow_dag.yaml.

Example 3: Vector Search with Skye Go SDK

Skye provides high-performance similarity search for recommendation and search use cases.

package main

import (
    "context"
    "fmt"
    "github.com/Meesho/BharatMLStack/go-sdk/skye"
)

func main() {
    // Initialize Skye client
    client, err := skye.NewClient("skye.ml-platform.svc:9092")
    if err != nil {
        panic(err)
    }
    defer client.Close()

    // Define search parameters for product recommendations
    searchReq := &skye.SearchRequest{
        Collection:    "product_embeddings",
        Vector:        []float32{0.1, 0.3, 0.5, /* ... 128 dims */},
        TopK:          50,  // Return top 50 similar products
        Filter:        "category IN ('electronics', 'gadgets') AND price < 50000",
        IncludeVectors: false,  // Don't return vectors to save bandwidth
    }

    // Execute search with latency SLO of 5ms
    ctx, cancel := context.WithTimeout(context.Background(), 5*time.Millisecond)
    defer cancel()
    
    results, err := client.Search(ctx, searchReq)
    if err != nil {
        // Handle timeout or error
        fmt.Printf("Search failed: %v\n", err)
        return
    }

    // Process results
    for _, result := range results {
        fmt.Printf("ProductID: %s, Score: %.4f\n", result.ID, result.Score)
        // Output: ProductID: prod_789, Score: 0.9876
    }

    // Batch search for multiple users
    batchReq := &skye.BatchSearchRequest{
        Searches: []*skye.SearchRequest{searchReq, searchReq2, searchReq3},
    }
    batchResults, _ := client.BatchSearch(ctx, batchReq)
    fmt.Printf("Processed %d searches in batch\n", len(batchResults))
}

Explanation: The Go SDK demonstrates Skye's batch search capability, essential for high-throughput recommendation systems. The Filter parameter enables metadata filtering during vector search, combining vector similarity with business rules. The 5ms context timeout enforces strict latency SLOs, typical of production requirements.

Advanced Usage & Best Practices

Performance Optimization Strategies

Cache Hierarchy Tuning: BharatMLStack uses three-tier caching—Redis for hot features (sub-millisecond), ScyllaDB for warm data (5-10ms), and S3 for cold storage. Monitor hit rates via Horizon's metrics endpoint and adjust TTLs dynamically. For features accessed >1000 QPS, keep them in Redis with 30-second TTL. For batch features, use ScyllaDB with 1-hour TTL.

Model Quantization with Numerix: Leverage Numerix's Rust engine for INT8 quantization. This reduces GPU memory by 4x and inference latency by 2x with minimal accuracy loss. Use the Numerix CLI to quantize models post-training:

numerix quantize --model fraud_detector_v3.pb --precision int8 --output fraud_detector_v3_int8.pb

DAG Parallelization in Inferflow: Design inference DAGs with parallel branches where possible. Feature fetching for user and item can run concurrently. Use the parallelism field in DAG specs:

spec:
  parallelism: 4  # Execute 4 nodes concurrently

Cost Management at Scale

Right-Sizing with Horizon Autoscaler: Horizon includes a custom Kubernetes autoscaler that scales based on QPS and latency SLOs, not just CPU. Configure it to maintain p99 latency under 10ms:

autoscaler:
  metrics:
    - type: latency
      target: 10ms
      percentile: 99
    - type: qps
      target: 10000

Spot Instance Orchestration: BharatMLStack natively supports spot/preemptible instances for batch workloads. Tag inference nodes with workload-type: batch and configure Inferflow to checkpoint progress every 30 seconds, enabling graceful migration when spots are reclaimed.

Comparison: BharatMLStack vs. Alternatives

Why Choose BharatMLStack? If you're serving models to millions of users, need sub-10ms latency, and want to avoid cloud vendor lock-in, BharatMLStack is unmatched. SageMaker and Vertex AI excel at getting started quickly but become cost-prohibitive at scale. Feast offers feature storage but lacks inference orchestration and vector search. BharatMLStack provides the complete package with proven performance.

Frequently Asked Questions

Q: Can BharatMLStack run on-premises? A: Absolutely. It's designed cloud-agnostic and runs seamlessly on bare-metal Kubernetes clusters. Meesho operates hybrid deployments across AWS and their own data centers.

Q: How does it achieve 60-70% cost reduction? A: Through Rust-based compute (Numerix), efficient caching, spot instance support, and eliminating managed service markups. You pay only for compute/storage, not platform fees.

Q: Is it suitable for small teams/startups? A: Yes. The quick-start setup runs on a single node. Start small and scale horizontally as you grow. The architecture grows with you.

Q: What about data privacy and compliance? A: Since you control the entire stack, data never leaves your infrastructure. This simplifies GDPR, HIPAA, and data residency compliance compared to SaaS solutions.

Q: How steep is the learning curve? A: If you know Kubernetes and basic ML concepts, you can be productive in a day. The SDKs feel familiar, and TruffleBox UI provides visual guidance.

Q: Can it handle both batch and streaming ML? A: Yes. The Online Feature Store ingests streaming data via Kafka/Kinesis, while the Interaction Store handles batch backfills. Inferflow DAGs support both synchronous and asynchronous execution modes.

Q: What's the catch with the BSL 1.1 license? A: It's source-available, not OSI-approved open source. You can use it freely, but if you offer BharatMLStack as a managed service, you need a commercial license from Meesho. For internal use, it's completely free.

Conclusion: The Future of ML Infrastructure is Bharat Scale

BharatMLStack represents a paradigm shift in machine learning infrastructure. It proves that open-source, community-driven platforms can outperform expensive managed services when built with real-world constraints in mind. The performance numbers—1M+ QPS, sub-10ms latency, 99.99% uptime—aren't theoretical; they're production metrics from serving India's diverse, demanding user base.

What excites me most is the economic efficiency. In an era where GPU costs can bankrupt AI initiatives, BharatMLStack's 60-70% cost reduction democratizes large-scale ML. The cloud-agnostic design future-proofs your architecture against vendor lock-in, while the integrated component ecosystem eliminates the painful stitching together of disparate tools.

If you're serious about deploying ML to production at scale, fork the repository today. Start with the quick-start guide, experiment with the SDKs, and join the Discord community. The documentation at meesho.github.io/BharatMLStack is comprehensive, and the community is actively helping newcomers.

The age of paying premium prices for basic ML infrastructure is over. BharatMLStack is here, it's proven, and it's ready to power your next billion-user application. Star the repo, try it out, and experience what true scale feels like.

Ready to build? Head to https://github.com/Meesho/BharatMLStack and start your journey toward ML infrastructure excellence.