Tag: vLLM

Parallax by Gradient is a distributed model serving framework that transforms personal devices into a decentralized AI cluster. Run 70B+ models across Mac, NVIDIA, and cloud nodes with pipeline parallelism, paged KV caching, and dynamic routing—no cloud API required.

GLM-OCR ranks #1 on OmniDocBench V1.5 with 94.62 score. This 0.9B parameter open-source multimodal OCR model delivers state-of-the-art formula recognition, table extraction, and layout understanding via vLLM, SGLang, and cloud deployment.

OpenCompass is the universal LLM evaluation platform endorsed by Meta AI, supporting 100+ datasets and models from GPT-4 to Llama3. Learn installation, real code examples, and advanced benchmarking strategies in this complete guide.

TriAttention achieves 10.7x KV cache compression and 2.5x throughput on long reasoning tasks with zero accuracy loss. Learn how trigonometric frequency-domain compression enables local deployment of large models on consumer GPUs.

Discover Chandra OCR 2, the open-source model topping benchmarks for complex document layouts, handwriting, and 90+ languages. Complete installation guide, real code examples, and performance comparison inside.

DeepSeek-OCR revolutionizes document AI by compressing visual text into efficient tokens for LLMs. Learn installation, real code examples, and why it's outperforming traditional OCR at 2500 tokens/second.