HftBacktest: 5 Revolutionary Features That Transform HFT Backtesting

The brutal truth about high-frequency trading? Most backtesting tools are dangerously misleading. They assume instant execution, ignore queue positions, and treat latency as an afterthought—costing firms millions in unexpected slippage. Enter HftBacktest, the open-source framework that’s rewriting the rules of algorithmic trading research. This isn’t just another backtesting library; it’s a precision-engineered simulator that mirrors real-market conditions with microscopic accuracy. Whether you’re building market-making bots or testing latency arbitrage strategies, HftBacktest delivers the granular fidelity that separates profitable algorithms from expensive failures.

In this deep dive, we’ll unpack why developers and quant traders are abandoning legacy tools for this game-changing framework. You’ll discover how its Numba-powered engine handles millions of ticks per second, why queue position simulation is non-negotiable for realistic fills, and how to deploy the same code from backtest to live trading on Binance and Bybit. We’ll walk through real code examples, compare it against alternatives, and show you exactly how to get started. By the end, you’ll understand why accurate backtesting isn’t just conservative—it’s the only foundation worth building on.

What Is HftBacktest and Why Is It Trending Now?

HftBacktest is a free, open-source high-frequency trading and market-making backtesting framework created by nkaz001 that fundamentally rethinks how algorithmic strategies are validated. Unlike traditional backtesters that operate on bar data or simplistic trade simulations, HftBacktest reconstructs full order book dynamics from Level-2 (Market-By-Price) and Level-3 (Market-By-Order) feeds, accounting for both feed latency and order latency in nanosecond resolution.

The framework emerged from a critical gap in the retail and professional trading toolset: existing solutions either cost tens of thousands of dollars or sacrifice accuracy for speed. nkaz001 built HftBacktest to democratize institutional-grade simulation, making it accessible to independent quants, crypto trading firms, and academic researchers. Its rise in popularity coincides with the crypto trading boom, where millisecond advantages translate to significant P&L, and regulators increasingly demand robust pre-trade risk validation.

What sets HftBacktest apart is its philosophical commitment to accuracy over optimism. The documentation explicitly warns against both overly pessimistic and optimistic simulations—because in HFT, small edges matter. A strategy showing 2% annualized alpha might be a goldmine or a mirage depending on whether your simulator properly models queue priority during volatile market events. This framework captures the microstructure reality: when your order arrives at the exchange, where it sits in the FIFO queue, and how long market data takes to reach your server.

The project currently supports Python 3.11+ with Numba JIT compilation for performance-critical paths and a Rust implementation for production live trading on Binance Futures and Bybit. This dual-language architecture lets researchers prototype rapidly in Python while deploying the same logic in Rust’s memory-safe, high-performance environment—a workflow that’s becoming the industry standard.

Key Features That Redefine HFT Simulation

1. Numba-Powered JIT Compilation for Python Speed

Performance is non-negotiable when processing billions of ticks. HftBacktest leverages Numba to compile Python functions to machine code at runtime, achieving C-like speeds without leaving the Python ecosystem. The @njit decorator transforms your strategy logic into highly optimized LLVM bytecode, eliminating interpreter overhead. This means you can run tick-by-tick simulations across months of Level-3 data in minutes rather than hours. The framework’s core engine is written in Numba-compatible Python, ensuring that even complex order book traversal and latency calculations execute at native speed.

2. Complete Tick-by-Tick Simulation with Nanosecond Precision

Most backtesters aggregate data into 1-minute or 1-second bars, hiding critical market microstructure. HftBacktest processes every single tick from both trade and order book feeds, maintaining an internal clock in nanoseconds. The hbt.elapse() method advances time precisely as it occurred historically, allowing strategies to react to events in their exact chronological sequence. This granularity reveals how your algorithm behaves during microbursts of volatility—those 50-millisecond periods where prices gap and queues evaporate. You can customize time intervals or sync simulation to actual feed arrival times, making it possible to model colocation scenarios versus remote server deployments.

3. Full Order Book Reconstruction from L2 and L3 Data

Level-2 data (Market-By-Price) shows aggregated quantities at each price level, while Level-3 data (Market-By-Order) provides individual order-level granularity. HftBacktest reconstructs the entire limit order book from either feed type, tracking every addition, modification, and cancellation. This enables realistic fill simulation—when your limit order is hit, the framework knows exactly how much size was ahead of you and how much behind. The hbt.depth(asset_no) object exposes tick size, lot size, best bid/ask, and complete order book state, allowing strategies to compute order book imbalance, depth-weighted mid-price, and other microstructure alphas with precision.

4. Sophisticated Latency Modeling (Feed and Order)

Latency arbitrage strategies live and die by microsecond differences. HftBacktest models two distinct latency types: feed latency (time from exchange to your server) and order latency (time from your server to exchange and back). You can provide custom latency distributions based on historical measurements or use built-in models that account for network jitter, exchange processing times, and geographic distance. The framework applies these latencies to every market data update and order action, ensuring your strategy sees delayed quotes and receives fills realistically. This prevents the classic backtesting pitfall where strategies exploit information they couldn’t possibly have had in real-time.

5. Queue Position-Aware Order Fill Simulation

Perhaps the most revolutionary feature: HftBacktest simulates your exact position in the FIFO queue at each price level. When you submit a limit order, the framework tracks how many orders arrived before yours and how many after. During a market order sweep, fills are allocated proportionally based on queue priority. This matters enormously—being 1st in queue versus 50th can mean the difference between a 100% fill and a 10% fill during fast markets. The provided fill models handle partial fills, queue jumping from hidden orders, and exchange-specific priority rules. You can also implement custom fill logic for venues with unique matching engines.

6. Multi-Asset, Multi-Exchange Backtesting

Modern HFT strategies often trade correlated assets across venues for hedging or arbitrage. HftBacktest supports simultaneous simulation of multiple assets on multiple exchanges, maintaining separate order books, latencies, and positions for each. The asset_no parameter lets you reference different instruments within the same strategy loop, enabling cross-market making or statistical arbitrage. The framework synchronizes clocks across venues, crucial for strategies that depend on inter-exchange latency differentials.

7. Rust-Powered Live Trading Deployment

Write once, run everywhere. The Rust implementation allows deploying your validated Python strategy directly to production with minimal changes. The live trading module connects to Binance Futures and Bybit WebSocket APIs, streaming real-time Level-2 data and executing orders with the same latency-aware logic used in backtests. This eliminates the dangerous gap between research and production code, where subtle implementation differences often cause unexpected losses. The Rust bot includes failover handling, rate limit management, and exchange-specific order type support.

Real-World Use Cases Where HftBacktest Shines

1. Market Making Strategy Development and Validation

Imagine you’re building a crypto market maker for BTC-USDT perpetual futures. Your strategy quotes tight spreads around a fair value model, but you need to know: What happens during a liquidation cascade? HftBacktest replays historical liquidation events with full order book depth, showing exactly how your quotes get eaten, whether you’re filled on the toxic side, and how queue position affects survival. You can test different skewing parameters, maximum position limits, and cancel thresholds against months of tick data, revealing which settings actually reduce adverse selection versus just looking good in aggregate backtests.

2. Latency Arbitrage Feasibility Studies

You suspect there’s a 5-millisecond arbitrage opportunity between Binance and Bybit during high volatility. Traditional backtesters can’t model this—they assume simultaneous data. HftBacktest lets you inject measured latency distributions for each exchange’s feed and order path. You can replay a volatile period, and the framework will show whether your arbitrage signals are actionable or already arbitraged away by the time your orders arrive. This prevents wasting capital on infrastructure for strategies that backtests show as profitable only under unrealistic assumptions.

3. Grid Trading Optimization with Queue Priority

Grid traders place dozens of limit orders at fixed intervals. HftBacktest’s queue position tracking reveals which grid levels actually get filled versus just sit at the back of the queue. You can optimize grid spacing based on historical order book depth profiles, and test whether using IOC (Immediate-Or-Cancel) orders for certain grid levels improves overall returns. The framework’s multi-asset support also lets you run correlated grids—e.g., ETH grids that hedge BTC exposure—something impossible in single-asset backtesters.

4. Cross-Exchange Inventory Risk Management

Running market makers on both Binance and Bybit requires careful inventory balancing. HftBacktest simulates hedging flows between venues, accounting for different latencies and fee structures. You can test algorithms that transfer inventory from the venue with high adverse selection to the one with better queue positioning, optimizing capital efficiency. The framework tracks margin requirements separately for each exchange, preventing the common mistake of underestimating collateral needs in multi-venue setups.

5. Pre-Trade Risk Model Validation

Before deploying a new alpha signal, you need to verify it doesn’t blow up during rare events. HftBacktest’s tick-level precision lets you stress-test risk models by replaying flash crashes, API outages (simulated by injecting extreme latency spikes), and order book gaps. You can validate that your position limits, auto-liquidation logic, and circuit breakers activate exactly when expected, providing regulatory-grade confidence before risking real capital.

Step-by-Step Installation and Setup Guide

Prerequisites

HftBacktest requires Python 3.11 or newer for optimal Numba compatibility. You’ll also need pip and a C compiler for Numba’s LLVM backend. For live trading, install Rust 1.70+ and Cargo.

Installation Method 1: PyPI Release (Recommended)

The fastest way to start is via pip. Open your terminal and run:

pip install hftbacktest

This installs the latest stable release with all Python dependencies including Numba, NumPy, and Pandas. Verify installation with:

python -c "import hftbacktest; print(hftbacktest.__version__)"

Installation Method 2: Development Build

For bleeding-edge features, clone the repository:

git clone https://github.com/nkaz001/hftbacktest.git
cd hftbacktest
pip install -e .

The -e flag installs in editable mode, letting you modify source code and see changes immediately. This is ideal for contributing or customizing core components.

Data Preparation

HftBacktest requires normalized tick data in a specific binary format for performance. The documentation provides conversion scripts for common sources. For crypto data, you can download pre-formatted samples from Stratosphere Capital’s data repository, which includes Binance and Bybit USDM futures data.

Data files must contain:

• Trade ticks: timestamp, price, quantity, side
• Order book updates: timestamp, price, quantity, operation (add/update/delete)

Place data in a data/ directory and reference it in your backtest configuration:

from hftbacktest import DataSource
data = DataSource('./data/btcusdt_2024.dat')

Environment Configuration

Create a virtual environment to isolate dependencies:

python -m venv hft-env
source hft-env/bin/activate  # On Windows: hft-env\Scripts\activate
pip install --upgrade pip

For Jupyter notebook development, install the kernel:

pip install ipykernel
python -m ipykernel install --user --name=hftbacktest

Rust Live Trading Setup

If deploying to production, install Rust and build the trading bot:

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
cd hftbacktest/bridge/rust
cargo build --release

Configure API keys in config.toml:

[binance]
api_key = "your_api_key"
api_secret = "your_api_secret"

Real Code Examples from the Repository

Let’s dissect the market making algorithm from the README to understand HftBacktest’s API design and best practices.

Example 1: Strategy Function Structure and Initialization

from numba import njit
import numpy as np

@njit  # Numba JIT compilation for performance
def market_making_algo(hbt):
    """
    Core market making strategy function.
    
    Parameters:
    hbt: HftBacktest instance containing market data and trading interface
    """
    asset_no = 0  # Primary asset identifier
    
    # Get static contract parameters from current order book depth
    tick_size = hbt.depth(asset_no).tick_size  # Minimum price increment
    lot_size = hbt.depth(asset_no).lot_size    # Minimum quantity increment
    
    # Main event loop: elapses time in nanoseconds (10ms per iteration)
    while hbt.elapse(10_000_000) == 0:  # Returns non-zero on data exhaustion
        # Clean up cancelled/filled orders from internal state
        hbt.clear_inactive_orders(asset_no)

Explanation: The @njit decorator is crucial—without it, the loop would run 100x slower. We initialize by fetching contract specifications (tick_size, lot_size) once, avoiding repeated lookups. The while hbt.elapse() pattern is the framework’s heartbeat, advancing simulation time and processing all events that occurred in that interval. The clear_inactive_orders() call prevents memory leaks and ensures accurate order state.

Example 2: Fair Value Pricing and Risk Adjustment

        # Strategy parameters (normally would be external configuration)
        a = 1  # Forecast coefficient
        b = 1  # Risk aversion coefficient  
        c = 1  # Base risk coefficient
        hs = 1  # Half-spread coefficient
        
        # Alpha generation: replace with actual signals
        forecast = 0  # Expected price drift (e.g., from order book imbalance)
        
        # Volatility estimation: critical for spread sizing
        volatility = 0  # Could be realized vol over last 100ms
        
        # Current position and risk calculation
        position = hbt.position(asset_no)  # Net position in contracts
        risk = (c + volatility) * position  # Risk scales with volatility
        
        # Dynamic half-spread based on volatility
        half_spread = (c + volatility) * hs
        
        # Risk limits to prevent overexposure
        max_notional_position = 1000  # USD value limit
        notional_qty = 100  # Target quote size in USD
        
        # Current market state
        depth = hbt.depth(asset_no)
        mid_price = (depth.best_bid + depth.best_ask) / 2.0
        
        # Avellaneda-Stoikov inspired reservation price
        reservation_price = mid_price + a * forecast - b * risk
        
        # Final quote prices
        new_bid = reservation_price - half_spread
        new_ask = reservation_price + half_spread
        
        # Convert to exchange-acceptable ticks (price rounding)
        new_bid_tick = min(np.round(new_bid / tick_size), depth.best_bid_tick)
        new_ask_tick = max(np.round(new_ask / tick_size), depth.best_ask_tick)
        
        # Calculate order quantity in contract units
        order_qty = np.round(notional_qty / mid_price / lot_size) * lot_size

Explanation: This implements a reservation price model that adjusts quotes based on inventory risk and market conditions. The forecast variable is where you’d plug in alpha signals like order book imbalance or short-term momentum. The risk term penalizes large positions, causing the strategy to skew quotes to reduce inventory. Critically, new_bid_tick is clamped to best_bid_tick to prevent quoting through the market—a common bug in naive implementations. The quantity calculation converts USD notional to contract units, respecting lot_size granularity.

Example 3: Smart Order Management and Queue Optimization

        # Simulate processing delay for quote calculation (1ms)
        if not hbt.elapse(1_000_000) != 0:
            return False  # Exit if data ends during processing
        
        last_order_id = -1
        update_bid = True  # Flag to submit new bid quote
        update_ask = True  # Flag to submit new ask quote
        
        # Check risk limits before quoting
        buy_limit_exceeded = position * mid_price > max_notional_position
        sell_limit_exceeded = position * mid_price < -max_notional_position
        
        # Iterate through existing orders to decide cancellations
        orders = hbt.orders(asset_no)
        order_values = orders.values()
        while order_values.has_next():
            order = order_values.get()
            
            if order.side == BUY:
                # Cancel if price changed or risk limit hit
                if order.price_tick == new_bid_tick or buy_limit_exceeded:
                    update_bid = False  # No need to replace
                if order.cancellable and (update_bid or buy_limit_exceeded):
                    hbt.cancel(asset_no, order.order_id, False)  # False = no wait for ack
                    last_order_id = order.order_id
            
            elif order.side == SELL:
                if order.price_tick == new_ask_tick or sell_limit_exceeded:
                    update_ask = False
                if order.cancellable and (update_ask or sell_limit_exceeded):
                    hbt.cancel(asset_no, order.order_id, False)
                    last_order_id = order.order_id

Explanation: This is where HftBacktest’s realism shines. The orders.values() iterator lets you examine every working order without Python list overhead (Numba compatibility). The logic avoids unnecessary cancellations—if your new bid price matches the existing order, you keep your queue position, which is critical for fill rates. The cancellable check prevents canceling orders that are already being processed. The False parameter in hbt.cancel() means “don’t wait for cancel confirmation,” which is realistic for HFT systems that fire-and-forget during fast markets.

Example 4: Aggressive Order Submission with Queue Priority

        # Submit new orders only if beneficial or risk limits allow
        if update_bid and not buy_limit_exceeded:
            # Use price tick as order ID for simplicity
            order_id = new_bid_tick
            # GTX = Good-Till-Cancelled, LIMIT order, no post-only requirement
            hbt.submit_buy_order(asset_no, order_id, 
                               new_bid_tick * tick_size,  # Convert back to price
                               order_qty, 
                               GTX, LIMIT, False)
            last_order_id = order_id
        
        if update_ask and not sell_limit_exceeded:
            order_id = new_ask_tick
            hbt.submit_sell_order(asset_no, order_id,
                                new_ask_tick * tick_size,
                                order_qty,
                                GTX, LIMIT, False)
            last_order_id = order_id

Explanation: The final step submits orders using the framework’s atomic submission API. Using new_bid_tick as the order_id is a clever trick for strategies that only need one order per price level—no need for a separate ID generator. The GTX (Good-Till-Cancelled) time-in-force keeps orders working until explicitly canceled, while LIMIT specifies order type. The False flag disables post-only, allowing aggressive takes if the price crosses (though the earlier clamping prevents this). This pattern ensures you’re always at the front of the queue when you do quote, maximizing fill probability.

Advanced Usage and Best Practices

Custom Latency Models

For ultra-realistic simulation, implement custom latency distributions:

@njit
def my_latency_model(hbt, asset_no):
    # Feed latency: normal distribution with 2ms mean, 0.5ms std
    feed_latency = np.random.normal(2_000_000, 500_000)
    
    # Order latency: varies by time of day (congestion)
    hour = hbt.timestamp(asset_no) // 3_600_000_000_000 % 24
    if 14 <= hour <= 16:  # Peak hours UTC
        order_latency = np.random.exponential(3_000_000)  # 3ms mean
    else:
        order_latency = np.random.exponential(1_000_000)  # 1ms mean
    
    return feed_latency, order_latency

Inject this into your backtest configuration to model network congestion patterns.

Optimizing Numba Performance

• Avoid Python objects: Use NumPy arrays and primitive types only
• Preallocate arrays: Declare np.zeros() outside loops
• Minimize function calls: Inline small calculations
• Use @njit(fastmath=True) for floating-point ops if precision allows

Data Efficiency

• Compress data: Use binary format with np.float32 instead of float64 where possible
• Subset data: Test on 1-hour slices before full-day runs
• Parallel backtests: Run multiple parameter sweeps using multiprocessing (Numba releases GIL)

Strategy Validation Workflow

1. Sanity check: Run on 1 hour of data, verify orders match expectations
2. Latency sensitivity: Test with ±1ms latency changes; stable P&L indicates robustness
3. Queue position stress: Artificially worsen your queue position by 10 places; observe drawdown
4. Live shadow trading: Run Rust bot in dry-run mode, compare fills to simulation

Comparison: HftBacktest vs. Alternatives

Why HftBacktest Wins for HFT: While QuantConnect and Backtrader excel at swing trading and portfolio management, they lack the microstructure fidelity critical for HFT. HftBacktest’s specialized design means you’re not fighting a general-purpose framework to model niche HFT dynamics. Compared to building a custom C++ engine, you get 90% of the performance with 10% of the development time, plus a clear path to production via Rust.

Frequently Asked Questions

What data format does HftBacktest require?

HftBacktest uses a high-performance binary format for tick data. You’ll need to convert CSV or JSON feeds into its structured format using provided utilities. The format stores timestamps as int64 nanoseconds, prices as float64, and quantities as float64. Level-3 data includes additional fields for order IDs and operation types. See the Data Preparation tutorial for conversion scripts.

How accurate is the latency simulation?

Highly accurate when provided with real measurements. The framework applies latency to each event individually, not as a blanket delay. You can model asymmetric latencies (different for data vs. orders), jitter, and even exchange-specific processing times. For best results, ping your exchange servers during different times of day and fit a distribution to the measurements.

Can I use HftBacktest for live trading directly?

Yes, via the Rust bridge. The Python code you write for backtesting contains the core logic, which you can port to Rust for production. The Rust implementation handles WebSocket connections, order management, and rate limiting. While you can’t run Python directly live (too slow for HFT), the translation is straightforward since both APIs are designed to mirror each other.

What’s the difference between L2 and L3 data support?

L2 (Market-By-Price) aggregates all orders at each price level—you see total size but not individual orders. L3 (Market-By-Order) shows each order separately, enabling exact queue position tracking. HftBacktest works with both, but L3 is strongly recommended for strategies where queue priority significantly impacts fill rates (e.g., market making). L2 is sufficient for latency arbitrage that only needs best bid/ask.

How does queue position simulation handle hidden orders?

The default fill model assumes pro-rata allocation with queue priority. Hidden orders are modeled as a configurable percentage of queue size that jumps ahead of visible orders. You can customize this in the fill model parameters. For exchanges with explicit hidden order priority rules (e.g., some equity venues), you can implement a custom fill model that respects those rules.

Is HftBacktest suitable for beginners in algorithmic trading?

Not for absolute beginners. The framework assumes you understand order book dynamics, latency impact, and HFT-specific risk management. However, if you’ve built basic trading bots and want to graduate to professional-grade simulation, the tutorials provide an excellent learning path. Start with the High-Frequency Grid Trading tutorial before attempting market making.

What performance can I expect?

On a modern 8-core CPU, HftBacktest processes ~5 million ticks per second in Numba mode. A full day of BTC-USDT L3 data (~50 million events) backtests in under 15 seconds. Memory usage is ~2GB per day of data. For parameter sweeps, you can parallelize across cores, achieving near-linear scaling.

Conclusion: Why HftBacktest Belongs in Your Toolkit

HftBacktest isn’t just another open-source project—it’s a paradigm shift. By forcing you to confront microstructure realities like queue position and latency jitter, it builds strategies that survive contact with live markets. The framework’s dual Python/Rust architecture bridges the research-production gap that destroys so many promising algorithms. Its Numba-powered engine makes institutional-grade simulation accessible without a C++ team.

The crypto trading examples for Binance and Bybit provide immediate, practical value, while the extensible design invites customization for equities, options, or any instrument with tick data. Whether you’re an independent quant validating a market-making strategy or a prop firm testing risk models, HftBacktest delivers the accuracy you need to trade with confidence.

Don’t let unrealistic backtests erode your edge. Clone the repository, run the market making example, and experience the difference that nanosecond-precision simulation makes. Your future P&L will thank you.

Start backtesting with HftBacktest today →