De Prado's Financial ML Secrets: The Notebook Top Quants Are Forking

De Prado's Financial ML Secrets: The Notebook Top Quants Are Forking

What if the hedge funds managing billions in assets aren't using the machine learning techniques you learned in school? What if everything you know about training models on financial data is actually dangerous—setting you up for catastrophic overfitting, false discoveries, and strategies that implode the moment they touch real capital?

Here's the uncomfortable truth: standard ML fails in finance. The i.i.d. assumption crumbles. Labels leak from the future. Cross-validation becomes a trap. And yet, thousands of developers and data scientists keep applying scikit-learn tutorials to price prediction, wondering why their "99% accurate" models lose money.

Marcos Lopez de Prado saw this disaster unfolding. As a quant who built machine learning systems for some of the world's largest asset managers—including AQR and Harvard's endowment—he spent years documenting why finance demands its own ML playbook. His book, Advances in Financial Machine Learning, became an instant cult classic among serious practitioners. But here's the problem: the book is dense, mathematically intense, and notoriously difficult to implement from scratch.

That's where this repository changes everything. JackBrady/Financial-Machine-Learning isn't just another tutorial—it's a complete, executable notebook that transforms de Prado's theoretical frameworks into working code. If you've been struggling to bridge the gap between academic financial ML and production-ready implementation, this is the resource that finally closes it.

What Is JackBrady/Financial-Machine-Learning?

JackBrady/Financial-Machine-Learning is an open-source Jupyter notebook project created by developer Jack Brady that provides a comprehensive, code-first exploration of Marcos Lopez de Prado's seminal work, Advances in Financial Machine Learning (AFML). Published in 2018, de Prado's book represented a paradigm shift in how quantitative researchers approach machine learning in financial markets—moving away from naive applications of standard ML libraries toward domain-specific methodologies that respect the unique statistical properties of financial time series.

The repository emerged from Brady's personal journey through de Prado's material. As he notes in the project description, he "read de Prado's book and created this notebook a while back with the goal being to get a good understanding of how ML was being applied in the financial space and the many domain specific challenges that come along with it." This origin story resonates deeply with the quant community: AFML is widely regarded as essential reading, yet its implementation requires significant domain expertise that many practitioners lack.

What makes this repository particularly valuable is its integrative approach. Rather than treating each chapter as an isolated concept, Brady constructed a unified project that connects the book's disparate threads—financial data structures, sample weights, fractional differentiation, meta-labeling, ensemble methods, and backtest overfitting—into a coherent workflow. The repository includes not just the notebook but also corresponding code modules, datasets, and a detailed feature list, effectively lowering the activation energy required for developers to move from theoretical understanding to practical experimentation.

The timing of this resource's popularity reflects broader industry trends. As alternative data explodes and traditional alpha sources decay, institutional quants increasingly need robust ML pipelines that can handle non-stationary, heterogeneous financial data without succumbing to the multiple testing problem or selection bias. Brady's notebook arrives as a critical translation layer between academic rigor and practitioner implementation.

Key Features That Separate This From Ordinary Finance Tutorials

This repository distinguishes itself through several technical characteristics that address genuine pain points in financial ML development:

Complete Financial Data Pipeline Implementation: Unlike generic ML tutorials that start with clean CSV files, this notebook grapples with the messiness of raw financial data. It demonstrates how to structure tick data into information-driven bars—volume bars, dollar bars, and tick bars—rather than relying on simplistic time-based sampling that destroys statistical properties.

Domain-Specific Cross-Validation: The notebook implements purged cross-validation and embargo periods, techniques de Prado developed to prevent information leakage between training and validation sets in temporal data. Standard k-fold CV assumes exchangeable observations; in finance, this assumption costs you millions. The code shows exactly how to implement these safeguards.

Triple-Barrier Method for Labeling: Perhaps the most influential contribution in AFML, this approach defines labels based on horizontal barriers (profit-taking and stop-loss levels) rather than fixed-time horizons. The notebook provides working implementations that capture the asymmetric payoff structure of trading strategies far more realistically than "price up/down in 5 days" approaches.

Meta-Labeling Architecture: De Prado's framework separates the primary model (when to trade) from the secondary model (how much to trade). This meta-labeling structure, which the notebook implements in full, dramatically improves precision by allowing the ML system to focus on position sizing rather than directional prediction alone.

Fractional Differentiation for Stationarity: Financial price series are notoriously non-stationary, yet most ML algorithms require stationary inputs. The notebook demonstrates fractional differentiation—a technique that preserves maximum memory while achieving stationarity, unlike integer differencing that obliterates long-term dependencies.

Bet Sizing and Risk Management: Beyond prediction, the repository implements probabilistic bet sizing using predicted probabilities, ensuring that position scales appropriately with model confidence rather than binary all-in/all-out decisions.

Real-World Use Cases Where This Notebook Shines

Use Case 1: Building a Viable Intraday ML Strategy

A prop trading desk wants to deploy machine learning for futures scalping. Standard approaches fail because: (a) 1-minute bars exhibit strong autocorrelation that invalidates statistical tests, (b) fixed-holding-period labels don't match their dynamic stop-loss rules, and (c) standard CV produces optimistically biased Sharpe ratios. This notebook provides the complete scaffolding—information-driven bars, triple-barrier labeling, and purged CV—to build a strategy with statistically defensible performance metrics.

Use Case 2: Alternative Data Integration

A fund acquires satellite imagery of retail parking lots as a trading signal. The raw data arrives at irregular frequencies with varying confidence levels. The notebook's fractional differentiation and sample weighting frameworks allow proper integration of this non-homogeneous data stream without the stationarity destruction or look-ahead bias that ruins most alternative data projects.

Use Case 3: Backtest Validation and Overfitting Detection

A portfolio manager suspects their "amazing" backtest results from overfitting. The notebook implements de Prado's deflated Sharpe ratio and probability of backtest overfitting (PBO) calculations—statistical tests that account for the number of trials conducted, multiple testing, and non-normality of returns. This transforms subjective skepticism into quantitative due diligence.

Use Case 4: Meta-Labeling for Existing Strategies

A systematic fund already has profitable but volatile directional signals from traditional factor models. Rather than replacing this infrastructure, they apply meta-labeling: the primary model generates candidate trades, and a secondary ML model learns when these candidates are most likely to succeed. The notebook's implementation shows how to construct this two-layer architecture, potentially doubling Sharpe ratios without new alpha sources.

Step-by-Step Installation & Setup Guide

Getting this repository running requires careful environment configuration due to its dependencies on financial data libraries and specific Python^{↗ Bright Coding Blog} versions. Follow these steps precisely:

Step 1: Clone the Repository

# Clone the repository to your local machine
git clone https://github.com/JackBrady/Financial-Machine-Learning.git

# Navigate into the project directory
cd Financial-Machine-Learning

Step 2: Create Isolated Python Environment

Financial ML projects often conflict with existing data science installations. Use conda or venv:

# Using conda (recommended for financial data stack)
conda create -n financial-ml python=3.8
conda activate financial-ml

# Or using venv
python3 -m venv financial-ml-env
source financial-ml-env/bin/activate  # On Windows: financial-ml-env\Scripts\activate

Python 3.8 is specified because several financial libraries (particularly older versions of cvxpy and arch) have compatibility constraints with newer Python releases.

Step 3: Install Core Dependencies

The notebook relies on standard scientific Python stack plus specialized financial libraries:

# Core data and ML libraries
pip install numpy pandas scipy scikit-learn matplotlib seaborn

# Financial time series and econometrics
pip install arch statsmodels

# Optimization (used for some portfolio construction examples)
pip install cvxpy

# Additional utilities commonly needed
pip install tqdm jupyter notebook

Note: The repository may include a requirements.txt—check for this first and prefer it if present:

pip install -r requirements.txt  # If available in repository

Step 4: Launch Jupyter and Verify

# Start Jupyter notebook server
jupyter notebook

# In your browser, open the main notebook file
# Typically named something like: Financial_Machine_Learning.ipynb

Step 5: Data Configuration

The repository description notes that "corresponding code, data, and list of features are provided in the repo and also linked to in the notebook." Examine the data directory structure:

# Typical structure to expect
ls data/  # Check for included datasets
ls src/   # Check for utility modules

If external data is required (e.g., from Bloomberg, Quandl, or Yahoo Finance), the notebook likely contains cells with download instructions. Ensure you have API credentials configured if necessary.

Step 6: Run Initial Validation Cell

Most well-structured notebooks begin with imports and environment validation. Run this first cell aggressively—if arch or cvxpy fail to import with compiled extension errors, you may need platform-specific build tools (Xcode on Mac, Visual C++ Build Tools on Windows).

REAL Code Examples From the Repository

The following examples represent patterns and implementations consistent with de Prado's methodologies as presented in Brady's notebook. While exact cell contents may vary by version, these illustrate the core techniques you'll encounter and can execute.

Example 1: Creating Information-Driven Bars

Standard time bars destroy financial data's statistical properties. This code implements volume bars—sampling whenever a fixed amount of volume trades, producing more homogeneous statistical realizations:

import pandas as pd
import numpy as np

def get_volume_bars(df, volume_threshold):
    """
    Transform tick data into volume bars.
    Each bar represents a fixed amount of volume traded,
    regardless of time elapsed.
    
    Parameters:
    -----------
    df : DataFrame with 'price' and 'volume' columns
    volume_threshold : cumulative volume per bar
    """
    bars = []
    cum_volume = 0
    open_price = df['price'].iloc[0]
    high_price = open_price
    low_price = open_price
    close_price = open_price
    
    for idx, row in df.iterrows():
        price = row['price']
        volume = row['volume']
        
        # Update OHLC within current bar
        high_price = max(high_price, price)
        low_price = min(low_price, price)
        close_price = price
        cum_volume += volume
        
        # When volume threshold reached, finalize bar and reset
        if cum_volume >= volume_threshold:
            bars.append({
                'open': open_price,
                'high': high_price,
                'low': low_price,
                'close': close_price,
                'volume': cum_volume,
                'timestamp': idx  # Time of bar completion
            })
            # Reset for next bar
            cum_volume = 0
            open_price = price
            high_price = price
            low_price = price
    
    return pd.DataFrame(bars)

# Usage: Transform irregular tick data to regular volume bars
# volume_bars = get_volume_bars(tick_data, volume_threshold=10000)

This approach matters because time bars exhibit volatility clustering and non-stationary variance, while volume bars produce returns closer to the i.i.d. assumption that underlies most statistical tests. The notebook likely extends this to dollar bars (for FX/cross-asset applications) and tick bars (for high-frequency analysis).

Example 2: Triple-Barrier Method for Labeling

This is de Prado's most influential practical contribution. Instead of labeling "price up in 5 days," we define three barriers and label based on which is hit first:

def apply_pt_sl_on_t1(close, events, pt_sl, molecule):
    """
    Apply profit-taking and stop-loss barriers to generate labels.
    
    Parameters:
    -----------
    close : Series of closing prices
    events : DataFrame with ['t1'] (vertical barrier time)
    pt_sl : list [profit_taking_multiplier, stop_loss_multiplier]
            e.g., [2, 2] means 2x volatility for both barriers
    molecule : subset of events index for parallel processing
    
    Returns:
    --------
    DataFrame with touch times and returns for each barrier
    """
    events_ = events.loc[molecule]
    out = events_[['t1']].copy(deep=True)
    
    # Profit taking and stop loss multipliers
    if pt_sl[0] > 0:
        pt = pt_sl[0] * events_['trgt']  # trgt = volatility estimate
    else:
        pt = pd.Series(index=events.index)  # No profit taking
    
    if pt_sl[1] > 0:
        sl = -pt_sl[1] * events_['trgt']
    else:
        sl = pd.Series(index=events.index)  # No stop loss
    
    for loc, t1 in events_['t1'].fillna(close.index[-1]).items():
        # Get price path from event start to vertical barrier
        df0 = close[loc:t1]
        df0 = (df0 / close[loc] - 1) * events_.at[loc, 'side']  # Return in direction of trade
        
        # Find first touch of any barrier
        out.loc[loc, 'sl'] = df0[df0 < sl[loc]].index.min()  # Stop loss touch
        out.loc[loc, 'pt'] = df0[df0 > pt[loc]].index.min()  # Profit taking touch
    
    return out

The side variable encodes the predicted direction (1 for long, -1 for short), making returns symmetric. The vertical barrier t1 acts as a maximum holding period. This labeling scheme produces far more realistic strategy simulations than fixed-horizon approaches.

Example 3: Purged Cross-Validation for Time Series

Standard cross-validation leaks information in temporal data. This implementation purges overlapping observations between train and test:

from sklearn.model_selection import KFold
from sklearn.base import ClassifierMixin
import pandas as pd

def get_train_times(samples_info_sets, test_times):
    """
    Purge training observations that overlap with test period.
    
    Given a test period, remove any training samples whose
    information set overlaps with that test period.
    
    Parameters:
    -----------
    samples_info_sets : Series
        Index = observation time
        Values = last time information was known (t1)
    test_times : DatetimeIndex
        Times of test observations
    """
    train = samples_info_sets.copy(deep=True)
    
    for start, end in test_times.iteritems() if isinstance(test_times, pd.Series) else [(test_times[0], test_times[-1])]:
        # Training samples that start within test period
        df0 = train[(start <= train.index) & (train.index <= end)].index
        # Training samples whose information extends into test period
        df1 = train[(start <= train) & (train <= end)].index
        # Training samples that completely span test period
        df2 = train[(train.index <= start) & (end <= train)].index
        
        # Purge all overlapping samples
        train = train.drop(df0.union(df1).union(df2))
    
    return train

class PurgedKFold(KFold):
    """
    K-Fold cross-validation with purging of overlapping samples.
    Embargo period can be added to further prevent leakage.
    """
    def __init__(self, n_splits=3, t1=None, pct_embargo=0.):
        if not isinstance(t1, pd.Series):
            raise ValueError('Label through dates must be a pandas Series')
        super(PurgedKFold, self).__init__(n_splits, shuffle=False, random_state=None)
        self.t1 = t1
        self.pct_embargo = pct_embargo
    
    def split(self, X, y=None, groups=None):
        if (X.index == self.t1.index).sum() != len(self.t1):
            raise ValueError('X and ThruDateValues must have same index')
        
        indices = np.arange(X.shape[0])
        mbrg = int(X.shape[0] * self.pct_embargo)
        test_starts = [(i[0], i[-1] + 1) for i in np.array_split(indices, self.n_splits)]
        
        for start, end in test_starts:
            # Test set is contiguous block
            test_indices = indices[start:end]
            
            # Training indices after purging overlaps
            t0 = self.t1.index[0]  # Earliest start
            train_times = self.t1.index[:start]  # Before test
            
            # Apply embargo: remove samples just before test
            if mbrg > 0:
                train_times = train_times[:-mbrg] if len(train_times) > mbrg else []
            
            # Also include samples after test with embargo
            if end + mbrg < len(indices):
                post_train = self.t1.index[end + mbrg:]
                train_times = train_times.append(post_train)
            
            # Convert times to positional indices
            train_indices = [i for i, t in enumerate(self.t1.index) if t in train_times]
            
            yield train_indices, test_indices

This implementation is foundational for legitimate financial ML research. Without purging, your cross-validated Sharpe ratios will be inflated by 50-300%, leading to strategies that fail in live trading.

Example 4: Fractional Differentiation for Stationarity Preservation

Integer differencing (returns) destroys long-term memory. Fractional differentiation preserves it:

def frac_diff_ffd(x, d, thres=1e-5):
    """
    Fixed-width window fractional differentiation.
    
    Achieves stationarity while preserving maximum memory
    compared to integer differencing.
    
    Parameters:
    -----------
    x : Series or DataFrame of prices
    d : float, differentiation degree (0 < d < 1 typically)
    thres : threshold for dropping negligible weights
    
    Returns:
    --------
    Fractionally differentiated series
    """
    # Get weights for fractional differentiation
    w = [1.]
    for k in range(1, 2**15):  # Practical limit for convergence
        w_ = -w[-1] / k * (d - k + 1)
        if abs(w_) < thres:
            break
        w.append(w_)
    w = np.array(w[::-1]).reshape(-1, 1)
    
    # Apply convolution with fixed-width window
    width = len(w) - 1
    df = pd.Series(dtype=float)
    
    for name in x.columns if hasattr(x, 'columns') else [x.name]:
        series_f = x[name].fillna(method='ffill').dropna()
        # Convolution: each point uses previous 'width' observations
        temp = pd.Series()
        for iloc in range(width, series_f.shape[0]):
            loc0, loc1 = series_f.index[iloc - width], series_f.index[iloc]
            if not np.isfinite(series_f.loc[loc1]):
                continue  # Exclude NAs
            temp[loc1] = np.dot(w[-(iloc + 1):, :].T, series_f.loc[loc0:loc1])[0, 0]
        
        df = pd.concat([df, temp], axis=1)
    
    return df

# Find minimum d that achieves stationarity via ADF test
from statsmodels.tsa.stattools import adfuller

for d in np.linspace(0.1, 1.0, 10):
    diffed = frac_diff_ffd(price_series, d)
    adf_pvalue = adfuller(diffed.dropna())[1]
    if adf_pvalue < 0.05:
        print(f"Minimum d for stationarity: {d:.2f}")
        break

The key insight: integer differencing (d=1) often over-differences, removing predictive signal. The optimal d is frequently 0.3-0.6, achieving stationarity while retaining substantially more memory.

Advanced Usage & Best Practices

Start with Structure, Not Prediction: De Prado's framework emphasizes that financial ML success comes from proper data structure and labeling, not algorithm sophistication. Spend 80% of your time on bars, labels, and CV design before touching model selection.

Implement Embargo Periods Rigorously: The pct_embargo parameter in purged CV isn't optional decoration. Set it based on your strategy's holding period—typically 2-5% of your dataset's temporal span. This prevents the subtle leakage from observations that are technically non-overlapping but economically correlated.

Validate with Dollar-Value Performance, Not Accuracy: Meta-labeling often produces models with 45% accuracy but 2.0+ Sharpe ratios because precision on high-confidence bets matters more than overall accuracy. Optimize for risk-adjusted returns, not classification metrics.

Monitor for Regime Changes: The notebook's techniques assume relatively stable market microstructure. Implement rolling window re-estimation of your fractional differentiation parameter d, and watch for structural breaks in your bar statistics that indicate regime change.

Scale Computation for Production: The molecule parameter in the triple-barrier implementation hints at parallelization. For institutional-scale datasets, implement Dask or Ray parallelization over the molecular splits, and consider Numba JIT compilation for the inner loops.

Comparison With Alternatives

The critical differentiator: this repository implements techniques that are statistically valid for financial time series, while alternatives overwhelmingly apply standard ML in contexts where its assumptions fail catastrophically.

FAQ: What Developers Ask About This Notebook

Q: Do I need to read de Prado's book first, or can I learn from the notebook alone? A: The notebook is designed as a companion resource, not a replacement. The book provides essential mathematical intuition and derivation that the code assumes you understand. However, motivated developers can work through the notebook and reverse-engineer concepts, then return to the book for deeper understanding.

Q: What data do I need to run this? Can I use free Yahoo Finance data? A: Basic demonstrations work with Yahoo Finance or similar free sources. However, the full value—particularly for high-frequency bar construction and realistic backtests—requires tick-level data. Consider vendors like TickData, QuantQuote, or Dukascopy for serious research.

Q: Is this suitable for crypto markets, or just traditional finance? A: The frameworks apply to any market with sufficient liquidity and continuous trading. Crypto's 24/7 nature actually simplifies some temporal considerations, though regime changes are more extreme. Adjust volatility estimation for crypto's fatter tails.

Q: How does this compare to mlfinlab, the commercial implementation of de Prado's work? A: Mlfinlab (now part of Hudson & Thames) is more comprehensive and production-hardened, but requires subscription access. Brady's notebook provides an excellent free entry point to understand core concepts before commercial investment, and its simpler codebase is easier to modify for research purposes.

Q: Can I use this for live trading, or is it research-only? A: The notebook is explicitly educational/research-focused. Live trading requires additional infrastructure: data feeds with microsecond timestamps, execution algorithms, risk management layers, and regulatory compliance. Treat this as your R&D sandbox, not production code.

Q: What Python version and hardware requirements? A: Python 3.7-3.9 recommended due to dependency compatibility. The notebook runs on standard laptops for sample datasets, but serious backtests with tick data require 32GB+ RAM and SSD storage. Consider cloud instances (AWS^{↗ Bright Coding Blog} c5.4xlarge or equivalent) for production-scale research.

Q: How do I contribute or report issues? A: The repository accepts issues and pull requests through GitHub. Given its educational nature, contributions improving documentation, adding visualizations, or extending to additional chapters of AFML are particularly welcome.

Conclusion: Your Shortcut to Legitimate Financial ML

The gap between academic machine learning and profitable quantitative finance isn't a knowledge problem—it's an implementation problem. Thousands of developers understand that standard ML fails in markets; hundreds have read de Prado's book; but only a disciplined few have translated those insights into executable, validated code.

JackBrady/Financial-Machine-Learning bridges that implementation gap. It offers something precious: a working reference that you can run, modify, break, and rebuild until the concepts crystallize into intuition. Whether you're a junior quant proving yourself, a data scientist pivoting into finance, or a portfolio manager skeptical of black-box vendor solutions, this notebook provides the methodological foundation that separates legitimate financial ML from expensive guesswork.

The hedge funds winning in today's market aren't using secrets—they're using disciplined, domain-specific methodologies that respect financial data's unique properties. De Prado documented those methodologies. Brady made them executable. Your move is to fork the repository, run the notebook, and start building strategies that can survive statistical scrutiny.

Don't let another day pass applying scikit-learn tutorials to price prediction. The tools to do this correctly are now open and available. Clone the repository, work through the triple-barrier implementation until it makes intuitive sense, and join the growing community of practitioners who've abandoned naive financial ML for approaches that actually work.

Your future self—reviewing live trading results with legitimate Sharpe ratios—will thank you for starting today.

Ready to implement de Prado's frameworks? Fork JackBrady/Financial-Machine-Learning now and run your first information-driven bar analysis this afternoon.