The Machine Learning Edge - Algorithmic Portfolio Management for Investors

Machine Learning (ML) has become one of the most talked about and least understood forces in modern portfolio management. While some investors dismiss it as the a fad dressed up in the latest technical language, others treat it as some all knowing black box with the ability to overwhelm any human judgment. The reality is, both views miss what's actually happening; a new tool is developing to analyze market interactions on a mass scale and people are scared of the implications. In this writing we'll define the important concepts and cut through the extreme narratives. We’ll explain how these systems are actually built and tested. What they can do and what they can't.  And where the real risks are.

Defining Terms

Current conversations about "AI investing" are often fast and loose with terms that mean very different things. Before going further, it's worth being precise.

• Artificial Intelligence (AI) - The broadest term — any system designed to perform tasks that would normally require human intelligence. This includes everything from a simple chess program to a large language model.

• Machine Learning (ML) - A subset of AI in which a system improves its performance on a task by learning patterns from data, rather than following instructions that a person wrote explicitly. It's the mechanism, not the outcome.

• Reinforcement Learning (RL) - A specific machine learning approach where an agent learns by taking actions, receiving a reward or penalty based on the outcome, and adjusting its future behavior accordingly — closer to how a person learns through trial and error than to traditional statistical modeling.

• Algorithm - A defined set of rules or steps for solving a problem. Every piece of software runs on algorithms — machine learning is a particular kind of algorithm that learns its own rules from data instead of having them hard-coded.

• Agent - A software system designed to act. In trading, an agent observes market conditions and decides what to do: buy, sell, or hold. The term is specific, it implies an ongoing feedback loop between the system and its environment, not a single prediction made in isolation.

• Model - The machine learning process — a set of learned parameters that maps an input (a stock's recent price and volume behavior) to an output (prediction or decision). 

Separating Fact from Fiction - Overhyped vs Demonized

Machine Learning in investment management is frequently misunderstood at both ends of the spectrum. Led by opposite and inaccurate narratives. The overhyped version goes like this “I’ve got a model that "predicts the market" with near-perfect accuracy, can auto execute with no human oversight, and gets better indefinitely the more data it sees”. In reality, markets are always being affected by information that isn't in any dataset — geopolitics, sentiment shifts, policy surprises — and no model sees all of it. Its been our experience that overhyped claims usually collapse the moment a strategy meets live, market conditions instead of a friendly back-test. 

The demonized version, “This is some dangerous black box that nobody understands, will inevitably blow up client capital, and represents an existential risk to markets.” This train of thought ignores the fact that quantitative and systematic strategies have run large pools of institutional capital for decades. The real risk isn't that a model is used — it's a model used without validation, without risk controls, or without human oversight.

The accurate picture sits between these polarities. Machine Learning is a powerful analytical tool with real and documented limitations. Not a crystal ball, and not an end of the world menace. It should be treated the way any disciplined investment process should be treated; validated, bounded, and monitored. The reality is, ML is simply a more capable way for human traders to approach the markets, bringing a disciplined, unemotional analysis at scale. We've found most human traders can skillfully monitor 30 - 40 individual positions at once. Layering ML analysis on top of existing human judgement, allows for an increase of approximately 10x.  Not replacing them, just expanding their reach.  

Gartner Hype Cycle - Where Are We Now?

Gartner's Hype Cycle is a way for tracking the expectations around a new technologies. Nearly every major technological shift has followed its pattern. First, an Innovation Trigger generates early proof-of-concept excitement, which accelerates into a Peak of Inflated Expectations, where coverage and capital outrun what the technology can actually deliver.

This is where we are now. This peak is always followed by a Trough of Disillusionment and we're preparing for it. The early adopters will run into real-world limitations, returns will disappoint, and enthusiasm dies. The private companies will make headlines for funding their subsequent rounds at lower valuations. Public companies will start going through a round of consolidations, especially in the semiconductor equipment providers. A few killer apps will get developed. Only then, after the shakeout does the technology begin to climb a Slope of Enlightenment, as practical, well-scoped use cases emerge — eventually reaching a Plateau of Productivity, where it becomes a mature, unremarkable part of how business gets done.

Using Machine Learning in Market Analysis

Throughout this process, we can expect financial media to flood us with opinion pieces debating the use of Machine Learning in portfolio management, pitting "Old School" managers against "Tech School" portfolio managers. In the noise, one thing will be lost. The growing utility of machine learning in finance. Stock markets generate enormous volumes of structured, repeating data — price, volume, volatility, sector relationships — updated continuously across thousands of securities. That combination of scale and structure is exactly the environment ML is suited for. A human analyst can develop deep expertise in a handful of names or a sector; a well-built ML process can apply a consistent analytical framework across an entire universe of securities, every day, without the framework degrading from bias or fatigue.

And unlike most traders, ML systems don't get emotionally attached to a position. They don't anchor to a purchase price, chase a name because it's been in the news, or hold a loser too long out of hope it will "come back." As global enterprise adoption of ML portfolio management accelerates across institutional finance, we'll see more investors deploying ML to capture performance, lower execution costs, and manage downside risk.

The Growth of ML Portfolio Management

Machine Learning was once the domain of a handful of large quant shops, Renaissance Technologies, Two Sigma, DE Shaw has now broadened considerably. Falling computational costs, more accessible AI infrastructure, and a decade of published research on reinforcement learning have brought the techniques within reach of far smaller, more specialized firms. Systematic and AI-assisted strategies now represent a meaningful and growing share of institutional trading activity, and that adoption curve shows no sign of reversing. For accredited investors evaluating managers today, the relevant question has shifted from "do they use these tools" to "how well do they use them."

The ML Process: Ingest, Process, Act

If we strip away the jargon; every ML-driven trading system follows roughly the same three-stage process.

1. Ingest - Raw market data — price, volume, volatility — is pulled in continuously across the security universe. Data quality matters enormously here: a model trained on noisy or incomplete data will learn noisy, unreliable patterns, no matter how sophisticated the algorithm layered on top of it. Models can be built to aggregate high-frequency tick data, order book dynamics, company regulatory filings, interest rate curves, and alternative datasets (e.g., satellite imagery, shipping logs, sentiment feeds).

2. Categorize & Process - Raw data is transformed into meaningful features — how a stock is performing relative to its sector, how its volatility compares to its own history, how volume is trending. This is where the analytical framework actually lives, the data is cleaned, normalized, and assigned a task. Features are engineered (e.g., volatility estimators, moving yield differentials, liquidity ratios) to prevent feature drift and eliminate look-ahead bias. The model doesn't see "AAPL," it sees a structured set of relative measurements that can be compared to previous sets.  

3. Output & Execute - The processed information produces a signal — a ranking, a buy/sell/hold decision, with a confidence score. The trained model evaluates the current market state and generates output that’s either acted on directly or displayed for human review. This is also where risk controls and human oversight are applied, while capital moves.

The Agents

Not all "AI trading agents" work the same way. Broadly, they fall into a few categories:

Rule Based Systems - Not really machine learning at all — a set of fixed conditions written by a human (e.g., "buy if the 50-day average crosses above the 200-day average"). Simple, transparent, but unable to adapt to changing market conditions on their own. Born in the wake of Bloomberg terminals spreadsheets, this was the standard for 20 years. 

Supervised Learning Models - Trained to predict a specific outcome, like next week's return based on historical examples. Useful for forecasting, but they don't inherently understand the consequences of acting on their own predictions. 

Reinforcement Learning Agents - Trained through interacting in a simulated market environment, where they receive rewards for profitable decisions and penalties for losing decisions or just doing nothing. Unlike supervised models, they learn a policy for acting, not just a forecast. 

Within reinforcement learning specifically, the choice of algorithm matters. Early approaches like Q-learning work well in simple environments but become unstable as complexity grows. Vanilla policy gradient methods can learn effectively but are prone to overly aggressive updates — essentially overreacting to a recent run of good or bad outcomes.

Proximal Policy Optimization (PPO) Agents

PPO agents were developed specifically to address that instability. It constrains how much an agent's strategy is allowed to change from one training update to the next, which prevents the kind of dramatic, erratic swings that make other reinforcement learning approaches risky when real capital is on the line. PPO agents can learn effectively from a more modest volume of data than some alternatives — a meaningful practical advantage when training many models across a diverse universe of securities rather than one model on a single asset. That combination of stability and efficiency is why they are one of the most widely used reinforcement learning agents in trading.

PPO agents don't start out knowing anything about markets. They learn entirely through iteration: taking an action in a simulated environment built from historical data, it observes the outcome, and receives a reward signal - positive for a profitable decision, negative for inaction or a losing decision, with the size of the reward or penalty typically scaled to the size of the outcome, adjusted to discourage excessive risk-taking, not just raw profit.

That single episode is repeated an enormous number of times; hundreds of thousands of simulated trading decisions, with the agent's underlying policy nudged slightly after each batch of experience to make profitable actions more likely and costly ones less likely. Over many iterations, a coherent decision-making policy emerges: not a fixed rule, but a learned tendency to act a certain way given a certain set of market conditions. This is also why training reinforcement learning agents is computationally intensive and time-consuming compared to simpler models — genuine iterative learning doesn't happen in a single pass over the data.

Blind spots - Risks Than Can’t Be Ignored

Overfitting - Robust ML models can easily fall victim to overfitting; looking brilliant in back-tests but falling apart the moment it hits real market data. PMs must take care to prevent the agent from “memorizing the walk” rather than learning from directional analysis. We guard against this in a few concrete ways: our different models view the market in different ways. In our reinforced learning algorithm; our 39 PPO trading agents are trained and validated on data the model never sees during training, so performance is always measured out-of-sample before a model is trusted with live signals. We also take the approach of training a number of narrowly focused, single-ticker models over one enormous model trying to generalize across the entire market to limit how much any one model can "cheat" by learning coincidental patterns specific to a handful of stocks.

Finally, every PPO signal has to be independently confirmed by our separate Relative Strength scoring system before a position advances. Because RS and PPO are built on different logic, an overfit PPO signal is unlikely to also show up as a genuine RS signal; this cross-validation acts as a check against any single model's blind spots.

Optimize in training, apply parameters in testing, then roll the window forward

Reducing overfitting is not a single fix, it’s a discipline. It typically involves several practices layered together: validating performance on data the model never saw during training (out-of-sample testing); favoring simpler, more narrowly scoped models over one enormous model trying to generalize across everything at once; and — critically — requiring independent confirmation from a second, differently-built analytical process before a signal is trusted. If two systems built on different logic both agree, an overfit pattern specific to one of them is far less likely to be the reason..

Drawdown Management - No model, however well-built, is immune to losing streaks. The goal isn't to eliminate losses but to keep them small, contained, and spread out. Our models are built to minimize and spread market risk. 

• Position level: every position carries a defined stop price and sizing limit, based on the stock’s individual volatility.  Position limits are established independent of the model itself, so no single signal, however confident the agent is, can put an outsized share of capital at risk. 

• Portfolio level: our algorithms are designed to read the broader market conditions; watching market trends, volatility and sector breadth measurements to  automatically reduce exposure when conditions turn hostile, rather than waiting for losses to accumulate first. 

• System-level: Experience has taught us fatal, large concentrated losses can come from anywhere. Our models are trained to spread position risk across all 11 major industry sectors to prevent concentration.  To avoid model blind spots and overfitting, our initiating positions require agreement between two independent signal sources.

Back-testing & Walk Forward Validation

A back-test asks how would this strategy have performed on historical data? It's a necessary first step, but on its own it's a weak form of evidence — it's relatively easy, intentionally or not, to design a strategy that looks good on one fixed stretch of history. 

Walk Forward Validation is a more rigorous standard. Rather than testing once on one historical window, the strategy is trained on one period, tested on the following period it has never seen, then the whole window rolls forward and the process repeats — many times, across many different market regimes. A strategy that only worked during one particular bull run tends to get exposed quickly under this approach. Walk-forward validation is closer to how the strategy will actually have to perform in live conditions: making decisions on data it has never encountered, over and over, across changing conditions.

Model Drift - models trained in one market regime, can slowly start to underperform as the underlying market changes over time. When an agent’s actual behavior begins to differ from its expected behavior, portfolio managers must have a methodology for retraining or retiring underperforming models/agents. At Blackridge, we retrain first, retire second. Our default response is not scrapping the agent, but retraining on more recent data. Markets evolve, and a model that's drifted from a regime it hasn't seen recently often just needs updated training data reflecting current conditions, not a redesign. Only when we see structural failure do we start considering retirement.

An agent gets retired when the underlying thesis itself seems to have broken; the stock’s price behavior has structurally changed, often due to a merger & acquisition change, sector reclassification, or a fundamental change in the market’s valuation, where its historical training data is no longer representative at all. This is where the experience of a seasoned portfolio manager comes into play.

Correlated Strategies - often known as “crowded trades”, similar strategies can quickly dislocate market values.  The remedy, build multi signal models, uncorrelated to each other. Also one must make sure models diversify across industries, to generate risk adjusted returns.  One or two sectors won’t do it. Traders models must diversify across all 11 GICS sectors.  

Our Models

Everything above describes the discipline we've built our process around, not just the theory behind it. Our reinforcement learning agents are built on a portfolio of 39 PPO models, each trained on a defined slice of our equity universe rather than one monolithic model trying to generalize across everything. This limits how much any single model can overfit to coincidental patterns.

The PPO signals don't act alone. They run alongside an independently built Relative Strength RSI scoring system that measures each stock's performance against its own sector benchmark across all eleven GICS sectors. A position only advances when both systems — built on different underlying logic — agree. That cross-validation serves as a primary structural defense against overfitting, and it reflects the broader principle discussed above: independent confirmation matters more than any single model's confidence.

Our validation process follows the walk-forward standard described above rather than a single static back-test, and our signal architecture includes a market regime layer that reads broader conditions — trend, volatility, sector breadth — and adjusts exposure accordingly, rather than assuming today's conditions will hold indefinitely.

Traders In the Loop

A prevalent misconception is that machine learning portfolios will eliminate the need for human managers. In practice, ML serves as a cognitive force multiplier for traders. We see it proven in real world practice, not just theory. Our models scan and score a far larger universe of securities, recognizing common patterns faster than any individual portfolio manager. Even with increased capability, we believe in pairing the technology to experienced human traders. And that experience doesn't sit outside the system, it's built into it. Position sizing limits, drawdown thresholds, and sector exposure caps are all set by our traders, based on the client consultation, independently of what any model recommends. Our traders successfully employ our ML signals in their Alpha Capture trading programs with larger funds . At Blackridge, signals inform decisions; but they don’t have the final say, our traders do. Our risk controls exist to limit agents position sizing and drawdown limits, keeping the final responsibility outside the model’s control.

This is the augmentation principle in practice, not just in theory: our models scan and score a far larger universe of securities, continuously, than any individual portfolio manager could track alone and experienced human judgment governs how that analysis actually translates into risk taken with client capital.

The Latest Evolution, Not a Break from the Past

Machine learning in portfolio management isn't a rupture from how markets have always worked in the past, it's the next step in a fin-tech evolution starting with the ticker tape, the calculator, then on to spreadsheet based modeling and the first generation of algorithmic execution with basket trading executions. While each iteration of technology reshaped what a skilled market participant could pay attention to, none of them changed the underlying job: allocate capital prudently, manage risk, and remain honest about what you don't know. Reinforcement learning is just the latest instrument in the long line of fin tech evolution, not a different one.

Much of what gets published about Machine Learning portfolio management is opinion dressed up as analysis shouting advocacy on one side, or reflexive skepticism on the other, both usually thinner on substance than they are on experience. At Blackridge Capital, we advocate another path. We choose to look honestly at where the real risks are, and explain to clients the specific computing and oversight structures that address them. That's the same standard we hold our own models to before a signal ever touches client money. Objective evaluation over narrative, every time.

Curious about how a tested ML portfolio might fit into your portfolio? If you’re an accredited investor looking to explore systematic approaches to your equity investing, reach out to us for a conversation.