By Matt Connor · Can an LLM Find Alpha Factors?
An LLM can write a hundred alpha factors in an hour. See what 240 coin flip factors score on ten years of real prices, and how to test the survivors.
An LLM can propose alpha factors all day long. Hand a capable model a data dictionary and a scoring harness, and it will write a hundred plausible factor expressions before lunch. The harder question sits underneath: how would anyone ever know whether one of them is real, when the search that produced it is a machine for manufacturing winners out of noise?
What is an alpha factor?
A factor is a rule that turns market data into one number for every stock on every date. Twelve-month price change is a factor. So is the ratio of debt to equity. A factor becomes a strategy when you rank a universe by it, buy the top slice, sell the bottom slice, and rebalance on a schedule. Alpha is whatever return is left after subtracting what a plain market exposure would have handed you anyway.
Candidates get scored with the Sharpe ratio: average return divided by the standard deviation of that return, scaled to a year. It is return per unit of wobble. A long-run Sharpe near 1 on a live strategy is respectable, which is worth holding in mind the next time a backtest quotes 3.
How LLM factor research actually works
Every project in this space runs a version of the same loop.
- The model writes factor expressions in a small language the harness can evaluate.
- A backtester scores each expression over a fixed history of prices and fundamentals.
- Expressions above a score threshold are kept. The rest are dropped.
- The keepers return to the model's context as worked examples, and the loop runs again.
Multi-agent trading systems split those jobs across separate roles, one proposing and one testing. The plumbing is genuinely useful, and the market data skills an AI agent needs are the same ones a person needs.
Nothing in the loop is dishonest. A search is how research gets done. The trouble is arithmetic, and it arrives the moment step 2 runs more than a handful of times.
Why an LLM alpha factor search manufactures winners
One price history. Thousands of cheap hypotheses. Every hypothesis is scored against the same finite sample, and that sample carries a great deal of luck. Test enough rules and some will fit the luck closely. The score cannot tell you which kind of fit you got, since a rule that matched noise and a rule that matched the market print the same number.
Here is the null hypothesis, drawn 240 times. Each "factor" below is a coin flip: a hash of the ticker, the month, and a trial number splits 40 large US names into two halves each month, and the strategy holds one half long and the other short. There is no information in it, by construction. Scored on real month-end returns from January 2016 through June 2021, the 240 trials land like this.
The exact SQL behind every number
WITH month_end AS (
SELECT ticker,
toStartOfMonth(toDate(toTimeZone(window_start, 'America/New_York'))) AS month_start,
argMax(toFloat64(close), window_start) AS close_px
FROM global_markets.delayed_stocks_minute_aggs
WHERE ticker IN ('AAPL','ADBE','AMZN','BA','CAT','COST','CRM','CSCO','CVX','DE',
'DUK','GE','GOOGL','HD','HON','IBM','INTC','JNJ','JPM','KO',
'LMT','MCD','MMM','MRK','MSFT','NKE','NVDA','ORCL','PEP','PFE',
'PG','QCOM','SO','T','TGT','TXN','UNP','VZ','WMT','XOM')
AND toDate(toTimeZone(window_start, 'America/New_York')) >= toDate('2015-12-01')
AND toDate(toTimeZone(window_start, 'America/New_York')) <= toDate('2021-06-30')
AND toDayOfMonth(toTimeZone(window_start, 'America/New_York')) >= 22
AND (toHour(toTimeZone(window_start, 'America/New_York')) * 60
+ toMinute(toTimeZone(window_start, 'America/New_York'))) BETWEEN 570 AND 959
GROUP BY ticker, month_start
),
lagged AS (
SELECT ticker,
month_start,
close_px,
lagInFrame(close_px) OVER (PARTITION BY ticker ORDER BY month_start
ROWS BETWEEN 1 PRECEDING AND CURRENT ROW) AS prev_px
FROM month_end
),
monthly_return AS (
SELECT ticker, month_start, close_px / prev_px - 1 AS ret
FROM lagged
WHERE prev_px > 0
AND month_start >= toDate('2016-01-01')
),
trial AS (
SELECT arrayJoin(range(1, 241)) AS n
),
factor_month AS (
SELECT t.n AS trial_id,
m.month_start AS month_start,
avgIf(m.ret, bitAnd(cityHash64(m.ticker, toString(m.month_start), t.n), 1) = 1)
- avgIf(m.ret, bitAnd(cityHash64(m.ticker, toString(m.month_start), t.n), 1) = 0) AS long_short_ret
FROM monthly_return AS m
CROSS JOIN trial AS t
GROUP BY trial_id, month_start
HAVING countIf(bitAnd(cityHash64(m.ticker, toString(m.month_start), t.n), 1) = 1) > 0
AND countIf(bitAnd(cityHash64(m.ticker, toString(m.month_start), t.n), 1) = 0) > 0
),
scored AS (
SELECT trial_id,
avg(long_short_ret) / stddevSamp(long_short_ret) * sqrt(12) AS sharpe
FROM factor_month
GROUP BY trial_id
HAVING stddevSamp(long_short_ret) > 0
)
SELECT multiIf(sharpe < -1.2, 'below -1.2',
sharpe < -0.8, '-1.2 to -0.8',
sharpe < -0.4, '-0.8 to -0.4',
sharpe < 0.0, '-0.4 to 0.0',
sharpe < 0.4, '0.0 to 0.4',
sharpe < 0.8, '0.4 to 0.8',
sharpe < 1.2, '0.8 to 1.2',
'1.2 and above') AS sharpe_bucket,
count() AS factor_count,
round(100 * count() / 240, 1) AS share_pct
FROM scored
GROUP BY sharpe_bucket
ORDER BY min(sharpe)The spread is the whole point. Nothing on that chart predicts anything, and yet 1 trials landed in the top band (1.2 and above), 0.4% of the search, and 1 in the bottom band (below -1.2). A researcher who ran one lucky trial and stopped would hold a chart and a Sharpe ratio, with no way to tell either apart from a discovery. Returns here are month-end close to month-end close; how monthly returns are measured covers that arithmetic.
The number that matters is how many you tried
A backtest reported on its own is missing its denominator. The same 240 trials, read as a search that keeps widening: at each step, the best score on the board beside the average of everything tried so far.
The exact SQL behind every number
WITH month_end AS (
SELECT ticker,
toStartOfMonth(toDate(toTimeZone(window_start, 'America/New_York'))) AS month_start,
argMax(toFloat64(close), window_start) AS close_px
FROM global_markets.delayed_stocks_minute_aggs
WHERE ticker IN ('AAPL','ADBE','AMZN','BA','CAT','COST','CRM','CSCO','CVX','DE',
'DUK','GE','GOOGL','HD','HON','IBM','INTC','JNJ','JPM','KO',
'LMT','MCD','MMM','MRK','MSFT','NKE','NVDA','ORCL','PEP','PFE',
'PG','QCOM','SO','T','TGT','TXN','UNP','VZ','WMT','XOM')
AND toDate(toTimeZone(window_start, 'America/New_York')) >= toDate('2015-12-01')
AND toDate(toTimeZone(window_start, 'America/New_York')) <= toDate('2021-06-30')
AND toDayOfMonth(toTimeZone(window_start, 'America/New_York')) >= 22
AND (toHour(toTimeZone(window_start, 'America/New_York')) * 60
+ toMinute(toTimeZone(window_start, 'America/New_York'))) BETWEEN 570 AND 959
GROUP BY ticker, month_start
),
lagged AS (
SELECT ticker,
month_start,
close_px,
lagInFrame(close_px) OVER (PARTITION BY ticker ORDER BY month_start
ROWS BETWEEN 1 PRECEDING AND CURRENT ROW) AS prev_px
FROM month_end
),
monthly_return AS (
SELECT ticker, month_start, close_px / prev_px - 1 AS ret
FROM lagged
WHERE prev_px > 0
AND month_start >= toDate('2016-01-01')
),
trial AS (
SELECT arrayJoin(range(1, 241)) AS n
),
factor_month AS (
SELECT t.n AS trial_id,
m.month_start AS month_start,
avgIf(m.ret, bitAnd(cityHash64(m.ticker, toString(m.month_start), t.n), 1) = 1)
- avgIf(m.ret, bitAnd(cityHash64(m.ticker, toString(m.month_start), t.n), 1) = 0) AS long_short_ret
FROM monthly_return AS m
CROSS JOIN trial AS t
GROUP BY trial_id, month_start
HAVING countIf(bitAnd(cityHash64(m.ticker, toString(m.month_start), t.n), 1) = 1) > 0
AND countIf(bitAnd(cityHash64(m.ticker, toString(m.month_start), t.n), 1) = 0) > 0
),
scored AS (
SELECT trial_id,
avg(long_short_ret) / stddevSamp(long_short_ret) * sqrt(12) AS sharpe
FROM factor_month
GROUP BY trial_id
HAVING stddevSamp(long_short_ret) > 0
),
ladder AS (
SELECT arrayJoin([1, 2, 5, 10, 25, 50, 100, 160, 240]) AS n
)
SELECT l.n AS factors_tried,
round(max(s.sharpe), 2) AS best_sharpe,
round(avg(s.sharpe), 2) AS average_sharpe
FROM ladder AS l
CROSS JOIN scored AS s
WHERE s.trial_id <= l.n
GROUP BY factors_tried
ORDER BY factors_triedA running maximum can only rise, which is exactly the trap. The first rule tested scored 0.44. After 240 trials the best on the board reads 1.59, while the average across all of them sits at 0.01. The headline improved without a single rule improving. A harness that evaluates ten thousand expressions is running this curve far to the right of anything drawn here, and the number it reports is the top of it.
What a held-out period does to the winners
The standard defense is a holdout: score on one period, then re-score the survivors on a later period the search never touched. Take the twelve best coin flips from the training window and run the identical rules across the five years that followed, July 2021 through June 2026.
The exact SQL behind every number
WITH month_end AS (
SELECT ticker,
toStartOfMonth(toDate(toTimeZone(window_start, 'America/New_York'))) AS month_start,
argMax(toFloat64(close), window_start) AS close_px
FROM global_markets.delayed_stocks_minute_aggs
WHERE ticker IN ('AAPL','ADBE','AMZN','BA','CAT','COST','CRM','CSCO','CVX','DE',
'DUK','GE','GOOGL','HD','HON','IBM','INTC','JNJ','JPM','KO',
'LMT','MCD','MMM','MRK','MSFT','NKE','NVDA','ORCL','PEP','PFE',
'PG','QCOM','SO','T','TGT','TXN','UNP','VZ','WMT','XOM')
AND toDate(toTimeZone(window_start, 'America/New_York')) >= toDate('2015-12-01')
AND toDate(toTimeZone(window_start, 'America/New_York')) <= toDate('2026-06-30')
AND toDayOfMonth(toTimeZone(window_start, 'America/New_York')) >= 22
AND (toHour(toTimeZone(window_start, 'America/New_York')) * 60
+ toMinute(toTimeZone(window_start, 'America/New_York'))) BETWEEN 570 AND 959
GROUP BY ticker, month_start
),
lagged AS (
SELECT ticker,
month_start,
close_px,
lagInFrame(close_px) OVER (PARTITION BY ticker ORDER BY month_start
ROWS BETWEEN 1 PRECEDING AND CURRENT ROW) AS prev_px
FROM month_end
),
monthly_return AS (
SELECT ticker, month_start, close_px / prev_px - 1 AS ret
FROM lagged
WHERE prev_px > 0
AND month_start >= toDate('2016-01-01')
),
trial AS (
SELECT arrayJoin(range(1, 241)) AS n
),
factor_month AS (
SELECT t.n AS trial_id,
m.month_start AS month_start,
avgIf(m.ret, bitAnd(cityHash64(m.ticker, toString(m.month_start), t.n), 1) = 1)
- avgIf(m.ret, bitAnd(cityHash64(m.ticker, toString(m.month_start), t.n), 1) = 0) AS long_short_ret
FROM monthly_return AS m
CROSS JOIN trial AS t
GROUP BY trial_id, month_start
HAVING countIf(bitAnd(cityHash64(m.ticker, toString(m.month_start), t.n), 1) = 1) > 0
AND countIf(bitAnd(cityHash64(m.ticker, toString(m.month_start), t.n), 1) = 0) > 0
),
scored AS (
SELECT trial_id,
avgIf(long_short_ret, month_start < toDate('2021-07-01'))
/ stddevSampIf(long_short_ret, month_start < toDate('2021-07-01')) * sqrt(12) AS in_sample_sharpe,
avgIf(long_short_ret, month_start >= toDate('2021-07-01'))
/ stddevSampIf(long_short_ret, month_start >= toDate('2021-07-01')) * sqrt(12) AS out_of_sample_sharpe
FROM factor_month
GROUP BY trial_id
HAVING countIf(month_start < toDate('2021-07-01')) >= 24
AND countIf(month_start >= toDate('2021-07-01')) >= 24
)
SELECT concat('trial ', toString(trial_id)) AS factor_label,
round(in_sample_sharpe, 2) AS in_sample_sharpe,
round(out_of_sample_sharpe, 2) AS out_of_sample_sharpe
FROM scored
ORDER BY in_sample_sharpe DESC
LIMIT 12Each pair of bars is one rule. The left bar is the score that earned it a place in the report. The right bar is the same rule over the next five years. The top-ranked trial scored 1.59 in training and -0.51 afterwards; the twelfth scored 0.74 and then 0.49.
Twelve rules is a small sample of its own. Sorting all 240 into fifths by training score, then averaging each fifth's holdout score, gives the cleaner view.
The exact SQL behind every number
WITH month_end AS (
SELECT ticker,
toStartOfMonth(toDate(toTimeZone(window_start, 'America/New_York'))) AS month_start,
argMax(toFloat64(close), window_start) AS close_px
FROM global_markets.delayed_stocks_minute_aggs
WHERE ticker IN ('AAPL','ADBE','AMZN','BA','CAT','COST','CRM','CSCO','CVX','DE',
'DUK','GE','GOOGL','HD','HON','IBM','INTC','JNJ','JPM','KO',
'LMT','MCD','MMM','MRK','MSFT','NKE','NVDA','ORCL','PEP','PFE',
'PG','QCOM','SO','T','TGT','TXN','UNP','VZ','WMT','XOM')
AND toDate(toTimeZone(window_start, 'America/New_York')) >= toDate('2015-12-01')
AND toDate(toTimeZone(window_start, 'America/New_York')) <= toDate('2026-06-30')
AND toDayOfMonth(toTimeZone(window_start, 'America/New_York')) >= 22
AND (toHour(toTimeZone(window_start, 'America/New_York')) * 60
+ toMinute(toTimeZone(window_start, 'America/New_York'))) BETWEEN 570 AND 959
GROUP BY ticker, month_start
),
lagged AS (
SELECT ticker,
month_start,
close_px,
lagInFrame(close_px) OVER (PARTITION BY ticker ORDER BY month_start
ROWS BETWEEN 1 PRECEDING AND CURRENT ROW) AS prev_px
FROM month_end
),
monthly_return AS (
SELECT ticker, month_start, close_px / prev_px - 1 AS ret
FROM lagged
WHERE prev_px > 0
AND month_start >= toDate('2016-01-01')
),
trial AS (
SELECT arrayJoin(range(1, 241)) AS n
),
factor_month AS (
SELECT t.n AS trial_id,
m.month_start AS month_start,
avgIf(m.ret, bitAnd(cityHash64(m.ticker, toString(m.month_start), t.n), 1) = 1)
- avgIf(m.ret, bitAnd(cityHash64(m.ticker, toString(m.month_start), t.n), 1) = 0) AS long_short_ret
FROM monthly_return AS m
CROSS JOIN trial AS t
GROUP BY trial_id, month_start
HAVING countIf(bitAnd(cityHash64(m.ticker, toString(m.month_start), t.n), 1) = 1) > 0
AND countIf(bitAnd(cityHash64(m.ticker, toString(m.month_start), t.n), 1) = 0) > 0
),
scored AS (
SELECT trial_id,
avgIf(long_short_ret, month_start < toDate('2021-07-01'))
/ stddevSampIf(long_short_ret, month_start < toDate('2021-07-01')) * sqrt(12) AS in_sample_sharpe,
avgIf(long_short_ret, month_start >= toDate('2021-07-01'))
/ stddevSampIf(long_short_ret, month_start >= toDate('2021-07-01')) * sqrt(12) AS out_of_sample_sharpe
FROM factor_month
GROUP BY trial_id
HAVING countIf(month_start < toDate('2021-07-01')) >= 24
AND countIf(month_start >= toDate('2021-07-01')) >= 24
),
ranked AS (
SELECT trial_id,
in_sample_sharpe,
out_of_sample_sharpe,
row_number() OVER (ORDER BY in_sample_sharpe DESC) AS in_sample_rank
FROM scored
)
SELECT multiIf(in_sample_rank <= 48, 'best fifth in training',
in_sample_rank <= 96, 'second fifth',
in_sample_rank <= 144, 'middle fifth',
in_sample_rank <= 192, 'fourth fifth',
'worst fifth in training') AS training_group,
round(avg(in_sample_sharpe), 2) AS avg_in_sample_sharpe,
round(avg(out_of_sample_sharpe), 2) AS avg_out_of_sample_sharpe
FROM ranked
GROUP BY training_group
ORDER BY min(in_sample_rank)In training the groups run from 0.66 at the top to -0.63 at the bottom, a wide and perfectly orderly ladder, which is guaranteed since the groups were cut by that very score. Over the holdout the same two ends average 0.01 and 0.13. The ladder flattens. A holdout is the only part of the pipeline that has not been optimized against, which is what makes it worth spending carefully.
Defenses that actually work
A holdout you spend once. Every look at it converts it into training data. Walk-forward testing, where the window rolls and each score comes from data after the fit, is the version that survives repeated use.
A multiple-testing adjustment. The deflated Sharpe ratio, introduced by Bailey and López de Prado in 2014, discounts an observed Sharpe for how many trials were run, how long the sample is, how skewed the returns are, and how fat their tails are. Feed it an honest trial count and a headline Sharpe out of a ten-thousand-expression search often deflates to nothing.
An audit trail covering every expression tried, including the ones thrown away. This is the load-bearing one, and it is why "auditable" is the interesting word in a factor-research project's description. The deflation needs a trial count. A pipeline that logs only winners has destroyed the input to its own correction. Discarded drafts, abandoned parameter sweeps, the researcher's own restarts, and every earlier version of the scoring code all count toward that number.
Cost and look-ahead bias checks before the score is believed. A factor ranked on a fundamentals field stamped with the date a vendor loaded it, rather than the date the market could see it, backtests beautifully and trades badly.
Reading the word "auditable"
New repositories in this corner appear most weeks, and a project with a few dozen stars is a prototype rather than a track record. Star counts also move faster than the code does, which is why this page scores the pattern instead of any one project. Here is what to open first in whichever one lands in front of you.
- Does it log every candidate with its expression and its score, timestamped, or only the keepers?
- Is the holdout enforced by the harness, or by the researcher's self-discipline?
- Does any reported score carry a trial count beside it?
- Which market was it built for? A library tuned on China A-shares inherits daily price limits and a restriction on selling shares bought the same session, and a factor's behavior under those rules does not carry over to US equities.
- Can you re-run it and reproduce the numbers? Pin the exact commit you read, since a project at this stage rewrites its scoring code between weekends.
None of this makes an LLM useless in factor research. Generating hypotheses is a real bottleneck and models are good at it. What changes is where the burden falls: onto the accounting for how many hypotheses were burned. Before any of it meets a live order book, paper trading is where the distance between a backtest and a fill becomes visible.
LLM alpha factor FAQ
Can an LLM find alpha factors?
It can propose them by the thousand, and a proposal is not a finding. The claim gets made at the scoring step, and a score drawn from a wide search carries a selection problem the score itself cannot see. Judge the holdout discipline and the recorded trial count ahead of the expression.
What is the deflated Sharpe ratio?
A correction that turns an observed Sharpe ratio into the probability that a search of that size would have produced it with no real edge present. Bailey and López de Prado published it in 2014. Its central input is the number of trials, which is exactly the number an unaudited research loop cannot supply.
How many backtests is too many?
There is no threshold, only an adjustment that has to be applied. One backtest scoring 1.0 and ten thousand backtests whose best scores 1.0 are different claims about the world. The coin flips above reached 1.59 across 240 trials with nothing in the data at all.
Why do published factors weaken after publication?
Academic work has tracked the decay of published anomalies over the years following their appearance. Crowding is one mechanism offered for it, and an original result overfit to its own sample is another; both draw the same shape on a chart. The efficient market hypothesis frames the first, and the trials above demonstrate the second.
Every panel here is a stored query over real month-end prices, with the SQL open underneath it. Copy one, raise the trial count, and watch the best number climb on the Strasmore terminal.