re: Does anyone here do the Jason Kelly investment strategy?

quote:

---

just found his whole system on google docs.

---

He's moved his latest off of Google Docs, but you likely found a copy that is fairly up to date.

quote:

---

9Sig is structurally different than LTCM or Niederhoffer's plans.

With normal 9Sig, you aren't borrowing money in your brokerage account. You own TQQQ and a bond fund. There isn't a broker standing there saying, "Send me another $500K by tomorrow morning or we're liquidating you."

Jason also built rules intended to preserve buying power. For example, the 9Sig buying-power throttle prevents a single quarterly buy from consuming more than 90% of the bond reserve, and the spike-reset rule can reduce an excessively high stock allocation after a huge rise.

So 9Sig doesn't have Niederhoffer's obvious Achilles heel of market plunging, margin call, forced liquidation, and game over.

We do have actual live stress tests, though. It was running during the February 2018 volatility explosion that killed XIV. It kept running.

---

Are you being deliberately obtuse? Every example I gave was them encountering their specific tail risk. It was never me accusing Mr. Kelly's systems of having the exact same flaws as them.

quote:

---

It also went through COVID. During March 2020 9Sig was roughly 36.5% below its target and signaled a strong buy rather than being forced to liquidate.

2022 was the more interesting test where TQQQ lost roughly 79% for the year. By July, after the first half of the bear market, 9Sig had deployed almost all of its buying power and was 96% TQQQ / 4% bonds.

---

quote:

---

Tail risk for 9Sig isn't a one-day crash. Imagine the 2000-2002 Nasdaq collapse.

---

So now you're admitting that Covid wasn't a specific tail risk for 9Sig. You know what I'm talking about, but you're dancing around it as best you can.

quote:

---

At that point 9Sig hasn't "blown up" like XIV, but economically it could get awfully close.

---

-99.7% according to this backtest, which is worse than XIV.

quote:

---

Was I lucky? Was the system just that good? Does it matter?

---

It's called overfitting, and yes, it matters enormously. It's why Mr. Kelly's backtests of 9Sig never go beyond 2010. He could do longer backtests for 3Sig that show it holds up fairly well through the Great Recession and tech bubble, but then he'd have to answer why he doesn't do the same for 6Sig and 9Sig.

FYI, Jason Kelly has done backtesting by simulating TQQQ data. He just doesn’t publish it publicly. I’ll find some time to go find it later.

quote:

---

Do you think I’m saying it’s risk-free?

---

No. Does something I've typed imply that?

quote:

---

What argument are we having?

---

Tail risks, specifically for the 9Sig system and effectively the risk premium of that particular system. From a glance, 3Sig doesn't have the same issue(s).

quote:

---

Your point about tail risk is fair, and Jason actually addressed that

---

I know; he somewhat addressed that in the original quote you gave. But he was also dismissive about the risk, disparaged metrics and methods to identify and understand such risks, and bragged about returns in that quote.

quote:

---

Roger generated 100 different simulated 30-year Nasdaq-like markets using historical limits for daily gains, losses, volatility, etc., then tested 972 different combinations of 9Sig rules across them -- 97,200 plan runs in total. The goal was specifically to avoid curve-fitting and see whether the system could survive very different sequences of returns.

---

Just to be clear about terminology, this is called a Monte Carlo permutation test, not to be confused with a regular Monte Carlo simulation. MCPT does indeed check to see if one is overfitting or not, while a regular Monte Carlo sim is to model uncertainty but cannot (directly, anyway) check for overfitting. The only danger of MCPT is if it was used to decide which variant of 9Sig to use, which is p-hacking. I'm not saying your post implied they did so, but since I can't see this information, I would recommend you make sure they didn't for your sake.

quote:

---

Some of those simulations got extremely ugly.

---

When you consider the historical backtest, that's natural and expected. Technically they should have created several thousands, or at least several hundreds, of simulated markets rather than just 100 for their testing, but the 100 still presents the problem. And if they truly only ran a Monte Carlo permutation test, all of the simulations are just rearranging historical data rather than creating new hypothetical markets like a regular Monte Carlo would do. Because the actual market data contained those declines, the MCPT simulations will contain similar ones.

quote:

---

9Sig was specifically tested against severe, randomized, path-dependent drawdowns rather than just assuming the historical market repeats itself.

---

Not really if they didn't do a regular Monte Carlo, and even if they did, 100 unique market simulations is too small for 30 years of market history. That said, you want a large number of simulations to make sure the system survives a wide variety of possible futures, but only one failed test can discredit a system. Since historical data already points to 9Sig's capability of catastrophic losses (even if it can eventually come back), a regular Monte Carlo is unnecessary. You don't need to concern yourself with testing for potential black swans because you have a known and dangerous white swan.

At this point, I'll just flatly state the problem. 9Sig's problem is market declines lasting multiple years and/or several quarters that drain both its bond sleeve and fresh capital contributions trying to catch falling knives and thus exasperating losses. Outside of that, you're either experiencing shorter versions of that or, more typically, collecting risk premium. It's just not good risk premium when you consider those drawdowns and their effects on the investor.

You will find very few investors capable of sticking to a system when down 80% or even 90%; depression, stress, anxiety, and psychosomatic pains all appear long before reaching those points. And since investors can't control when those declines happen, they can occur at already bad times for some of the investors. Many can say right now that it's the risk they choose for the returns they're chasing, but their tune will change when that risk is staring them in the face.

That's why I said on the first page that I don't trust Jason Kelly. He disparaged metrics and methods that are used to choose systems with good risk premiums and determine exposure levels people can realistically handle because those metrics and methods show the weakness of his 9Sig system. Then he had the gall to say that those metrics don't define superiority, perhaps implying 9Sig's returns since 2010 do. It would have been great if he had just acknowledged the serious risk his subscribers were taking and that he wanted to mitigate that with an improved version, but that's not what he did.

And after saying all of that, I still find the base idea of 3Sig, not 6Sig and 9Sig, interesting. It's given me some ideas to test, even if they don't look similar to that system.

Jason extracted a price-behavior profile from historical data and fed it into a "restricted randomizer that generates daily changes within this profile to simulate many markets." They then generated random Nasdaq-like daily returns. So I don't think the description that "all of the simulations are just rearranging historical data" matches what they actually did. These were synthetic simulated markets, not simply permutations of the historical return series.

More importantly, Jason's own research recognizes the specific failure mode you're describing. Where I think you and Jason differ is what conclusion to draw from that. The existence of an 80% or 90% drawdown demonstrates a very real risk. I don't think it automatically demonstrates that the investment plan is unsound. Whether that risk is acceptable depends on the investor, the amount allocated, the investor's circumstances, and whether he can actually stick with the plan.

Speaking for myself, I certainly didn't go "all in." I allocated what I considered a reasonable portion of my portfolio and gave the strategy a chance. I've experienced the ugly periods over roughly the past decade and never deviated from the plan. I'm not alone. A number of people in the Kelly community did the same thing. Was it easy? No. Jason and the community probably had a lot to do with helping people maintain discipline during those periods. But it demonstrates that sticking with it through severe declines isn't merely theoretical or impossible.

For me, the risk/reward fit what I was trying to accomplish. I was willing to accept unusually high risk with a portion of my portfolio in exchange for the possibility of unusually high returns. That obviously won't be appropriate for everybody, and I wouldn't tell somebody else what percentage, if any, they should put into it.

I think there's plenty of room to criticize 9Sig, question its risk-adjusted returns, or decide that the drawdown potential makes it unacceptable for you. Where I disagree is going from "this strategy has a potentially catastrophic drawdown" to "therefore it's an unsound investment plan." Those aren't the same conclusion.

It's a rollercoaster. I hope you start your ride off in a valley and not at the top of a mountain. Look at the historical chart and recognize how happy or sad you might be. I've known people who experienced the pain early -- but they stuck with it and are glad they did (but it took two years to recover before seeing gains).

I don't know what your source of info is for the rules, but if its source is from his site then you probably have access to all of the rules. Not that it's overly complicated, but some are really important when things go to the extremes.

I have one friend who was just happy that TQQQ was doing so well and decided they were not going to rebalance. It cost them about $200K of profit and they told me "lesson learned".

If you have any questions, let me know. I am more comfortable talking trade mechanics and aspects of the plan that are public domain (VCA), but I try not to share the details that come out of JK's research because that's how he makes his living.

Good luck.

I'm a long-time subscriber which means I'm grandfathered into a much lower price. I guess it's Jason's way to give back to his early supporters. I have found the Letter invaluable to stay on track. He's an excellent writer and provides a lot of excellent information on what's going on in the market and economy. His personal End Notes are worth the price of admission, in my opinion. I'm also getting lazy in my old age and so it's easy for me to follow the plan using his calculators.

For the current price, I wouldn't say it's worth subscribing annually. If I were you, I might consider subscribing a few times a year but time it where you get the End of Quarter Letters. That way, you'll see how he does the math and he'll actually provide the final stock/bond split after taking action. If you're not adding significant amounts of cash, then you can just match his stock/bond allocation to get the same result. In other words, you don't have to do the rigorous math with your specific numbers because at the end of the day the percentages are the same.

After subscribing a few times you'll be able to decide if it's worth your continuing or going on your own.

EDIT: To be clear, the advice is if you subscribe just to match the Letter. Otherwise, if you just do your own math then your percentages may differ significantly from the Letter just because you're on your own path and may have started with the base allocation while the Letter may have been running away from that for a while.

I think your regime point is interesting, but I’m not sure your test demonstrates overfitting of the Sig system.

You’re testing whether a quarter that fails to earn 3% is followed by an unusually strong quarter. I don’t think that’s really the premise of 3Sig or 9Sig. The 3% target is part of a value-path/rebalancing mechanism, not a prediction that the next quarter will rebound. If the next quarter is weak again, the system buys again. If it eventually recovers, it trims. The result comes from that repeated interaction among leverage, volatility, buying, trimming and the reserve – not just the return of the immediately following quarter.

You’re also using SPX rather than the Nasdaq 100, which may or may not produce the same behavior given the differences in volatility and composition.

That said, I do think you’ve shown something worth paying attention to: the post-2010 environment has been much more favorable to buying after weak quarters than earlier periods. That’s a legitimate regime-dependence concern, and I’d be interested in seeing the full Sig rules tested through those earlier periods.

Where I think we still differ is how far the evidence takes us. To me, your numbers raise a good question about whether recent conditions have been unusually favorable. I don’t think they establish that 9Sig is an overfit or unsound system without actually testing the complete system.

And for what it’s worth, I don’t think you have any ulterior motive here. I assume you’re analyzing it because you find the subject interesting, just as I do. Most criticism of this system usually does not evaluate the full system, which is what you have to do to fully appreciate how choosing a stock with appropriate volatility and growth can do. Both characteristics are required along with enough time to allow the system to work.