Book a demoSign in
Algo Creator School
November 16, 2026

You are the leak. Seal the holdout (Algo Creator School #6)

Anthony Garcia, QuantVue CTO
CTO Anthony Garcia
Author
You are the leak. Seal the holdout

Here's the failure mode nobody instruments. You split your data, build on the training set, and check the test result. It's mediocre, so you tweak a filter and run the test again. And again. Forty tweaks later the test set has leaked into the strategy through you. The data never moved; the selection process is contaminated. As the machine-learning people put it: hand-tuning against the test score is calling fit on the test set with a human optimizer.

The structural fix is a three-way split. Train is where you fit. Validation is where you iterate, and you're allowed to overfit it, which is exactly why it can't be the number you report. Test is touched once, at the end, for a go or no-go. If you burned your test set tweaking, you don't have a test set.

Seal the holdout before you start. Set aside the most recent stretch of data, write down the date you locked it, and never load it, plot it, or score against it during development. Make sure it spans more than one kind of market, including the ugly stretches. A holdout of easy conditions tests nothing.

Then pre-register: before opening the holdout, write the full spec. Instruments, entry and exit rules, exact parameters, sizing, costs, and the pass/fail metric with its threshold, date-stamped, no edits after. You can't fork paths once the path is fixed. And keep the parameter budget honest: a useful rule of thumb is a year of data per optimized parameter, and fewer parameters beats more, every time.

That's the whole gauntlet across these six emails: honest data, computable rules, leak-free tests, and selection that discounts its own luck. It's how we hold our own strategies to account. If you'd rather start from ones that already passed, that's ATS.

This lesson was originally sent to the Algo Creator School email list on July 24, 2026. New lessons go out by email regularly. Join the list at quantvue.io.

Educational content only, not financial advice. Futures trading involves substantial risk of loss and is not suitable for all investors. Individual results vary, and past performance, actual or simulated, does not guarantee future results.

Read next

Free 20,000+ member Discord!

faqs

Your questions answered

What happens after I purchase?

After check out, you will receive a Welcome Email with your login details.

Use these details to Login to our dashboard, where you'll then submit your TradingView or NinjaTrader information.

Access is granted automatically if the form is filled out properly.

If you are having issues with access being granted, please submit a support ticket here.

Can I use this if I'm a total beginner?

No. Our core system is designed for those with baseline trading knowledge and live-trading experience. Do not sign up if you're a beginner trader. Intermediate to advanced experience is recommended.

Will I win every trade with this?

No. There is no algorithm or indicator that can accurately predict with 100% certainty the financial markets.

QuantVue is not responsible for trading losses. Past results are never indicative of future performance. Use QuantVue and trade at your own risk.

Which markets & timeframes does QuantVue work on?

Our tools are designed for futures. However, clients can tweak our toolkit to any chart or market.

Do I need a paid TradingView plan?

Most clients use a paid TradingView plan, and the most popular among users is TradingView premium.

How do I cancel my membership?

You can manage your account in the "My Account" tab or submit a support ticket here and we will process your cancellation immediately!

What is the refund policy?

Our monthly & yearly Pro / Elite plans are backed by a 30-day money back guarantee. If you are not 100% satisfied within 30 days from the time of purchase, you may request a full refund.

Beyond the 30-day period, all sales are final.

The 30 day money back guarantee only applies to first time customers. If you have used our system or had a plan in the past, you are not eligble for a refund. This means if you have used the PRO plan in the past and sign up for ELITE at a later date, you will not be elgible for the refund guarantee and vice versa.

This refund policy does not apply to our ATS program.

See our full refund policy here.

Does the 30 day money back guarantee apply to the ATS?

There is a separate money back guarantee for the ATS Program. The ATS refund policy has certain conditions that must be met to receive a full refund.

See our full ATS refund policy here.

Does the Elite plan include prop automation?

The Elite Plan on it's own does not automate into cash or prop firm accounts.

If you want to automate our tools into cash or prop accounts, you can do so via our alert connector QuantLynk here.

Keep in mind QuantLynk is detectable by prop firms, so we recommend using an automation-friendly prop firm.

To automate our tools into prop account undetected, we recommend our ATS Program here.

Is there a free trial on the ATS Program?

No. There is no free trial on the ATS program.