What Have We Actually Learned?

After saving the 2023 results, Hoppy scrolled back to the first question he'd written down:

“Am I more likely to make money if I buy after a downward Nine-Beat Count completes?”

Back then, it was just one line. Now it had rules, experimental results, trade records, and a few rather unflattering curves attached to it.

“I thought we'd run a test and find out whether to buy.”

“And now?” asked Dr. Hop.

“Now I know I was asking several different things.”

Hoppy paused.

“Still, after all that work, we haven't found a way to make money that I'd feel confident using. So what did we get out of it?”

That's worth talking about before we close the project.

There was a lot hiding inside that one question

Let's look back at how we got here.

First, make the idea specific.

“Buy after a downward Nine-Beat Count” sounds clear enough. But a program needs more: what counts as reaching nine? When is it confirmed? Which bar and price do we use to enter? Agreeing on those details let us pin down what we were actually testing.

Second, check whether those dates deserve a closer look.

We temporarily ignored the size of our wallet. We measured each opportunity separately and compared it with random dates for the same stock. The question was “Are these dates different?”—not yet “How much will my account make?”

Third, try to improve the setup.

Which opportunities should we take, and which should we skip? Once we're in, might a different exit work better? We tried a limited set of conditions and accepted that a new rule might not improve on the old one.

Fourth, give it a limited amount of money.

We can calculate a return for every opportunity, but the same money can't buy them all at once. So we let 100 units follow the rules, accounting for positions, waiting, fees, and slippage. Then we compared the account with an index over the same period.

Fifth, keep the rules and test them on another period.

We had already looked at results and made choices. For holdout validation, we carried the chosen setup over unchanged to see how it behaved outside the data that helped us choose it.

These steps weren't there to make things complicated.

They answered different parts of the original question. Skip one, and it's easy to mistake an answer to one question for an answer to another.

For example: a positive mean event return must mean the account will make money. Beating the index must mean we didn't lose money. A good result on the old data must mean the next period will look the same.

Having worked through the process, we no longer have to lump those things together.

No money-making formula. So what are we taking away?

“I think” has become something we can check

At the beginning, if someone disagreed that prices tend to rebound after count nine, we might each have offered a few examples and convinced nobody.

Now we can open the rules and records. How we found signals, calculated returns, and chose comparisons is there to inspect.

Someone can still disagree, but the discussion can be more specific. Is the entry timing unreasonable? Does this data fail to answer the question? Did the program calculate something incorrectly, or did a correct calculation simply not support our idea?

We know where to look, instead of arguing “I believe it” versus “I don't.”

We can set some changes aside—for now

Hoppy had expected conditions based on MACD or volume to make the exit rule a little smarter.

But in our agreed comparison, neither of the two indicator-based exits met the improvement criteria. We kept the simple 10-bar exit.

That doesn't mean indicators are always useless or complex rules are always bad. It means the changes we tried in this experiment didn't give us enough reason to replace the original rule.

With that result on record, we don't have to add conditions just to make the project look more sophisticated.

Research can point toward ideas worth pursuing. It can also help us spend less time circling ideas that don't yet have evidence behind them.

We have a clearer sense of how far a result can take us

When we opened 2023, the selected setup's mean and median event returns turned negative. The primary account lost money too, yet still outperformed its same-period index reference.

That doesn't add up to a simple “It worked!” Nor does it need to become “None of this was useful.”

It tells us that the positive returns seen in the earlier period didn't fully repeat in the new one. Why they didn't repeat, or whether a different condition would change the outcome, would need further research. We can't fill those gaps with guesses.

“So we can decide to pause here?” asked Hoppy.

Of course. Or you can note a specific question you genuinely want to investigate and return to it later. What matters is knowing why you're continuing—or why you're stopping for now.

This study is finished. It isn't a live trading system

We've now worked through a teaching study. But don't rush to connect its program to a real account.

We kept several things deliberately simple along the way.

The companies are fictional, and the industries were defined for teaching. We tried only a few filters and exits, not every possibility. Our account started with 100 units and held one company at a time; we didn't design a personal position-sizing system. Fees, slippage, and execution followed simplified assumptions, not every constraint a real market can impose.

Those simplifications helped us work through the process. They also limit what its findings can be used for: this experiment is not a basis for real-world investment decisions.

“So if we fill in all those gaps, we'll make money?”

Not necessarily. More complete research may help us understand a setup better. It may also reveal more problems.

This isn't a “finish these tasks and become profitable” checklist. We just need to know what we've examined and what we haven't.

You don't need to sign up for a long list of advanced topics right now. Being able to explain this study clearly matters more than making the project bigger.

Next time you have an idea, you know how to begin

Think back to how we worked with AI.

When the program produced an error, we could ask AI to investigate, fix it, and try again. When a chart was confusing, we could ask it to explain the chart using actual records. When a prompt was ambiguous, we could keep talking until the rules were clearer.

You don't have to write every line of code yourself. Your program doesn't have to be identical to the course's program, either.

But you do need to make some things clear: What am I trying to find out? What are we keeping fixed this time? Does this output answer my question? AI says it's done—what did it actually produce?

AI can help us write programs and check them. But “AI says it's fine” still needs records and calculations behind it. That statement isn't a substitute for checking.

If you'd like, you can have one short closing conversation with Codex or WorkBuddy in your current project:

Reflect on the study with AI | Optional

Copy

First, read this project's existing research agreements, experiment summaries, and backtest records. Don't browse the web, run new experiments, or change the original results. If necessary records are missing, ask me rather than inventing them.

Help me reflect on this study, asking just one short question at a time: what I originally wanted to know, which tests we actually ran, what the results tell us and what remains unknown, and whether I want to pause or pursue a specific further question.

Follow up based on my answers and the project records. If something I say conflicts with the records, point out the specific difference in plain language. Don't fill in my answers for me or grade me on profitability. When we're done, help me put together a short closing note for confirmation in this conversation. Don't overwrite existing files.

This isn't asking AI to write a polished success story for you.

You might say, “I found a question worth investigating further.” Or, “The current results haven't given me a reason to spend more time on this.” If you're unsure, keep a note of what you haven't understood yet.

You don't need to force a positive conclusion just to finish.

Beside his original question, Hoppy added a line:

“Tested under agreed rules. The results weren't quite what I expected. The records are all here.”

“What about the next impressive-sounding indicator you come across?” asked Dr. Hop.

“I'll still be curious.”

Hoppy closed his laptop, then added:

“But this time, I know how to start checking.”