Why few wins break a model
With twelve wins, one unusual deal is eight percent of everything the model knows about winning. It will find patterns that are really one customer's quirks, and they will look strong on the deals it learned from.
The test that catches this is holding deals back: fit on most of your history, then ask the model to rank the deals it never saw. A model that does well on its own deals and poorly on the held-back ones has learned nothing about your next quarter.
What to do until you have enough
Start from benchmark weights for your industry and deal type. They are learned from many companies, so they are less specific to you and far more stable.
Score on buyer behavior instead of rep-entered fields. Replies, meetings attended and new stakeholders exist from day one, while a probability field means nothing until someone has calibrated it.
Refit every time a deal closes, and only switch when your own model wins on held-back deals.
How Kaypo answers this for your pipeline
Every open deal, scored on what the buyer did, with the move that worked in your own wins.
Kaypo reads your CRM, email and calls, checks every signal against the deals you have already won and lost, and puts the answer on each deal: the score, why it moved, and the next step with the email written.
How Kaypo handles it
Kaypo fits a model on your closed deals once there are about 30, tests it on deals it was never shown, and keeps the industry benchmark when your own model does not beat it. The Model page says which one is in use and why, in plain words.
Questions people ask
How many closed deals do you need for predictive deal scoring?
About 30, with at least 8 wins and 8 losses. Below that, a fitted model mostly describes noise and benchmark weights are the more honest choice.
Can an early-stage startup use predictive scoring?
An early-stage startup can use predictive scoring if it starts from industry benchmark weights and scores buyer behavior. Switch to a model fitted on your own deals once it beats the benchmark on deals it was not trained on.
How do I know if my deal score is overfitting?
Compare how well it ranks the deals it learned from with how well it ranks deals it never saw. A large gap means it memorized your history.
Reviewed October 27, 2026 against the product as it behaves today.