Machine Learning for Analysts

There is a quiet anxiety among analysts when machine learning comes up. The dashboards you build are respected, your SQL is sharp, and yet “ML” sits in a separate room marked for data scientists only. The truth is less intimidating. The hard part of applied machine learning is not the mathematics; it is the judgment, and judgment is exactly what a good analyst already has. The model is the smallest, most replaceable piece of the whole endeavor.

The shift is from explaining to predicting

Everything you do today describes the past. A report says revenue fell in the northeast; a chart shows which products lagged. Machine learning does a different job: it makes a prediction about a single case it has never seen. Which of these customers will churn next month? Is this transaction fraud? You are no longer summarizing history, you are scoring the future, one row at a time.

That distinction tells you when to reach for ML and when not to. If a quarterly summary answers the question, build the summary. ML earns its complexity only when a per-case prediction will actually change a decision, when you have many such cases, and when you have historical examples of the outcome. A model with no decision attached to it is a science project, not analysis.

Framing beats algorithms

The most consequential work happens before you import a single library. Four questions decide whether your project succeeds: What is the decision this will inform? What exactly is the target, defined down to its time window? What will you actually know at the moment of prediction? And what does a wrong answer cost, in each direction?

That last pair matters more than beginners expect. A false alarm that wastes a five-dollar coupon is cheap; a missed fraud is not. The ratio between those costs decides which metric you optimize and where you set your decision threshold later. Skip this framing and you will build a technically correct model that solves the wrong problem beautifully.

Respect the test set like it is sacred

If there is one habit that separates trustworthy modeling from self-deception, it is this: judge the model only on data it has never seen. Before you do anything else, you hold a slice of your data aside and you do not look at it. You train on the rest, you tune on the rest, and only at the very end do you unseal the test set to get an honest estimate of how the model will perform in the wild.

The temptation to peek is constant, and peeking is corrosive. Every time you tweak the model to do better on data you have already seen, you are memorizing that particular data rather than learning the underlying pattern. The result is a model that dazzles in development and falls apart in production. The companion sin is leakage: letting information that would not exist at prediction time slip into training, like predicting churn using the cancellation date that only exists because the customer churned. When a result looks too good to be true, it almost always is, and leakage is usually the culprit.

Always beat a stupid baseline

Before celebrating any accuracy number, build the dumbest possible predictor and measure it. For a yes/no problem, always guess the majority class. If only five percent of customers churn, a model that smugly predicts “no churn” for everyone is ninety-five percent accurate and utterly worthless. Without a baseline you cannot tell a genuinely good model from a flattering number. The baseline is the bar; every real model has to clear it, and reporting both keeps you honest.

This is also why accuracy alone is a trap on imbalanced data. The questions that matter are sharper: of the cases we flagged, how many were right (precision)? Of the cases that mattered, how many did we catch (recall)? The confusion matrix lays out exactly what the model gets right and wrong, and from it every other metric flows. An analyst who reads a confusion matrix fluently is already most of the way to using ML responsibly.

Know your toolkit, and keep it small

You do not need fifty algorithms. Linear and logistic regression give you fast, interpretable models whose coefficients you can read aloud to a stakeholder. A single decision tree is a readable flowchart but tends to overfit; a random forest, which averages hundreds of varied trees, fixes that and is the reliable default for tabular data. For finding groups with no labels, k-means is enough. In scikit-learn they all share the same fit and predict shape, so trying another is a one-line change. Start with the simple, explainable model. Reach for the forest only if the extra accuracy is worth the lost clarity, and keep the simple model when the difference is marginal.

Good enough is a business decision, not a math one

The goal is never maximum accuracy. It is enough accuracy to beat the current process and clear the cost-benefit bar. Does the model beat the baseline and the human or rule it replaces? Is the value of its correct predictions greater than the cost of its mistakes plus the cost of maintaining it? Is it stable when you cross-validate? If yes, ship it, and stop chasing a percentage point that no decision depends on.

And do not forget the two things that make a model real rather than a notebook curiosity: check it for fairness across subgroups, because a model trained on biased history will faithfully reproduce that bias, and translate the result into the language of decisions. “Work the top hundred accounts the model flags and about seventy will actually churn” is worth more than any AUC. The math was never the hard part. The judgment was, and you already have it.