
Key takeaways
A 95%-accurate model can still create no business value, usually because the team skipped business understanding and solved a surface request instead of the real problem.
Four questions close the gap: What is the real problem and why does it matter? Who is the end user? How will the solution fit into the workflow? How will success be measured?
Two traps await when measuring success: accuracy is meaningless on imbalanced data, and predicting who will churn is not the same as knowing who can be kept (uplift); a model's value lies in the extra customers it creates over a control group.
Louis Dorard's Machine Learning Canvas puts every answer on one page in ten blocks across Learn, Predict and Evaluate. Four worked examples show it for classification, regression, clustering and recommendation.
When several requests compete, score each on value and feasibility using the canvas blocks; the matrix separates strategic bets, quick wins and projects that need foundations first.
The canvas complements CRISP-DM: CRISP-DM describes the sequence of work, while the canvas records the design decisions.
In data science, we are often captivated by the complexity of algorithms, the glamour of "state-of-the-art" models and the thrill of squeezing out another percentage point of accuracy. We rush to collect data, preprocess it, build a model and deploy it. Then a sad truth emerges: a model with 95% accuracy brings the business no value, or worse, nobody uses it.
This familiar story comes from skipping a quiet but critical foundation: business understanding. This stage is not about code or algorithms but about understanding the business problem deeply. Before writing the first line of code, you must be able to answer: "What problem are we really trying to solve?" Skimping on this stage leads to wasted resources, products nobody adopts and stakeholders who lose faith in data science.
The article has seven parts. Section 1 describes the gap between a "surface request" and the "real problem". Section 2 sets out four core questions that close it, each with two examples. Section 3 examines two common traps in measuring success, including the gap between response models and uplift models. Section 4 introduces the Machine Learning Canvas, a tool that captures all the answers on one page, and Section 5 applies it to four types of problem: classification, regression, clustering and recommendation. Section 6 prioritises when several problems arrive at once, using a value and feasibility matrix. Section 7 places business understanding within the CRISP-DM cycle.
1. The gap between request and problem
Most data projects begin with a short sentence from the business: "We need a customer churn prediction model," or "Build us a recommendation system." It sounds like a clear technical requirement but is only a surface request. It does not say who will use the result, for what, when, or what counts as success.
Business understanding requires the data professional to act as a consultant: to listen and ask the right questions to move from the surface request to the real problem. Figure 1 summarises that journey through the four core questions detailed in Section 2.

2. Four core questions for business understanding
Each question below is illustrated with two running examples, customer churn prediction and product recommendation, each in three steps: the surface request, the probing questions and the real problem uncovered.
2.1. What is the real problem, and why does it matter?
This is the most fundamental question. Never accept the initial request mechanically. Dig for the business's real "pain" and why solving it is urgent.
Example 1: churn prediction. Surface request: "We need a churn prediction model." Probing questions: "Why does predicting churn matter now? How much revenue is churn actually costing? Is the end goal just to know who will leave, or to keep them?" Real problem (illustrative figures): the company is losing 15% of revenue each quarter as high-value customers leave. The goal is not "prediction" but cutting churn among VIP customers to 5% by detecting it early and offering suitable care.
Example 2: product recommendation. Surface request: "Build a product recommendation system to increase revenue." Probing questions: "How do we want to increase revenue? Up-selling more expensive products, or cross-selling related ones? Where will recommendations appear on the site?" Real problem (illustrative figures): customers usually buy one product and leave. The goal is to raise average order value (AOV) by 10% by suggesting products frequently bought together, right on the cart page.
2.2. Who will be the end user?
However perfect a technical solution is, it fails if users cannot or will not use it.
Example 1. Surface request: "Just export an Excel file of customers at risk of churning." Probing questions: "Who will read this file? Marketing or customer care? What is their current workflow? What will they do with the list?" Real problem: the customer care team needs the list built straight into their CRM. More importantly, they need to know why a customer is flagged, such as "usage down 50%" or "called support three times this week", to personalise the retention call.
Example 2. Surface request: "Show recommended products." Probing questions: "Who sees these recommendations? At what stage of the shopping journey? Are they in a hurry?" Real problem: the shopper is the end user, on the cart page and about to pay. Recommendations must be fast, truly relevant and non-distracting; otherwise they may abandon the cart altogether.
2.3. How will the solution be integrated and used?
This question determines the technical architecture and how the final product operates.
Example 1. Surface request: "We need a model." Probing questions: "How often must results be updated, daily or weekly? How will they reach users?" Real problem: a batch-processing system that runs at the start of each month, scans all customers and pushes the highest-risk list into the CRM through an API.
Example 2. Surface request: "A recommendation system." Probing questions: "Must predictions happen the moment a user adds an item to the cart? What latency is acceptable?" Real problem: a real-time API with latency under 100 milliseconds that takes the cart's products as input and returns a ranked list of recommended products.
2.4. How will we measure success?
This is where we move from model metrics to business metrics.
Example 1. Surface request: "The model needs over 90% accuracy." Probing questions: "What if the model reaches 90% accuracy but churn does not fall? What does a wrong prediction cost, for example flagging a loyal customer as about to leave?" Real problem: success is measured by an A/B test: actual churn among customers who receive model-driven intervention must be at least 15% lower than in a control group with no intervention.
Example 2. Surface request: "Click-through on recommendations must be high." Probing questions: "What if users click a lot but don't buy? Isn't the end goal more revenue?" Real problem: success is measured by an A/B test: average order value for customers who see recommendations must be at least 10% higher than for those who don't.
Table 1 summarises the four questions and the real problems found in the two examples.
The fourth question is where projects stumble most, because two traps are very easy to fall into. The next section analyses them.
3. Two traps in measuring success
Once the business problem has become a data problem, the next step is choosing a measure of success. Many projects go wrong here without realising it. The two common traps are trusting a high accuracy figure on imbalanced data, and confusing predicting who will churn with knowing who can be kept.
3.1. Why "90% accuracy" can be meaningless
Return to the request "the model needs over 90% accuracy". Suppose the company has 10,000 customers and only 5%, or 500, churn each month. Figure 2 compares two models (illustrative figures).

Model A is a lazy "model" that always answers "no churn". It is right for 9,500 of 10,000 customers, 95% accuracy, above the requirement. But it finds not a single customer to retain. Model B flags 1,000 customers, of whom 300 really churn. Its accuracy is only 91%, lower than A's. But it finds 60% of the customers about to leave.
With imbalanced data like this, look at precision (of those flagged, how many really churn: 30% for model B) and recall (of those who churn, how many the model finds: 60%). Then convert them into money: how much a retention call costs and how much losing a customer costs. If each call costs about USD 2 and each retained customer is worth about USD 80, model B's 1,000 calls cost USD 2,000, and keeping just 25 of the 300 customers about to leave breaks even (illustrative figures).
3.2. Predicting who will churn is not predicting who can be kept
The second trap is subtler. Even a very accurate churn model can waste the budget because it answers the wrong question. Figure 3 divides customers into four groups by how they respond to being contacted (Radcliffe & Surry, 2011).

Some customers are sure things: calling them is wasted money. Some are lost causes: they have already decided to switch provider, and no offer will keep them. Some are persuadables, who stay only if contacted: this is where the budget should go. And some are sleeping dogs: a call reminds them their contract is ending, prompting them to compare prices and leave.
A conventional churn model finds high-risk customers, lumping persuadables and lost causes together. Uplift models are built to find the persuadables specifically, and they need data from experiments with control groups. That is why question 4 almost always leads to an A/B test.
Haughton et al. (2026) tell a real story showing how this trap plays out inside an organisation. A large company had three teams: a modelling team that built response models to select customers, a marketing team that designed campaigns, and a measurement team that evaluated results. Customers in the model's top two deciles were randomly split into a group that received a mailed offer and a group that did not; a randomly chosen set of customers was split in the same way. The results are in Table 2: customers chosen by the model responded at 1.0%, more than three times the 0.3% of random customers, so the modelling team declared the model a success. But the measurement team pointed out that among model-selected customers, those who received the mail and those who did not both responded at 1.0%: the campaign created no extra customers at all. The three teams blamed one another until a new modelling manager saw that the problem lay in the model's design. It had been built to find people who would buy, whereas the business needed people who buy because they were contacted.
Table 2 shows why the right metric matters. Lift over random (bottom row, first column: 0.7 percentage points) proves only that the model finds people inclined to buy. The model's business value is a difference of differences: the effect of the offer among model-selected customers minus the effect of the offer among random customers (the bottom-right cell). That is the quantity uplift models are built to maximise, and the one that belongs in the answer to "how will we measure success?". Uplift matters most when the campaign's effect is small relative to natural purchasing: if 5% of customers buy anyway and the offer adds only 0.01 percentage points, a conventional response model learns almost nothing but natural buying behaviour. The best-known application is Barack Obama's 2012 re-election campaign: rather than contacting voters certain to vote for him or certain to vote for his opponent, the analytics team used uplift models to find the most persuadable voters and concentrated resources on them.
Asking questions is necessary, but the answers easily get lost in meeting notes and emails. A tool is needed to keep them on one page.
4. A tool to systematise: the Machine Learning Canvas
Inspired by the famous Business Model Canvas from the start-up world, Louis Dorard created the Machine Learning Canvas (MLC), a visual one-page blueprint that helps all stakeholders, from business and product to engineering, look in the same direction, use a shared language and make sure no important aspect is overlooked (Dorard, n.d.). Figure 4 shows Dorard's canvas.

The canvas has a central block and three sections, ten blocks in all. Table 3 summarises the question each block must answer; the subsections below explain them.
4.1. The central block: value proposition
This is the heart of the canvas, where the answers to the four questions in Section 2 crystallise. It forces you to state the problem, its importance and the end user clearly. An effective formula for writing it comes from Geoffrey Moore's Crossing the Chasm (Moore, 2014):
For (target customers) who (have a need or opportunity), (our product or service) is a (product category) that (delivers a benefit).
4.2. LEARN: the data and training foundation
This section covers everything to do with data and how the model "learns".
Data Sources lists every usable source: internal databases (transaction systems, data warehouses), third-party APIs, static files (CSV, Excel), web data, even the output of another machine-learning system. Naming sources clearly helps estimate the complexity of collection and integration.
Collecting Data answers: how will the system keep learning and avoid going stale (model decay)? In particular, how will we get new labels for the output? Do people need to label manually (human in the loop), and at what cost?
Features describes how raw data becomes signals the model understands. This is feature engineering: from transaction dates, we create days since last purchase, purchase frequency and so on. Consulting domain experts is especially valuable here.
Building Models sets the cycle and constraints of retraining: how often (hourly, daily, weekly), how long it takes and hidden costs such as cloud resources.
4.3. PREDICT: the core logic of the system
This section describes how the system uses the model to create results and impact.
Prediction Task defines the technical problem precisely. Despite its name, the block covers any machine-learning task: classification, regression, clustering, recommendation. Define the inputs, outputs and acceptable complexity.
Decisions is the most important bridge between results and value. It answers "So what?": how does a model's output become a business action? For example, if an email is classified as spam, the system automatically moves it to the spam folder.
Making Predictions sets when and how the system produces results for new data: every time a user opens the app or on a fixed schedule; for each data point (real time) or for a whole batch.
4.4. EVALUATE: measurement and validation
This section ensures the project's success can be measured before and after deployment.
Offline Evaluation sets criteria for assessing the model in a test environment before it affects real users: technical metrics such as precision, recall and F1, and more importantly business metrics, such as simulating revenue if the model were applied to the test set (like the break-even calculation in Section 3.1).
Live Evaluation & Monitoring tracks real-world performance and measures the business value created, usually with an A/B test: has actual churn fallen?
Ten blocks sound abstract. The best way to understand them is to fill them in for real projects.
5. Four worked examples
The four examples below correspond to four common types of machine-learning problem. Each starts with a surface request and ends with a ten-block canvas.
5.1. Classification: predicting customer churn
Situation: the leadership of a telecoms company says: "We're losing too many customers. Build a churn prediction model to fix this."
Value Proposition: "For the customer care team, who need to identify high-value subscribers at risk of leaving, our early-warning system is an analytics tool that helps them reach out proactively with suitable offers to reduce churn."
Data Sources: CRM (customer information), billing database (charge history), customer service system (support calls), usage data (data volume, call minutes).
Collecting Data: each month, the system automatically labels customers who do not renew or pay as "churned"; this data is used to retrain the model.
Features: tenure, average bill over the last three months, days overdue on payment, number of support calls, percentage change in data usage versus the previous month.
Building Models: retrained on the 1st of each month with the previous month's data, running overnight.
Prediction Task: binary classification. Input: the customer's feature vector. Output: a "churn" or "no churn" label and a probability score.
Decisions: when a customer's churn score exceeds 0.8, a high-priority task is created automatically in the CRM and assigned to a care agent, with the main reasons.
Making Predictions: batch processing on the 1st of each month, scanning all current customers.
Offline Evaluation: precision and recall on the test set, emphasising recall so high-risk customers are not missed while keeping precision high enough that agents don't waste time on customers with no intention of leaving; a simulation of retention cost versus revenue retained.
Live Evaluation & Monitoring: an A/B test comparing actual churn between the intervention group and a control group. Target (illustrative): 15% lower.
Result: a shift from "prediction" to an automated, measurable "retention action" process that can be continuously improved.
5.2. Regression: predicting house prices
Situation: a property company wants a tool "to predict house prices for sales staff".
Value Proposition: "For property sales staff, who spend a lot of time setting asking prices, the price estimator is a feature in the internal software that gives them a reliable reference price instantly, speeding up quotes."
Data Sources: internal data on completed sales, data from property listing sites, open data on zoning, schools and hospitals.
Collecting Data: each week, the system automatically updates newly listed and sold properties.
Features: floor area, bedrooms, bathrooms, district, distance to the nearest metro station, building age, parking.
Building Models: retrained every Sunday for the new working week.
Prediction Task: regression. Input: the property's feature vector. Output: an estimated value.
Decisions: the model provides a "suggested price range" (for example, USD 190,000 to 210,000, illustrative), which staff use as the basis for the official asking price (human in the loop).
Making Predictions: on demand: when staff enter a property's details, the system calls the model's API and returns the result at once.
Offline Evaluation: mean absolute percentage error (MAPE); illustrative target under 10%.
Live Evaluation & Monitoring: average time from request to quote (expected to fall by 30%) and the gap between quoted and actual sale price.
Result: a "smart assistant" that does not replace people but helps sales staff work more efficiently. Leaving the final price to staff is a deliberate choice: the model doesn't know things you only learn by visiting, such as which way the house faces or what the neighbours are like.
5.3. Clustering: customer segmentation
Situation: the marketing team of an e-commerce site proposes: "Segment our customers so we can market better."
Value Proposition: "For the marketing team, who currently send the same message to every customer, the automated segmentation system is a data source that helps them design personalised email and push campaigns to raise engagement and conversion."
Data Sources: the order database, the customer database, web behaviour data.
Collecting Data: behavioural and transaction data are collected continuously.
Features: the RFM model: Recency (last purchase), Frequency (how often), Monetary (total spend).
Building Models: rerun the clustering algorithm (such as K-Means) at the start of each week.
Prediction Task: clustering. Input: each customer's RFM vector. Output: the customer's cluster, named for example "VIP", "loyal" or "dormant".
Decisions: each segment gets a different campaign: discount codes for VIPs, a "We miss you!" email for dormant customers.
Making Predictions: weekly batch processing, updating segment labels for all customers.
Offline Evaluation: cluster-quality metrics such as the silhouette score; marketing experts review each cluster's characteristics to ensure they make business sense.
Live Evaluation & Monitoring: compare open, click and conversion rates of personalised campaigns with the previous one-size-fits-all campaigns.
Result: a vague request to "segment" becomes a data-driven marketing strategy with clear actions and metrics.
5.4. Recommendation: cross-selling in the cart
Situation: an e-commerce company asks: "Build a product recommendation system to increase revenue."
Value Proposition: "For customers viewing their cart, the 'Frequently bought together' feature is a product list that helps them find useful items easily and get more value from this purchase."
Data Sources: detailed history of successfully paid orders.
Collecting Data: new order data is collected in real time.
Features: no complex features needed; the input is the set of products in each cart.
Building Models: rerun an association-rule mining algorithm (such as Apriori) weekly to find new rules (for example, illustratively: customers who buy milk have a 60% chance of buying nappies).
Prediction Task: recommendation. Input: the products in the current cart. Output: a list of recommended products ranked by rule strength.
Decisions: when a user adds product A and a sufficiently strong rule {A} → {B} exists, the system shows B under "Frequently bought together".
Making Predictions: real time, looking up precomputed rules.
Offline Evaluation: evaluate rules by support, confidence and lift.
Live Evaluation & Monitoring: an A/B test with average order value as the core metric.
Result: choosing the right technical method goes straight at the business goal (higher AOV) and avoids building an unnecessarily complex system.
Table 4 sets the four canvases side by side on the blocks where they differ most.
Table 4 shows one thing clearly: the "live evaluation" row is always a business metric, not a model metric. The canvas forces every project to answer question 4 from the start. But the canvas is only the first step of a larger cycle.
6. When there are many problems: which to tackle first
The four examples in Section 5 show how a canvas clarifies one project. In practice a data team rarely receives one request; it receives five or six at once, each with a sponsor who believes theirs matters most. This section uses the canvas blocks themselves to set priorities.
6.1. Scoring value and feasibility
Each problem is scored from 1 to 5 on five criteria in two groups. Value covers financial impact (weight 30%) and strategic fit (20%); these scores come from the Value Proposition and Decisions blocks. Feasibility covers data readiness (20%), technical ease (15%) and low legal or reputational risk (15%); these come from the Data Sources, Collecting Data and Features blocks. The value and feasibility scores are weighted averages within each group. Table 5 scores six illustrative requests received by the data team of a telecom operator in one quarter.
Figure 5 places the six requests from Table 5 on a matrix with a value threshold of 3.5 and a feasibility threshold of 3.0.

6.2. Four quadrants, four responses
Each quadrant in Figure 5 calls for a different response. Strategic (high value, feasible enough), here request A: invest seriously and accept a longer timeline. A's high value is realised only if the project measures the right thing, that is, targets the persuadable customers of the uplift model in Section 3.2 rather than sending offers to everyone with a high churn score. Quick wins (moderate value, easy), here B and D: do them early to build trust and to let the team learn. Bernard Marr advises a priority list of one to three big strategic opportunities plus one to three quick wins, because quick wins build confidence while larger projects are still under way (Marr, 2025). Foundations first (high value, not yet feasible), here C and E: do not drop them, but do not start now either; invest in the missing conditions, mostly data. Park (low value, hard) is F.
The matrix is not a decision machine. The scores are human judgements written down for transparency, so its greatest value is to expose those judgements to debate: if the sales director scores dynamic pricing 5 on financial impact and the finance director scores it 2, the reasons for the gap are what deserve discussion. Request F sitting close to the feasibility threshold is a reminder that the boundaries between quadrants are conventions; a small change in weights moves it to another quadrant.
7. Business understanding within the CRISP-DM cycle
CRISP-DM (Cross-Industry Standard Process for Data Mining) is the most widely used process framework for structuring data-science projects (Chapman et al., 2000). It has six iterative phases that need not be followed rigidly in sequence, as Figure 6 shows.

Table 6 links each CRISP-DM phase to the Machine Learning Canvas blocks.
Table 6 shows that the canvas does not replace CRISP-DM but complements it: CRISP-DM describes the sequence of work, while the canvas records the design decisions on one page. Business understanding is the first phase of CRISP-DM and also the phase you return to after each deployment: every A/B test result feeds the questions for the next cycle.
Conclusion
Business understanding is not an unnecessary delay. It is the wisest investment you can make in a data project. By taking time to listen, ask questions and use structured tools such as the Machine Learning Canvas, you move from technician to strategic partner, someone who not only builds models but creates real value for the business.
The four questions sound simple, but each can transform a project: a "churn prediction model" can become a retention programme measured by experiment; a "recommendation system" may need only a few association rules; and a "95% accuracy" figure may find not a single customer.
Next time you receive a request, don't rush to open a Jupyter notebook. Open a conversation and a canvas. The success of the project, and sometimes saving the company from a costly mistake, depends on it.
References
Chapman, P., Clinton, J., Kerber, R., Khabaza, T., Reinartz, T., Shearer, C., & Wirth, R. (2000). CRISP-DM 1.0: Step-by-step data mining guide. SPSS.
Dorard, L. (n.d.). The Machine Learning Canvas. https://www.ownml.co/machine-learning-canvas
Haughton, D., Haughton, J., & Lo, V. S. Y. (2026). Cause and effect business analytics and data science. CRC Press.
Marr, B. (2025). AI strategy: Unleash the power of artificial intelligence in your business. Kogan Page. https://www.koganpage.com/business-strategy/ai-strategy-9781398619562
Moore, G. A. (2014). Crossing the chasm (3rd ed.). HarperBusiness. https://www.amazon.com/s?k=Crossing+the+chasm+Moore&i=stripbooks
Radcliffe, N. J., & Surry, P. D. (2011). Real-world uplift modelling with significance-based uplift trees. Stochastic Solutions White Paper. https://www.stochasticsolutions.com/pdf/sig-based-up-trees.pdf

