Turning machine learning ideas into products

It’s no exaggeration to say that my team and I are obsessed with getting the most out of machine learning. That’s why we built the Capital One Machine Learning Business Model Canvas. Our Canvas — loosely based on Strategyzer’s concept aimed at more strategically approaching business model development and implementation — is updated to focus specifically on machine learning-based business projects, including the inputs, training, outputs, and success criteria that we believe can help turn machine learning ideas into actual products. Further, we hope this approach can help advance you and/or your company’s machine learning product ideation by putting potential products to the test and making them more viable.

To overcome the common (and uncommon) mistakes that have killed so many machine learning ideas, we’ve developed a step-by-step approach to increasing your potential product’s chances of success. In this post, we’re going to give you steps from our updated Canvas to get this process started.

Let’s go.

The first thing to figure out is how your machine learning product will improve the company — we’re talking here about things like reducing costs, saving time, increasing intelligence. Think big!

Next, identify everyone who could benefit from your ML product. Who will use it? Who will benefit? Besides the most obvious people, whose work will improve because of your product?

Finally, consider all the ripple effects. Imagine not only how this will affect your team, but also how it will shape decision making throughout the company.

Your goal should be to determine what the full impact of the ML product will be, which will increase the viability of your project.

To figure out if your ML product is viable, you need to understand the problem you’re addressing — what’s the business challenge? Explore how this problem is creating trouble. Answer why this problem has not been solved yet.

The next phase is to define — step by step — what is required to solve the problem manually. Identify all the pain involved with accomplishing what you need to do. For example, would it require 100 people to listen to a set of customer service calls, and to subsequently search millions of transactions to identify those raised by the customer?

If you don’t do a good job of precisely defining your problem early on, you could waste valuable time and resources. So rather than saying, “customers aren’t happy enough,” you need to zero in on a specific problem that your machine learning tool can address in an observable and measurable way.

Also: make sure your problem can’t be solved through traditional means like regression. Keep in mind, machine learning solutions work best when confronting problems that are complex and have many interconnected variables.

When thinking about how a machine will learn to solve a problem, make sure you understand the data sources needed. What information will be essential? How must the data points be combined? How will you get access and permission to the data?

Also, remember — humans and machines learn differently. Humans need just a few examples of target training samples to learn. Machines often require thousands. Be prepared to create a very large body of training samples. And don’t forget, the success of the machine learning system will depend in large part on the training set. Does it reflect predictions made of the larger sample?

People who are new to machine learning sometimes seek a “magic” solution that “just knows” how to get it right. But you can’t just want an ML system to become a reality, you must envision what the system will look like.

On top of that, it’s critical that you decide what success is when it comes to your product. What will it have to do to be deemed a success? How will success be measured? And is there a role for humans in this ML product?

It’s easy for ML beginners to fail to consider problems — that is, when something goes wrong and the model makes a mistake. What would happen, for instance, if your machine learning product gives the right answer to the wrong question? Is that a serious mistake? Or is it no big deal? There can be a huge difference depending on your goals. Consider the consequences of errors and what precisely you will require for overall success.

The steps in our Canvas will help you to think realistically about the product you’re developing, to work well with your technical teams to achieve faster outcomes, and to develop a convincing business case that will help earn your product the greenlight. The result will be a successful machine learning product that solves a specific problem.

Our goal here is simple: unleash the smartest, most efficient and most effective ML products in the industry. And by helping turn a vision like yours into a reality, we’re getting that done.

By Jie “JZ” Zhang, Product Manager, Capital One and John Whalen, Principal, Psychological Insight & Innovation at Brilliant Experience