Dear friends,
With the rise of software engineering over several decades, many principles of how to build traditional software products and businesses are clear. But the principles of how to build AI products and businesses are still developing. I’ve found that there are significant differences, and I’ll explore some of them in this and future letters.
That AI enables new categories of products and businesses is a familiar theme. However, using this new technology — whether in a startup going from 0 to 1 or a large company incubating a new product — brings special challenges:
Unclear technical feasibility. It’s relatively well understood what a traditional mobile app or web app can do. If you can draw a reasonable wireframe, you can probably build it. But until you’ve examined the data and run some experiments, it’s hard to know how accurate an AI system can be in a given application. For example, many technologists overestimated how easy it would be to build an acceptably safe self-driving car. Generally, AI startups bring higher technical risk than traditional software startups because it’s harder to validate in advance if a given technology proposal is feasible.
Complex product specification. The specification for a traditional web app might come in the form of a wireframe, but you can’t draw a wireframe to indicate how safe a self-driving car must be. It’s extremely complex to specify operating conditions (sometimes also called the operational design domain) and acceptable error rates under various conditions. Similarly, it can be hard to write a spec for a medical diagnosis tool, depending on how acceptable different types of errors are (since not all errors are equally severe). Further, product specs often evolve as the team discovers what is and isn’t technically feasible.
Need for data. To develop a traditional software product, you might (a) interview users to make sure they want what you aim to build, (b) show them a wireframe to make sure your design meets their needs, and (c) dive into writing the code. If you’re building an AI product, you need to write code, but you also need access to data to train and test the system. This may not be a big challenge. For a consumer product, you may be able to start with a small amount of data from an initial cohort of users. But for a product aimed at business customers — say, AI to optimize shipping or help a hospital manage its medical records — how can you get access to shipping data or medical records? To work around this chicken-and-egg problem, some AI startups start by doing consulting or NRE (non-recurring engineering) work. Those activities are hard to scale, but they afford access to data that can shape a scalable product.
Additional maintenance cost. For traditional software, the boundary conditions — the range of valid inputs d — are usually easy to specify. Indeed, traditional software often checks the input to make sure, for example, it’s getting an email address in a field dedicated to that input. But for AI systems, the boundary conditions are less clear. If you have trained a system to process medical records, and the input distribution gradually changes (data drift/concept drift), how can you tell when it has shifted so much that the system requires maintenance?
Because of these differences between traditional software and AI, the best practices for building AI businesses are different. I’ll dive deeper into these differences in future letters. Meanwhile, please ask your business friends to subscribe to The Batch if they want to understand how to build an AI business!
Keep learning!
Andrew