How many startups did OpenAI kill during that event?

In other words, what startups are no longer necessary/relevant given what OpenAI launched.

The related question that comes up is:

How does one build a defensible A.I. startup?

Let’s take a step back. The way you build a defensible A.I. startup is the way you build any kind of defensible startup. A.I. startups are interesting now because of the new opportunities that generative A.I. has unlocked — and because the speed at which things are happening.

Also, recognize that this question of defensibility is not a binary one. It’s not that one startup is defensible and another is not. It’s a spectrum of defensibility. And, there are different factors that will increase the defensibility.

Factor 1: Creating Enough Customer Value

The first thing to recognize: The most common threat your startup has of being rendered irrelevant is not that OpenAI (or someone else) renders it irrelevant because of a new launch. The most common cause of death is that your startup wasn’t relevant enough in the first place because it didn’t create enough customer value. Note: Chances are, you’re creating some customer value (no entrepreneur intentionally starts a company that doesn’t create customer value). The issue is that you have to create enough customer value to overcome the natural inertia of the market. In other words, the energy of your startup has to overcome the friction people have to go through to even consider your product.

Factor 2: Doing Hard, Helpful Things

To increase defensibility, you need to do things that are relatively hard to do which makes your product more helpful to customers. The reason they need to be hard things is that otherwise others can readily do it and you have a hard time differentiating.

This is usually where the new crop of A.I. startups often fail. Often, they are described as “thin wrappers around the GPT APIs”.

Here’s an example: You build an A.I. powered tool that helps the recruiting department write effective job postings. A couple of years ago, that would have seemed magical because nobody had really figured out how to understand natural language — or to write in natural language. But now, with the popularity of ChatGPT (and the GPT APIs more broadly), this is a widely available technology. So, the hard thing you were doing is no longer hard. In fact, it’s easy enough that customers can reasonably just use ChatGPT to get a decent approximation of what they need. Perhaps not as good as yours, with its fancy prompt-engineering, but often good enough.

So, to increase your defensibility, you need to do things of value that are going to be hard for others to do for some period of time. If all you’re effectively doing is supplying GPT a really fancy prompt, that’s unlikely to be enough if your market opportunity is large enough.

There’s an important point embedded in there: The more customer value you are creating, the more economic opportunity your startup has. And, the more opportunity there is, the more competition that opportunity will draw. So, you’re trying to balance doing something interesting enough to be valuable, but have a counter-balancing difficulty that will limit the number of competitors that also attempt it.

This one’s worth breaking down, in terms of the types of hard things you can do.

1. You have access to a proprietary asset (like data) that others don’t have easy access to. In our “write job postings” example, perhaps you have a corpus of thousands of job postings including some outcome scores (as to how well they did). You could use this data to create better job postings. Others don’t have ready access to this data. Note: The asset doesn’t have to be data. It could be prior code that you can leverage. It could be hard-to-gain partnerships with suppliers. Anything that’s valuable and hard.

2. You have efficient access to customers. This is an oft-ignored one. If you have built channels to reach customers in efficient ways, that’s an advantage. Even if others build a similar (or even slightly better) product, you can still have defensibility if you have a way of more easily gaining users/customers. Specifically in the A.I. world, often access to users/customers provides a related advantage: more data. It’s possible to create a virtuous loop whereby the more users of your product, the better the product gets (because of feedback loops) thereby making it easier to get even more users. That’s often hard for others to replicate.

3. You have a network effect. A simple and common example is if you’re creating a marketplace of some sort. These are often hard to do — but that’s precisely what makes them so valuable, because once you have a strong network effect in place, it’s really hard for others to replicate or displace it.

Ok, so let’s dig in a bit specifically to A.I. startups. Some additional thoughts:

1. Building foundational LLMs is exceptionally hard (and expensive). So, there’s an opportunity to create some defensibility there. But the challenge is that what is hard today may be rendered not-so-hard tomorrow either through open source (which is moving at a torrid pace) or a release by one of the existing LLM providers (like OpenAI).

2. If you’re building dev tools that make it easier for developers to leverage generative A.I., you are at the risk of the LLM providers themselves building that into their offering. They have every incentive to make their platform easy and useful. They’re not intentionally trying to kill you, they’re just trying to better serve the developers on their platform.

3. If you’re building a “ChatGPT for [x]”, it’s important to be mindful of whether the differentiation you have on top of ChatGPT is really enough. Especially now that OpenAI is launching custom “GPTs” with their own app/GPT store.

To close out, let’s summarize.

Here’s what I’d ask myself: How valuable is what I’m doing? What makes this hard for others to do? Could this get dramatically easier because of what others launch or how the industry evolves? If this does get easier, do I have a “Plan B” to create differentiated value?

The important thing is to be honest with yourself.

And now, with a slew of updates and improvements launched at their first Dev Day event yesterday (Nov 6 2023).

Lots has been written about all the new features and capabilities, so I’m not going to dig into that here. But, not a lot has been written about the strategic benefits of some of these launches.

LLMs Are Somewhat Of a Commodity

GPT-4 is the most capable LLM out there. But that does not confer absolute strategic advantage as one might think.

Why?

Because if you put aside the power of an individual model (like GPT-4), the underlying interface is actually very simple.

LLMs are simply a function that takes text in (the prompt) and provides text back out. Now granted, there’s a lot of variability in power and capability, but the interface is quite consistent.

This is why it’s relatively easy to move from one model to the other to try it out. If you’re using GPT-3.5 or GPT-4, trying out Anthropic’s Claude (to gain access to the 100k tokens) or to the open source Mistral LLM is relatively straight-forward. You’re still passing text in and getting text out. And there are new, quite capable LLMs coming out seemingly every week.

Because the interface is simple and consistent, as new models come out, it’s not too hard to actually try out a new model or in some cases, use model X for some use cases and model Y for other use cases.

Strategic Impact Of Dev Day Launches

Now, let’s look at some of the key new capabilities that OpenAI launched. I want to focus on the ones that don’t make the foundational model better per se, but just make the model easier to work with.

Anything that raises the level of abstraction that developers can work at has an advantage, because the higher the level of abstraction, the higher the number of developers that you get on the platform (all things being equal).

But, there’s a subtle but significant side-effect of these improved abstractions: In order to benefit from these better abstractions: The underlying interface is no longer simply pass text-in, get text-out.

For example: In the new Assistants API, OpenAI will now do the memory management for you. So you can implement a conversational-style interaction in your chatbot without having to worry about context windows, sliding windows of memory, selective summarization or a bunch of other things. You just use the new APIs to interact with the LLM and it manages all the memory for you.

Why is this a big deal?

Because the more developers start using this new API with memory management, the less of a commodity GPT-4 becomes. Now, you can’t just willy-nilly switch to another model that comes out next week without first considering whether it supports memory management and even if it does, you have to figure out whether your code has to change to match however that new model supports memory management. OpenAI does it elegantly with the notion of assistants, threads and messages. But there’s no requirement that other LLMs have to use those same concepts.

Same with the new Retrieval features in the Assistants API. You get a lot of power, and you get it simply, but you have to use the feature in the way OpenAI designed it.

Same with Code Interpreter and data analysis.

All of these are massively powerful features — and all make OpenAI’s platform different from the other models that are out there.

Different not just in what OpenAI can do, but different in terms of how you interact with the platform and use the capabilities.

Net result: The switching costs go up the more people use these features as a result of which the moat around OpenAI widens. That’s the strategic benefit.

Now, of course, there will be open source libraries like LangChain that will emerge to help abstract away these differences and help you move across models while still preserving some of these new features — but as well intentioned and well executed as those implementations will be, there will always be leaks in the abstractions. Things won’t always work exactly the way you want. Switching models will still require some thought and consideration.

So, people will stay in the warm confines of OpenAI’s platform longer because there’s no reason to try out other models and increasing reason not to. It’s cold and chaotic out there.

And just like nobody got fired for buying IBM back in the day, nobody gets fired for building with OpenAI.

This makes OpenAI the company more valuable. It’s strategy 101.

p.s. Apologies for the wonky image/illustration. I was lazy and just gave this post to DALL-E 3 and let it create a visual. It’s not awful, but there’s clearly work to do.

Before we dig in, quick disclosures:
1. I’ve been a long-time fan of OpenAI — and am also an investor.

2. I have been tinkering with alpha versions of the APIs for a little while now.

Here’s what I’m personally most excited about.

#1 Memory/Thread Management

If you’ve done any development of a chat bot (like ChatSpot.com) using GPT, you’ve likely wanted to implement some sort of “memory” so that users can have a conversation with your chat bot (similar to what can be done in ChatGPT). This allows users to ask “follow-up” questions.

The problem is that implementing such memory is non-trivial. First off, all memory has to be squeezed into the context window and it’s often hard to know a priori how much context window you should be consuming for the memory vs. the prompt output. Then, you likely need to “summarize” some of the conversation in order to compress more of the previous conversation into the context window. And, if the kinds of things you have in memory is actually data (like the results of a data query), life is really hard, because there’s no good way to summarize that.

Now, with the new Assistants API, OpenAI does all of the heavy lifting for you. You just create a “thread” and add messages to that thread. All the management of memory is done for you.

#2 Retrieval Support in API

One of the most common approaches to implementing LLM apps that need to have access to custom knowledge is using what’s called Retrieval Augmented Generation (RAG). With RAG, you take the custom knowledge/data you have and create vector embeddings of it and store in a vector database of some sort. Then, when a user submits a query, you do a semantic search to find the most relevant documents based on the user query. You then pass *those* documents along to the LLM inside the context window along with the user query. This all works pretty well, but takes some effort.

OpenAI has now made that easy with support for retrieval right in the API. When building out your bot/assistant, you can first up[load your custom knowledge. GPT will then access that knowledge as needed in order to respond to user prompts. It basically takes care of the RAG for you.

#3 Code Interpreter Support

With the new Assistants API you can now also enable “code interpreter” support (also known as data analysis support). This lets you leverage a Python runtime engine right inside GPT so GPT can generate and run code to do data analysis or otherwise respond to a user prompt. Very, very powerful.

That’s what I’m most excited about. There’s also of course the new 128k token context window (huge!), support for multi-modal input/output, etc. But these are the 3 things I’m most excited for as a developer right now.