The year 2023 was an inflection point in the development of broadly useful AI systems across text, image, video, audio, and other modalities. At Runway alone, we saw the release of video-generation models such as Gen-1 and Gen-2, as well as tools that enable new forms of creative control with those models. In the coming year, here are some areas where I expect to see continued progress:
- Video generation: Over the past year, generative video models (text-to-video, image-to-video, video-to-video) became publicly available for the first time. In the coming year, the quality, generality, and controllability of these models will continue to improve rapidly. By the end of 2024, a nontrivial percentage of video content on the internet will take advantage of them in some capacity.
- Real-time interactivity: As large models become faster to run and we develop more structured ways to control them, we’ll start to see more novel user interfaces and products emerge around them that go beyond the usual prompt-to-x or chat-assistant paradigms.
- Automating AI research: Developers have embraced coding assistants based on large language models such as GitHub Copilot. But little tooling has been designed to accelerate AI research workflows specifically; for instance, automating a lot of the repetitive work involved in developing and debugging model code, training and evaluating models, and so on. More of these tools will emerge in the coming year.
- More emphasis on systems: Much conversation has focused on the capabilities of individual networks trained end-to-end. In practice, however, AI systems deployed in real-world settings are usually powered by a pipeline of models. More frameworks will appear for building such modular systems.
Beyond technological advancements, the most rewarding part of building these systems is that, with every update and increase in capabilities, new audiences are introduced to them and new stories are told that weren’t told before. I’m excited to see how that will continue to happen in the coming year.
Anastasis Germanidis is co-founder and CTO of Runway, an applied AI research company shaping the next era of art, entertainment, and human creativity.