Apr 24, 2024
Why We Need More Compute for Inference: Today, large language models produce output primarily for humans. But agentic workflows produce lots of output for the models themselves — and that will require much more compute for AI inference.
Much has been said about many companies’ desire for more compute (as well as data) to train larger foundation models.