Letters
From Prompts to Mega-Prompts: Best practices for developers of LLM-based applications in the era of long context and faster, cheaper token generation
In the last couple of days, Google announced a doubling of Gemini Pro 1.5's input context window from 1 million to 2 million tokens, and OpenAI released GPT-4o, which generates tokens 2x faster and 50% cheaper than GPT-4 Turbo and natively accepts and generates multimodal tokens.