Meta

36 Posts

Tuning LLMs for Better RAG: Meta’s RA-DIT boosts language model output by optimizing text retrieval
Meta

Tuning LLMs for Better RAG: Meta’s RA-DIT boosts language model output by optimizing text retrieval

Retrieval-augmented generation (RAG) enables large language models to generate better output by retrieving documents that are relevant to a user’s prompt. Fine-tuning further improves RAG performance.
Swiss Army LLM
Meta

Swiss Army LLM

The combination of  language models that are equipped for retrieval augmented generation can retrieve text from a database to improve their output. Further work extends this capability to retrieve information from any application that comes with an API. 
The Big Picture and the Details: I-JEPA, or how vision models understand the relationship between parts and the whole
Meta

The Big Picture and the Details: I-JEPA, or how vision models understand the relationship between parts and the whole

A novel twist on self-supervised learning aims to improve on earlier methods by helping vision models learn how parts of an image relate to the whole.
Dozens of phones display Meta's apps makeover
Meta

Facebook’s Generative Facelift: All about the AI upgrades of Meta's Messenger, WhatsApp, and Instagram

Meta is rolling out AI-powered upgrades to its social platforms. Meta announced a chat interface, image generator, and celebrity tie-ins for Facebook, Instagram, Messenger, and WhatsApp. The new capabilities take advantage of LLaMa 2 and an unnamed image generator, presumably CM3leon.
Different Media, Similar Embeddings: ImageBind, the AI model that binds data from seven data types at once
Meta

Different Media, Similar Embeddings: ImageBind, the AI model that binds data from seven data types at once

The ability of OpenAI’s CLIP to produce similar embeddings of a text phrase and a matching image opened up applications like classifying images according to labels that weren’t in the training set. A new model extends this capability to seven data types.
Text-To-3D Animation: MAV3D, a method for generating 3D dynamic scenes from text descriptions
Meta

Text-To-3D Animation: MAV3D, a method for generating 3D dynamic scenes from text descriptions

Text-to-video generation is so 2022! A new system takes in text and generates an animated 3D scene that can be viewed or rendered from any angle.
AI Firms Agree to Voluntary Guidelines: U.S. companies agree to uphold a list of responsible AI commitments.
Meta

AI Firms Agree to Voluntary Guidelines: U.S. companies agree to uphold a list of responsible AI commitments.

In the absence of nationwide laws that regulate AI, major U.S. tech companies pledged to abide by voluntary guidelines — most of which they may already be following.
Where Is Meta’s Generative Play?: Why Meta still lacks a flagship generative AI service
Meta

Where Is Meta’s Generative Play?: Why Meta still lacks a flagship generative AI service

While Microsoft and Google scramble to supercharge their businesses with text generation, Meta has yet to launch a flagship generative AI service. Reporters went looking for reasons why.
Example of interactive editing sessions with Meta's text generator PEER
Meta

Collaborative Text Generator: A language model that collaborates with human writers

Text from current language models can be useful as a rough draft, but that leaves the polishing to human writers. A language model learned how to generate and respond to editorial directions.
Runaway LLaMA: How Meta's LLaMA NLP model leaked
Meta

Runaway LLaMA: How Meta's LLaMA NLP model leaked

Meta’s effort to make a large language model available to researchers ended with its escape into the wild. Soon after Meta started accepting applications for developer access to LLaMA, a family of trained large language models...
Architecture for PointGoal Navigation on a legged robot
Meta

Streamlined Robot Training: Robots trained in lo-fi simulation perform better in reality.

Autonomous robots trained to navigate in a simulation often struggle in the real world. New work helps bridge the gap in a counterintuitive way.
Graph with difference in test error in keeping hard versus easy examples
Meta

Unsupervised Data Pruning: New method removes useless machine learning data.

Large datasets often contain overly similar examples that consume training cycles without contributing to learning. A new paper identifies similar training examples, even if they’re not labeled.
Dataset FOLIO example based on the Wild Turkey Wikipedia page
Meta

Language Models Defy Logic: Large NLP models struggle with logical reasoning.

Who would disagree that, if all people are mortal and Socrates is a person, Socrates must be mortal? GPT-3, for one. Recent work shows that bigger language models are not necessarily better when it comes to logical reasoning.
Alon Halevy next to a big computer screen
Meta

Alon Halevy: Facebook AI director Alon Halevy envisions your personal data timeline

The important question of how companies and organizations use our data has received a lot of attention in the technology and policy communities. An equally important question that deserves more focus in 2023 is how...
Diagram explaining Atlas, a retrieval-augmented language model that exhibits strong few-shot performance on knowledge tasks
Meta

Memorize Less; Retrieve More: How small language models can perform specialized tasks.

Large language models are trained only to predict the next word based on previous ones. Yet, given a modest fine-tuning set, they acquire enough information to learn how to perform tasks such as answering questions.
Load More

Subscribe to The Batch

Stay updated with weekly AI News and Insights delivered to your inbox