Get ready for a game-changer! We're thrilled to unveil SAM 3 and SAM 3D, the latest additions to our Segment Anything Collection. These models are set to revolutionize how we interact with visual content and open up a world of creative possibilities.
But here's where it gets exciting: SAM 3 allows you to detect, segment, and track objects in images and videos with ease. And it's not just about the visuals; SAM 3 understands text prompts too! Imagine being able to describe an object with words, and having the model identify and isolate it in an image or video. It's like having a superpower for content creation.
Now, let's talk about the elephant in the room: AI models have traditionally struggled with linking language to specific visual elements. Most models can only handle simple labels like "bus" or "car", but what about more detailed descriptions? This is where SAM 3 shines. It accepts a wide range of text prompts, from "red baseball cap" to more complex phrases like "people sitting down, but not wearing a red baseball cap". It's like having a personal assistant for your visual content needs!
And this is the part most people miss: SAM 3 can work hand-in-hand with multimodal large language models, taking text prompts to a whole new level. It's like giving your AI a more detailed instruction manual, allowing it to understand and execute your vision with precision.
With SAM 3, we're building a new generation of creative media tools. Our video creation app, Edits, will soon introduce effects that can be applied to specific people or objects in videos. And that's not all; new SAM 3-enabled creation experiences are coming to Vibes on the Meta AI app and meta.ai. Get ready to unleash your creativity!
But wait, there's more! SAM 3D is a game-changer in its own right. It consists of two open-source models that can reconstruct a 3D object from a single image. This is a huge leap forward in AI-guided 3D reconstruction, setting a new standard for understanding the physical world.
SAM 3D Objects and SAM 3D Body are not just about performance; they're about pushing the boundaries of what's possible. These models significantly outperform existing methods, and we've collaborated with artists to create SAM 3D Artist Objects, a unique evaluation dataset that measures research progress in 3D.
The potential applications of SAM 3D are endless. From robotics and science to sports medicine and creative pursuits like AR/VR and game development, SAM 3D opens up a world of possibilities. And for those curious about AI-enabled 3D modeling, SAM 3D is a gateway to exploring and understanding the visual world in a whole new dimension.
We're already putting SAM 3D to work with our new View in Room feature on Facebook Marketplace. This feature helps people visualize home decor items like lamps or tables in their own spaces before purchasing, bringing a whole new level of convenience and confidence to online shopping.
You can explore these cutting-edge models on our new Segment Anything Playground platform. No technical expertise is needed; just upload an image or video and start experimenting. Prompt SAM 3 with a text phrase to cut out specific objects, or use SAM 3D to view scenes from new perspectives, rearrange them, or add awesome 3D effects. We've also got templates for practical and fun video edits, from pixelating sensitive information to creating spotlight effects and motion trails.
As part of this release, we're sharing valuable resources: the SAM 3 model weights, a new evaluation benchmark dataset for open vocabulary segmentation, and a research paper detailing how we built SAM 3. We're also partnering with the Roboflow annotation platform, allowing you to annotate data and fine-tune SAM 3 to your specific needs.
For SAM 3D, we're sharing model checkpoints and inference code, and introducing a novel benchmark for 3D reconstruction. This dataset is a diverse collection of images and objects, offering a level of realism and challenge that surpasses existing benchmarks. It's a new standard for measuring research progress in 3D, and a step towards a deeper understanding of the physical world.
We can't wait to see what you create with these innovative models! Explore, build, and push the boundaries of what's possible. Your creativity knows no limits!
