Key Takeaways
- Meta has released Muse Spark Open, an open-weight AI model with 816 billion parameters.
- The model supports text, images, audio, and video as input and generates text as output.
- Muse Spark Open can run on high-end Apple Silicon Macs and Nvidia GPUs.
Meta has unveiled Muse Spark Open, a groundbreaking open-weight AI model with 816 billion total parameters, making it the largest in the company’s Muse family.
The model is designed to handle a wide range of tasks, including coding, tool use, and other agentic tasks, with a context window of up to 1,048,576 tokens, shared between text, images, audio, and video.
Muse Spark Open employs a Mixture-of-Experts (MoE) architecture, with 256 experts in each MoE layer, but only eight are activated per token, allowing the model to use around 42 billion active parameters per token.
The model supports various input formats, including text, images, audio, and video, and generates text as output. It includes separate encoders for visual and audio input, with the audio system processing 24kHz audio at around 12.5 tokens per second.
Meta demonstrated the model’s capabilities by showing it generating and improving 3D objects in Three.js, controlling a browser from screenshots, and performing a voice-driven shopping task.
The BF16 and NVFP4 checkpoint formats are available, and the model can run on high-end Apple Silicon Macs through llama.cpp, oMLX, and mlx-lm. A 2-bit version tested on an M3 Ultra with 512GB of unified memory used around 304GB and generated roughly 18 tokens per second.
Nvidia GPUs are also supported, with the BF16 version running across multiple H200 systems and the NVFP4 version supporting newer Blackwell GPUs such as the B200 and GB300.
The model works with frameworks including SGLang, vLLM, and Hugging Face Inference Endpoints, allowing for flexible deployment options.
Meta has made the weights of Muse Spark Open available for download from platforms such as Hugging Face, Amazon SageMaker AI, Google Cloud, and Microsoft Foundry, with the model receiving extra safety post-training and meeting the thresholds of Meta’s internal risk framework.
