SenseTime released the SenseNova-Loop-9B-Preview open-source multimodal model on 30 October.
The model is built on a looped Transformer design and accepts text and image inputs to generate text outputs. SenseTime claims that the 9 billion parameter model can match the performance of models with approximately 27 billion parameters. SenseTime also reported that the model beats other systems across office, coding, and agent tasks.
The architecture features 32 distinct decoder layers, where the first four and last four layers run once while layers 4 through 27 execute twice in sequence with shared weights. This configuration results in 56 decoder-layer executions per forward pass and stores 32 sets of layer weights. The model contains about 9 billion total parameters and approximately 14.6 billion inference parameters.
In each group of four, the decoder interleaves three linear-attention layers with one full-attention layer. SenseTime integrated two-stream manifold-constrained hyper-connections (mHC) inside the looped section. SenseTime said the mHC feature raised scores on TeamBench by 13.34 points and on GPQA Diamond by 4.42 points.
Training recipes for the model used sequences of up to 262,144 tokens including code, images, text, and tool trajectories. The model supports MTP for faster inference, while Dspark and Dflash2 remain in the training phase. The weights are available on Hugging Face under the Apache 2.0 license, and the code is hosted on GitHub.
SenseTime said it will next explore adaptive iteration counts and DSA to improve task performance and efficiency. A technical report for the model is still pending. The benchmark chart for the model covers T3-Banking, Terminal Bench 2.1, AIDABench, and BIRD-Dev.
