Byte discusses training a model with over 5 trillion parameters, and the Seed base model team adjusts the architecture
Comparatively, according to a late LatePost report, ByteDance is currently discussing a large model with training parameters exceeding 5 trillion yuan, surpassing Ali Qwen 3.8-Max (2.4 trillion yuan) and Dark Side of the Moon K3 (2.8 trillion yuan), making it the largest solution currently known in China. The plan is still in its early stages, which doesn't mean it will eventually be released. The new model is to be led by Xiang Liang, head of Seed Foundation, and in collaboration with Shen Ke, the head of pre-training data for the big language model. Seed is reorganizing the organization, dividing responsibilities, and allocating resources based on this.
Two weeks ago, ByteDance founder Zhang Yiming and Seed head Wu Yonghui held a full staff meeting. Zhang Yiming's appeasement team said that it was already very difficult to train the big model. He hoped to pursue the upper limit of intelligence and rank first in the world. He acknowledged that programming is currently the key direction, advocates integrating volcano engines, Feishu, and Doubao resources to build computing power and data advantages, and reminded not to be led by a single hotspot. He praised Seedance's differentiated leadership and clearly opposed distillation, arguing that it is difficult to truly surpass, that the AGI barrier should be built from a lower level, and said the company will continue to invest more in AI.
Seed's multi-modal performance has been outstanding in the past six months. Seedance 2.0, Seedream, etc. drive volcanic engine MaaS, but the market response of the language model Seed 2.0 has been limited, and poor coding capabilities affect the revenue structure. Byte has introduced Guo Daya to specialize in coding with a high salary, and has collected related resources. Facing the general trend of increasing model sizes in the industry, Byte hopes to seek to overtake cars on a larger scale, push for the cancellation of horse racing, break down departmental walls, and concentrate efforts.
