ByteDance pushes toward 10-trillion-parameter AI model, aiming past Anthropic

ByteDance is reportedly in the early stages of training an AI model that could rival the scale of Anthropic's most advanced systems, signaling how far Chinese labs have come in closing the gap with top US players.

According to three people familiar with the matter, the Chinese tech giant is pre-training a model with as many as 10 trillion parameters - three times the size of Moonshot's Kimi K3, the largest Chinese model released so far. Pre-training typically takes three to six months, after which the model would be fine-tuned and released if all goes well. The final size would only be set later.

Anthropic does not publicly disclose model sizes, but industry estimates place its top Mythos 5 at around 8 trillion parameters and Fable 5 at roughly 5 trillion. While parameter count sets a model's storage capacity, real performance also hinges on data quality and training methods.

ByteDance's ambition to build one of the world's largest models reflects a broader push among Chinese labs to not just catch up but overtake US peers. In recent weeks, models from Moonshot and Alibaba have posted strong benchmark scores, trailing only Anthropic's Fable 5 in some areas. Mythos 5, meanwhile, remains restricted to approved organizations after a temporary ban in June over security concerns.

Industry insiders say several Chinese labs are already training models on the scale of Fable 5, but ByteDance is currently the most aggressive in aiming for the largest.

Source: Ars Technica