{"url":"https://newsforagent.com/news/2026-11-18/zeng-yan-colaz-bytedance-ai-roadmap","markdown_url":"https://newsforagent.com/news/2026-11-18/zeng-yan-colaz-bytedance-ai-roadmap.md","title":"ByteDance Researcher Zeng Yan's Cola2 Paper Maps 10-Trillion-Parameter AI Push","headline":"Zeng Yan's Cola2 Paper Backs ByteDance's 10-Trillion-Parameter Model Roadmap","description":"Zeng Yan's Cola2 paper proposes diffusion-based text generation and backs ByteDance's 10-trillion-parameter model plan.","section":"Research & Clinical Trials","published_at":"2026-11-18T16:18:48+00:00","modified_at":"2026-11-18T16:18:48+00:00","author":"News Desk","key_takeaways":["Zeng Yan's Cola2 paper proposes a diffusion-based text generation method that compresses text into semantic outlines.","Semantic splitting scored 8.2% higher accuracy than fixed-length splitting in the paper's experiments.","ByteDance is pre-training a model with a maximum parameter count of up to 10 trillion, according to foreign media cited by 36kr.","Cola2 used about 346 billion tokens in training and has not converged, with an average score of 60.10 on 8 tasks."],"companies":[{"name":"ByteDance","is_primary":true,"profile_url":"https://akta.pro/company-profiles/00003uu/bytedance","website":"http://bytedance.com"}],"source":{"publisher":"36kr","title":"Zeng Yan’s New Paper Unveils ByteDance’s Ambitious 100-Trillion-Scale AI Model Roadmap","url":"https://eu.36kr.com/en/p/4075099096258182"},"other_sources":[],"tags":["Research & Clinical Trials","ByteDance"],"body_markdown":"Zeng Yan, a ByteDance researcher, published the Cola2 paper proposing a diffusion-based text generation method that compresses text into semantic outlines, with weights released on Hugging Face in the first half of November. [36kr](https://eu.36kr.com/en/p/4075099096258182) reported that the paper aligns with a roadmap toward a model with a maximum parameter count of up to 10 trillion.\n\n## Inside Zeng Yan's Cola2 Paper\n\n[ByteDance](https://akta.pro/company-profiles/00003uu/bytedance) is a Chinese-founded technology company operating TikTok, Douyin, the Doubao assistant, Seedance video generation and the Volcano Engine cloud, according to akta.pro [company records](https://akta.pro/company-data). The Cola2 paper splits text generation into two layers: a first layer that fixes meaning in a continuous latent space, and a second that turns that meaning into characters. The paper reports that semantic splitting scored 8.2% higher accuracy than fixed-length splitting, and that a compressed score of 29.4% was slightly better than 28.7% without compression.\n\nCola2 trained its outline and full-text modules together. With the compressor fixed, the score was 47.6%; joint training raised it to 53.5%, and adding a target that lets the compressor predict subsequent content raised it to 58.2%. The model used about 346 billion tokens in training, has not converged, and averaged 60.10 points on 8 tasks, surpassing Llama-3-8B on HellaSwag and StoryCloze. The paper states the training scale was \"limited by available resources.\"\n\n## ByteDance's 10-Trillion-Parameter Ambition\n\nOn November 12, The Paper reported that Zhang Yiming told an all-hands meeting for the AI team that ByteDance has the ability to compete with OpenAI and Anthropic, but that catching the cutting edge \"the earliest time will be 2029 or 2030.\" Zhang said the Seed model's pre-training data scale is \"still insufficient.\" Foreign media reported that Seed is pre-training a model with a maximum parameter count of up to 10 trillion, and 36kr reported that ByteDance has moved from a multi-line horse-racing mechanism to concentrated betting.\n\n## Earlier ByteDance AI Moves\n\nOn November 17, Pandaily reported that ByteDance's Volcano Engine announced on November 16 that the Seedance 2.5 4K API is generally available to developers and enterprises, with 10-bit color depth and a Draft mode that generates a 480p preview before final 4K rendering. On November 16, Music Business Worldwide reported that EVOX Productions sued ByteDance in California, alleging it copied 7,095 vehicle photographs to train the Seedream image generator. On November 11, Crypto Briefing reported that ByteDance is preparing to release a personal AI agent called Spell, integrated into its Doubao assistant.\n\n## What Comes Next For ByteDance\n\nZeng Yan's paper states that reaching the upper limit of its path requires larger models, longer training, more data and more computing power. Zhang Yiming said the earliest time to catch the cutting-edge level will be 2029 or 2030.\n","publisher":{"name":"News for Agent","url":"https://newsforagent.com"}}