Yao Shunyu reorganizes Tencent's multi-modal route: closer to Liang Wenfeng and away from Li Feifei
Text | Miao Zheng Editor | Wang Jing Source | Letter AI Tencent's mixed element multi-modal team has undergone another personnel change. According to media reports, Lin Xudong, who was responsible for xAI's multi-modal understanding, has left xAI and joined Tencent's mixed element as the head of the multi-modal content generation algorithm. The reason this personnel news is worth paying attention to is that it takes place in the context of continuous adjustments of mixed and multi-modal teams. Over the past period of time, news of the departure of the person in charge, the transfer of researchers, and the addition of new members came out one after another within the mixed yuan. Hu Han, the former head of multimodal understanding, left his career to start a business, and Tian Yonglong and others joined Tencent. The reporting relationship between the original multi-modal team also changed with the integration of the big language model department and the multimodal model department. However, does this mean that Tencent's multi-modal team is “changing the dynasty” is currently unable to draw a direct conclusion. What can be confirmed by public information is that mixed forces have indeed experienced personnel movements and organizational restructuring. The rumor of Lin Xudong's addition is more like a new signal in this adjustment: Tencent is recombining the two routes of multimodal understanding and content generation. So the question is, what exactly did Lin Xudong come from, and what abilities can he add to Tencent? And is Tencent's multi-modal approach shifting from “generating content” to Yao Shunyu's more biased “understanding context and acting in the world”? What is Lin Xudong's origin and what can he do after joining Tencent? According to public information, Lin Xudong graduated from Tsinghua University in 2018 and then went to Columbia University to study for his doctorate. While studying at the blog, his research interests included embedded learning, video analysis, and generative models. He also participated in the Vx2Text project in collaboration with Columbia University and Facebook AI. V indicates video, x indicates unknown, can be sound, voice, or even ambient sound. 2 represents TO, and Text represents subtitles. Its logic is to first convert different modes such as video and sound into vectors similar to “language tokens”, then uniformly feed the language model for fusion, and finally generate open text by an autoregressive decoder. Transformer can only understand tokens, so AI essentially doesn't understand video and audio file formats, making it even less likely to convert them into text. For example, if a dog jumps into the water next to a swimming pool, Vx2Text's video recognizer (V) will output keywords: dog, jump, pool; sound reader (x) will output: sound of water, fluttering. Although the product function of Vx2Text is “generation,” the core difficulty of the product is “understanding.” Of course, Vx2Text doesn't simply “translate” a screen into a few sentences. Models need to recognize people, objects, movements, and events from videos, understand how these things change over time, and finally organize visual information into language. After graduating from his PhD, Lin Xudong joined DeepMind and participated in Gemini-related multi-modal pre-training and post-training work. In 2025, he also joined xAI. According to public information, it is responsible for the direction of multimodal understanding and participating in the training of multimodal content understanding and generation models. Now that he has joined Tencent Hybrid, he will be responsible for the hybrid multi-modal content generation algorithm. Lin Xudong was added not so much to improve the performance of mixed-element multi-modal generation, but rather to solve a problem that plagues all multimodals — understanding. The previous generation model was more like a picture maker. Give it a hint, and it can generate an image or a video. But as long as users make more complex requests, the model just can't keep up. For example, the characters change in the long video, the shape of the object is not consistent before and after, the camera movement does not match the spatial relationship, etc. It's not because the model doesn't generate, but because it doesn't remember and understand the world steadily. Therefore, putting Lin Xudong in the position of multi-modal content generation is probably because he “translated” multi-modality into something AI can understand. Lin Xudong's addition can only be clearly seen in a larger context. That is, now Tencent's mixed element is reorganizing its multi-modal route. In January 2025, Tencent Outstanding Scientist (Tencent Distinguished Scientist) Hu Han succeeded Liu Wei, who had previously left his job, and was fully responsible for the research and development of mixed-element multi-modal models, and also served as Tencent's mixed-element big model Tech Lead. Tencent's internal organization was adjusted in the second half of 2025. He transferred from the Multimodal Model Department to the “Frontier” Frontier Technology Research Group under the Big Language Model Department. The title was changed to Head of the Multimodal Understanding Direction, and the reporting line was also changed to report to Yao Shunyu. The actual position changed from “the head of an independent department” to a “big language model...
