I am currently a first year Ph.D. student at Gaoling School of Artificial Intelligence in Renmin University of China, supervised by Prof. Yankai Lin. My research interests focus on foundation language models pre-training, particularly Mixture-of-Experts (MoE).

News within this year

• 2026.06: We propose “Routers with Manifold Power Iteration”, a fresh perspective to address the inherent flaw in MoE router design.
• 2026.05: “EmbedFilter” is accepted by KDD 2026 Oral, a linear filter designed to refine zero-shot text embeddings.
• 2026.04: Two papers, “Union-of-Experts” and “AlignX”, are accpeted by ACL 2026.
• 2025.09: “PolarQuant” is accepted by NeurIPS 2025, we propose a polar transformation perspective for KV Quant for the first time.
• 2025.06: I am selected by 2025 CCF-Tencent Rhino-Bird Elite Talent Program.

Publications & Preprint

Foundation Language Models

• Redesign Mixture-of-Experts Routers with Manifold Power Iteration. Songhao Wu et al., (Preprint)
• Union-of-Experts: Neurons in Mixture-of-Experts are Secretly Routers. Songhao Wu et al.
  Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics, (ACL’26).
• Autonomy-of-Experts Models. Ang Lv et al. Proceedings of the 42nd International Conference on Machine Learning, (ICML’25).
• PEAR: Position-Embedding-Agnostic Attention Re-weighting Enhances Retrieval-Augmented Generation with Zero Inference Overhead.
  Tao Tan*, Yining Qian* and Ang Lv* et al. Proceedings of the ACM on Web Conference 2025, (WWW’25 Oral).

Efficient LLM

• PolarQuant: Leveraging Polar Transformation for Efficient Key Cache Quantization and Decoding Acceleration.
  Songhao Wu and Ang Lv* et al. Thirty-ninth Annual Conference on Neural Information Processing Systems, (NeurIPS’ 25).

Information Retrieval

• Your UnEmbedding Matrix is Secretly a Feature Lens for Text Embeddings. Songhao Wu and Zhongxin Chen et al.
  Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining, (KDD’26 Oral).
• Bridge the Gap between Past and Future: Siamese Model Optimization for Context-Aware Document Ranking.
  Songhao Wu et al. Proceedings of the 33rd ACM International Conference on Information and Knowledge Management, (CIKM’24).
• Unify Graph Learning with Text: Unleashing LLM Potentials for Session Search.
  Songhao Wu et al. Proceedings of the ACM on Web Conference 2024, (WWW’24).

Other Publications

• From 1,000,000 Users to Every User: Scaling Up Personalized Preference for User-level Alignment.
  Jia-Nan Li et al. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics, (ACL’26).

Academic Services

• Conference Reviewer: ICML (Silver), NeurIPS, ICLR, ARR Reviewer

Internships

• 2025.6- Now, Research Intern, Pretraining Team, Tencent Hunyuan. Mentor: Ruobing Xie.
• 2024.12 - 2025.5, Meituan
• 2023.9 - 2024.11, Ant Group