Ziqian Zhong
Hi! I'm Ziqian Zhong (approximately zea-chain-drone). I studied computer science and math in MIT. In January 2025, I will be starting my PhD at Carnegie Mellon
University where I am very fortunate to be advised by
Aditi Raghunathan. Before that, I will be working as a research scientist at Pika.
Research interest: Understanding and improving large language models. Large language models like transformers, are complex structures with much yet to be unraveled. Due to their extensive search space and high training cost, improving, controlling, and aligning them requires a profound and principled approach. My research aims to gain insights on large language models by exploring their mechanisms, behaviors, and limitations. [Resume]
![]( "Email")
![]( "Twitter")
![]( "Github")
![]( "DBLP")
Research interest: Understanding and improving large language models. Large language models like transformers, are complex structures with much yet to be unraveled. Due to their extensive search space and high training cost, improving, controlling, and aligning them requires a profound and principled approach. My research aims to gain insights on large language models by exploring their mechanisms, behaviors, and limitations. [Resume]
Figure 1: Positional Query-Key Pattern of Some Transformer
Publications
For theoretical computer science papers, conventionally authors are
listed in alphabetical order; *: equal contribution
The Clock and the Pizza: Two Stories in Mechanistic Explanation of
Neural Networks
[Arxiv]
Ziqian Zhong*, Ziming Liu*, Max Tegmark, Jacob Andreas (Oral at NeurIPS 2023)
Grokking as Compression: A Nonlinear Complexity Perspective
[Arxiv]
Ziming Liu*, Ziqian Zhong*, Max Tegmark (NeurIPS UniReps Workshop 2023)
On Problems Related to Unbounded SubsetSum: A Unified Combinatorial
Approach
[PDF]
Mingyang Deng*, Xiao Mao*, Ziqian Zhong* (Presented at SODA 2023)
New Additive Approximations for Shortest Paths and Cycles
[PDF]
Mingyang Deng*, Yael Kirkpatrick*, Victor Rong*, Virginia
Vassilevska Williams*, Ziqian Zhong* (ICALP 2022)
New Lower Bounds and Upper Bounds for Listing Avoidable Vertices
[PDF]
Mingyang Deng*, Virginia Vassilevska Williams*, Ziqian
Zhong* (MFCS 2022)
Selected Awards
Gold Medal, Fourth Place
@ International Olympiad in Informatics 2019
Second Place
@ 46th ICPC World Final
Fourth Place
@ Universal Cup Summer Summit 2024
First Place
@ ICPC North America Championship 2022
Fourth Place
@ Meta Hacker Cup 2023
Second Place
@ MIT Weblab Web Programming Competition 2022
Second Place
@ Baidu AStar Programming Contest 2020
Honorable Mention
@ Alibaba Global Mathematics Competition 2022
Honorable Mention
@ Putnam Mathematical Competition 2022
Side Projects
CP Ideas
[Link]
Generate competitive programming problems with GPT-3
Is my problem new?
[Link]
Semantic search demo for competitive programming problems.
Mosaic Detective
[Link]
Guess the word from an image downscaled
Light Synth
[Link]
Synthesizer, but with balls and whistles
Pitch Trainer
[Link]
Microtonal ear training (in Chinese)
Reedle
[Link]
Wordle, but with regex