Zhiyuan You | 尤志远 Ph.D. Candidate The Chinese University of Hong Kong zhiyuanyou@foxmail.com | zhiyuanyou.ai@gmail.com

Biography

I am a Ph.D. candidate at Multimedia Laboratory (MMLab), The Chinese University of Hong Kong (CUHK). I am supervised by Prof. Tianfan Xue and Prof. Chao Dong. I also work closely with Dr. Jinjin Gu.

Prior to that, I received my M.Eng. from Shanghai Jiao Tong University (SJTU) in 2023, supervised by Prof. Xinyi Le and Prof. Yu Zheng, and my B.Eng. from the same university in 2020.

I previously conducted research at several leading institutions, including ByteDance Seed, Adobe, Shanghai AI Lab, Horizon Robotics, and SenseTime.

My current research interest mainly lies in generation models, multi-modal systems, vision-centric agent, and low-level vision.

News

[2026.09]  Three papers (One Position Track) to appear in NeurIPS 2026.
[2026.07]  One paper to appear in ACM MM 2026.
[2026.05]  One paper to appear in ICML 2026 (Oral).
[2026.03]  One paper to appear in ACL 2026.
[2026.03]  One paper to appear in PR.
[2026.02]  Three papers (One Findings Track) to appear in CVPR 2026.
[2025.11]  One paper to appear in TIP.
[2025.11]  End of my internship at Adobe — heartfelt thanks to all the teams.
[2025.09]  Two papers (One Datasets & Benchmarks Track) to appear in NeurIPS 2025.
[2025.08]  One paper to appear in TOG, SIGGRAPH Asia 2025 (Journal Track).
[2025.07]  We win the Championship in the VQualA 2025 DIQA Challenge (ICCV 2025 Workshop).
[2025.03]  One paper to appear in TPAMI.
[2025.02]  Two papers to appear in CVPR 2025 (One Highlight). See you in Nashville.
[2025.01]  One paper to appear in ICLR 2025.
[2024.12]  One paper to appear in AAAI 2025 (Oral).
[2024.09]  One paper to appear in NeurIPS 2024 (Spotlight).
[2024.08]  One paper to appear in TMLR.
[2024.07]  One paper to appear in ECCV 2024. See you in Milan.
[2023.08]  I become a Ph.D. student at MMLab in The Chinese University of Hong Kong.
[2023.03]  I graduate and receive my Master's degree from Shanghai Jiao Tong University.
[2022.09]  One paper to appear in NeurIPS 2022 (Spotlight).
[2022.09]  One paper to appear in ICONIP 2022 (Oral).
[2022.08]  One paper to appear in WACV 2023.
[2020.07]  I graduate and receive my Bachelor's degree from Shanghai Jiao Tong University.

[Show More]

Publications

*: Equal Contribution, †: Corresponding Author

Harnessing Image Diffusion Prior for Photo-Realistic Video Restoration

Fanghua Yu, Hongyu An, Jinfan Hu, Xinqi Lin, Zhiyuan You, Jian Wang, Hongyang Li, Chao Dong†, Jinjin Gu†

Neural Information Processing Systems (NeurIPS), 2026

[Paper]

We propose HYPVR, which transfers powerful image generation priors to video restoration while synthesizing fine-grained details with temporal consistency.

LongSCOP: Semantically Consistent Long Video Outpainting

Yu Tang*, Zhiyuan You*, Zhixiao Wang, Fanghua Yu, Qingyu Zhang, Xiang Yin, Chao Dong†, Jinjin Gu†

Neural Information Processing Systems (NeurIPS), 2026

[Paper]

We introduce LongSCOP, a framework for stable and semantically consistent long-video outpainting with long-horizon generation and VLM-based reference retrieval.

Evaluation of Visual Processing Should Be Human-Centered, Not Metric-Centered

Fanghua Yu*, Jinfan Hu*, Zhiyuan You, Xiang Yin, Hongyu An, Xinqi Lin, Chao Dong†, Jinjin Gu†

Neural Information Processing Systems (NeurIPS), 2026, Position Paper Track

[Paper]

We argue that visual processing evaluation should move beyond metric-centered benchmarks toward human-centered, context-aware, and fine-grained assessment.

Generalizable Blind Image Quality Assessment with Retrieval Augmentation and Contrastive Vision-Text Local Meta-Training

Yan Zhong, Zhiyuan You, Xinping Zhao, Li Zhang, Xinyuan Song, Ruoyu Zhao, Lei Shi, Tingting Jiang

ACM Multimedia (ACM MM), 2026

[Paper]

We propose Relocat-IQA, a retrieval-enhanced blind image quality assessment framework with contrastive vision-text learning and local meta-training for improved generalization.

PhotoAgent: Agentic Photo Editing with Exploratory Visual Aesthetic Planning

Mingde Yao, Zhiyuan You, King-Man Tam, Menglu Wang, and Tianfan Xue†

International Conference on Machine Learning (ICML), 2026, Oral

[Project Page] [Paper] [Code]

We propose PhotoAgent, an autonomous photo editing agent that performs multi-step aesthetic planning and refinement to transform images with minimal user input.

RouteMoA: Dynamic Routing without Pre-Inference Boosts Efficient Mixture-of-Agents

Jize Wang, Han Wu, Zhiyuan You, Yiming Song, Yijun Wang, Zifei Shan, Yining Li, Songyang Zhang, Xinyi Le†, Cailian Chen†, Xinping Guan, Dacheng Tao

Association for Computational Linguistics (ACL), 2026

[Paper] [Code]

We propose RouteMoA, an efficient mixture-of-agents framework with dynamic routing. RouteMoA uses a lightweight scorer to predict coarse-grained performance, finding high-potential candidates without inference.

DA-VAE: Plug-in Latent Compression for Diffusion via Detail Alignment

Xin Cai, Zhiyuan You, Zhoutong Zhang, Tianfan Xue†

Computer Vision and Pattern Recognition (CVPR), 2026

[Project Page] [Paper] [Code]

We introduce Detail-Aligned VAE (DA-VAE), a method that increases the compression ratio of a pretrained VAE while requiring only lightweight adaptation for the pretrained diffusion backbone.

PhotoFramer: Multi-modal Image Composition Instruction

Zhiyuan You, Ke Wang, He Zhang, Xin Cai, Jinjin Gu, Tianfan Xue†, Chao Dong†, Zhoutong Zhang

Computer Vision and Pattern Recognition (CVPR), 2026

[Project Page] [Paper] [Code]

We introduce PhotoFramer, a multi-modal composition instruction framework, to provide composition guidance. Given a poorly composed image, PhotoFramer first describes how to improve the composition in natural language and then generates a well-composed example image.

How Far Have We Gone in Generative Image Restoration? A Study on Its Capability, Limitations and Evaluation Practices

Xiang Yin, Jinfan Hu, Zhiyuan You, Kainan Yan, Yu Tang, Chao Dong, Jinjin Gu

Computer Vision and Pattern Recognition (CVPR), 2026, Findings

[Project Page] [Paper] [Code]

We present a large-scale and multi-dimensional study on generative image restoration, covering diverse architectures, revealing critical performance disparities.

Revisiting the Generalization Problem of Low-level Vision Models Through the Lens of Image Deraining

Jinfan Hu*, Zhiyuan You*, Jinjin Gu, Kaiwen Zhu, Tianfan Xue, Chao Dong†

Pattern Recognition (PR), 2026

[Paper]

We investigate the underlying mechanism of the generalization failure to unseen degradations in low-level vision, and show that the key to better generalization lies in guiding the network to learn the image content rather than the degradation.

RadarQA: Multi-modal Quality Analysis of Weather Radar Forecasts

Xuming He*, Zhiyuan You*, Junchao Gong, Couhua Liu, Xiaoyu Yue, Peiqin Zhuang, Wenlong Zhang†, Lei Bai†

Neural Information Processing Systems (NeurIPS), 2025

[Paper] [Code] [Data]

We introduce an MLLM-based weather forecast quality analysis method, RadarQA, integrating key physical attributes with detailed assessment reports.

ReinAD: Towards Real-world Industrial Anomaly Detection with a Comprehensive Contrastive Dataset

Xu Wang, Jingyuan Zhuo, Zhiyuan You, Zhiyu Tan, Yikuan Yu, Siyu Wang, Xinyi Le†

Neural Information Processing Systems (NeurIPS), 2025, Datasets & Benchmarks Track

[Project Page] [Paper] [Code] [Data]

We introduce ReinAD dataset, a comprehensive, contrast-based, fine-grained, unaligned, and large-scale dataset towards real-world industrial anomaly detection.

Harnessing Diffusion-Yielded Score Priors for Image Restoration

Xinqi Lin, Fanghua Yu, Jinfan Hu, Zhiyuan You, Wu Shi, Jimmy S. Ren, Jinjin Gu†, Chao Dong†

ACM Transactions on Graphics (TOG), ACM SIGGRAPH Asia, 2025, Journal Track

[Project Page] [Paper] [Code] [Online App] [CCTV News (央视新闻直播间报道)]

We introduce HYPIR, a simple yet powerful paradigm: fine-tuning a pre-trained diffusion model with adversarial (GAN) loss — no diffusion sampling, no extra adapters, achieving an unprecedented balance of speed, fidelity, and quality.

DeQA-Doc: Adapting DeQA-Score to Document Image Quality Assessment

Junjie Gao, Runze Liu, Yingzhe Peng, Shujian Yang, Jin Zhang, Kai Yang†, Zhiyuan You†

International Conference on Computer Vision Workshop (ICCVW), 2025

[Paper] [Code]

Our DeQA-Doc wins the Championship in the VQualA 2025 DIQA (Document Image Quality Assessment) Challenge, ICCV 2025 Workshop.

Teaching Large Language Models to Regress Accurate Image Quality Scores using Score Distribution

Zhiyuan You, Xin Cai, Jinjin Gu, Tianfan Xue†, Chao Dong†

Computer Vision and Pattern Recognition (CVPR), 2025

[Project Page] [Paper] [Code] [Data]

We introduce DeQA-Score, a distribution-based depicted image quality assessment model for score regression.

UltraFusion: Ultra High Dynamic Imaging using Exposure Fusion

Zixuan Chen*, Yujin Wang*, Xin Cai, Zhiyuan You, Zheming Lu, Fan Zhang, Shi Guo, Tianfan Xue†

Computer Vision and Pattern Recognition (CVPR), 2025, Highlight, Best Demo Honorable Mention

[Project Page] [GitHub Page] [Paper] [Code] [Demo: HuggingFace / OpenXLab]

We propose UltraFusion, the first exposure fusion technique that can merge input with 9 stops differences.

Interpreting Low-level Vision Models with Causal Effect Maps

Jinfan Hu, Jinjin Gu, Shiyao Yu, Fanghua Yu, Zheyuan Li, Zhiyuan You, Chaochao Lu, Chao Dong†

IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 2025

[Paper] [Code]

We propose Causal Effect Map (CEM), a model-agnostic and task-agnostic interpreting method for low-level vision models.

An Intelligent Agentic System for Complex Image Restoration Problems

Kaiwen Zhu*, Jinjin Gu*, Zhiyuan You, Yu Qiao, Chao Dong†

International Conference on Learning Representations (ICLR), 2025

[Project Page] [Paper] [Code]

We introduce AgenticIR, an LLM-based agentic system that utilize various tools for complex image restoration problems.

SAIL: Sample-Centric In-Context Learning for Document Information Extraction

Jinyu Zhang*, Zhiyuan You*, Jize Wang, Xinyi Le†

Association for the Advancement of Artificial Intelligence (AAAI), 2025, Oral

[Paper] [Code]

We introduce SAIL, a sample-centric approach selecting tailored in-context examples for training-free document information extraction.

Enhancing Descriptive Image Quality Assessment with A Large-scale Multi-modal Dataset

Zhiyuan You, Jinjin Gu, Xin Cai, Zheyuan Li, Kaiwen Zhu, Chao Dong†, Tianfan Xue†

IEEE Transactions on Image Processing (TIP), 2025

[Project Page] [Paper] [Code] [Data]

We introduce DepictQA-Wild, also named Enhanced DepictQA (EDQA), a multi-functional in-the-wild descriptive image quality assessment model.

PhoCoLens: Photorealistic and Consistent Reconstruction in Lensless Imaging

Xin Cai, Zhiyuan You, Hailong Zhang, Wentao Liu, Jinwei Gu, Tianfan Xue†

Neural Information Processing Systems (NeurIPS), 2024, Spotlight

[Project Page] [Paper] [Code]

We introduce PhoCoLens, a novel two-stage approach for consistent and photorealistic lensless image reconstruction.

Depicting Beyond Scores: Advancing Image Quality Assessment through Multi-modal Language Models

Zhiyuan You*, Zheyuan Li*, Jinjin Gu*, Zhenfei Yin, Tianfan Xue†, Chao Dong†

European Conference on Computer Vision (ECCV), 2024

[Project Page] [Paper] [Code] [Data]

We introduce DepictQA, leveraging multi-modal large language models, allowing for detailed, language-based, and human-like evaluation of image quality.

MaskMA: Towards Zero-Shot Multi-Agent Decision Making with Mask-Based Collaborative Learning

Jie Liu*, Yinmin Zhang*, Chuming Li, Zhiyuan You, Zhanhui Zhou, Chao Yang, Yaodong Yang†, Yu Liu, Wanli Ouyang

Transactions on Machine Learning Research (TMLR), 2024

[Paper]

We release MaskMA, a masked pretraining framework for multi-agent decision-making.

Few-shot Object Counting with Similarity-Aware Feature Enhancement

Zhiyuan You, Kai Yang, Wenhan Luo, Xin Lu, Lei Cui, Xinyi Le†

Winter Conference on Applications of Computer Vision (WACV), 2023

[Paper] [Code] [Video]

We propose a novel SAFECount block, equipped with a similarity comparison module and a feature enhancement module for few-shot object counting.

A Unified Model for Multi-class Anomaly Detection

Zhiyuan You, Lei Cui, Yujun Shen, Kai Yang, Xin Lu, Yu Zheng, Xinyi Le†

Neural Information Processing Systems (NeurIPS), 2022, Spotlight

[Paper] [Code]

We present UniAD that accomplishes anomaly detection for multiple classes with a unified framework.

ADTR: Anomaly Detection Transformer with Feature Reconstruction

Zhiyuan You, Kai Yang, Wenhan Luo, Lei Cui, Yu Zheng, Xinyi Le†

International Conference on Neural Information Processing (ICONIP), 2022, Oral

[Paper]

We propose ADTR, a transformer that reconstructs pre-trained features for anomaly detection and extends to image- and pixel-level labels.

Education

	Ph.D. Candidate at Multimedia Laboratory (MMLab) @ The Chinese University of Hong Kong Aug. 2023 - Current GPA: 3.82 / 4.0 Advisor: Prof. Tianfan Xue and Prof. Chao Dong
	M.Eng. with Honor @ Shanghai Jiao Tong University Sep. 2020 - Mar. 2023 GPA: 3.76 / 4.0 Advisor: Prof. Xinyi Le and Prof. Yu Zheng
	B.Eng. with Honor @ Shanghai Jiao Tong University Sep. 2016 - Jun. 2020 GPA: 89.36 / 100, Ranking: 5 / 148 Advisor: Prof. Xinyi Le

Experience

	Research Intern @ ByteDance Seed Current Team: Seedance Team Mentor: Dr. Zhaoyang Huang and Dr. Feng Wang
	Research Scientist Intern @ Adobe Jul. 2025 - Nov. 2025 Team: Marc Levoy's NextCam Team Mentor: Dr. Zhoutong Zhang, Dr. Ke Wang, and Dr. He Zhang
	Research Intern @ Shanghai Artificial Intelligence Laboratory May. 2024 - Apr. 2025 Mentor: Prof. Chao Dong
	Research Intern @ Horizon Robotics Nov. 2022 - Mar. 2023 Mentor: Dr. Yuelong Yu
	Research Intern @ SenseTime Dec. 2020 - Nov. 2022 Mentor: Mr. Kai Yang

Academic Service

Journal Reviewer

• IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI)
• International Journal of Computer Vision (IJCV)
• IEEE Transactions on Image Processing (TIP)
• IEEE Transactions on Multimedia (TMM)
• IEEE Transactions on Knowledge and Data Engineering (TKDE)
• IEEE Transactions on Neural Networks and Learning Systems (TNNLS)
• IEEE Transactions on Systems, Man, and Cybernetics: Systems (TSMCS)
• IEEE Transactions on Circuits and Systems for Video Technology (TCSVT)
• Pattern Recognition (PR)
• Neural Networks (NN)

Conference Reviewer

• Computer Vision and Pattern Recognition (CVPR)
• International Conference on Computer Vision (ICCV)
• European Conference on Computer Vision (ECCV)
• Neural Information Processing Systems (NeurIPS)
• International Conference on Machine Learning (ICML)
• International Conference on Learning Representations (ICLR)
• Association for the Advancement of Artificial Intelligence (AAAI)
• ACM Multimedia (MM)
• Winter Conference on Applications of Computer Vision (WACV)
• British Machine Vision Conference (BMVC)

Awards

• SJTU Excellent Master Dissertation, 2023
• SJTU Outstanding Graduate (Postgraduate), 2023
• National Scholarship, 2022
• SJTU Outstanding Graduate (Undergraduate), 2020