Zhiyuan You (尤志远)

I am a second-year first-year Ph.D. student at Multimedia Laboratory (MMLab), The Chinese University of Hong Kong (CUHK). I am supervised by Prof. Chao Dong and Prof. Tianfan Xue. I also work closely with Dr. Jinjin Gu.

Prior to that, I received my M.Eng. from Shanghai Jiao Tong University (SJTU) in 2023, supervised by Prof. Xinyi Le and Prof. Yu Zheng, and my B.Eng. from the same university in 2020. I spent a wonderful time in Horizon Robotics and SenseTime as a research intern, mentored by Dr. Yuelong Yu and Mr. Kai Yang, respectively.

My current research interest mainly lies in multi-modality vision for low-level vision, anomaly detection, few-shot learning, and industrial visual inspection.

CV  /  Google Scholar  /  Github
Email: zhiyuanyou [at] foxmail [dot] com

profile photo
News

[2024.09]    One paper to appear in NeurIPS 2024 (Spotlight).
[2024.08]    One paper to appear in TMLR.
[2024.07]    One paper to appear in ECCV 2024. See you in Milan.
[2023.08]    I become a Ph.D. student at MMLab in The Chinese University of Hong Kong.
[2023.03]    I graduate and receive my Master's degree from Shanghai Jiao Tong University.
[2022.09]    One paper to appear in NeurIPS 2022 (Spotlight).
[2022.09]    One paper to appear in ICONIP 2023 (Oral).
[2022.08]    One paper to appear in WACV 2023 (Early Accept).
[2021.12]    One paper to appear in NN.
[2020.07]    I graduate and receive my Bachelor's degree from Shanghai Jiao Tong University.

Research

*: Equal Contribution, †: Corresponding Author

arXiv, preprint
Descriptive Image Quality Assessment in the Wild
Zhiyuan You, Jinjin Gu, Zheyuan Li, Xin Cai, Kaiwen Zhu, Chao Dong†, Tianfan Xue†
arXiv, 2024
paper / project page / code

We introduce DepictQA-Wild, a multi-functional in-the-wild descriptive image quality assessment model.


Conference
PhoCoLens: Photorealistic and Consistent Reconstruction in Lensless Imaging
Xin Cai, Zhiyuan You, Hailong Zhang, Wentao Liu, Jinwei Gu, Tianfan Xue
Neural Information Processing Systems (NeurIPS), 2024, Spotlight
paper / project page

We introduce PhoCoLens, a novel two-stage approach for consistent and photorealistic lensless image reconstruction.

Depicting Beyond Scores: Advancing Image Quality Assessment through Multi-modal Language Models
Zhiyuan You*, Zheyuan Li*, Jinjin Gu*, Zhenfei Yin, Tianfan Xue†, Chao Dong†
European Conference on Computer Vision (ECCV), 2024
paper / project page / code

We introduce DepictQA, leveraging Multi-modal Large Language Models, allowing for detailed, language-based, and human-like evaluation of image quality.

Few-shot Object Counting with Similarity-Aware Feature Enhancement
Zhiyuan You, Kai Yang, Wenhan Luo, Xin Lu, Lei Cui, Xinyi Le†
Winter Conference on Applications of Computer Vision (WACV), 2023, Early Accept
paper / code / video

We propose a novel SAFECount block, equipped with a similarity comparison module and a feature enhancement module for few-shot object counting.

A Unified Model for Multi-class Anomaly Detection
Zhiyuan You, Lei Cui, Yujun Shen, Kai Yang, Xin Lu, Yu Zheng, Xinyi Le†
Neural Information Processing Systems (NeurIPS), 2022, Spotlight
paper / code

We present UniAD that accomplishes anomaly detection for multiple classes with a unified framework.

ADTR: Anomaly Detection Transformer with Feature Reconstruction
Zhiyuan You, Kai Yang, Wenhan Luo, Lei Cui, Yu Zheng, Xinyi Le†
International Conference on Neural Information Processing (ICONIP), 2022, Oral
paper

We propose ADTR to apply a transformer to reconstruct pre-trained features for anomaly detection, and propose novel losses to extend ADTR to anomaly-available case (both image-level & pixel-level labeled).


Journal
MaskMA: Towards Zero-Shot Multi-Agent Decision Making with Mask-Based Collaborative Learning
Jie Liu*, Yinmin Zhang*, Chuming Li, Zhiyuan You, Zhanhui Zhou, Chao Yang, Yaodong Yang†, Yu Liu, Wanli Ouyang
Transactions on Machine Learning Research (TMLR), 2024
paper

We release MaskMA, a masked pretraining framework for multi-agent decision-making.

UTRAD: Anomaly detection and localization with u-transformer
Liyang Chen, Zhiyuan You, Nian Zhang, Juntong Xi, Xinyi Le†
Neural Networks (NN), 2022
paper / code

We introduce UTRAD, a U-TRansformer based Anomaly Detection framework.

An accurate star identification approach based on spectral graph matching for attitude measurement of spacecraft
Zhiyuan You, Junzheng Li, Hongcheng Zhang, Bo Yang, Xinyi Le†
Complex & Intelligent Systems (CAIS), 2021
paper / code

My undergraduate thesis and my first first-author paper.

We propose a novel star identification approach based on spectral graph matching.

Education
Ph.D. Student at Multimedia Laboratory (MMLab) @ The Chinese University of Hong Kong
Aug. 2023 - Current
Advisor: Prof. Chao Dong and Prof. Tianfan Xue
M.Eng. with Honor in Mechanical Engineering @ Shanghai Jiao Tong University
Sep. 2020 - Mar. 2023
GPA: 3.76 / 4.0
Advisor: Prof. Xinyi Le and Prof. Yu Zheng
B.Eng. with Honor in Mechanical Engineering @ Shanghai Jiao Tong University
Sep. 2016 - Jun. 2020
GPA: 89.36 / 100, Ranking: 5 / 148
Advisor: Prof. Xinyi Le
Experience
Horizon Robotics
Research Intern
Shanghai, China
Dec. 2022 - Mar. 2023
Mentor: Dr. Yuelong Yu
Perception Algorithm for Autonomous Driving
SenseTime
Research Intern
Shanghai, China
Dec. 2020 - Nov. 2022
Mentor: Mr. Kai Yang
Anomaly Detection & Few-Shot Learning
Services

• Journal Reviewer

    International Journal of Computer Vision (IJCV)
    IEEE Transactions on Neural Networks and Learning Systems (T-NNLS)
    IEEE Transactions on Systems, Man and Cybernetics: Systems (T-SMCS)
    IEEE Transactions on Multimedia (T-MM)
    IEEE Transactions on Circuits and Systems for Video Technology (T-CSVT)
    Pattern Recognition (PR)
    Knowledge Based Systems (KBS)
    Journal of Intelligent Manufacturing
    Neurocomputing

• Conference Reviewer

    IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, 2025
    Annual Conference on Neural Information Processing Systems (NeurIPS), 2024
    International Conference on Learning Representation (ICLR), 2025
    International Conference on Artificial Intelligence and Statistics (AISTATS), 2025

Awards

    [2023.03]    Excellent Master Dissertation
    [2023.03]    Outstanding Graduate (Postgraduate)
    [2022.09]    National Scholarship
    [2020.06]    Outstanding Graduate (Undergraduate)


Template from JonBarron