Jianjie Luo

News

NEW! (September 2024) I will attend ECCV 2024 conference onsite in Milan, Italy. Nice to meet you!

NEW! (August 2024) One paper accepted to IEEE Transactions on Circuits and Systems for Video Technology(TCSVT).

NEW! (July 2024) One paper accepted to ECCV 2024.

NEW! (May 2023) I will attend CVPR 2023 conference onsite in Vancouver, Canada. Nice to meet you!

NEW! (February 2023) One paper accepted to CVPR 2023.

Bio

I am currently a Ph.D candidate of School of Computer Science and Engineering in Sun Yat-sen University(SYSU), a member of a joint Ph.D. program between SYSU and JD AI Research(JDAIR). My advisors are Dr. Ting Yao, Dr. Tao Mei, Prof. Jianlin Feng and Prof. Hongyang Chao.

Research Interests

Computer Vision
Multimodal Learning
Representation Learning

Education

08/2019 - 12/2024, Sun Yat-Sen University (SYSU)

Ph.D. in Computer Science and Technology (Expected December 2024)
Joint Ph.D. Program with JD.com
Thesis topic: Describing Multimedia with Semantic Alignment

08/2015 - 07/2019, Sun Yat-sen University (SYSU)

B.Eng. in Software Engineering
Recipient of the National Scholarship Award, Outstanding Undergraduate Award

Experiences

03/2024 - 09/2024, HiDream.ai Inc., Beijing

Research Assistant
Mentor: Ting Yao

07/2020 - 05/2023, Computer Vision and Multimedia Lab at JD Explore Academy, Beijing

Research Intern (Star Intern Award)
Mentor: Ting Yao

07/2018 - 08/2019, Computer Vision and Multimedia Lab at JD AI Research, Beijing

Research Intern (Star Intern Award)
Mentor: Ting Yao

03/2018 - 06/2018, Pixtalks Technology, Guangzhou

Research Intern
Mentor: Shengyong Ding

Publications [Google Scholar]

2024

Unleashing Text-to-Image Diffusion Prior for Zero-Shot Image Captioning

Jianjie Luo, Jingwen Chen, Yehao Li, Yingwei Pan, Jianlin Feng, Hongyang Chao, Ting Yao

In ECCV, 2024.

PDF Website

Exploring Vision-Language Foundation Model for Novel Object Captioning

Jianjie Luo, Yehao Li, Yingwei Pan, Ting Yao, Jianlin Feng, Hongyang Chao, Tao Mei

In IEEE Transactions on Circuits and Systems for Video Technology, 2024.

PDF

2023

Semantic-Conditional Diffusion Networks for Image Captioning

Jianjie Luo, Yehao Li, Yingwei Pan, Ting Yao, Jianlin Feng, Hongyang Chao, Tao Mei

In CVPR, 2023.

PDF Code

2022

Boosting Vision-and-Language Navigation with Direction Guiding and Backtracing

Jingwen Chen, Jianjie Luo, Yingwei Pan, Yehao Li, Ting Yao, Hongyang Chao, Tao Mei

In ACM Transactions on Multimedia Computing, Communications, and Applications, 2022.

PDF

2021

CoCo-BERT: Improving Video-Language Pre-training with Contrastive Cross-modal Matching and Denoising

Jianjie Luo, Yehao Li, Yingwei Pan, Ting Yao, Hongyang Chao, Tao Mei

In ACM Multimedia, 2021.

PDF

2020

Auto-captions on GIF: A Large-scale Video-sentence Dataset for Vision-language Pre-training

Yingwei Pan, Yehao Li, Jianjie Luo, Jun Xu, Ting Yao, Tao Mei

In Arxiv, 2020. Finally accepted in ACM Multimedia, 2022.