Ziyue Feng

Ziyue Feng

World & Spatiotemporal Modeling

Apple

Biography

I am a Machine Learning Engineer at Apple working on world and spatiotemporal modeling, with a research focus on enabling machines to represent, understand, and predict real-world physical environments. My work centers on learning-based models that capture geometric structure, temporal dynamics, and physical consistency from visual data.

Across academia and industry, my research spans 3D reconstruction, active perception, and generative world models, with applications in autonomous systems, robotics, and spatial computing. I have contributed original research published in leading computer vision and robotics venues, and actively serve as a peer reviewer for top-tier AI and robotics conferences and journals.

Interests

  • World and spatiotemporal modeling
  • Learning-based 3D reconstruction & scene representation
  • Active perception for autonomous systems

Education

  • PhD in Computer Vision, 2019 - 2024

    Clemson University

  • BEng in Computer Science, 2015 - 2019

    Xi'an Jiaotong University

Selected Publications

*
All Depth Prediction 3D Reconstruction NeRF Motion Planning

NARUTO: Neural Active Reconstruction from Uncertain Target Observations

[CVPR 2024] NeRF style active reconstruction system with uncertainty learning. Project page.
Ziyue Feng, Huangying Zhan, etc...
Preprint PDF Code Project Poster Video

CVRecon: Rethinking 3D Geometric Feature Learning For Neural Reconstruction

[ICCV 2023] Proposed a novel cost volume based 3D geometric feature representation. Project page: https://cvrecon.ziyue.cool/
Ziyue Feng, Leon Yang, Pengsheng Guo, Bing Li
Preprint PDF Code Project Slides

Disentangling Object Motion and Occlusion for Unsupervised Multi-frame Monocular Depth

[ECCV 2022] Solve the dynamic object problem in multi-frame monocular depth prediction.
Ziyue Feng, Liang Yang, Longlong Jing, Haiyan Wang, Yingli Tian, Bing Li
Preprint PDF Code Project Poster Slides Video

Advancing Self-supervised Monocular Depth Learning with Sparse LiDAR

[CoRL 2021] Use sparse LiDAR to improve depth prediction
Ziyue Feng, Longlong Jing, Peng Yin, Yingli Tian, Bing Li
Preprint PDF Code Poster Slides Video

PSE-Match: A Viewpoint-Free Place Recognition Method With Parallel Semantic Embedding

[IEEE T-ITS 2021] A viewpoint invariant point cloud global localization descriptor.
Peng Yin, Lingyun Xu, Ziyue Feng, Anton Egorov, Bing Li
Preprint PDF

Model-based decision making with imagination for autonomous parking

[IEEE IV 2018] Improved RRT for autonomous parking motion planning.
Ziyue Feng, Shitao Chen, Yu Chen, Nanning Zheng
Preprint PDF Code Video

Experience

 
 
 
 
 

Machine Learning Engineer — World & Spatiotemporal Modeling

Apple

Oct 2024 – Present San Diego, CA
  • Conduct research on learning-based world and spatiotemporal modeling, developing generative and predictive models that enable machines to represent, understand, and forecast real-world environments from visual data, with an emphasis on geometric consistency, temporal coherence, and physical plausibility.
 
 
 
 
 

Senior Machine Learning Engineer

Matterport

Apr 2024 – Aug 2024 Sunnyvale, CA
  • Worked on learning-based large-scale 3D reconstruction and spatial data systems for digitizing and modeling real-world physical environments, advancing robust scene representation and geometric understanding in real-world deployments.
 
 
 
 
 

Research Intern

Google

Sep 2023 – Dec 2023 San Francisco, CA
  • Conducted advanced research on learning-based approaches to visual computing and modeling of complex real-world environments, focusing on neural representations supporting consistent geometric and spatiotemporal understanding.
 
 
 
 
 

Research Intern

OPPO US Research

Jun 2023 – Sep 2023 Palo Alto, CA
  • Developed active perception and NeRF-based SLAM methods enabling embodied agents to explore, localize, reconstruct, and plan in unknown environments. Published at CVPR 2024.
 
 
 
 
 

Research Assistant

Clemson University

Sep 2019 – Present South Carolina
  • Proposed a novel 3D geometric feature learning paradigm for neural reconstruction, advancing learning-based world representation through improved cost-volume reasoning (ICCV 2023).
  • Developed self-supervised multi-frame depth estimation frameworks addressing object motion and occlusion, enabling temporally consistent depth prediction in dynamic scenes (ECCV 2022).
  • Introduced learning-based multi-modal depth modeling integrating sparse LiDAR with monocular vision, advancing world modeling for autonomous systems (CoRL 2021).
 
 
 
 
 

Research Intern

MEGVII (FACE++) Research

Jan 2019 – May 2019
  • Early Research in Learning-based Perception: Conducted early research on learning-based gaze estimation, investigating model generalization under domain shift in real-world visual sensing scenarios.
 
 
 
 
 

Research Intern

Institute of Artifitial Intelligence and Robotics (IAIR at XJTU)

Oct 2016 – Jan 2019
  • Autonomous Parking: Proposed a model-based decision-making framework for autonomous parking, including an imagination-based module for RRT path planning and kinematic-aware trajectory refinement. Published at IEEE IV 2018.

Contact

…