News | People | Research | Publications | Datasets | Positions | Teaching | Contact

Robotics & Embodied AI Lab

Guang Chen

Professor, Ph.D Advisor

Department of Computer Science and Technology

Tongji University

Email: guangchen(at)tongji.edu.cn or guang(at)in.tum.de

Address: NO.4800, Cao'an Road, Jiading District, Shanghai


Welcome to the website of the Robotics & Embodied AI Lab led by Prof. Dr. rer. nat. Guang Chen. Our lab is affiliated with the CAE academician Prof. Changjun Jiang' research team at Tongji University. Our lab members are from Tongji University and Technical University of Munich, and jointed advised by Prof. Guang Chen, Prof. Alois Knoll, and Prof. Changjun Jiang.

I am looking for highly motivated and self-driven students, postdocs and research assistants, please drop me an email if you are interested in working with me.


News

Pre-Prints

A Review of Safe Reinforcement Learning: Methods, Theory and Applications
Shangding Gu, Long Yang, Yali Du, Guang Chen, Florian Walter, Jun Wang, Yaodong Yang, Alois Knoll
Arxiv, 2022
code / video / arXiv / bibtex 
@misc{gu2022,
      title={A Review of Safe Reinforcement Learning: Methods, Theory and Applications}, 
      author={Shangding Gu and Long Yang and Yali Du and Guang Chen and Florian Walter and Jun Wang and Yaodong Yang and Alois Knoll},
      year={2022},
      eprint={2205.10330},
      archivePrefix={arXiv},
      primaryClass={cs.AI}
}

Peer-Reviewed Paper (Selected, 2020-)

TMA: Temporal Motion Aggregation for Event-based Optical Flow
Haotian Liu, Guang Chen*, Sanqing Qu, Yanping Zhang, Zhijun Li, Alois Knoll, Changjun Jiang
IEEE International Conference on Computer Vision (ICCV), 2023
Project / code / arXiv / bibtex 
@misc{liu2023tma,
      title={TMA: Temporal Motion Aggregation for Event-based Optical Flow}, 
      author={Haotian Liu and Guang Chen and Sanqing Qu and Yanping Zhang and Zhijun Li and Alois Knoll and Changjun Jiang},
      conference={IEEE International Conference on Computer Vision (ICCV)},
      year={2023},
}
Most existing learning-based approaches for event optical flow estimation directly remould the paradigm of conventional images by representing the consecutive event stream as static frames, ignoring the inherent temporal continuity of event data. In this paper, we argue that temporal continuity is a vital element of event-based optical flow and propose a novel Temporal Motion Aggregation (TMA) approach to unlock its potential. TMA comprises three components: an event splitting strategy to incorporate intermediate motion information underlying the temporal context, a linear lookup strategy to align temporally fine-grained motion features and a novel motion pattern aggregation module to emphasize consistent patterns for motion feature enhancement. Extensive experiments on DSEC-Flow and MVSEC datasets verify the effectiveness and superiority of our approach.
UMC: A Unified Bandwidth-efficient and Multi-resolution based Collaborative Perception Framework
Tianhang Wang, Guang Chen*, Kai Chen, Zhengfa Liu, Bo Zhang, Alois Knoll, Changjun Jiang
International Conference on Computer Vision (ICCV), 2023
Project / code / arXiv / bibtex 
@inproceedings{wang2023umc,
        title     = {UMC: A Unified Bandwidth-efficient and Multi-resolution based Collaborative Perception Framework},
        author    = {Tianhang, Wang and Guang, Chen and Kai, Chen and Zhengfa, Liu, Bo, Zhang, Alois, Knoll, Changjun, Jiang},
        booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
        year      = {2023}
        },
}
We aim to propose a Unified Collaborative perception framework named UMC, optimizing the communication, collaboration, and reconstruction processes with the Multi-resolution technique. The communication introduces a novel trainable multi-resolution and selective-region (MRSR) mechanism, achieving higher quality and lower bandwidth. Then, a graph-based collaboration is proposed, conducting on each resolution to adapt the MRSR. Finally, the reconstruction integrates the multi-resolution collaborative features for downstream tasks. Since the general metric can not reflect the performance enhancement brought by MCP systematically, we introduce a brand-new evaluation metric that evaluates the MCP from different perspectives.
Urban Radiance Field Representation with Deformable Neural Mesh Primitives
Fan Lu, Yan Xu, Guang Chen*, Hongsheng Li, Kwan-Yee Lin*, Changjun Jiang
International Conference on Computer Vision (ICCV), 2023
Project / code / arXiv / bibtex 
@misc{lu2023dnmp,
      title={Urban Radiance Field Representation with Deformable Neural Mesh Primitives}, 
      author={Fan Lu and Yan Xu and Guang Chen and Hongsheng Li and Kwan-Yee Lin and Changjun Jiang},
      conference={IEEE/CVF International Conference on Computer Vision (ICCV)},
      year={2023},
}
We propose a novel neural rendering framework for urban scenes. We represent the urban scene as a set of deformable neural mesh primitives (DNMPs). The DNMP is a flexible and compact neural variant of classic mesh representation, which enjoys both the efficiency of rasterization-based rendering and the powerful neural representation capability for photo-realistic image synthesis.
Sparse-to-Dense Matching Network for Large-scale LiDAR Point Cloud Registration
Fan Lu, Guang Chen*, Yinlong Liu, Yibing Zhan, Zhijun Li, Dacheng Tao, Changjun Jiang
IEEE Transactions on Pattern Analysis and Machine Intelligence (T-PAMI), 2023
code / video / arXiv / bibtex 
@misc{lu2023SDMNet,
      title={Sparse-to-Dense Matching Network for Large-scale LiDAR Point Cloud Registration }, 
      author={ Fan Lu and Guang Chen and Yinlong Liu and Yibing Zhan and Zhijun Li and Dacheng Tao and Changjun Jiang },
      conference={IEEE Transactions on Pattern Analysis and Machine Intelligence},
      year={2023},
}
We propose a novel Sparse-to-Dense Matching Network (SDMNet) for large-scale outdoor LiDAR point cloud registration. Specifically, SDMNet performs registration in two sequential stages: sparse matching stage and local-dense matching stage. We design a novel neighborhood matching module to incorporate local neighborhood consensus, significantly improving performance. The local-dense matching stage is followed for fine-grained performance. Extensive experiments on three large-scale outdoor LiDAR point cloud datasets demonstrate that the proposed SDMNet achieves state-of-the-art performance with high efficiency.
NeuralPCI: Spatio-temporal Neural Field for 3D Point Cloud Multi-frame Non-linear Interpolation
Zehan Zheng, Danni Wu, Ruisi Lu, Fan Lu, Guang Chen*, Changjun Jiang
IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023
Project / code / arXiv / bibtex 
@misc{zheng2023neuralpci,
      title={NeuralPCI: Spatio-temporal Neural Field for 3D Point Cloud Multi-frame Non-linear Interpolation}, 
      author={Zehan Zheng and Danni Wu and Ruisi Lu and Fan Lu and Guang Chen and Changjun Jiang},
      conference={IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)},
      year={2023},
}
We propose NeuralPCI: an end-to-end 4D spatio-temporal Neural field for 3D Point Cloud Interpolation, which implicitly integrates multi-frame information to handle nonlinear large motions for both indoor and outdoor scenarios. And we also construct a new multi-frame point cloud interpolation dataset called NL-Drive for large nonlinear motions in autonomous driving scenes. Furthermore, NeuralPCI tends to be a flexible unified framework to conduct both the interpolation and extrapolation, facilitating several applications as well.
Upcycling Models under Domain and Category Shift
Sanqing Qu, Tianpei Zou, Florian Röhrbein, Cewu Lu, Guang Chen*, Dacheng Tao, Changjun Jiang
IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR), 2023
Project / Code / arXiv / bibtex 
@misc{qu2023GLC,
      title={Upcycling Models under Domain and Category Shift }, 
      author={Sanqing Qu and Tianpei Zou and Florian Röhrbein and Cewu Lu and Guang Chen and Dacheng Tao and Changjun Jiang},
      conference={IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR)},
      year={2023},
}
Deep neural networks (DNNs) often perform poorly in the presence of domain shift and category shift. To address this, in this paper, we explore the Source-free Universal Domain Adaptation (SF-UniDA). SF-UniDA is appealing in view that universal model adaptation can be resolved only on the basis of a standard pre-trained closed-set model, i.e., without source raw data and dedicated model architecture. To achieve this, we develop a generic global and local clustering technique (GLC). GLC equips with an inovative one-vs-all global pseudo-labeling strategy to realize "known" and "unknown" data samples separation under various category-shift. Remarkably, in the most challenging open-partial-set DA scenario, GLC outperforms UMAD by 14.8% on the VisDA benchmark.
Modality-Agnostic Debiasing for Single Domain Generalization
Sanqing Qu, Yingwei Pan, Guang Chen*, Ting Yao, Changjun Jiang, Tao Mei
IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR), 2023
code / video / arXiv / bibtex 
@misc{qu2023MAD,
      title={Modality-Agnostic Debiasing for Single Domain Generalization}, 
      author={Sanqing Qu and Yingwei Pan and Guang Chen and Ting Yao and Changjun Jiang and Tao Mei},
      conference={IEEE / CVF Computer Vision and Pattern Recognition Conference (CVPR)},
      year={2023},
}
Existing single-DG techniques commonly devise various data-augmentation algorithms, and remould the multi-source domain generalization methodology to learn domain-generalized (semantic) features. Nevertheless, these methods are typically modality-specific, thereby being only applicable to one single modality (e.g.,image). In contrast, we target a versatile Modality-Agnostic Debiasing (MAD) framework for single-DG, that enables generalization for different modalities. We have evaluated the effectiveness and superiority of MAD for single-DG via various empirical evidences on a series of tasks, including recognition on 1D texts, 2D images, 3D point clouds, and semantic segmentation on 2D images.
GSC: A Graph and Spatio-temporal Continuity Based Framework for Accident Anticipation
Tianhang Wang, Kai Chen, Guang Chen*, Bin Li, Zhijun Li, Zhengfa Liu, Changjun Jiang
IEEE Transactions on Intelligent Vehicles (TIV), 2023
code / video / arXiv / bibtex 
@misc{wang2023GSC,
      title={ GSC: A Graph and Spatio-temporal Continuity Based Framework for Accident Anticipation }, 
      author={Tianhang Wang and Kai Chen and Guang Chen and Bin Li and Zhijun Li and Zhengfa Liu and Changjun Jiang},
      conference={IEEE Transactions on Intelligent Vehicles (TIV)},
      year={2023},
}
we propose a Graph and Spatio-temporal Continuity based framework for accident anticipation called GSC, which takes the missing agents into account. Specifically, the proposed GSC maintains the spatio-temporal continuity of missing agents, which are in the occluded spatial state in the process of the graph convolution operation.
D2IFLN: Disentangled Domain-Invariant Feature Learning Networks for Domain Generalization
Zhengfa Liu, Guang Chen*, Zhijun Li, SanQing Qu, Alois Knoll, Changjun Jiang
IEEE Transactions on Cognitive and Developmental Systems, 2023
code / video / arXiv / bibtex 
@article{'liu2023DDIFLN,
  title={D2IFLN: Disentangled Domain-Invariant Feature Learning Networks for Domain Generalization},
  author={Liu, Zhengfa and Chen, Guang and Li, Zhijun and Qu, Sanqing and and Alois Knoll and Jiang, Changjun},
  journal={IEEE Transactions on Cognitive and Developmental Systems},
  year={2023},
  publisher={IEEE}
}
Domain generalization (DG) aims to learn a model that generalizes well to an unseen test distribution. Mainstream methods follow the domain-invariant representational learning philosophy to achieve this goal. However, due to the lack of priori knowledge to determine which features are domain-specific and task-independent, and which features are domain-invariant and task-relevant, existing methods typically learn entangled representations, limiting their capacity to generalize to the distribution-shifted target domain. To address this issue, in this paper, we propose novel Disentangled Domain-Invariant Feature Learning Networks (D2IFLN), to adapt feature realize feature disentanglement and facilitate domain-invariant feature learning.
A Discrete Soft Actor-Critic Decision-Making Strategy with Sample Filter for Freeway Autonomous Driving
Jiayi Guan, Guang Chen*, Jin Huang, Zhijun Li, Lu Xiong, Jing Hou, Alois Knoll
IEEE Transactions on Vehicular Technology, 2022
paper / code / bibtex 
@misc{guan2022a,
      title={ A Discrete Soft Actor-Critic Decision-Making Strategy with Sample Filter for Freeway Autonomous Driving }, 
      author={ Jiayi Guan, Guang Chen, Jin Huang, Zhijun Li, Lu Xiong, Jing Hou, Alois Knoll },
      journal={ IEEE Transactions on Vehicular Technology },
      year={2022},
      publisher={IEEE}
}
In this work, we design a discrete decision-making strategy based on the discrete soft actor-critic with a sample filter algorithm (DSAC-SF) to improve driving efficiency and safety on freeways with dynamic traffic. Experimental results indicate that our strategy obtains a high success rate and a fast vehicle speed in the decision-making tasks on freeways.
PIPO: Policy Optimization with Permutation Invariant Constraint for Distributed Multi-Robot Navigation
Ruiqi Zhang, Guang Chen*, Jing Hou, Zhijun Li, Alois Knoll
IEEE International Conference on Multisensor Fusion and Integration, 2022

Best Student Paper Award


paper / bibtex 
@inproceedings{zhang2022PIPO,
  author={Zhang, Ruiqi and Chen, Guang and Hou Jing and Li, Zhijun and Knoll, Alois},
  journal={IEEE International Conference on Multisensor Fusion and Integration}, 
  title={PIPO: Policy Optimization with Permutation Invariant Constraint for Distributed Multi-Robot Navigation}, 
  year={2022}}
In this study, we propose a decentralized multi-agent reinforcement learning method through constructing the representation constraint via the graph convolutional network. Meanwhile, leverage the permutation-invariant property shuffling observation to enhance the representation and generalization ability of the actor-critic structure. Our method is much safer than centralized MARL baselines and can be generalized to an arbitrary number of agents in different scenarios.
BMD: A General Class-balanced Multicentric Dynamic Prototype Strategy for Source-free Domain Adaptation
Sanqing Qu, Guang Chen*, Jing Zhang, Zhijun Li, Wei He, Dacheng Tao
European Conference on Computer Vision (ECCV), 2022
code / video / arXiv / bibtex 
@misc{qu2022BMD,
      title={BMD: A General Class-balanced Multicentric Dynamic Prototype Strategy for Source-free Domain Adaptation}, 
      author={Sanqing Qu and Guang Chen and Jing Zhang and Zhijun Li and Wei He and Dacheng Tao},
      conference={European Conference on Computer Vision (ECCV)},
      year={2022},
}
we propose a general class-Balanced Multicentric Dynamic prototype (BMD) strategy for the SFDA task. Specifically, for each target category, we first introduce a global inter-class balanced sampling strategy to aggregate potential representative target samples. Then, we design an intra-class multicentric clustering strategy to achieve more robust and representative prototypes generation. We further introduce a dynamic pseudo labeling strategy to incorporate network update information during model adaptation.
PSDC: A Prototype-Based Shared-Dummy Classifier Model for Open-Set Domain Adaptation
Zhengfa Liu, Guang Chen*, Zhijun Li, Yu Kang, SanQing Qu, Changjun Jiang
IEEE Transactions on Cybernetics, 2022
code / video / arXiv / bibtex 
@article{liu2022psdc,
  title={PSDC: A Prototype-Based Shared-Dummy Classifier Model for Open-Set Domain Adaptation},
  author={Liu, Zhengfa and Chen, Guang and Li, Zhijun and Kang, Yu and Qu, Sanqing and Jiang, Changjun},
  journal={IEEE Transactions on Cybernetics},
  year={2022},
  publisher={IEEE}
}
Open Set Domain Adaptation (OSDA) aims to achieve knowledge transfer in the presence of both domain shift and label shift, which assumes that there exist additional unknown target classes not presented in the source domain. To solve the OSDA problem, most existing methods introduce an additional unknown class to the source classifier and represent the unknown target instances as a whole. However, it is unreasonable to treat all unknown target instances as a group, since these unknown instances typically consist of distinct categories and distributions. It is challenging to identify all unknown instances with only one additional class. In addition, most existing methods directly introduce marginal distribution alignment to alleviate distribution shift between source and target domain, failing to learn discriminative class boundaries in the target domain since they ignore categorical discriminative information in the adaptation. To address these problems, in this paper, we propose a novel Prototype-based Shared-Dummy Classifier (PSDC) model for the OSDA.
Robot Policy Improvement with Natural Evolution Strategies for Stable Nonlinear Dynamical System
Yingbai Hu#, Guang Chen#, Zhijun Li*, Alois Knoll
IEEE Transactions on Cybernetics, 2022
Link / code / video / bibtex 
@misc{hucyber2022,
      title={Robot Policy Improvement with Natural Evolution Strategies for Stable Nonlinear Dynamical System}, 
      author={Yingbai Hu and Guang Chen and Zhijun Li and Alois Knoll},
      journal={IEEE Transactions on Cybernetics},
      year={2022},
      publisher={IEEE}
}
This paper focuses on improving the adaptability and robustness of robot learning. We propose a policy improvement-based hierarchical learning strategy to imitate and motor skills from human demonstration. The low-level learning method only focuses on behavioural cloning, while the high-level one aims to enhance adaptability and robustness through policy improvement.
Residual Policy Learning Facilitates Efficient Model-Free Autonomous Racing
Ruiqi Zhang, Jing Hou, Guang Chen*, Zhijun Li, Jianxiao Chen, Alois Knoll
IEEE Robotics and Automation Letters, 2022
arXiv / code / video / bibtex 
@misc{zhang2022resrace,
      title={Residual Policy Learning Facilitates Efficient Model-Free Autonomous Racing}, 
      author={Ruiqi Zhang and Jing Hou and Guang Chen and Zhijun Li and Jianxiao Chen and Alois Knoll},
      journal={IEEE Robotics and Automation Letters},
      year={2022},
      publisher={IEEE}
}
In this study, we propose an efficient residual policy learning method for high-speed autonomous racing named ResRace, which leverages only the real-time raw observation of LiDAR and IMU for low-latency obstacle avoiding and navigation. Experiments illustrate our method outperforms the leading algorithms and reaches the comparable level of professional human players on the five F1Tenth tracks.
NeuroGrasp: Multi-modal Neural Network with Euler Region Regression for Neuromorphic Vision-based Grasp Pose Estimation
Hu Cao, Guang Chen*, Zhijun Li, Yingbai Hu, Alois Knoll
IEEE Transactions on Instrumentation & Measurement, 2022
Link / code / video / bibtex 
@misc{tim2022,
      title={NeuroGrasp: Multi-modal Neural Network with Euler Region Regression for Neuromorphic Vision-based Grasp Pose Estimation}, 
      author={Hu Cao and Guang Chen and Zhijun Li and Yingbai Hu and Alois Knoll},
      journal={IEEE Transactions on Instrumentation & Measurement},
      year={2022},
      publisher={IEEE}
}
In this paper, we construct a dynamic robotic grasping dataset named \emph{NeuroGrasp}. To the best of our knowledge, it is the first event-based multi-modality robotic grasping dataset. Based on this dataset, we introduce a multi-modal deep neural network for grasping pose estimation with combing frame-based vision and event-based vision.
Learning Local Event-based Descriptor for Patch-based Stereo Matching
Peigen Liu, Guang Chen*, Zhijun Li, Huajin Tang, Alois Knoll
IEEE International Conference on Robotics and Automation (ICRA), 2022
Link / code / video / bibtex 
@misc{liu2022learning,
      title={Learning Local Event-based Descriptor for Patch-based Stereo Matching}, 
      author={Peigen Liu and Guang Chen and Zhijun Li and Huajin Tang and Alois Knoll},
      conference={IEEE International Conference on Robotics and Automation},
      year={2022},
      publisher={IEEE}
}
In this paper, we propose two novel patch-based stereo matching methods, which focus on improving matching accuracy and reducing running time respectively. We present accuracy representation that encodes events distribution information based on multiple spatial-temporal planes. Then, we design efficient and accuracy networks and train them using the proposed corresponding loss, the matching scores are computed based on the output local event-based feature descriptors. Finally, we utilize cost aggregation and left right consistency check to generate and refine disparity map.
Globally Optimal Linear Model Fitting with Unit-Norm Constraint
Yinlong Liu, Yiru Wang, Manning Wang, Guang Chen*, Alois Knoll, Zhijian Song
International Journal of Computer Vision, 2022
Link / code / video / bibtex 
@misc{ijcv2022,
      title={Globally Optimal Linear Model Fitting with Unit-Norm Constraint}, 
      author={Yinlong Liu and Yiru Wang and Manning Wang and Guang Chen and Alois Knoll and Zhijian Song},
      journal={International Journal of Computer Vision},
      year={2022},
      publisher={Springer}
}
In this paper, we develop a globally optimal algorithm aiming at consensus set maximization to solve the robust linear model fitting problems with the unit-norm constraint, which is based on the branch-and-bound optimization framework.
Neuromorphic Vision-based Fall Localization in Event Streams with Temporal Spatial Attention Weighted Network
Guang Chen, Sanqing Qu, Zhijun Li, Haitao Zhu, Jiaxuan Dong, Min Liu, Jorg Conradt
IEEE Transactions on Cybernetics, 2022
Link / code / video / bibtex 
@misc{cyber2022,
      title={Neuromorphic Vision-based Fall Localization in Event Streams with Temporal Spatial Attention Weighted Network}, 
      author={Guang Chen and Sanqing Qu and Zhijun Li and Haitao Zhu and Jiaxuan Dong and Min Liu and Jorg Conradt},
      journal={IEEE Transactions on Cybernetics},
      year={2022},
      publisher={IEEE}
}
In this paper, we proposed a bio-inspired event-camera based falls temporal localization framework, which is also the first falls temporal localization framework, to our best of knowledge. For the proposal generation, we propose the first event density-based action proposal generation scheme, which is simple yet more effective than existing sliding-window based schemes. To realize proposal temporal modeling, we introduce a temporal-spatial weighted network to extract more robust motion representations.
MoNet: Motion-based Point Cloud Prediction Network
Fan Lu, Guang Chen*, Zhijun Li, Lijun Zhang, Yinlong Liu, Sanqing Qu, Alois Knoll
IEEE Transactions on Intelligent Transportation Systems, 2021
arXiv / code / video / bibtex 
@misc{lu2020monet,
      title={MoNet: Motion-based Point Cloud Prediction Network}, 
      author={Fan Lu and Guang Chen and Yinlong Liu and Zhijun Li and Lijun Zhang and Yinlong Liu and Sanqing Qu and Alois Knoll},
      journal={IEEE Transactions on Intelligent Transportation Systems},
      year={2021},
      publisher={IEEE}
}
In this paper, we propose a novel motion-based neural network named MoNet. The key idea of the proposed MoNet is to integrate motion features between two consecutive point clouds into the prediction pipeline. The introduction of motion features enables the model to more accurately capture the variations of motion information across frames and thus make better predictions for future motion.
HRegNet: A Hierarchical Network for Large-scale Outdoor LiDAR Point Cloud Registration
Fan Lu, Guang Chen*, Yinlong Liu, Lijun Zhang, Sanqing Qu, Shu Liu, Rongqi Gu
International Conference on Computer Vision (ICCV), 2021
project / code / video / bibtex 
@article{ispc:hregnet,
  author  = {Fan Lu and Guang Chen and Yinlong Liu and Lijun Zhang and Sanqing Qu and Shu Liu andand Rongqi Gu},
  title   = {HRegNet: A Hierarchical Network for Large-scale Outdoor LiDAR Point Cloud Registration},
  journal = {International Conference on Computer Vision (ICCV)},
  year    = {2021},
}
we propose an efficient hierarchical network named HRegNet for large-scale outdoor LiDAR point cloud registration. Instead of using all points in the point clouds, HRegNet performs registration on hierarchically extracted keypoints and descriptors. Bilateral consensus and neighborhood consensus are introduced to improve the robustness and accuracy.
HSTA: A Hierarchical Spatio-Temporal Attention Model for Trajectory Prediction
Ya Wu, Guang Chen*, Zhijun Li, Lu Xiong, Zhengfa Liu, Alois Knoll
IEEE Transactions on Vehicular Technology, 2021
project / code / video / bibtex 
@article{ispc:tvt-l-2021,
  author  = {Ya Wu and Guang Chen and Zhijun Li and Lu Xiong and Zhengfa Liu and Alois Knoll},
  title   = {HSTA: A Hierarchical Spatio-Temporal Attention Model for Trajectory Prediction},
  journal = {IEEE Transactions on Vehicular Technology (TVT)},
  year    = {2021},
}
we propose a Hierarchical Spatio-Temporal Attention architecture (HSTA), which activates the utilization of spatial interactions with different weights, and jointly considers the temporal interactions across time steps of all agents. More specially, the graph attention mechanism (GAT) is presented to capture spatial interactions, the multi-head attention mechanism (MHA) is conducted to encode temporal correlations of interactions and a state gated fusion (SGF) layer is used to integrate spatial and temporal interactions..
Pole-Curb Fusion based Robust and Efficient Autonomous Vehicle Localization System with Branch-and-Bound Global Optimization and Local Grid Map Method
Guang Chen, Fan Lu, Zhijun Li*, Yinlong Liu, Jinhu Dong, Junqiao Zhao, Junwei Yu, Alois Knoll
IEEE Transactions on Vehicular Technology, 2021
project / code / video / bibtex 
@article{ispc:tvt-l-2021,
  author  = {Guang Chen and Fan Lu and Zhijun Li and Yinlong Liu and Jinhu Dong and Junqiao Zhao and Junwei Yu and Alois Knoll},
  title   = {Pole-Curb Fusion based Robust and Efficient Autonomous Vehicle Localization System with Branch-and-Bound Global Optimization and Local Grid Map Method},
  journal = {IEEE Transactions on Vehicular Technology (TVT)},
  year    = {2021},
}
we propose a novel lightweight LiDAR-based localization system for autonomous vehicle in this paper. The proposed system only relies on lightweight poles and curbs landmark map, which is highly robust and efficient compared to other localization systems.
Fusion-based Feature Attention Gate Component for Vehicle Detection based on Event Camera
Hu Cao, Guang Chen*, Jiahao Xia, Genghang Zhuang, Alois Knoll
IEEE Sensors Journal, 2021
project / code / video / bibtex 
@article{ispc:sensors2021,
  author  = {Hu Cao and Guang Chen and Jiahao Xia and Genghang Zhuang and Alois Knoll},
  title   = {Fusion-based Feature Attention Gate Component for Vehicle Detection based on Event Camera},
  journal = {IEEE Sensors Journal},
  year    = {2021},
}
we introduce a fully convolutional neural network with feature attention gate component (FAGC) for vehicle detection by combining frame-based and event-based vision. Both grayscale features and event features are fed into feature attention gate component (FAGC) to generate the pixel-level attention feature coefficients to improve the feature discrimination ability of the network.
Active Safety Control of Automated Electric Vehicles at Driving Limits: A Tube-based MPC Approach
Peng Hang, Xin Xia, Guang Chen, Xinbo Chen
IEEE Transactions on Transportation Electrification, 2021
project / code / video / bibtex 
@article{ispc:tai21,
  author  = {Peng Hang and Xin Xia and Guang Chen and Alois Knoll},
  title   = {Active Safety Control of Automated Electric Vehicles at Driving Limits: A Tube-based MPC Approach},
  journal = {IEEE Transactions on Transportation Electrification},
  year    = {2021},
}
The motion control problem of AEVs at driving limits is studied in this paper. To address this issue, an integrated controller is designed based on Tube-based MPC control algorithm. To advance AEVs’ path tracking performance and handling stability simultaneously, 4WS and DYC techniques are applied.
KAM-Net: Keypoint-Aware and Keypoint-Matching Network for Vehicle Detection from 2D Point Cloud
Tianpei Zou, Guang Chen*, Zhijun Li, Wei He, Sanqing Qu, Shangding Gu, Alois Knoll
IEEE Transactions on Artificial Intelligence (minor revision), 2021
project / code / video / bibtex 
@article{ispc:tai21,
  author  = {Tianpei Zou and Guang Chen and Zhijun Li and Wei He and Sanqing Qu and Shangding Gu and Alois Knoll},
  title   = {KAM-Net: Keypoint-Aware and Keypoint-Matching Network for Vehicle Detection from 2D Point Cloud},
  journal = {IEEE Transactions on Artificial Intelligence },
  year    = {2021},
}
...
Globally Optimal Consensus Maximization for Relative Pose Estimation With Known Gravity Direction
Yinlong Liu, Guang Chen*, Rongqi Gu, Alois Knoll
IEEE Robotics and Automation Letters ( Volume: 6, Issue: 3, July 2021)
project / code / video / bibtex 
@article{ispc:ral21,
  author  = {Yinlong Liu and Guang Chen and Rongqi Gu and Alois Knoll},
  title   = {Globally Optimal Consensus Maximization for Relative Pose Estimation With Known Gravity Direction},
  journal = {IEEE Robotics and Automation Letters },
  year    = {2021},
}
We propose a globally optimal algorithm for relative pose estimation with known gravity direction. Specifically, the proposed method employs the branch-and-bound algorithm to solve a consensus maximization problem, and thus it is able to obtain the global solution with a provable guarantee.
VCANet: Vanishing-Point-Guided Context-Aware Network for Small Road Object Detection
Guang Chen*, Kai Chen, Lijun Zhang*, Liming Zhang, Alois Knoll
Automotive Innovation, 2021
project / code / video / bibtex 
@article{ispc:vcanet21,
  author  = {Guang Chen and Kai Chen and Lijun Zhang and Liming Zhang and Alois Knoll},
  title   = {VCANet: Vanishing-Point-Guided Context-Aware Network for Small Road Object Detection},
  journal = {Automotive Innovation},
  year    = {2021},
}
This paper is the first attempt to establish a large TJ-LDRO dataset, which consists of 109,337 images from real and virtual simulation environment, labeled in detail. Besides, the Vanishing-point-guided Context-Aware Network (VCANet) is introduced for small object detection.
PointINet: Point Cloud Frame Interpolation Network
Fan Lu, Guang Chen*, Sanqing Qu, Zhijun Li, Yinlong Liu, Alois Knoll
Thirty-Fifth AAAI Conference on Artificial Intelligence, 2021
project / code / video / bibtex 
@article{ispc:pointinet,
  author  = {Fan Lu and Guang Chen and Sanqing Qu and Zhijun Li and Yinlong Liu and Alois Knoll},
  title   = {PointINet: Point Cloud Frame Interpolation Network},
  journal = {AAAI},
  year    = {2021},
}
We propose a novel framework, namely Point Cloud Frame Interpolation Network (PointINet). Based on the proposed method, the low frame rate point cloud streams can be upsampled to higher frame rates.
RSKDD-Net: Random Sample-based Keypoint Detector and Descriptor
Fan Lu, Guang Chen*, Yinlong Liu, Zhongnan Qu, Alois Knoll
Advances in Neural Information Processing Systems (NeurIPS), 2020
code / video / bibtex 
@article{ispc:rskddnet,
  author  = {Fan Lu and Guang Chen and Yinlong Liu and Zhongnan Qu and Vittorio Murino},
  title   = {RSKDD-Net: Random Sample-based Keypoint Detector and Descriptor},
  journal = {NeurIPS},
  year    = {2020},
}
This paper proposes Random Sample-based Keypoint Detector and Descriptor Network (RSKDD-Net) for large scale point cloud registration. The key idea is using random sampling to efficiently select candidate points and using a learning-based method to jointly generate keypoints and descriptors.
Globally Optimal Camera Orientation Estimation from Line Correspondences by BnB algorithm
Yinlong Liu, Guang Chen*, Alois Knoll
IEEE Robotics and Automation Letters (RA-L), 2020
code / video / bibtex 
@article{liu2020globally,
  title={Globally Optimal Camera Orientation Estimation from Line Correspondences by BnB algorithm},
  author={Liu, Yinlong and Chen, Guang and Knoll, Alois},
  journal={IEEE Robotics and Automation Letters},
  volume={6},
  number={1},
  pages={215--222},
  year={2020},
  publisher={IEEE}
}
We propose a globally optimal camera orientation estimation algorithms. We decouple the rotation and translation estimation of a PnL problem by considering the geometrical property. The BnB algorithm is applied and it globally searches the entire rotation space to obtain the optimal camera orientation.
Pseudo-Image and Sparse Points: Vehicle Detection With 2D LiDAR Revisited by Deep Learning-Based Methods
Guang Chen*, Fa Wang, Sanqing Qu, Junwei Yu, Xiangyong Liu, Lu Xiong, Alois Knoll
IEEE Transactions on Intelligent Transportation Systems, 2020
code / video / dataset / bibtex 
@article{chen2020pseudo,
  title={Pseudo-Image and Sparse Points: Vehicle Detection With 2D LiDAR Revisited by Deep Learning-Based Methods},
  author={Chen, Guang and Wang, Fa and Qu, Sanqing and Chen, Kai and Yu, Junwei and Liu, Xiangyong and Xiong, Lu and Knoll, Alois},
  journal={IEEE Transactions on Intelligent Transportation Systems},
  year={2020},
  publisher={IEEE}
}
We propose a learning based method with the input of pseudo-images, named Cascade Pyramid Region Proposal Convolution Neural Network (Cascade Pyramid RCNN), and a hybrid learning method with the input of sparse points, named Hybrid Resnet Lite.
NeuroIV: Neuromorphic Vision Meets Intelligent Vehicle Towards Safe Driving With a New Database and Baseline Evaluations
Guang Chen*, Fa Wang, Weijun Li, Lin Hong, Jorg Conradt, Jieneng Chen, Zhenyan Zhang, Yiwen Lu, Alois Knoll
IEEE Transactions on Intelligent Transportation Systems, 2020
code / video / bibtex 
@article{chen2020neuroiv,
  title={NeuroIV: Neuromorphic vision meets intelligent vehicle towards safe driving with a new database and baseline evaluations},
  author={Chen, Guang and Wang, Fa and Li, Weijun and Hong, Lin and Conradt, J{\"o}rg and Chen, Jieneng and Zhang, Zhenyan and Lu, Yiwen and Knoll, Alois},
  journal={IEEE Transactions on Intelligent Transportation Systems},
  year={2020},
  publisher={IEEE}
}
In this work, we build the first-ever database, NeuroIV, and provides some baseline evaluations that bridges the gap between neuromorphic engineering and intelligent vehicle research. The NeuroIV introduces new ways to sense and perceive the environment that brings new revolution of vision-based perception system in intelligent vehicle. It will serve as a standardized and open-source platform on which new neuromorphic vision based methods can be developed and evaluated.
NeuroBiometric: An Eye Blink Based Biometric Authentication System Using an Event-Based Neuromorphic Vision Sensor
Guang Chen*, Fa Wang, Xiaoding Yuan, Zhijun Li, Zichen Liang, Alois Knoll
IEEE/CAA Journal of Automatica Sinica
code / video / bibtex 
@ARTICLE{guang-biometrics,
  author={G. {Chen} and F. {Wang} and X. {Yuan} and Z. {Li} and Z. {Liang} and A. {Knoll}},
  journal={IEEE/CAA Journal of Automatica Sinica}, 
  title={NeuroBiometric: An eye blink based biometric authentication system using an event-based neuromorphic vision sensor}, 
  year={2021},
  volume={8},
  number={1},
  pages={206-218},
  doi={10.1109/JAS.2020.1003483}}
Existing biometric authentication systems based on explicit and static features bear the risk of being attacked by mimicked data. This work proposes a highly efficient biometric authentication system based on transient eye blink signals that are precisely captured by a neuromorphic vision sensor with microsecond-level temporal resolution.
Globally Optimal Vertical Direction Estimation in Atlanta World
Yinlong Liu, Guang Chen*, Alois Knoll
IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020
code / video / bibtex 
@article{liu2020globally,
  title={Globally optimal vertical direction estimation in Atlanta World},
  author={Liu, Yinlong and Chen, Guang and Knoll, Alois},
  journal={IEEE Transactions on Pattern Analysis and Machine Intelligence},
  year={2020},
  publisher={IEEE}
}
In this paper, we propose a novel method for estimating the vertical direction in Atlanta world. It obtains the globally optimal solution by applying a BnB algorithm, without requiring any prior knowledge of the number of frames.
Multi-Objective Scheduling Strategy with Genetic Algorithm and Time Enhanced A* Planning for Autonomous Parking Robotics in High-Density Unmanned Parking Lots
Guang Chen, Jin Hou, Jinhu Dong, Zhijun Li, Shangding Gu, Bo Zhang, Junwei Yu, Alois Knoll
IEEE/ASME Transactions on Mechatronics, 2020
code / video / bibtex 
@article{chen2020multi,
  title={Multi-Objective Scheduling Strategy with Genetic Algorithm and Time Enhanced A* Planning for Autonomous Parking Robotics in High-Density Unmanned Parking Lots},
  author={Chen, Guang and Hou, Jing and Dong, Jinhu and Li, Zhijun and Gu, Shangding and Zhang, Bo and Yu, Junwei and Knoll, Alois},
  journal={IEEE/ASME Transactions on Mechatronics},
  year={2020},
  publisher={IEEE}
}
This paper provides an efficient and convenient scheduling solution for the implementation of the high-density unmanned parking lot.
A Novel Illumination-Robust Hand Gesture Recognition System with Event-based Neuromorphic Vision Sensor
Guang Chen, Zhongcong Xu, Zhijun Li, Huajin Tang, Sanqing Qu, Kejia Ren, Alois Knoll
IEEE/ASME Transactions on Automation Science and Engineering, 2020
code / video / bibtex 
@article{chen2021novel,
  title={A novel illumination-robust hand gesture recognition system with event-based neuromorphic vision sensor},
  author={Chen, Guang and Xu, Zhongcong and Li, Zhijun and Tang, Huajin and Qu, Sanqing and Ren, Kejia and Knoll, Alois},
  journal={IEEE Transactions on Automation Science and Engineering},
  volume={18},
  number={2},
  pages={508--520},
  year={2021},
  publisher={IEEE}
}
We propose an event-based gesture recognition system to overcome the detriment constraints and enhance the robustness of the recognition performance.
A Survey of the Four Pillars for Small Object Detection: Multiscale Representation, Contextual Information, Super-Resolution, and Region Proposal
Guang Chen, Haitao Wang, Kai Chen, Zhijun Li, Zida Song, Yinlong Liu, Wenkai Chen, Alois Knoll
IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2020
link / bibtex 
@article{chen2020survey,
  title={A survey of the four pillars for small object detection: Multiscale representation, contextual information, super-resolution, and region proposal},
  author={Chen, Guang and Wang, Haitao and Chen, Kai and Li, Zhijun and Song, Zida and Liu, Yinlong and Chen, Wenkai and Knoll, Alois},
  journal={IEEE Transactions on Systems, Man, and Cybernetics: Systems},
  year={2020},
  publisher={IEEE}
}
In this article, the first-ever survey of recent studies in deep learning-based small object detection is presented.
Event-based Neuromorphic Vision for Autonomous Driving: A paradigm shift for bio-inspired visual sensing and perception
Guang Chen*, Hu Cao, Jorg Conradt, Huajin Tang, Florian Rohrbein, Alois Knoll
IEEE Signal Processing Magazine, 2020
video / bibtex 
@article{chen2020event,
  title={Event-based neuromorphic vision for autonomous driving: a paradigm shift for bio-inspired visual sensing and perception},
  author={Chen, Guang and Cao, Hu and Conradt, Jorg and Tang, Huajin and Rohrbein, Florian and Knoll, Alois},
  journal={IEEE Signal Processing Magazine},
  volume={37},
  number={4},
  pages={34--49},
  year={2020},
  publisher={IEEE}
}
This article serves as a starting point for new research-ers and engineers in the autonomous driving field and provide a bird’s-eye view to both neuromorphic vision and autonomous driving research communities.
NeuroAED: Towards Efficient Abnormal Event Detection in Visual Surveillance With Neuromorphic Vision Sensor
Guang Chen*, Peigen Liu, Zhengfa Liu, Huajin Tang, Lin Hong, Jinhu Dong, Jorg Conradt, Alois Knoll
IEEE Transactions on Information Forensics and Security, 2020
dataset / code / bibtex 
@article{chen2020neuroaed,
  title={Neuroaed: Towards efficient abnormal event detection in visual surveillance with neuromorphic vision sensor},
  author={Chen, Guang and Liu, Peigen and Liu, Zhengfa and Tang, Huajin and Hong, Lin and Dong, Jinhu and Conradt, J{\"o}rg and Knoll, Alois},
  journal={IEEE Transactions on Information Forensics and Security},
  volume={16},
  pages={923--936},
  year={2020},
  publisher={IEEE}
}
Existing methods usually rely on standard frame-based cameras to record the data and process them with computer vision technologies. In contrast, this paper presents a novel neuromorphic vision based abnormal event detection system.

I like this website and this