First in-depth survey on the topic of deep transfer learning for intelligent vehicle perception

An international group of scientists has published a paper in the journal Green Energy and Intelligent Transportation, summarizing a comprehensive review of deep transfer learning for intelligent vehicle perception.

In recent years, perception has been viewed as a critical component in intelligent vehicles for precise localization, safe motion planning, and robust control. The perception system provides intelligent vehicles with immediate environmental information about surrounding pedestrians, vehicles, traffic signs, and other items and helps to avoid possible collisions.

Deep learning-based intelligent vehicle perception has been developing prominently to provide a reliable source for motion planning and decision making in autonomous driving. Many powerful deep learning-based methods can achieve excellent performance in solving various perception problems of autonomous driving.

However, these deep learning methods still have several limitations; for example, the assumption that lab-training (source domain) and real-testing (target domain) data follow the same feature distribution may not be practical in the real world. There is often a dramatic domain gap between them in many real-world cases.

As a solution to this challenge, deep transfer learning can handle situations excellently by transferring knowledge from one domain to another. Deep transfer learning aims to improve task performance in a new domain by leveraging the knowledge of similar tasks previously learned in another domain.

There are currently no survey papers on the topic of deep transfer learning for intelligent vehicle perception. This new survey paper aims to make contributions to introduce and explain the deep transfer learning techniques for intelligent vehicle perception, offering invaluable insights and directions for future research.

For intelligent vehicles or autonomous driving, perception plays a crucial role in receiving data from sensors and extracting meaningful information from the surrounding environment, so as to make meaningful decisions for precise motion planning by identifying obstacles, traffic signs/markers, and available driving areas. The researchers grouped these intelligent vehicle perception tasks into two classes (object detection, semantic/instance segmentation).

Despite the remarkable achievements of the intelligent vehicle perception algorithms on benchmark datasets, there are still significant challenges in the real world due to the large variations in the sensor types and settings, data in diverse style, environment, weather and illumination, trained epoch, and architecture.

Based on these observations, the researchers divided the domain distribution discrepancy for intelligent vehicle perception into three types: sensor difference, data difference, and model difference.

With the rapid advancement of autonomous driving techniques, there is now an abundance of driving scene images available. Deep learning methods are booming in the application of autonomous driving with high performance perception.

Transfer learning (TL) is a machine learning method to largely apply the knowledge acquired from one task or domain to another related task or domain. Researchers classified deep transfer learning into several main types: Supervised TL, Unsupervised TL, Weakly-and-semi Supervised TL, Domain Generalization.

The main challenges of deep transfer learning for the current intelligent vehicle perception include sensor robustness, methodology limitation, realism of synthetic data, scarcity of annotated benchmarks in complex scenarios, international standards for hardware sensors, and international standards for software packages.

In order to address the above challenges, the following work will be undertaken in the future:

• Firstly, more research should be focused on improving the sensor robustness.
• Secondly, researchers could make efforts to develop more advanced deep transfer learning methods.
• Thirdly, the realism of the synthetic data can be improved by more advanced computer game engines.

Additionally, more high-quality benchmark datasets in complex driving scenarios could be collected and publicized. Finally, multiple companies from different countries can collaborate to promote international standards for hardware sensors and software packages.