Automotive Autonomous Driving Sensor: Type and Development

With the rapid development of the automobile industry, the problem of traffic congestion is becoming more and more serious. The development of autonomous vehicles is one of the ways to solve the current traffic congestion. Against the background of high-quality economic development, the automobile industry is developing in the direction of intelligence, electrification, and connectivity. The electric automobile platform has created a hardware foundation for the implementation of autonomous driving technology, and has laid a solid foundation for the realization of autonomous driving at L2 and above levels. Good conditions were created. Cars equipped with autonomous driving technology greatly reduce the driver's driving burden and greatly improve the level of active safety of the car. The rapid development of autonomous driving technology has catalyzed the development and application of autonomous driving sensors. This article will sort out the types of autonomous driving sensors and evaluate the advantages, disadvantages and development prospects of various sensors.

Electric cars rely on self-driving sensors for overtaking capabilities

1. What is Autopilot sensor?

Autopilot sensors are specialized devices used in autonomous and semi-autonomous systems, such as self-driving cars, drones, and aircraft, to gather real-time data from the environment and assist in navigation and control tasks. These sensors include a variety of technologies like cameras, lidar (Light Detection and Ranging), radar, ultrasonic sensors, GPS (Global Positioning System), IMUs (Inertial Measurement Units), and more. They work collectively to provide the system with information about the vehicle's surroundings, speed, orientation, and position. This data is crucial for the autopilot system to make informed decisions, adjust trajectories, and ensure safe and accurate navigation without human intervention. Autopilot sensors play a pivotal role in enabling vehicles and machines to sense their environment, avoid obstacles, maintain proper paths, and ultimately enhance overall operational efficiency and safety.

2. Autopilot sensor type

To realize autonomous driving, the automotive system needs to perceive two levels of information: first, the vehicle's own status information, such as the vehicle's forward direction, position coordinates, speed, acceleration, steering angle, etc.; second, information about the environment in which the vehicle is located, such as the road Information, traffic signs, other vehicles, pedestrians, etc. According to the current state and environmental information, the car analyzes and makes decisions, and completes the automatic driving operation of the vehicle through the executive mechanism. Therefore, for the category of autonomous driving sensors, it can also be divided into two categories according to this standard:

One is the sensor that perceives the state of the vehicle itself, mainly the vehicle positioning sensor;

One category is sensors that perceive environmental information, mainly based on visual perception and radar perception.

2.1 Sensors for Vehicle Positioning

Satellite positioning system. The satellite positioning system uses electromagnetic wave communication between multiple satellites and ground receivers to solve the three parameters of position, time, and speed. In principle, only 4 satellites are needed. However, due to various propagation errors, it is necessary to Multiple satellite redundant data are corrected. The United States first used the satellite positioning system for the military, and later expanded it to the civilian field, which is the well-known GPS (Global Positioning System) system. The satellite positioning system can output the longitude and latitude position information of the receiver, altitude information, direction information, speed information, etc. Through RTK differential positioning technology, the positioning accuracy can be greatly improved to the centimeter level, which is ideal for outdoor positioning of cars with good electromagnetic wave propagation conditions. important means. The research and development and application of satellite positioning systems rely on the development of aerospace technology. Only a few countries and regions in the world can build their own satellite positioning systems. In addition to the earliest GPS system, Russia also built the "GLONASS" system (GLONASS) during the Soviet period, and the European Union also developed the Galileo satellite positioning system (GALILEO). China's Beidou Satellite Positioning System (BDS) started late, but it currently has preliminary regional navigation, positioning and timing capabilities. The positioning accuracy is decimeter and centimeter levels, the speed measurement accuracy is 0.2m/s, and the timing accuracy is 10ns. The satellite navigation system with global coverage has outstanding advantages: global all-weather positioning, high positioning accuracy, short observation time, and can provide globally unified three-dimensional geocentric coordinates. But the shortcomings are also prominent: the system requires a good outdoor environment for normal operation without obvious building obstruction, otherwise it will cause signal loss. Therefore, the satellite navigation system is suitable for outdoor navigation, but for tunnels and indoor environments, its application capabilities are greatly reduced.

Vehicle positioning sensor and satellite communication signal simulation diagram

inertial navigation system. The inertial navigation system measures the acceleration of the carrier in the inertial reference system, performs an integral operation, and obtains the instantaneous speed and position data of the carrier, and transforms it into the navigation coordinate system to obtain the speed and yaw angle in the navigation coordinate system. and location information. The common inertial navigation system is the six-axis gyroscope sensor of the IMU inertial navigation module. Given an initial condition of the inertial navigation system, the system can calculate the current state of the vehicle by integral calculation without relying on external reference. Because of this feature of the inertial navigation system, it has two very obvious advantages: First, the inertial navigation system does not depend on the external reference system, so there is no requirement for the environment. Compared with GPS positioning, the inertial navigation system can be used indoors 1. Working in a closed environment; Second, the inertial navigation system is generally loaded inside the vehicle, and it can still not be damaged when the vehicle is scratched or damaged by collision. However, because the inertial navigation system needs to be a continuous integrating system, it is an open-loop system without external reference conditions. Therefore, any small system error may cause the error to diverge after time integration. Because of this shortcoming, the inertial navigation system does not work reliably for a long time without reference conditions. It often requires references from other systems to correct errors. It is suitable for backup navigation when the GPS signal is lost for a short period of time.

Indoor wireless positioning system. Because satellite positioning has the disadvantage of not being able to navigate indoors, and there is a problem of error divergence through the inertial navigation system, it is necessary to find a new solution for indoor working conditions. At present, the indoor wireless positioning system is widely used. The wireless link can be Wi-Fi, ZigBee, or 5G signal. The accuracy of these positioning systems is not high, and the common Wi-Fi positioning accuracy is only 3-5m. In recent years, a high-precision indoor positioning method has been gradually promoted. It is called a new indoor high-precision wireless positioning system UWB, which can accurately obtain 10cm-level precision positioning data through wireless pulse time measurement technology.

2.2 Vision sensor

For the perception of the environment in which the car is located, a very simple idea is: to imitate the perception process of people driving, that is, the visual perception of the eyes. Therefore, the simulation of human eyes through single or multiple cameras has become an important route for autonomous driving perception. Cameras are divided into panoramic cameras, fisheye cameras, depth cameras, night vision cameras, night infrared cameras, etc. The basic principle is similar to the eyes. It aims at the object you want to observe and can automatically focus on the object you want to pay attention to. Advances in image processing technology have greatly promoted the development of automotive visual navigation.

Vision sensor circuit board diagram

Current image processing technology can not only quickly identify lane lines, but also identify traffic signs such as speed limits, other vehicles, and pedestrians on the road. It can be said that in an environment with good lighting conditions, using a camera to perceive the environment can identify most of the elements in the traffic environment.

Cars equipped with L2-level assisted driving functions are generally equipped with a front camera to recognize road lines, provide the deviation of the vehicle's driving direction and the target path for the road keeping (LKA) function, and realize the vehicle's centered driving through negative feedback adjustment. In addition to detecting the environment ahead, environmental detection in the side and rear directions can provide more necessary information for the vehicle to make decisions such as changing lanes and overtaking. Xpeng P7 Pengyi Edition, which has reached L2+ level of autonomous driving, is equipped with up to 14 cameras to achieve 360° panoramic visual perception of the vehicle; Tesla Model 3, a global leader in smart cars, has a total of 8 cameras in the car, on the left and right wings There is a side-facing and rear-facing camera on each board, which is integrated with the side turn signal. There is a forward-facing and downward camera on the left and right sides of the upper part of the B-pillar. There are three forward-facing cameras above the front windshield. There are three forward-facing cameras above the license plate. A rearward-facing camera, its rich camera configuration provides the Autopilot autonomous driving system with comprehensive environmental awareness.

The deployment of multiple cameras can provide vehicles with a full 360° environmental perception. Driven by the rapid development of artificial neural networks, especially deep learning algorithms, visual navigation integrating multiple cameras has become an inevitable choice for high-end autonomous driving. However, cameras also have their natural flaws. For example, in environments such as heavy rain, night, heavy fog, and strong reflections, their environmental detection capabilities are greatly reduced, and their reliability is also greatly reduced.

2.3 Radar perception sensor

Among the categories of environmental sensing sensors, there is a category that specifically uses the principle of reflection ranging to detect obstacles, and is collectively called radar. According to different transmission media, it can be divided into ultrasonic radar using ultrasonic waves, millimeter wave radar using millimeter waves, and laser radar using lasers.

Ultrasonic radar. Ultrasonic radar is a kind of radar that uses ultrasonic waves to reflect off obstacles, and then calculates the distance between itself and obstacles based on the speed of ultrasonic waves, and the cost is low. At present, ultrasonic radar is relatively mature, has a high market penetration rate, and the price has dropped to a low level, with a single cost of less than 10 yuan. The propagation speed of sound waves is low, so the effective detection range of ultrasonic radar is short. The maximum detection distance of common ultrasonic radars on the market does not exceed 6m. Due to its short detection range but low cost, reversing assistance and automatic parking functions are the best application scenarios for ultrasonic radar. Cars with reverse assist and automatic parking functions will be equipped with multiple ultrasonic radars, usually located on the front and rear bumpers of the vehicle, with the number up to six.

 

Radar Sensing Sensors: Utilizing Reflection Ranging Principles to Detect Obstacles(Image Source: Just Auto)

Millimeter wave radar. Millimeter wave radar is a radar that operates in the millimeter wave band. Usually millimeter waves refer to electromagnetic waves in the 30-300GHz frequency domain (wavelength 1-10mm). Because the wavelength is shorter, the resolution is higher. Millimeter wave radar can provide location and speed information of obstacles, and the longest detection range can reach 1000m. In autonomous driving technology, front millimeter wave radar is often used for adaptive cruise (ACC), and side radar is often used for side collision avoidance and lane change assistance. Millimeter-wave radar has the ability to penetrate smoke and dust, and has the ability to work all-weather. Compared with cameras, millimeter-wave radar can work at night, in heavy fog, etc., and can make up for the shortcomings of visual navigation. Therefore, in autonomous driving technology at L2 and above, camera and millimeter wave radar data are integrated with each other. At present, the competition concentration in the global millimeter-wave radar market is relatively low, and a dominant player has not yet formed. At present, traditional Tier manufacturers occupy a dominant position. The disadvantage of millimeter wave radar is that its static ranging is too complicated and its judgment of tangential motion is poor.

LiDAR. Lidar is a radar system that emits a laser beam to detect target position, speed and other characteristics. The longest detection distance can reach 200m. Compared with millimeter-wave radar, lidar can not only obtain the position and speed of obstacles, but also obtain the three-dimensional shape characteristics of obstacles. Because of this, lidar can be used to perform three-dimensional modeling and identification of the environment in which the vehicle is located. Various static and dynamic obstacles. Lidar can provide such rich environmental information, which can greatly improve the ability of autonomous driving to avoid obstacles. In addition to the ranging module, the lidar system often also integrates the inertial measurement system and the GPS positioning system. It is a comprehensive sensor for environmental perception and positioning. Because the laser wavelengths are mainly 905nm and 1550nm, it can provide high-precision obstacle detection. Usually the angular resolution is not less than 0.1mrad, which means that the lidar can distinguish two targets 0.3m apart at a distance of 3km, and can simultaneously Track multiple targets with distance resolution up to 0.lm. The signal propagation of lidar is greatly affected by the medium. It can achieve good detection results in clear and highly visible air.

However, lidar errors become larger in weather conditions such as thunderstorms, heavy fog, and heavy rain. Early use of lidar in autonomous vehicles was relatively rare, mainly because the cost was too high, which limited industrial promotion. Take the HDL-64E lidar launched by Velodyne in 2007 as an example, the price is as high as US$80,000. As domestic manufacturers including Huawei and DJI begin to invest in research and development, the cost of vehicle-mounted lidar is falling rapidly. Sagitar announced the MEMS solid-state lidar RS-LiDAR-M1 in January 2020, priced at US$1,898. If the order volume is between 100,000 and 1 million units, the price will drop to between US$200 and US$500; The lidar technology developed by Huawei's Wuhan Optoelectronics Technology Research Center plans to reduce the cost of lidar to US$200 or even US$100 in the future; DJI's lidar brand Livox product HAP will be mass-produced in 2021 and has successfully obtained approval from Xpeng Motors Mounted in mass production. In March 2022, the Xpeng P5 was launched, becoming the world's first mass-produced model equipped with lidar. NIO's high-end sedan ET7 will also be equipped with lidar and will be launched in 2022. Models equipped with lidar can generally achieve L3 autonomous driving capabilities.

3. Prospects for the development of autonomous driving sensors

With the improvement of the level of autonomous driving technology and the evolution of multi-technical routes for autonomous driving, the sensors used for autonomous driving are also developing in the direction of multi-sensor fusion, neuromorphic and networked.

3.1 Multi-sensor fusion

Autopilot sensor suppliers will further provide a set of distinctive autopilot solutions while providing autopilot sensors. This set of solutions is often based on the fusion of multiple sensors. Some manufacturers try to integrate GPS, lidar, camera and inertial navigation sensors, and realize the integration of multi-sensors at the hardware product level. The redundant information between sensors supports each other, which can greatly improve the robustness of environment perception. For example, HiDefLiDAR, a product of the long-range image-level lidar company Innovusion, is equipped with a 1080P camera. On the real-time point cloud perceived by the lidar, each pixel contains not only RGB color information, but also spatial coordinates and reflection value information. , are three-dimensional color point clouds. The fusion of lidar and camera video data at the hardware level is more efficient than pure software fusion. Data from different sensors can maintain good real-time performance, and the efficiency of computing and processing will be greatly improved.

3.2 Neuromorphic sensor

The basic idea of autonomous driving sensors is to imitate human perception. Human perception is intelligent and pays differential attention to the environment. Therefore, researchers start from this concept to develop neuromorphic sensors to make the sensors more intelligent.

Take vision as an example. Vision is a basic function of intelligent creatures and agents. More than 80% of the information in the human perception system comes from the visual system. The human visual system has the advantages of low redundancy, low power consumption, high dynamics and strong robustness. . The current camera sensor collects a large amount of environmental information indiscriminately. In order to achieve better perception effects, the trend is often towards increasing the resolution. The increase in resolution puts forward higher requirements on the computing power of the on-board processor, which greatly increases the cost of the system.

From this perspective, the current camera sensor provides a large amount of redundant information to the computing center, which is not the mechanism of the human eye. Therefore, how to make camera perception closer to human eyes and more intelligent has become an important research and development direction. Some researchers have proposed neuromorphic vision sensors. Neuromorphic vision is a visual perception system that includes hardware, software, and biological neural models. Target. The biologically-inspired vision sensor "event camera" developed with this idea works in a completely different way than standard cameras. Instead of outputting intensity image frames at a constant rate, event cameras only output information about local pixel-level brightness changes. When these pixel-level brightness changes (called events) cross a set threshold, the event camera timestamps with microsecond resolution and outputs an asynchronous event stream. Event cameras do not require high-computing AI chips. They only require 1% or less of the computing power of traditional high-speed case AI chips to complete the work, and the power consumption is greatly reduced.

3.3 Networking of sensors

With the rapid development of new-generation information technologies such as 5G, cloud computing, and the Internet of Things, the technical route of intelligent connected vehicles has been highly valued. As of March 2022, nearly 30 cities across the country have issued more than 900 A road test license has been issued, more than 5,000 kilometers of intelligent connected vehicle test roads have been opened, and the cumulative mileage of safe road tests has exceeded 13 million kilometers. The technical route of intelligent network connection puts forward new requirements for autonomous driving sensors. In the context of intelligent networked vehicles, the sensors mounted on the vehicle no longer serve the current vehicle itself, but can also be used by cloud computing centers or other vehicles through the Internet of Things. The characteristics of the network connection are obvious. Taking 5G-V2X as an example, the positioning of the vehicle itself can be obtained through the communication between the vehicle and the environmental buildings or terminals installed on the road, while the perception of other vehicles on the road can be obtained more accurately through the communication between V2V Vehicle behavior prediction; connected vehicles in the traffic system can communicate with the cloud computing center through 5G communication to achieve a more overall optimal path planning and help the development of smart transportation and smart cities. Different from the current closed network implemented by systems such as radar and sensors mounted on a single vehicle, 5G-V2X enables the collection of information of all traffic participants on the road to form a real intercommunication network, so that the vehicle has a "God's perspective" and can Achieving global optimal planning for various information is an important basis for realizing L4 level driverless driving.

4. Conclusion

Autonomous driving sensors are an important basis for the realization of autonomous driving technology, and the development of autonomous driving sensors determines the development of autonomous driving technology. Autonomous driving sensors are divided into two categories: sensors that sense the vehicle's own status, represented by GPS systems, and sensors that sense environmental information, represented by vision and radar sensors. With the improvement of the level of autonomous driving technology and the evolution of autonomous driving technology routes, the sensors used for autonomous driving are also developing in the direction of multi-sensor fusion, neuromorphic and networked.