Obstacle Avoidance on ASR-D501 with USB Camera
This guide demonstrates how to build an obstacle avoidance pipeline on the ASR-D501, using a USB camera and flight controller.
Overview
- Model Optimization & NPU Acceleration: Use Qualcomm AI Hub to export a YOLO object detection model tailored specifically for the QCS6490 chipset. This seamless deployment enables the device to run high-speed, hardware-accelerated inference directly on its NPU.
- Scalable ROS 2 Architecture: Use the OrbbecSDK ROS 2 to publish real-time RGB and depth image topics from the 3D camera. This publish-subscribe method ensures modular, low-latency sensor communication that easily integrates with drone control systems.
- Analysis & Decision: This demo combines the detection bounding boxes with the depth data to estimate the distance to obstacles, ultimately determining the required avoidance direction (Left, Center, or Right) for the flight controller.
System Requirements and Components
| Component | Specification |
|---|---|
| Platform | ASR-D501 (Qualcomm QCS6490) |
| OS Image | ASRD501A1-Ubuntu24.04-x07_v1.0.0 |
| 3D Camera | Orbbec Gemini 335Lg (USB) |
| Flight Controller | Pixhawk® 6C |
| Model | YOLOv8-Detection (from Qualcomm AI Hub Models) |
Workflow
- Step 1: Refer to the Downloading Qualcomm AI Hub Models guide for model acquisition
- Step 2: Connect and identify the USB camera
- Step 3: Environment setup (Install OrbbecSDK and OrbbecSDK ROS 2)
- Step 4: Download the Python file and run live inference
- Step 5: Connect to the flight controller
Step 1: Refer to the Downloading Qualcomm AI Hub Models guide for model acquisition
Please refer to the Downloading Qualcomm AI Hub Models to obtain the object detection model. Follow the instructions there to export the YOLO model for the QCS6490 chipset.
Step 2: Connect and identify the USB camera
Connect the Orbbec Gemini 335Lg to a USB Type-C port on the ASR-D501.
You can verify that the system detects the camera hardware by running the following command:
lsusb
Look for the Orbbec device in the output list to confirm a successful physical connection.
Step 3: Environment setup (Install OrbbecSDK and OrbbecSDK ROS 2)
Since we use an Orbbec camera and ROS 2 for this project, we need to install the official OrbbecSDK and OrbbecSDK ROS 2 to get the camera data.
- OrbbecSDK v2 - v2.9.3 [Official GitHub Repository ↗]
Please refer to the GitHub repository and follow the instructions to set up your workspace first.
After successfully installing the Orbbec SDK v2,
you can launch the OrbbecViewer.
This all-in-one graphical user interface (GUI) offers a suite of features including data stream preview, camera configuration, and post-processing.
The OrbbecViewer interface is shown below.

- OrbbecSDK ROS 2 - v2.9.3 [Official GitHub Repository ↗]
Please refer to the GitHub Repository to clone the source code into the src directory of your ROS 2 workspace (~/ros2_ws/src).
Before building the package, you must modify the gemini_330_series.launch.py file
(in ~/ros2_ws/src/OrbbecSDK_ROS2/orbbec_camera/launch folder)
to set the specific resolution and frame rate for both the RGB and Depth:
- Width: 640, Height: 480
- FPS: 30
Please refer to the image below for the exact parameters to modify:


After saving the changes in the launch file, build the workspace so the configurations take effect:
cd ~/ros2_ws
colcon build --event-handlers console_direct+ --cmake-args -DCMAKE_BUILD_TYPE=Release
With the package successfully built, you are now ready to launch the camera node to publish the topics and monitor the live streams.
- On terminal 1 - Launch the camera node
source ~/ros2_ws/install/setup.bash
ros2 run orbbec_camera list_devices_node # Check if the camera is connected
ros2 launch orbbec_camera gemini_330_series.launch.py
- On terminal 2 - Use RViz2 to visualize the camera output
source ~/ros2_ws/install/setup.bash
rviz2
Once RViz2 opens, you can add an Image display to view the live RGB and depth topics.
You can follow the below image to add topics:

- On terminal 3 - List active ROS 2 topics
source ~/ros2_ws/install/setup.bash
ros2 topic list
Ensure that your camera successfully publishes the RGB and Depth topics, as shown below.

Step 4: Download the Python file and run live inference
Please use the link below to download the code and place it in the same directory as your exported model:
📥 Download obstacle_det.py
This script processes the live ROS 2 image streams, detects birds (obstacles) and persons, and calculates the obstacle's location (Left, Center, or Right) in the frame using the depth data.
Run the inference script:
source ~/ros2_ws/install/setup.bash
python3 obstacle_det.py
You should now see a window displaying the real-time bounding boxes and distance estimation!
Step 5: Connect to the flight controller
In our complete physical drone system,
once obstacle_det.py identifies where the obstacle is (Left, Center, or Right),
it sends a message to the flight controller.
The flight controller then triggers the appropriate motors to move the drone and avoid the obstacle.
Here is a simple workflow of the process:
Demo Video
Check out our real-world flight test video below to see the obstacle avoidance in action!