The first part of the blog covered the hardware capabilities of the MRD5165 Eagle Kit. This part discusses various software packages and the benefits they bring when using the Eagle Kit for your drones.

Software for AI-Enabled Drones

Software development for AI-enabled drones involves creating and implementing the necessary algorithms, control systems, and interfaces to harness the power of the QRB5165 SOC and its AI capabilities. Here’s a more detailed exploration of the key aspects of software development for AI-enabled drones.

Operating System and Device Drivers:

Eagle Kit comes fully integrated with Ubuntu core and ROS framework packages along with Vision and Robotics SDK from Qualcomm Technologies. Linux-based operating systems are common choices for drone development considering its low-latency performance and efficiency. Developers can easily integrate various device drivers for sensors, cameras, and communication interfaces are crucial for seamless functionality.

Flight Control Software:

The flight control software is a critical component that manages and coordinates various subsystems of a drone to ensure stable, precise, and safe flight. MRD5165 Eagle Kit integrates crucial software stacks for Navigation and Path Planning, Auto Pilot, AI/ML, Robotics Vision, etc. This software also includes algorithms for flight stabilization, control systems, and communication with ground control stations. We will look at various Software stacks and SDKs in the later part of the article.

SLAM (simultaneous localization and mapping) is a method used for autonomous vehicles that lets you build a map and localize your vehicle in that map at the same time. The Eagle Kit supports VSLAM enabling the drone in simultaneous mapping of the environment and determining location using visual data by combining computer vision, image processing, and sensor fusion. Engineers use the map information to carry out tasks such as path planning and obstacle avoidance and to create 3D maps of its environment in real time, while in flight.

MRD565 Eagle Kit – SDK Family

GStreamer

GStreamer is a widely used open-source multimedia framework that provides a pipeline-based system for handling various multimedia tasks such as capturing, processing, and rendering audio and video streams. Drone developers can utilize GStreamer on Mistral’s MRD5165 Eagle Kit for various multimedia processing tasks.

• Capture: Acquiring audio/video streams from onboard sensors
• Real-Time Processing: Implementing immediate processing tasks
• Streaming: Broadcasting multimedia content locally or over networks
• Integration: Merging multimedia processing with flight control
• Onboard Applications: Developing multimedia apps directly on the drone
• Customization: Tailoring GStreamer for specific needs

Qualcomm Hexagon SDK

The Qualcomm Hexagon SDK equips developers with tools tailored for Mistral’s MRD5165 Eagle Kit. The Eagle Kit offers,

• The In-built Hexagon DSP architecture offers compilers, libraries, and IDEs for efficient software development.
• Hardware acceleration for multimedia and DSP tasks, machine learning support, power efficiency, and low-latency processing.
• Write code in C/C++/OpenCL, compile with Hexagon tools, optimize for performance, debug with SDK tools.

Qualcomm Neural Processing SDK

The Qualcomm Neural Processing SDK in Mistral’s MRD5165 Eagle Kit is tailored to streamline AI development for drone developers. The SDK enables,

• Quickly deploy AI models entirely on-device, leveraging Qualcomm AI products.
• Run trained neural networks efficiently on Qualcomm platforms, including CPU, GPU, and Hexagon Processor.
• Compatibility with TensorFlow, PyTorch, Keras, and ONNX formats, ensuring flexibility in model selection.
• Provides tools for model conversion, execution, and APIs for runtime control, along with desktop tools for conversion and performance benchmarking.
• Developers can use familiar frameworks like TensorFlow and PyTorch for network design and training, minimizing the learning curve.
  

  Source: Qualcomm

Computer vision SDK

The MRD5165 Eagle Kit includes essential tools and resources for drone developers to kick-start their projects, streamlining the development process and accelerating time-to-market for innovative drone applications. The Qualcomm Computer Vision SDK, integrated into Mistral’s Eagle Kit, offers drone developers a powerful toolset for enhancing their applications:

• Enable gesture recognition, face detection, tracking, recognition, text recognition, and augmented reality within drone-based applications, enriching user experiences and expanding functionality.
• The SDK offers a mobile-optimized computer vision library, ensuring efficient performance on ARM-based processors, ideal for drone platforms.
• Drone developers can integrate Qualcomm Computer Vision functions directly into their applications, leveraging the SDK’s capabilities to enhance drone vision systems.

ROS 2.0

Mistral’s MRD5165 Eagle Kit, comes with ROS 2.0, provides a robust platform for designing and deploying advanced autonomous drones. Let’s see how MRD5165 Eagle Kit enables drone designers to leverage the full potential of the ROS 2.0.

• The modular architecture of ROS 2.0 and high compute power of MRD5165 Eagle Kit allow developers to easily integrate new sensors, actuators, or algorithms into their drone systems. This modularity ensures scalability, enabling designers to adapt the drone’s capabilities to various applications without overhauling the entire software stack.
• With ROS 2.0’s efficient real-time communication capabilities, the MRD5165 Eagle Kit facilitates seamless data exchange between components, ensuring fast and reliable control and coordination essential for autonomous flight and navigation.
• The MRD5165 Eagle Kit supports a wide range of hardware configurations, from lightweight embedded systems to powerful computing units. ROS 2.0’s compatibility with diverse hardware platforms allows drone designers to choose the configuration that best suits their requirements, whether it’s for agile maneuverability or long-range capabilities.
• With the MRD5165 Eagle Kit, developers can easily integrate various algorithms, software and prototype drones, ensuring safety and reliability while reducing development time and costs.
• The MRD5165 Eagle Kit ensures support for ROS 2.0’s extensive ecosystem of libraries, tools, and pre-existing packages contributed by the ROS community. Drone designers can leverage these resources to accelerate development and focus on implementing higher-level functionalities, enhancing the capabilities of their autonomous drones.

Qualcomm Machine Vision SDK

By leveraging the MRD5165 Eagle Kit loaded with the Qualcomm Machine Vision SDK, drone developers can accelerate the development of advanced autonomous drones with enhanced perception, navigation, and control capabilities, enabling them to perform a wide range of tasks reliably and efficiently in diverse environments. Here’s how the Eagle Kit, combined with the Machine Vision SDK, facilitates drone development:

• The VSLAM algorithm from the Machine Vision SDK enables the Eagle Kit to accurately determine the drone’s position and orientation in real-world coordinates, crucial for autonomous navigation and mapping tasks. With the SDK’s capabilities, the Eagle Kit can create detailed 3D maps of the environment using data from onboard cameras, IMUs, and optionally GPS, providing valuable spatial awareness for the drone.
• By leveraging the DFS algorithm, drones equipped with the Eagle Kit can generate dense depth maps from stereo camera setups, allowing for precise distance estimation to objects in the environment. This depth sensing capability, combined with obstacle detection algorithms, enables the drone to navigate safely through complex environments, avoiding collisions with obstacles in its path.
• The DFT algorithm in the Machine Vision SDK provides an optic-flow-like tracking algorithm, which is essential for maintaining stability and accuracy during critical flight maneuvers, such as landing or following terrain contours. With the Eagle Kit and the SDK, developers can ensure that their drones maintain stable flight even when flying close to the ground or in challenging conditions.
• Through the VM algorithm, the Eagle Kit can generate volumetric representations of the environment, incorporating depth information from cameras to perform collision checking and ensure safe navigation through obstacles. This capability enhances the drone’s ability to navigate autonomously in complex environments, minimizing the risk of collisions and ensuring safe operation.

MAVLink SDK

The MRD5165 Eagle Kit comes with integrated MAVlink SDK. The MAVLink SDK serves as a fundamental element for drone software development on the MRD5165 Eagle Kit, empowering developers to construct robust and interoperable software solutions across various aspects of UAV operation, from flight control and navigation to mission planning and payload management.

The MAVSDK allows developers to communicate with and control drones, and other aerial robotic systems that utilize the MAVLink protocol. MAVSDK provides a high-level API that abstracts away the complexities of the MAVLink protocol, enabling developers to focus on building applications and payloads for various unmanned systems.

• MAVLink provides a consistent communication protocol for seamless data exchange between the MRD5165 Eagle Kit, ground control stations, and other components within the drone ecosystem, ensuring interoperability and reliability.
• The SDK furnishes developers with APIs and libraries designed to simplify MAVLink integration into drone software, abstracting the complexities of the protocol and expediting development efforts.
• Supporting various programming languages and platforms, including C/C++, Python, and mobile platforms like Android and iOS, the MAVLink SDK empowers developers to craft MAVLink-enabled applications tailored to diverse environments.
• Developers have the flexibility to customize and extend MAVLink functionality to meet specific project requirements, enabling the creation of custom message types, additional features, and integration with other protocols and systems.
• Seamlessly integrating with popular autopilot software such as PX4 and ArduPilot, the MAVLink SDK facilitates the development of companion computer applications, ground control stations, and mission planning tools that communicate effortlessly with the MRD5165 Eagle Kit through the MAVLink protocol.

Conclusion:

The Mistral MRD5165 Eagle Kit, equipped with the Qualcomm QRB5165 SoC and integrated with advanced software development tools, offers a robust platform for building AI-powered autonomous drones. Its support for ROS 2.0 and MAVLink SDK ensures interoperability, scalability, and community support for accelerated development and collaboration.

Mistral’s MRD5165 Eagle Kit stands as a formidable platform for drone developers to create innovative and high-performance AI-enabled aerial systems. With its powerful hardware, advanced software tools, and seamless integration capabilities, the Eagle Kit empowers developers to build next-generation autonomous drones that excel in navigation, perception, and mission execution across a wide range of applications.

To know more about MRD5165 Eagle Kit, visit the product page or write to us to talk to a technical expert.