ModalAI’s Evolutionary Development of World Class Flight Controllers
Written by Vinny Kemler
Wow, what a ride! (for our hardware)
This Summer, ModalAI witnessed a huge inflection point in the history of robotics. The Snapdragon autopilot the original team at ModalAI worked on during our time at Qualcomm enabled NASA’s Ingenuity helicopter to take its first flight on Mars (and many since!)
During my 23+ year hardware engineering career, I’ve been a part of and directly influenced many technological firsts and advancements in cellular, IoT, wearables, and robotics. Like several of us at ModalAI, I spent the bulk of my career at Qualcomm. While there I was developing best in class hardware for a breadth of product lines. I lived through the birth of the smartphone powered by the Snapdragon system on a chip (SoC) with 3G, led hardware development of the first 3G connected smartwatch and the first wireless charging always-on display wrist watch. Then, was privileged to work on and lead the hardware efforts to create the first, second, third, and fourth generation of Snapdragon based autopilots, one of which ultimately enabled NASA’s Ingenuity, and the latest two which now complement ModalAI’s broad portfolio of powerhouse AI driven flight controllers. We are on the cusp of more fantastic and impressive robotic capabilities, and ModalAI’s history and future are part of this driving force.
What is a Flight Controller?
ModalAI designs and develops hardware, software, and mechanical systems for ground and aerial systems, with the primary focus on quadcopters, so let’s describe briefly what a flight controller is and what it does.
A flight controller is the brains of a drone that enables it to fly (or drive for ground robots). Flight is performed by controlling the thrust of propeller motors while simultaneously monitoring the status of several inertial and position sensors to maintain steady and stable positioning (the drone’s attitude). When a pilot wishes to perform a mission either manually or autonomously those controls are updated to affect the movement of the drone and the flight controller ensures the movement is done to adjust the position precisely while maintaining a safe attitude.
Additional sensory inputs such as GPS and image sensors help navigation and obstacle detection or avoidance, and many radio links such as 4G or 5G help keep the pilot in touch with the craft during the mission or to get critical mission data from the drone. It is the job of the flight controller to take all these inputs simultaneously and achieve the desired mission (even if it is just for a selfie or something more crucial like preventing our soldiers from being put into harm’s way).
If a user wants the flight controller to just focus on stable and precision flight while freeing up compute power for image processing or artificial intelligence applications, they can use a companion computer. Our VOXL Flight hardware design is a perfect example where we pair up a 32-bit Arm running open source PX4 flight control software (on our Flight Core) alongside the Snapdragon 821 (on our VOXL) which is able to focus on all the mission critical applications like video processing and data links.
The ModalAI portfolio offers VOXL and Flight Core as independent flight controllers or the combined VOXL Flight companion computer solution to offer our customers ultimate scalability and flexibility. And now, with our newest release of the Qualcomm Flight RB5 5G Platform, we have expanded our portfolio to include the world’s first AI and 5G-enabled drone solution.
How did ModalAI achieve this level of integration and excellence for a new start-up? Technically speaking, VOXL is the team’s third generation flight controller design, with the new Qualcomm Flight RB5 5G Platform as the fourth generation. At Qualcomm, where a large chunk of this team originated from, we developed three generations of Snapdragon powered flight controllers.
Evolution of the Snapdragon Flight Controller
ModalAI leverages over five years of prior research and development in the drone and robotics space from Qualcomm. I joined the Qualcomm robotics team in 2014 when it was just forming and was asked to be the hardware lead. I never thought from that moment we would drive so many advancements in robotics, autonomy, and the dozens of applications our customers are now developing using the hardware I designed. Not to mention the advancement of space exploration by achieving the first controlled flight on Mars!
When our team started, we had no experience designing drone hardware and had no idea what the market wanted or how we could influence the hobbyist community to trust and desire our Snapdragon based solutions. We were competing with simple 8-bit and cheap single core 32-bit hardware platforms many were already comfortable with. Was there really a place for multi-core smartphone chips? We sure hoped so! Our CTO, Donald Hutson, and CEO, Chad Sweet, were at that time very eager to prove what could be done. We embodied a team mantra to develop state-of-the-art hardware and software that could support a multitude of industries and demonstrate new and unique features currently not capable. It was the old adage “build it and they will come”.
Turning a cellphone into a robot is not an easy feat. It took a lot of outside the box thinking and clever adaptation to change a phone, which mostly stays in a pocket, into something that can be compelling for quadrotors, fixed wing, or ground based robotics that is always on the move and processing its environment. In the engineering world, this is a very difficult yet fun challenge.
2015 - Snapdragon Cargo
The first flight controller hardware we developed, known internally as “Constitution”, used a Snapdragon 600 class chipset and powered our first “break-out demonstration”, the Snapdragon Cargo. A multi-functional robot designed by Donald Huston, that could seamlessly navigate on land and in the air. The robot could fly and drive like a tank! Also note, it was an Octo-rotor (yes, that’s right… eight propellers, not the four propeller quad-rotors that are ubiquitous now).
This was unveiled at CES in 2015 and illustrated to the world that Qualcomm was officially into the business of drones. We were all very proud of Donald’s ingenious design and the hustle by my hardware team and Chad’s software team to make this all come together. We still have fond memories of that platform which propelled us into the spotlight.
2016 - Snapdragon Flight
As a direct result of the marketing and technical success of the Snapdragon Cargo, we had immediate demands to do more. We iterated upon the Constitution and developed the first 800-class Snapdragon chipset platform, our second generation flight controller, the Snapdragon Flight. Internally we called this “Eagle”. The Eagle design was a significant advancement to Constitution with the Quad-Core 801 32-bit Arm-core and advanced DSP (digital signal processor) capabilities. Snapdragon Flight showcased a more traditional Quad-rotor design and included a plethora of direct mounted camera sensors, on-board inertial sensors, WiFi and GPS and a ton of flash memory storage and expansion capabilities. All in just a 40mm x 58mm package (smaller than a business card and much smaller than Constitution). Prior to this design, all “state of the art” drones had at least three and sometimes up to seven different hardware circuit boards. We achieved this level of integration in just one circuit board. This might have helped NASA’s JPL to decide they were interested in this design, and ultimately, after an exciting technical exchange meeting and tour with the Pasadena Helicopter team and years of further development, Snapdragon Flight took a 250 million mile journey as part of the Mars Ingenuity Helicopter and paired link on the Perseverance Rover. The image captured here shows Chad (middle-left) and I (middle-right) directly next to the Rover model at JPL HQ, with our other Qualcomm colleagues on the edges.
2017 - Qualcomm Flight Pro
Our team was satisfied with what we had done, but we knew there was more to come. After the success of the Snapdragon Flight in 2016, we heard about a new Qualcomm chip coming that promised even more AI and vision processing capabilities called the Snapdragon 821. We took the best features of Snapdragon Flight and evolved them into the Qualcomm Flight Pro, an 820-class chipset that was more than twice as powerful with a 64-bit Quad-core CPU and independent DSP that alone was more powerful than most video game chips. This would be the world's first 64-bit flight controller. Coming in at 36mm x 75mm, (still smaller than a business card) we showcased the impressive camera and control capabilities of this platform at CES 2017 where we had a 360-degree camera drone flying over 20-feet in the air paired with a Virtual Reality headset to give conference goers a unique First-Person-View of the show floor. We also highlighted swarm capabilities where several drones at once completed an autonomous mission through an obstacle course. Yes, designing all this hardware was fun and cool!
2018 - VOXL
In 2018, Chad Sweet and Donald Hutson determined that we still had more to do for this market. They founded ModalAI to continue the success and capabilities of the Qualcomm Flight Pro. The first order of business was to make several improvements to that design’s hardware, and continue to develop the software into a much more flexible and customer friendly open-source and scalable platform. This is what is now known as our flagship VOXL companion computer. In order to garner the interest of the robotics developer market, we enabled VOXL to support popular open source applications, like PX4, TensorFlow Lite, and OpenCV. By making our autopilots open, it enables our customers to achieve best-in-class computer development features and deep learning AI capabilities with a buildable Linux kernel, cross-compile SDKs, LTE/5G, time-of-flight cameras, Docker images for development, and more.
2020 - VOXL Flight
In 2020 we partnered with the Defense Innovation Unit (DIU) to create the VOXL Flight, a PCB that combines our VOXL companion computer with our Flight Core PX4 flight controller. As part of the evolution, along with around the clock support, VOXL Flight is NDAA ‘20 Section 848 compliant for U.S. Military and Government applications. This gives our customers the assurance that this hardware is assembled and tested in the U.S.A. (right here in Southern California) allowing a bevy of applications where national security is of interest.
We have since expanded our VOXL family line to include flight controller PCBs all the way up to fully built development drones. The VOXL companion computer is now being used by 200+ partners, and most notably was awarded the 2021 RBR50 Innovation Award, 2021 Artificial Intelligence Excellence Award, and helped Chad win San Diego’s 2021 Future of Tech Award.
2021 - Qualcomm Flight RB5 5G Platform
Our mission at ModalAI is to accelerate autonomy for smaller, safer, and smarter drones and robots with open, NDAA-compliant autopilots. We are thrilled to see the success of our platforms in many products and robotic applications across the globe. We are committed to continuing that mission and driving (or flying) forward. Most recently, we collaborated with Qualcomm to launch the Qualcomm Flight™ RB5 5G Platform; the world’s first 5G and AI-enabled drone platform and reference design.
Qualcomm Flight RB5 5G Platform
The Qualcomm Flight RB5 5G Platform allows developers and Original Equipment Manufacturers (OEMs) to build drone and flight applications based on the Qualcomm® QRB5165 System-on-a-Chip (SoC). This platform is the latest evolution of the Snapdragon-based flight controllers, with the powerful QRB5165 SoC onboard that supports autonomous flight using 5G cellular technology and the 5th generation Qualcomm® Artificial Intelligence (AI) Engine. This design was many months in the making, and our team managed to design this new world-class hardware, software, and mechanical solution during the height of the global pandemic. We are all proud of this solution and know this is just the beginning of even more advanced platforms yet to come. The Qualcomm Flight RB5 5G Platform is available for purchase now.
I’m honored to be a part of this great team as the Director of Hardware Engineering where I get to talk about our amazing history of achievements. And, you guessed it, we aren’t done designing and innovating yet! At ModalAI, we are constantly iterating and advancing robotic capabilities. As the robotics space and our customers demand more, ModalAI is committed to developing the next generations of best-in-class, Size, Weight and Power (SWAP) optimized, Blue UAS Autopilots.
Sign up for our newsletter to stay in the loop about new products, demos, and releases.
