Your agency recently purchased a small unmanned aircraft system (sUAS), commonly referred to as a drone. The chief assigned Firefighter Wright as the pilot because he professed to have “Hundreds of hours of model aircraft experience.” Firefighter Wright possesses a Federal Aviation Administration Remote Pilot Certificate but has never completed any type of flying skills evaluation.
All is going well until Wright responds to a two alarm fire at an industrial complex adjacent to Interstate Highway 29 (I-29). The winds are a bit high, 15 mph (24 kph) gusting to 20 mph (32 kph), not impossible for a sUAS, but challenging. Wright is an enthusiastic young firefighter intent on impressing his company captain and the on-scene battalion chief. He launches the sUAS intending to stream video of the fire scene to the battalion chief. Five minutes into the flight, Wright receives a message on his sUAS hand controller indicating that the Global Positioning System (GPS) receiver in his aircraft has failed. Without GPS, Wright must “hand fly” the aircraft without the assistance of an autopilot. Wright struggles with maintaining control of the aircraft ultimately losing the battle and striking the windshield of a vehicle travelling northbound on I-29 at 75 mph (121 kph). The driver, blinded by the opaqueness of his suddenly shattered windshield, loses control of the vehicle, travels across the center median, and strikes a school bus traveling southbound. The final tragic toll is 14 fatalities (including both drivers and 12 students) and 26 injured students. Could this accident have been prevented? If so, how?
The utilization of small unmanned aircraft systems (sUAS) by U.S. fire departments has grown exponentially over the last six years. In 2014 less than 12 agencies had acquired sUAS. Today, an estimated 566 fire departments, and an estimated 1134 law enforcement agencies and search and rescue teams, are using sUAS. A number of factors affected this growth including the cost effectiveness of the technology when compared to traditional manned aircraft; a general realization that drone use in public safety is not inherently an invasion of people’s privacy; and the Federal Aviation Administration’s enactment, in August of 2016, of 14 CFR Part 107, “The Small UAS Rule”. Part 107 created a new FAA certificate: Remote Pilot. Applicants for a Remote Pilot Certificate must: pass an FAA examination consisting of 60 questions covering 12 aeronautical subject matter areas and apply for the certificate in-person or online with the FAA. The FAA Remote Pilot Examination covers a broad spectrum of topics including airspace, aeronautical charts, meteorology, aeronautical decision making, and the specifics of Part 107 regulations. Interestingly, an individual can obtain an FAA Remote Pilot Certificate without ever having flown a sUAS!
Flying safely in our national air space requires knowledge and skill. While the FAA’s Remote Pilot Examination is a good evaluation of remote pilot knowledge, the lack of a practical examination leaves a skill evaluation void that potentially increases the liability exposure of individuals and agencies using sUAS. There is a clear need for an accompanying skills evaluation to ensure the safety of the remote pilot, bystanders, property, and manned aircraft in the area.
According to Adam Jacoff , a project manager at the National Institute of Standards and Technology (NIST), “The first step toward credentialing remote pilot skills is to get everybody onto the same measuring stick. That’s where standard test methods can play a key role. Especially across public safety, industrial, commercial, and even recreational pilots. All need to demonstrate essential maneuvers to maintain positive aircraft control while performing whatever payload functionality is necessary to successfully perform the intended tasks.”
Adam Jacoff is leading an international effort to develop standard test methods for small unmanned aircraft systems. The initial suites for Maneuvering and Payload Functionality can be used to quantitatively evaluate various system capabilities and remote pilot proficiency. They are being standardized through the ASTM International Standards Committee on Homeland Security Applications; Response Robots (ASTM E54.09).They are also referenced as Job Performance Requirements in the National Fire Protection Association Standard for Small Unmanned Aircraft Systems Used For Public Safety Operations (NFPA 2400) and the ASTM Standard Guide for Training for Remote Pilot in Command of Unmanned Aircraft Systems Endorsement (ASTM F38.03 F3266-18).
Participants in NIST’s test method validation exercises learn how to fabricate apparatuses, conduct trials, and embed them into their own training and credentialing programs. The NIST test methods are already being used as the basis for state-wide credentialing of emergency responders in Colorado and Texas. Many other state and local emergency response organizations are also adopting the test methods. Canada is moving quickly to implement these tests as the basis for credentialing their emergency responders nationwide. Others will certainly follow. The Airborne Public Safety Accreditation Commission (APSAC) is strongly considering their adoption, as is the Civil Air Patrol, an auxiliary of the U.S. Force, as they seek to standardize their pilot credentialing across 52 wings consisting of over 1200 sUAS pilots.
Ben Miller, Director of the Colorado Center of Excellence for Advanced Technology Aerial Firefighting, has followed NIST’s sUAS Standard Test Methods project from the inception. “NIST was one of the very first evaluation groups to show interest during the early days of UAS in public safety. The rigor that today’s Standard Test Methods show is a direct result of their years of work into the project. The applicability of the method supports acquisition decisions as well as employment considerations. The NIST sUAS Standard Test Methods produce data that can be used to answer the questions of what system do I buy and what system do I use for which mission?” The Colorado Center of Excellence for Advanced Technology Aerial Firefighting provides this certification process to stakeholder public safety agencies within the State of Colorado. To date, 16 agencies and 42 UAS operators have gone through the process.
The NIST sUAS Test Methods include four different “test lanes”: Basic Proficiency Evaluation for Remote Pilots (Part 107 qualification); Open Test Lane; Obstructed Test Lane; and Confined Test Lane. These test methods can all be used to evaluate sUAS capabilities and sensor systems, or remote pilot proficiency for credentialing. The tests are easy to conduct alone or in groups, and inexpensive enough to set up multiple concurrent lanes. They are quick to perform, typically less than 30 minutes to conduct all the tests in a given lane. NIST has done an excellent job of creating a comprehensive user guide, scoring forms, and apparatus targets that can be printed and placed in the buckets. It’s that simple. But the results are more than the sum of their parts. Fly the lane once and you immediately see how easy it is to evaluate precise station keeping (with or without GPS, downward image flow, windy conditions, etc.). There are various repeatable maneuvering flight paths, and tests for zoom lenses and any additional sensors. The flight paths get incrementally harder but all use the same essential bucket alignment tasks so you can evaluate yourself, know your range to various targets, then compare your results over time or against others.
The Basic Proficiency Evaluation for Remote Pilots (BPERP) is the entry level test method. It is designed to complement the Federal Aviation Administration’s Part 107 Remote Pilot Certificate by providing an inexpensive, easily duplicatable, mechanism for assessing remote pilot flying skills. The BPERP can be administered in 10 minutes utilizing 3 omni bucket stands, a 50’ tape measure, and a stop watch. The BPERP requires a compact test area of 50’x 20’ so can easily be administered indoors or outdoors. The BPERP requires the remote pilot to conduct 3 takeoffs and landings from a 12” radius circle, climb to specified altitudes of 10’ and 20’ AGL, conduct yawing turns, and conduct forward, reverse, and transverse flight maneuvers. The goal is to capture still images of 36 targets that are placed within 2 gallon buckets that are fastened to three omni bucket test stands that are constructed from 2”x 4” and 4”x 4” lumber. The bucket stands are easy to assemble and can be transported in a couple of nylon golf club bags or simply stacked and placed in a vehicle. The test consists of one maneuvering phase and two transverse flight phases. Pilots earn one point for each accurately captured target image, 2 points for an accurate first landing, and one point each for accurate second and third landings. Scoring sheets are available from NIST. Agencies set their own benchmark scores for passing the test.
NIST Standard sUAS Test Methods represent an excellent way for organizations to “raise the bar” on their remote pilot credentialing with more rigorous and comparable evaluations. Fire agencies can also utilize them to get more informed about what different sUAS equipment can reliably do. The combination of pilot skills and equipment capabilities, with tracked scores over time, provide an essential measure of “readiness” for any given mission. Each set of tests, either conducted in a standard test lane or embedded into an operational training scenario, enables each pilot and organization to evaluate their readiness more rigorously, while practicing their procedures, data collection and logging. Quantitative scores captured in standard test methods can provide the rationale for changes that need to be made. By establishing minimum thresholds of remote pilot proficiency, agencies will further insulate themselves from potential civil liability by demonstrating due diligence in vetting their sUAS pilots. The NIST sUAS Test Methods can provide that missing element in every organization’s training program, an easy to implement “measuring stick” for systems and pilots. While almost nothing is certain, it is highly likely that the sUAS accident described in the preamble could have been prevented if Firefighter Wright had been subjected to a rigorous sUAS flying skills evaluation. Such an evaluation could have made him aware of shortcomings in his manual flight skills and encouraged him to improve those skills. Secondarily, implementation of a flying skills evaluation, such as the NIST sUAS Standard Test Methods, would be further evidence that Firefighter Wright’s agency had been diligent in their pilot screening and training. Such evidence is extremely valuable in the defense of a civil liability lawsuit.
For more information, go to RobotTestMethods.nist.gov
About the Author
Alan Frazier currently serves as a Senior Fellow at Georgetown University. He is working with the National Institute of Standards and Technology (NIST) to validate and implement these and other test methods with any interested public safety organization. He hosts “train the trainer” workshops and exercises to help such organizations get started. He is also involved in developing and validating new test methods for sUAS. Alan previously served as an Associate Professor of Aviation at the University of North Dakota. He is a 40 year law enforcement professional having served as a sworn officer and supervisor with city, county, state and federal law enforcement agencies. He founded, and served 10 years as the officer-in-charge of, the Northeast Regional sUAS Team in Grand Forks, North Dakota. He is an FAA Airline Transport Pilot rated to fly single and multi-engine airplanes, helicopters, gliders, and small unmanned aircraft systems.
