Testing vs. Assessing

As aircraft systems, technology, communications, and the processes used to manage an aviation operation continue to improve, so must the industry advance the way it measures pilot proficiency and conducts competency assessments. Training just to get ready for a flight test fails to maximize the potential for learning in an environment increasingly filled with next-generation pilots. That’s where training-to-proficiency comes in.

Having just returned from a recurrent simulator training session, it’s a good time to reflect on the considerable advantages of training-to-proficiency versus training solely for the purpose of preparing for a flight test. To illustrate the difference between the two, private air operators in Canada are trained and assessed to Transport Canada flight test standards and beyond in a productive cognitive learning environment, and the resulting knowledge and skills achieved clearly proves the merits of the training-to-proficiency program. Many commercial operators, however, are still mandated to undergo a prescribed pilot proficiency check (PPC). This method of preparatory training and competency testing provides little time and resources for acquiring new knowledge and skills.  

As the aviation industry progresses technologically, are our methods of flight crew assessment advancing at the same pace? Is the prevalent method to evaluate a pilot’s abilities, aptitude, skill and performance through a PPC regime the best method—or is there a better way?

Typical preparatory training and the subsequent PPC normally comprises two simulator training sessions devoted to the review of emergency and abnormal procedures and standard operating procedures. The final sim session exists solely to give the candidate an opportunity to demonstrate competency during a flight test. This method of recurrent training and testing—although mandated for years—is questionable in terms of its effectiveness in achieving utmost proficiency. The question could be asked: “If the pilot candidate has already proven proficiency in the preparatory PPC training, then why does he/she need to be assessed again in a testing environment?” 

Essentially, the PPC program focuses specifically on getting ready for the test, with little quality time devoted to learning imperative skills such as problem-solving, decision-making, situational awareness, communications and workload management. 

Training-to-proficiency is focused on a competency objective to proficiently conduct complex manoeuvres and procedures, while at the same time acquiring knowledge, experience and skill. Conversely, progressive learning is nominal in the traditional preparatory training and PPC model.

Private operators have the flexibility during recurrent simulator training to not only accomplish all the Transport Canada-prescribed training elements, but also to devote further time during the third sim session, to do scenario-based training in areas tailored specifically to the operator. Curricula can be developed where abnormal and emergency scenarios are accomplished during the time normally devoted to the PPC. As an example, controlled flight into terrain (CFIT) escape manoeuvres could be simulated with departures out of critical airports such as Aspen, Colo., with high and hot conditions, or complex area navigation (RNAV) approaches and standard instrument departures (SIDs) can be performed to competency. 

Training-to-proficiency also allows time for lesson plans that expose pilots to current industry occurrences, including up-to-date research and post-accident reports from the Flight Safety Foundation, Transport Canada Aviation Safety Letters, Transportation Safety Board, manufacturers, and others.

As an example, operators may incorporate elements into the training-to-proficiency exercises related to pitot/static system failures using the Air France 447 final accident report, and/or aerodynamic stall recovery techniques and proper stall warning response learned from the 2009 Colgan Air 3407 accident report. 

When learning and assessing is reflective of current aviation industry events, the results are considerable, especially with regard to comprehensible awareness and applied new knowledge. Replacing a regulated PPC with a simulator session devoted to current industry events is certainly compelling, especially when you consider the imperative of our SMS culture: constant learning. 

Rudy Toering, president and CEO of the Canadian Business Aviation Association, recently stated: “In the Private 604 sector, operators must train at least once a year in a simulator and the competency is determined during the training as professional evaluators attest to the proficiency progressively through the training. No prescriptive check ride is therefore required, and as a result the 604 operators use the test ride slot in the simulator to practice and master select competencies as well as focusing the training on specific operator needs. These factors clearly have created one of the most accident-free sectors in aviation worldwide.”

Progress has been made toward providing a better cognitive learning environment for commercial pilots, including line-oriented flight training (LOFT). LOFT is a method wherein operators can substitute a semi-annual PPC session for training. A LOFT curriculum incorporates representative flight segments containing normal, abnormal, and emergency procedures that may typically be expected in daily operations. However, the other semi-annual session is devoted to the old methodology of training followed by a PPC. 

Another program that has advanced pilot competency assessment is the advanced qualification program (AQP). When the Canadian Aviation Regulations (CARs) were initially drafted, Transport Canada provided the AQP provision. This is a voluntary program and an alternative method of training and evaluation. AQP uses a systematic methodology for developing proficiency-based training and evaluation programs in lieu of traditional training programs. AQP was also established to permit a greater degree of regulatory flexibility in the approval of innovative pilot training programs. Based on a documented analysis of operational requirements, an air operator under AQP may propose to depart from traditional practices with respect to what, how, when, and where training and testing is conducted. The program requires significant administration and as such is not practical for smaller operators, although for larger commercial operators, AQP is a progressive step toward better cognitive learning.

According to Capt Mike Zorychta, vice president of flight operations for CanJet Airlines, “LOFT and AQP are advancing the quality of training and proficiency in airline operations.” 

These training and testing program advances certainly help to alleviate the intense stress typically experienced by pilots during a PPC. What role does this episodic stress play in pilot performance? Consider what the Transport Canada Pilot Proficiency Check – Flight Test Guide states: “A PPC Flight Check is always a jeopardy ride for the individuals involved.” This profound statement certainly could have the effect of creating unproductive stress during a flight test, as it becomes a prevailing negative thought while the candidate simultaneously is doing their utmost to proficiently demonstrate multi-tasking skills, resource management and flying expertise. 

Cognitive learning is the method wherein pilots process information, reason, remember, and relate information. Performance results are more effective when learned in a positive cognitive environment where “jeopardy” is not looming. The capacity to produce desired results is inherent in every pilot; however, the teaching environment will ultimately have the greatest effect on confidence and thus human factors performance. 

The Transport Canada publication, Human Factors for Aviation – Basic Handbook, Transport defines human factors as follows: “Human Factors is the study of how people interact with their environments. In the case of aviation, it is the study of how pilot performance is influenced by such issues as the design of the cockpits, temperature and altitude, the functions of the organs of the body, the effect of the emotions, and interaction and communication with the other participants in the aviation community, such as crew members and air traffic control personnel.” 

A positive learning and assessment environment influences and reinforces pilot performance. The current industry system of testing pilot performance through a PPC eliminates many of the core elements contained in the human factors definition, whereas training-to-proficiency can accommodate all the elements. 

A Boeing article entitled The Role of Human Factors in Aviation Safety states: “By continuously studying the interface between human performance and commercial airplanes, Boeing continues to help operators apply the latest human factors knowledge for increased flight safety.” The high level of pilot performance required to proficiently interface knowledge and understanding of ever-increasing complex systems is best assessed through extensive training and exposure. A one-shot PPC can have the result of evaluating how the pilot performs at that moment, and may not be an accurate assessment of his/her human ability to manage complex systems and resources. 

As aircraft systems become more complex, as communications evolve and processes advance, our industry must also move forward when it comes to pilot proficiency and competency assessments. The antiquated methodology of training for the sole purpose of preparing for a flight test does not provide an environment for maximum learning, nor will it equip the next generation of pilots with the knowledge and skills required to keep current with industry progression and innovation.