Reliability of Assessors' Competence Ratings in Air Traffic Control Training

Luchtverkeersleiding Nederland, which is responsible for air traffic control in the Netherlands, has designed a competence-based assessment system for air traffic controller training. In this paper, inter-rater agreement and rating errors (leniency and severity, range restriction and central tendency, halo errors) were examined to evaluate the reliability of assessor's ratings. The findings have shown that assessors' competence ratings were interchangeable to a certain extent while rating errors were minimally present. Although assessors agree with each other in overall performance level, they do not necessarily completely agree in singular competences. The reliability of simulator tests was higher than the reliability of progression reports filled in over time. As a result of these findings, refresher training of assessors has been organized to reduce the occurrence of rating errors and to increase inter-rater agreement.

• Availability:
  • Find a library where document is available. Order URL: http://worldcat.org/oclc/47250898
• Authors:
  • Oprins, Esther
  • Burggraaff, Ernst
  • van Weerdenburg, Hans
• Publication Date: 2006

Language

• English

Media Info

• Media Type: Print
• Features: References; Tables;
• Pagination: pp 305-321
• Serial:
  • Human Factors and Aerospace Safety
  • Volume: 6
  • Issue Number: 4
  • Publisher: Ashgate Publishing Limited
  • ISSN: 1468-9456
  • Serial URL: http://www.ashgate.com/subject_area/aviation/aviation_journals.htm

Subject/Index Terms

• TRT Terms: Air traffic controllers; Case studies; Evaluation and assessment; Performance evaluations; Ratings; Reliability (Statistics); Training
• Identifier Terms: Luchtverkeersleiding Nederland
• Uncontrolled Terms: Competence
• Geographic Terms: Netherlands
• Subject Areas: Aviation; Data and Information Technology; Education and Training; Operations and Traffic Management;

Filing Info

• Accession Number: 01091352
• Record Type: Publication
• Files: TRIS
• Created Date: Apr 23 2008 9:22AM